Vehicle control method and system, electronic equipment and storage medium

Abstract

The invention discloses a vehicle control method and system, electronic equipment and a storage medium, and belongs to the technical field of intelligent automobiles. In the method, when a vehicle-mounted terminal recognizes a keyword for starting a vehicle according to voice data, under the condition that a driver sits on a current driver seat, the voice data and a video picture of the driver seat are sent to a server for further safety verification; when the server determines that both the voice data and the driver are successfully matched with the related information provided by the target user, the server notifies the vehicle-mounted terminal to control the vehicle to start, so that the vehicle starting safety is ensured on the basis of realizing voice control on the vehicle starting, the situation that the user forgets to take a vehicle key or the vehicle cannot be started when the vehicle key fails is avoided, and the user experience is improved. And the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a vehicle control method, system, electronic device and storage medium. Background Art

[0002] With the rapid development of smart car technology, users can unlock or lock their vehicles without using car keys, for example, by touching the unlocking area on the inside of the door handle.

[0003] However, the above method easily causes the user to forget to bring the car key, while the car key is still required to start the vehicle. Once the user forgets to bring the car key, the vehicle will not be able to start, resulting in a poor user experience. Summary of the invention

[0004] The embodiment of the present application provides a vehicle control method, system, electronic device and storage medium, and the technical solution is as follows:

[0005] In a first aspect, a vehicle control method is provided, which is applied to a vehicle control system, wherein the vehicle control system includes a vehicle-mounted terminal and a server, and the method includes:

[0006] The vehicle-mounted terminal acquires first voice data and a first video picture of a driver's seat in the vehicle;

[0007] The vehicle-mounted terminal performs voice recognition on the first voice data, and when the first voice data successfully matches a first keyword among target keywords provided by a target user and the first video image indicates that there is a driver in the driver's seat, the first voice data and the first video image are sent to the server, and the first keyword indicates starting the vehicle;

[0008] The server matches the first voice data with the target voice data provided by the target user, and matches the facial information of the driver in the first video screen with the target facial information provided by the target user, and sends a first notification message of successful matching to the vehicle terminal when the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information;

[0009] The vehicle-mounted terminal controls the vehicle to start based on the first notification message.

[0010] In some embodiments, the server matches the first voice data with target voice data provided by a target user in a voice database, and matches the facial information of the driver in the first video screen with the target facial information provided by the target user, including:

[0011] The server extracts features from the first voice data to obtain timbre features of the first voice data, obtains a first matching degree between the timbre features and the timbre features of the target voice data, and determines that the first voice data successfully matches the target voice data when the first matching degree is greater than or equal to a first threshold;

[0012] The server performs facial recognition on the first video image to obtain facial information of the driver, obtains a second matching degree between the facial information and the target facial information, and determines that the facial information successfully matches the target facial information when the second matching degree is greater than or equal to a second threshold.

[0013] In some embodiments, the method further comprises:

[0014] The server sends a second notification message of matching failure to the vehicle terminal when the first voice data fails to match the target voice data and the facial information successfully matches the target facial information, or when the first voice data successfully matches the target voice data and the facial information fails to match the target facial information;

[0015] The vehicle-mounted terminal sends a security verification message to the target user based on the second notification message, and controls the vehicle to start when the target user performs a confirmation operation on the security verification message.

[0016] In some embodiments, when the target user performs a confirmation operation on the security verification message, the method further includes any of the following:

[0017] The server updates the target voice data based on the matching result between the first voice data and the target voice data when the first voice data fails to match the target voice data and the face information successfully matches the target face information;

[0018] When the first voice data successfully matches the target voice data and the facial information fails to match the target facial information, the server updates the target facial information based on the matching result between the facial information and the target facial information.

[0019] In some embodiments, the method further comprises:

[0020] The server sends a third notification message of matching failure to the vehicle terminal when the first voice data fails to match the target voice data and the facial information fails to match the target facial information;

[0021] The vehicle-mounted terminal triggers an anti-theft alarm based on the third notification message and notifies the target user.

[0022] In some embodiments, after the vehicle is started, the method further comprises:

[0023] The vehicle-mounted terminal acquires second voice data and performs voice recognition on the second voice data, and sends the second voice data to the server when the second voice data successfully matches the second keyword in the target keyword, wherein the second keyword indicates shutting down the vehicle;

[0024] The server matches the second voice data with the target voice data in the voice database, and sends a fourth notification message of successful matching to the vehicle terminal when the second voice data successfully matches the target voice data;

[0025] The in-vehicle terminal controls the vehicle to shut down based on the fourth notification message.

[0026] In some embodiments, after the vehicle is started, the method further comprises:

[0027] The vehicle-mounted terminal acquires a second video picture of the vehicle, where the second video picture includes a video picture inside the vehicle and a video picture outside the vehicle;

[0028] When the second video screen indicates that the vehicle is in the target scene, the vehicle-mounted terminal obtains third voice data and performs voice recognition on the third voice data. When the third voice data successfully matches the second keyword in the target keyword, the vehicle is controlled to be closed, and the second keyword indicates to close the vehicle.

[0029] In a second aspect, a vehicle control system is provided, the vehicle control system comprising a vehicle-mounted terminal and a server;

[0030] The vehicle-mounted terminal is used to: obtain first voice data and a first video picture of a driver's seat in a vehicle; perform voice recognition on the first voice data, and when the first voice data successfully matches a first keyword in a target keyword provided by a target user and the first video picture indicates that there is a driver in the driver's seat, send the first voice data and the first video picture to the server, and the first keyword indicates starting the vehicle;

[0031] The server is used to match the first voice data with the target voice data provided by the target user, and match the facial information of the driver in the first video screen with the target facial information provided by the target user, and send a first notification message of successful matching to the vehicle terminal when the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information;

[0032] The vehicle-mounted terminal is further used to control the start-up of the vehicle based on the first notification message.

[0033] In a third aspect, a vehicle control device is provided, which is applied to a vehicle-mounted terminal in a vehicle control system, the vehicle control system also including a server, and the device includes:

[0034] An acquisition module for first voice data and a first video picture of a driver's seat in a vehicle;

[0035] a sending module, configured to perform voice recognition on the first voice data, and when the first voice data successfully matches a first keyword in a target keyword provided by a target user and the first video screen indicates that there is a driver in the driver's seat, send the first voice data and the first video screen to the server, wherein the first keyword indicates starting the vehicle; the server is configured to match the first voice data with the target voice data provided by the target user, and match the facial information of the driver in the first video screen with the target facial information provided by the target user, and when the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information, send a first notification message of successful matching to the vehicle-mounted terminal;

[0036] A control module is used to control the vehicle to start based on the first notification message.

[0037] In a fourth aspect, a vehicle control device is provided, which is applied to a server in a vehicle control system, the vehicle control system also including a vehicle-mounted terminal, and the device includes:

[0038] A receiving module, used for receiving first voice data sent by the vehicle-mounted terminal and a first video picture of a driver's seat in the vehicle;

[0039] a matching module, configured to match the first voice data with the target voice data provided by the target user, and to match the facial information of the driver in the first video screen with the target facial information provided by the target user;

[0040] A sending module, configured to send a first notification message of successful matching to the vehicle terminal when the first voice data successfully matches the target voice data and the face information successfully matches the target face information;

[0041] The vehicle-mounted terminal is used to perform voice recognition on the first voice data, and when the first voice data successfully matches the first keyword among the target keywords provided by the target user and the first video image indicates that there is a driver in the driver's seat, the first voice data and the first video image are sent to the server, and the first keyword indicates to start the vehicle; and the vehicle start is controlled based on the first notification message.

[0042] In a fifth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the steps executed by the vehicle terminal in the vehicle control method provided in the first aspect; or, to implement the steps executed by the server in the vehicle control method provided in the first aspect.

[0043] In the sixth aspect, a computer-readable storage medium is provided, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor to implement the steps executed by the vehicle terminal in the vehicle control method provided in the first aspect; or, to implement the steps executed by the server in the vehicle control method provided in the first aspect.

[0044] In summary, in the vehicle control method provided in the embodiment of the present application, when the vehicle-mounted terminal recognizes the keyword used to start the vehicle based on the voice data, if there is a driver in the current driving seat, the voice data and the video image of the driver's seat are sent to the server for further security verification. When the server determines that the voice data and the driver are successfully matched with the relevant information provided by the target user, the vehicle-mounted terminal is notified to control the vehicle start-up. On the basis of realizing voice-controlled vehicle start-up, the safety of vehicle start-up is ensured, the situation where the user cannot start the vehicle when he forgets to bring the car key is avoided, and the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0047] Figure 2 is a flow chart of a vehicle control method provided by an embodiment of the present application;

[0048] Figure 3 is a structural schematic diagram of a vehicle control device provided in an embodiment of the present application;

[0049] Figure 4 is a structural schematic diagram of another vehicle control device provided in an embodiment of the present application;

[0050] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards. For example, the voice data, video images, facial information, etc. involved in this application are all obtained with full authorization.

[0053] Figure 1 Schematic diagram of an implementation environment provided by the embodiment of the present application. Figure 1 As shown, the implementation environment includes a vehicle control system, which includes a vehicle terminal 100 and a server 200. The vehicle terminal 100 and the server 200 are communicatively connected to each other and perform data transmission and interaction through a network.

[0054] The vehicle terminal 100 is deployed in a physical vehicle and runs a client to provide multiple functions, such as vehicle control and status monitoring, navigation and positioning, multimedia entertainment, communication and social interaction, personalized settings and user management, etc., which are not limited in this application. In some embodiments, the vehicle terminal 100 includes the following multiple components to achieve multiple functions.

[0055] The power management component 101 is used to manage the power of the vehicle. The power management component 101 is connected to the processor 102, the microphone 103 and the network card 104 respectively.

[0056] The processor 102 is used to process various data and instructions. The processor 102 is connected to other components through multiple interfaces. Schematically, the processor 102 includes multiple types of processors, such as a microcontroller unit (MCU) and a system on chip (SOC). The MCU and the SOC work together to implement the functions of the processor 102.

[0057] Microphone 103 is used to collect voice signals.

[0058] The network card 104 is used to provide a network connection function. The vehicle terminal 100 transmits various vehicle data to the server 200 via the network card 104 .

[0059] The external interface 105 is used to provide a connection interface with external devices.

[0060] The communication interface 106, for example, includes a Universal Serial Bus (USB) interface for connecting USB devices, such as storage devices or other USB peripherals; a Controller Area Network (CAN) interface for realizing internal network communication of the vehicle; and a Local Interconnect Network (LIN) interface, which is an auxiliary bus network that cooperates with the CAN bus to realize communication between different Electronic Control Units (ECU) in the vehicle.

[0061] The memory 107 is used to store data and programs. Schematically, the memory 107 includes but is not limited to a non-volatile memory (such as a FLASH memory), a static random access memory, and the like.

[0062] The server 200 is used to provide background services for the vehicle-mounted terminal 100. In some embodiments, the server 200 includes a database 201, a database operation module 202, a network transmission module 203 and a control module 204. Among them, the database 201 is used to store voice data, such as voice data obtained after signal enhancement processing of the original voice signal, or the original voice signal, which is not limited in this application; the database 201 is also used to store face information, such as images or videos containing faces, or face features obtained after feature extraction of images, which is not limited in this application. The database operation module 202 is used to read data from the database 201, store data in the database 201, etc. The network transmission module 203 is used to provide a network connection function. The server 200 receives various data sent by the vehicle-mounted terminal 100 through the network transmission module 203, and sends various data to the vehicle-mounted terminal 100. The control module 204 is used to control the remaining modules to ensure the coordinated work of each module.

[0063] Schematically, the server 200 is, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The number of servers 200 can be more or less, and the embodiments of the present application are not limited to this. Of course, the server 200 can also include other functional servers to provide more comprehensive and diversified services.

[0064] In the present application, the vehicle terminal 100 and the server 200 together constitute a vehicle control system for implementing the vehicle control method provided in the present application. The vehicle control method is implemented through interaction between the vehicle terminal 100 and the server 200.

[0065] In some embodiments, the above network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network. In some embodiments, the data exchanged through the network is represented using technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0066] Reference below Figure 2 , taking the interaction between the vehicle-mounted terminal and the server in the vehicle control system as an example, the vehicle control method provided in this application is introduced.

[0067] Figure 2 : is a flow chart of a vehicle control method provided by an embodiment of the present application. Figure 2 As shown, the method is applied to a vehicle control system, the vehicle control system includes a vehicle-mounted terminal and a server, and taking the interaction between the vehicle-mounted terminal and the server as an example, the method includes the following steps 201 to 208.

[0068] 201. The vehicle-mounted terminal obtains first voice data and a first video image of a driver's seat in a vehicle.

[0069] In the embodiments of the present application, the vehicle may be a fuel vehicle, an electric vehicle, or a hybrid vehicle, which is not limited in the present application. Schematically, the vehicle is equipped with a microphone and a camera, and the microphone may be distributed in multiple locations in the vehicle, such as the center of the roof, in front of the driver's seat, etc. The camera is located in front of or to the side of the driver's seat to ensure that the video image containing the driver's seat can be captured. The vehicle-mounted terminal collects the voice signal in the vehicle through the microphone, converts the collected voice signal into a digital signal, and obtains the first voice data for subsequent processing in the digital domain. In addition, the vehicle-mounted terminal is connected to the camera in communication, such as through a USB interface communication connection, or through a wireless network communication connection, etc., which is not limited in the present application. The camera shoots the driver's seat and transmits the first video image of the driver's seat to the vehicle-mounted terminal.

[0070] 202. The vehicle-mounted terminal performs voice recognition on the first voice data, and when the first voice data successfully matches the first keyword in the target keyword provided by the target user and the first video image indicates that there is a driver in the driver's seat, the first voice data and the first video image are sent to the server.

[0071] In an embodiment of the present application, the target user has the authority to control the start of the vehicle, such as the owner of the vehicle. The vehicle terminal stores the target keywords provided by the target user, and the target keywords instruct the vehicle to perform various operations, such as starting the vehicle, shutting down the vehicle, unlocking the vehicle, locking the vehicle, opening the door, closing the door, opening the window, heating the seat, etc. The present application does not limit the number of target keywords. Among them, the target keywords include a first keyword, and the first keyword indicates starting the vehicle, for example, the first keyword is "starting the vehicle", "vehicle ignition" or "starting the vehicle", etc. In the present application, starting the vehicle is to instruct the vehicle engine to change from a stationary state to a running state, that is, to enter the driving preparation stage. Taking a fuel vehicle as an example, starting the vehicle is to control the ignition system to perform an ignition operation so that the engine starts smoothly. Taking an electric vehicle as an example, starting the vehicle is to drive the motor to start running through the motor control unit, and transmit power to the wheels so that the vehicle starts smoothly.

[0072] Schematically, the vehicle terminal performs voice recognition on the first voice data to convert the first voice data into text, matches the recognized text with the target keyword, and determines whether the recognized text contains the first keyword provided by the target user. If so (for example, the recognized text includes the first keyword), it is determined that the match is successful. In addition, the vehicle terminal performs image recognition on the first video screen to determine whether there is a driver in the driver's seat.

[0073] In this step, when the first voice data successfully matches the first keyword and the first video screen indicates that there is a driver in the driver's seat, it indicates that the driver is in place in the current vehicle and the vehicle needs to be started. Based on this, the vehicle terminal sends the first voice data and the first video screen to the server to perform a safety verification on whether the driver meets the conditions for starting the vehicle, thereby ensuring the safety of the vehicle starting under voice control. In some embodiments, when there is a driver in the driver's seat, the vehicle terminal can also perform a posture analysis on the driver to determine whether the driver is in a normal sitting position, whether the body is obviously blocked, etc., so as to further confirm whether the driver's state is safe for driving.

[0074] In addition, when the vehicle terminal sends the first voice data and the first video image to the server, it also sends a vehicle identification code (VIN) to the server to identify the vehicle. In some embodiments, the vehicle terminal may also send the acquisition time of the first voice data and the first video image, etc., so that the server can correctly parse and process the data, which is not limited in this application.

[0075] 203. The server matches the first voice data with the target voice data provided by the target user, and matches the facial information of the driver in the first video image with the target facial information provided by the target user.

[0076] In an embodiment of the present application, the database of the server stores the target voice data and target face information provided by the target user, wherein the number of target voice data may be one or more, such as the voice data of the car owner user, the voice data of other users authorized by the car owner user, etc. The number of target face information may be one or more, such as the face information of the car owner user, the face information of other users authorized by the car owner user, etc. Schematically, the database uses a relational database (such as MySQL) or a non-relational database (such as MongoDB) to store the target voice data and target face information, and each record corresponds to the identity information of a target user and its pre-registered target voice data and target face information.

[0077] In some embodiments, the server performs feature extraction on the first voice data to obtain the timbre features of the first voice data, obtains a first degree of match between the timbre features and the timbre features of the target voice data, and determines that the first voice data and the target voice data are successfully matched when the first degree of match is greater than or equal to a first threshold. The first threshold is a preset threshold, which can be set according to the algorithm used by the server for feature extraction and business requirements, for example, the first threshold is 90%. Schematically, the server compares the timbre features of the first voice data with the target voice data one by one. During the comparison process, a distance measurement algorithm (such as Euclidean distance or cosine similarity) is used to calculate the first degree of match between different timbre features. In addition, the present application does not limit the manner in which the server determines whether the first voice data and the target voice data are successfully matched. For example, a speech feature extraction model based on deep learning (such as a convolutional neural network or a recurrent neural network) is configured on the server, and the server inputs the first voice data and the target voice data into the speech feature extraction model respectively to obtain the timbre features of the first voice data and the timbre features of the target voice data. The timbre features extracted by the speech feature extraction model can accurately characterize the uniqueness of the user from which the voice data comes, such as information such as vocal tract shape and pronunciation habits.

[0078] In some embodiments, the server performs face recognition on the first video screen, obtains the driver's face information, obtains the second matching degree between the face information and the target face information, and determines that the face information matches the target face information successfully when the second matching degree is greater than or equal to the second threshold. Wherein, the second threshold is a preset threshold, which can be set according to the face recognition algorithm adopted by the server and business requirements, for example, the second threshold is 95%. Schematically, the server compares the driver's face information with the target face information one by one. During the comparison process, a distance measurement algorithm (such as Euclidean distance or cosine similarity) is used to calculate the second matching degree between different face information. In addition, the present application does not limit the face recognition algorithm adopted by the server. For example, a face feature extraction model based on deep learning (such as a convolutional neural network or a recurrent neural network) is configured on the server. The server inputs the first video screen into the face feature extraction model, extracts the key features of the face, such as the position, shape, texture and other feature descriptions of the facial features, and uses the key features as the driver's face information to match the target face information provided in advance by the target user.

[0079] In some embodiments, the server records detailed information of the match during the matching process, including the time of the match, the similarity value of the match, the vehicles and owner information involved, etc., and stores this information in the operation log table of the database for subsequent statistical analysis and troubleshooting.

[0080] 204. When the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information, the server sends a first notification message of successful matching to the vehicle terminal.

[0081] In the embodiment of the present application, when the first voice data successfully matches the target voice data and the face information successfully matches the target face information, the server generates a first notification message of successful matching, and uses an encrypted transmission mechanism to send the first notification message to the vehicle terminal through the network. Schematically, the first notification message includes an identification field (such as a status code indicating a successful match) and additional information (such as an identification of the driver's identity verification, authorization information, etc.), so that the vehicle terminal can perform corresponding operations and processing later.

[0082] 205. The vehicle-mounted terminal controls the vehicle to start based on the first notification message.

[0083] In an embodiment of the present application, after receiving the first notification message, the vehicle-mounted terminal verifies the legitimacy and integrity of the first notification message, such as checking whether the source of the message is legal, whether it has been tampered with, etc., and ensures the reliability of the message by verifying the digital signature or checksum in the message. After confirming that the first notification message is valid, the vehicle-mounted terminal communicates with the electronic control unit ECU of the vehicle and sends a start instruction to the relevant ECU responsible for starting the vehicle (such as the engine control unit of a fuel vehicle or the motor control unit of an electric vehicle). After receiving the start instruction, these ECUs operate according to the vehicle's established start process.

[0084] After the above steps 201 to 205, when the vehicle-mounted terminal recognizes the keyword for starting the vehicle based on the voice data, if there is a driver in the current driving seat, the voice data and the video image of the driver's seat are sent to the server for further security verification. When the server determines that the voice data and the driver are successfully matched with the relevant information provided by the target user, the vehicle-mounted terminal is notified to control the vehicle start. Since the data stored on the server is not easy to tamper with and has high security, security verification is performed through the server. On the basis of realizing voice-controlled vehicle start-up, the safety of vehicle start-up can be ensured, thereby avoiding the situation where the user forgets to bring the car key or the car key fails and cannot start the vehicle, thereby improving the user experience.

[0085] In some embodiments, the server sends a second notification message of failed matching to the vehicle terminal when the first voice data fails to match the target voice data and the facial information successfully matches the target facial information, or when the first voice data successfully matches the target voice data and the facial information fails to match the target facial information. The second notification message includes the reason for the failed matching. Based on the second notification message, the vehicle terminal sends a security verification message to the target user, and controls the vehicle to start when the target user performs a confirmation operation on the security verification message. In schematic form, when any one of the first voice data and the driver's facial information matches successfully, it indicates that the current driver may meet the conditions for voice control of vehicle start-up. Based on this, the vehicle terminal sends a security verification message to the target user to obtain the authorization of the target user. For example, the vehicle terminal sends a security verification message to the mobile phone terminal used by the target user by sending a text message or making a call, and accordingly, the target user performs a confirmation operation on the security verification message, such as replying to a confirmation text message or replying to a confirmation during a call, etc., which is not limited in this application. In this way, various situations of voice-controlled vehicle start-up are fully considered. For example, when the driver has a hoarse voice due to a cold, the voice data matching may fail. For example, when the driver has a facial injury or is wearing makeup, the facial information matching may fail. The above method can reduce the false rejection rate and improve the user experience through further security verification.

[0086] In some embodiments, when the target user performs a confirmation operation on the security verification message, the vehicle terminal feeds back the confirmation operation to the server. Then, when the first voice data fails to match the target voice data and the facial information successfully matches the target facial information, the server updates the target voice data based on the matching result between the first voice data and the target voice data; and, when the first voice data successfully matches the target voice data and the facial information fails to match the target facial information, the server updates the target facial information based on the matching result between the facial information and the target facial information. This process is that after any one of the first voice data and the driver's facial information matches successfully and the target user has verified the driver's identity, it indicates that the first voice data or facial information that failed to match meets the conditions for voice-controlled vehicle startup. Based on this, the server promptly updates this information to the target voice data and target facial information provided by the target user, thereby improving the accuracy and generalization ability of subsequent matching.

[0087] In other embodiments, when the first voice data fails to match the target voice data and the facial information fails to match the target facial information, the server sends a third notification message of failed matching to the vehicle terminal. The third notification message includes the reason for the failed matching. Based on the third notification message, the vehicle terminal triggers the anti-theft alarm and notifies the target user. In this way, the mechanism of voice-controlled vehicle startup is closely integrated with the vehicle's safety system (such as anti-theft system, driving safety assistance system, etc.). If a non-owner voice is detected trying to start the vehicle, not only will the startup be refused, but the anti-theft alarm will also be triggered, and the owner will be notified to ensure the safety of the vehicle.

[0088] 206. The vehicle-mounted terminal obtains the second voice data and performs voice recognition on the second voice data. When the second voice data successfully matches the second keyword in the target keyword, the second voice data is sent to the server.

[0089] In the embodiment of the present application, after the vehicle terminal controls the vehicle to start, it obtains the second voice data and performs voice recognition on the second voice data. Based on the introduction of the aforementioned step 202, it can be seen that the target keyword instructs the vehicle to perform various operations. In this step, the target user also has the authority to control the vehicle to shut down, such as the owner user. The target keyword includes the second keyword, and the second keyword instructs to shut down the vehicle. For example, the second keyword is "turn off the vehicle", "turn off the vehicle" or "turn off the engine", etc. In the present application, shutting down the vehicle is to instruct the vehicle engine to change from a running state to a stationary state.

[0090] Schematically, the vehicle-mounted terminal performs voice recognition on the second voice data to convert the second voice data into text, matches the recognized text with the target keyword, and determines whether the recognized text contains the second keyword provided by the target user. If so (for example, the recognized text includes the second keyword), it is determined that the match is successful and the second voice data is sent to the server. It should be understood that since shutting down the vehicle does not necessarily require the driver to be in the driver's seat, this step may not consider the video picture of the driver's seat. Of course, in some scenarios, the video picture of the driver's seat can also be obtained, and the second voice data can be sent to the server when the video picture indicates that there is a driver in the driver's seat.

[0091] 207. The server matches the second voice data with the target voice data provided by the target user, and sends a fourth notification message of successful matching to the vehicle terminal if the second voice data matches the target voice data successfully.

[0092] In the embodiment of the present application, the way in which the server matches the second voice data with the target voice data is the same as step 203, so it will not be repeated. When the second voice data matches the target voice data successfully, the server generates a fourth notification message of successful matching, and uses an encrypted transmission mechanism to send the fourth notification message to the vehicle terminal through the network. Schematically, the fourth notification message includes an identification field (such as a status code indicating a successful match) and additional information (such as an identification of the driver's identity verification, authorization information, etc.), so that the vehicle terminal can perform corresponding operations and processing later.

[0093] 208. The vehicle-mounted terminal controls the vehicle to shut down based on the fourth notification message.

[0094] In an embodiment of the present application, after receiving the fourth notification message, the vehicle-mounted terminal verifies the legitimacy and integrity of the fourth notification message, such as checking whether the source of the message is legal, whether it has been tampered with, etc., and ensures the reliability of the message by verifying the digital signature or checksum in the message. After confirming that the fourth notification message is valid, the vehicle-mounted terminal communicates with the vehicle's electronic control unit ECU and sends a shutdown command to the relevant ECU responsible for shutting down the vehicle (such as the engine control unit of a fuel vehicle or the motor control unit of an electric vehicle). After receiving the shutdown command, these ECUs operate according to the vehicle's established shutdown process. Specifically, when shutting down the vehicle, for a fuel vehicle, the engine stops running, and the fuel supply and ignition systems stop working; for an electric vehicle, the drive motor stops running, the high-voltage battery pack stops supplying power to the motor, and the various control units of the vehicle also enter a low-power or dormant state.

[0095] After the above steps 206 to 208, when the vehicle-mounted terminal recognizes the keyword for shutting down the vehicle based on the voice data, the voice data is sent to the server for further security verification. When the server determines that the voice data and the relevant information provided by the target user match successfully, the vehicle-mounted terminal is notified to control the vehicle to shut down. On the basis of realizing voice-controlled vehicle shutdown, the safety of vehicle shutdown is ensured, avoiding the situation where the user forgets to bring the car key or is unable to shut down the vehicle when the car key malfunctions, thereby improving the user experience.

[0096] In some embodiments, after controlling the vehicle to start, the vehicle terminal obtains a second video screen of the vehicle, and the second video screen includes a video screen inside the vehicle and a video screen outside the vehicle; when the second video screen indicates that the vehicle is in the target scene, the vehicle terminal obtains third voice data and performs voice recognition on the third voice data, and controls the vehicle to shut down when the third voice data successfully matches the second keyword in the target keyword. Among them, the vehicle is equipped with multiple cameras for shooting video screens inside and outside the vehicle. The target scene refers to a scene that requires emergency parking, such as a traffic accident or other emergency. In the above manner, after the vehicle is started, the vehicle terminal obtains a real-time video screen of the vehicle. When it is recognized that the vehicle is in the target scene, when the voice data matches the second keyword, there is no need to perform security verification through the server, that is, the vehicle terminal can directly control the vehicle to shut down, avoiding the delay caused by the information interaction between the vehicle terminal and the server in the target scene affecting the safety of vehicle driving.

[0097] In summary, in the vehicle control method provided by the present application, when the vehicle terminal recognizes the keyword for starting the vehicle based on the voice data, if there is a driver in the current driving seat, the voice data and the video picture of the driver's seat are sent to the server for further security verification. When the server determines that the voice data and the driver are successfully matched with the relevant information provided by the target user, the vehicle terminal is notified to control the vehicle to start. On the basis of realizing the voice control of the vehicle start, the safety of the vehicle start is ensured, and the situation that the user forgets to bring the car key or the car key is faulty and cannot start the vehicle is avoided, thereby improving the user experience. In addition, after the vehicle is started, when the vehicle terminal recognizes the keyword for shutting down the vehicle based on the voice data, the voice data is sent to the server for further security verification. When the server determines that the voice data and the relevant information provided by the target user are successfully matched, the vehicle terminal is notified to control the vehicle to shut down. On the basis of realizing the voice control of the vehicle shut down, the safety of the vehicle shut down is ensured, and the situation that the user forgets to bring the car key or the car key is faulty and cannot shut down the vehicle is avoided, thereby improving the user experience.

[0098] See also Figure 3 An embodiment of the present application provides a vehicle control device, which is configured in a vehicle terminal of a vehicle control system. The vehicle control system also includes a server. The device includes an acquisition module 301, a sending module 302 and a control module 303.

[0099] Acquisition module 301, for first voice data and a first video picture of a driver's seat in a vehicle;

[0100] The sending module 302 is used to perform voice recognition on the first voice data, and when the first voice data successfully matches the first keyword in the target keyword provided by the target user and the first video screen indicates that there is a driver in the driver's seat, the first voice data and the first video screen are sent to the server, and the first keyword indicates starting the vehicle; the server is used to match the first voice data with the target voice data provided by the target user, and match the facial information of the driver in the first video screen with the target facial information provided by the target user, and when the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information, a first notification message of successful matching is sent to the vehicle terminal;

[0101] The control module 303 is used to control the vehicle startup based on the first notification message.

[0102] In some embodiments, the apparatus further comprises:

[0103] A receiving module, configured to receive a second notification message of a failed match, wherein the second notification message is sent by the server to the vehicle-mounted terminal when the first voice data fails to match the target voice data and the facial information successfully matches the target facial information, or when the first voice data successfully matches the target voice data and the facial information fails to match the target facial information;

[0104] The control module 303 is further configured to send a safety verification message to the target user based on the second notification message, and control the vehicle to start when the target user performs a confirmation operation on the safety verification message.

[0105] In some embodiments, the apparatus further comprises:

[0106] The security module is used to trigger the anti-theft alarm and notify the target user based on the third notification message of matching failure sent by the server. The third notification message is sent by the server to the vehicle terminal when the first voice data fails to match the target voice data and the facial information fails to match the target facial information.

[0107] In some embodiments, after the vehicle is started, the acquisition module 301 is further used to:

[0108] Acquire the second voice data and perform voice recognition on the second voice data, and send the second voice data to the server if the second voice data successfully matches the second keyword in the target keyword, and the second keyword indicates to shut down the vehicle;

[0109] The control module 303 is also used to control the vehicle to shut down based on a fourth notification message of failed matching. The fourth notification message is sent to the vehicle terminal by the server when the server matches the second voice data with the target voice data in the voice database and the second voice data successfully matches the target voice data.

[0110] In some embodiments, after the vehicle is started, the acquisition module 301 is further used to:

[0111] Acquire a second video picture of the vehicle, the second video picture including a video picture inside the vehicle and a video picture outside the vehicle;

[0112] The control module 303 is also used to obtain third voice data and perform voice recognition on the third voice data when the second video screen indicates that the vehicle is in the target scene. When the third voice data successfully matches the second keyword in the target keyword, the vehicle is controlled to be closed, and the second keyword indicates to close the vehicle.

[0113] In this device, when the vehicle-mounted terminal recognizes the keyword for starting the vehicle based on the voice data, if there is a driver in the current driver's seat, the voice data and the video image of the driver's seat are sent to the server for further security verification. When the server determines that the voice data and the driver are successfully matched with the relevant information provided by the target user, the vehicle-mounted terminal is notified to control the vehicle start. On the basis of realizing voice-controlled vehicle start-up, the safety of vehicle start-up is ensured, and the situation in which the user forgets to bring the car key or cannot start the vehicle when the car key malfunctions is avoided, thereby improving the user experience.

[0114] It should be noted that: the vehicle control device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the vehicle control. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle control device provided in the above embodiment and the vehicle control method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0115] See also Figure 4 An embodiment of the present application provides a vehicle control device, which is configured on a server of a vehicle control system. The vehicle control system also includes a vehicle-mounted terminal. The device includes a receiving module 401, a matching module 402 and a sending module 403.

[0116] The receiving module 401 is used to receive the first voice data sent by the vehicle terminal and the first video picture of the driver's seat in the vehicle;

[0117] A matching module 402, for matching the first voice data with the target voice data provided by the target user, and matching the face information of the driver in the first video screen with the target face information provided by the target user;

[0118] The sending module 403 is used to send a first notification message of successful matching to the vehicle terminal when the first voice data successfully matches the target voice data and the face information successfully matches the target face information;

[0119] The vehicle-mounted terminal is used to perform voice recognition on the first voice data. When the first voice data successfully matches the first keyword in the target keyword provided by the target user and the first video screen indicates that there is a driver in the driver's seat, the first voice data and the first video screen are sent to the server, and the first keyword indicates to start the vehicle; and the vehicle start is controlled based on the first notification message.

[0120] In some embodiments, the matching module 402 is used to:

[0121] Extracting features from the first voice data to obtain timbre features of the first voice data, obtaining a first matching degree between the timbre features and the timbre features of the target voice data, and determining that the first voice data successfully matches the target voice data when the first matching degree is greater than or equal to a first threshold;

[0122] Perform face recognition on the first video image to obtain the driver's face information, obtain a second matching degree between the face information and the target face information, and determine that the face information matches the target face information successfully when the second matching degree is greater than or equal to a second threshold.

[0123] In some embodiments, the sending module 403 is also used to send a second notification message of matching failure to the vehicle terminal when the first voice data fails to match the target voice data and the facial information successfully matches the target facial information, or when the first voice data successfully matches the target voice data and the facial information fails to match the target facial information.

[0124] In some embodiments, when the target user performs a confirmation operation on the security verification message, the device further includes an updating module for any of the following:

[0125] If the first voice data fails to match the target voice data and the face information successfully matches the target face information, updating the target voice data based on the matching result between the first voice data and the target voice data;

[0126] When the first voice data successfully matches the target voice data and the facial information fails to match the target facial information, the target facial information is updated based on the matching result between the facial information and the target facial information.

[0127] In some embodiments, the sending module 403 is also used to: when the first voice data fails to match the target voice data and the facial information fails to match the target facial information, send a third notification message of matching failure to the vehicle terminal.

[0128] In this device, when the vehicle-mounted terminal recognizes the keyword for starting the vehicle based on the voice data, if there is a driver in the current driver's seat, the voice data and the video image of the driver's seat are sent to the server for further security verification. When the server determines that the voice data and the driver are successfully matched with the relevant information provided by the target user, the vehicle-mounted terminal is notified to control the vehicle start. On the basis of realizing voice-controlled vehicle start-up, the safety of vehicle start-up is ensured, and the situation in which the user forgets to bring the car key or cannot start the vehicle when the car key malfunctions is avoided, thereby improving the user experience.

[0129] It should be noted that: the vehicle control device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the vehicle control. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle control device provided in the above embodiment and the vehicle control method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0130] refer to Figure 5 , the embodiment of the present application further provides an electronic device, Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. The electronic device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 501 and one or more memories 502, wherein the memory 502 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 501 to implement the steps performed by the vehicle terminal or server in the vehicle control method provided in the above method embodiment. Of course, the electronic device can also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device can also include other components for realizing the functions of the device, which will not be repeated here.

[0131] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0132] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0133] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that: Applied to a vehicle control system, the vehicle control system includes a vehicle-mounted terminal and a server, and the method includes: The vehicle-mounted terminal acquires first voice data and a first video picture of a driver's seat in the vehicle; The vehicle-mounted terminal performs voice recognition on the first voice data, and when the first voice data successfully matches a first keyword among target keywords provided by a target user and the first video image indicates that there is a driver in the driver's seat, the first voice data and the first video image are sent to the server, and the first keyword indicates starting the vehicle; The server matches the first voice data with the target voice data provided by the target user, and matches the facial information of the driver in the first video screen with the target facial information provided by the target user, and sends a first notification message of successful matching to the vehicle terminal when the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information; The vehicle-mounted terminal controls the vehicle to start based on the first notification message.

2. The method according to claim 1, characterized in that The server matches the first voice data with target voice data provided by a target user in a voice database, and matches the face information of the driver in the first video screen with the target face information provided by the target user, including: The server extracts features from the first voice data to obtain timbre features of the first voice data, obtains a first matching degree between the timbre features and the timbre features of the target voice data, and determines that the first voice data successfully matches the target voice data when the first matching degree is greater than or equal to a first threshold; The server performs facial recognition on the first video image to obtain facial information of the driver, obtains a second matching degree between the facial information and the target facial information, and determines that the facial information successfully matches the target facial information when the second matching degree is greater than or equal to a second threshold.

3. The method according to claim 1, characterized in that: The method further comprises: The server sends a second notification message of matching failure to the vehicle terminal when the first voice data fails to match the target voice data and the facial information successfully matches the target facial information, or when the first voice data successfully matches the target voice data and the facial information fails to match the target facial information; The vehicle-mounted terminal sends a security verification message to the target user based on the second notification message, and controls the vehicle to start when the target user performs a confirmation operation on the security verification message.

4. The method according to claim 3, characterized in that: In the case where the target user performs a confirmation operation on the security verification message, the method further includes any one of the following: The server updates the target voice data based on the matching result between the first voice data and the target voice data when the first voice data fails to match the target voice data and the face information successfully matches the target face information; When the first voice data successfully matches the target voice data and the facial information fails to match the target facial information, the server updates the target facial information based on the matching result between the facial information and the target facial information.

5. The method according to claim 1, characterized in that The method further comprises: The server sends a third notification message of matching failure to the vehicle terminal when the first voice data fails to match the target voice data and the facial information fails to match the target facial information; The vehicle-mounted terminal triggers an anti-theft alarm based on the third notification message and notifies the target user.

6. The method according to claim 1, characterized in that After the vehicle is started, the method further includes: The vehicle-mounted terminal acquires second voice data and performs voice recognition on the second voice data, and sends the second voice data to the server when the second voice data successfully matches the second keyword in the target keyword, wherein the second keyword indicates shutting down the vehicle; The server matches the second voice data with the target voice data in the voice database, and sends a fourth notification message of successful matching to the vehicle terminal when the second voice data successfully matches the target voice data; The in-vehicle terminal controls the vehicle to shut down based on the fourth notification message.

7. The method according to claim 1, characterized in that After the vehicle is started, the method further includes: The vehicle-mounted terminal acquires a second video picture of the vehicle, where the second video picture includes a video picture inside the vehicle and a video picture outside the vehicle; When the second video screen indicates that the vehicle is in the target scene, the vehicle-mounted terminal obtains third voice data and performs voice recognition on the third voice data. When the third voice data successfully matches the second keyword in the target keyword, the vehicle is controlled to be closed, and the second keyword indicates to close the vehicle.

8. A vehicle control system, characterized in that: The vehicle control system includes a vehicle-mounted terminal and a server; The vehicle-mounted terminal is used to: obtain first voice data and a first video picture of a driver's seat in a vehicle; perform voice recognition on the first voice data, and when the first voice data successfully matches a first keyword in a target keyword provided by a target user and the first video picture indicates that there is a driver in the driver's seat, send the first voice data and the first video picture to the server, and the first keyword indicates starting the vehicle; The server is used to match the first voice data with the target voice data provided by the target user, and match the facial information of the driver in the first video screen with the target facial information provided by the target user, and send a first notification message of successful matching to the vehicle terminal when the first voice data successfully matches the target voice data and the facial information successfully matches the target facial information; The vehicle-mounted terminal is further used to control the start-up of the vehicle based on the first notification message.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the steps executed by the vehicle terminal in the vehicle control method as described in any one of claims 1 to 7, or to implement the steps executed by the server in the vehicle control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by the processor to implement the steps executed by the vehicle terminal in the vehicle control method as described in any one of claims 1 to 7, or to implement the steps executed by the server in the vehicle control method as described in any one of claims 1 to 7.

Images