A voice control method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the field of vehicle control technology, and provides a voice control method, device, terminal device and storage medium. In the embodiment of the present application, a voice command is obtained; when there is missing information in the voice command, auxiliary information is obtained; the auxiliary information includes visual information and / or vehicle information; the voice command is compensated according to the auxiliary information to obtain a compensated voice command; the compensated voice command is executed, so that the vehicle accurately responds to the voice command issued by the passenger.

Description

Technical Field

[0001] The present application belongs to the field of vehicle control technology, and in particular, relates to a voice control method, apparatus, terminal device and storage medium. Background Art

[0002] With the development of society, vehicles are becoming more and more common in people's lives. Existing vehicles have added voice control functions to improve the convenience of people's control of the vehicles. However, since everyone's speaking style and way are different in real scenarios, the control devices in the vehicle will often receive some ambiguous commands with unclear intentions, resulting in the vehicle being unable to accurately respond to the voice commands issued by the passengers. Summary of the invention

[0003] The embodiments of the present application provide a voice control method, apparatus, terminal device and storage medium, which can solve the problem that the vehicle cannot accurately respond to the voice commands issued by the passengers.

[0004] In a first aspect, an embodiment of the present application provides a voice control method, comprising:

[0005] Get voice commands;

[0006] When there is missing information in the voice command, obtaining auxiliary information; the auxiliary information includes visual information and / or vehicle information;

[0007] Performing information compensation on the voice command according to the auxiliary information to obtain a compensated voice command;

[0008] Execute the above compensated voice command.

[0009] In a second aspect, an embodiment of the present application provides a voice control device, including:

[0010] A command acquisition module, used to acquire voice commands;

[0011] An information acquisition module, used to acquire auxiliary information when there is missing information in the voice command; the auxiliary information includes visual information and / or vehicle information;

[0012] An information compensation module, used to perform information compensation on the voice command according to the auxiliary information to obtain a compensated voice command;

[0013] The execution module is used to execute the compensated voice command.

[0014] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned voice control methods when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned voice control methods are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the voice control methods in the first aspect.

[0017] In the embodiment of the present application, a voice command is obtained and a corresponding judgment is performed. When there is missing information in the voice command, auxiliary information is obtained; the auxiliary information includes visual information and / or vehicle information, and the voice command is compensated according to the auxiliary information to obtain a compensated voice command, that is, a more accurate voice command, so as to execute the compensated voice command, so that the vehicle accurately responds to the voice command issued by the passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a schematic diagram of a first flow chart of the voice control method provided in an embodiment of the present application;

[0020] Figure 2 This is a second flow chart of the voice control method provided in the embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0022] Figure 4 This is a third flow chart of the voice control method provided in the embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the structure of a voice control device provided in an embodiment of the present application;

[0024] Figure 6It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] Figure 1 FIG. 1 is a flow chart of a voice control method in an embodiment of the present application. The execution subject of the method may be a terminal device, which may be arranged in a vehicle. Figure 1 As shown, the above voice control method may include the following steps:

[0031] Step S101: Acquire voice commands.

[0032] In this embodiment, the passengers in the vehicle can wake up the voice assistant in the terminal device through sound, so that the voice assistant detects the voice command issued by the passengers, so as to understand the intention of the passengers by analyzing the voice command, that is, the operation that the passengers expect the terminal device to perform, so as to achieve the purpose of voice control of the vehicle, and the terminal device realizes the interaction process between the vehicle and the passengers by executing the voice command. Specifically, the terminal device can obtain the voice command issued by the passengers through SDK detection.

[0033] Specifically, the terminal device can pre-set the wake-up word so that when it detects that the wake-up word exists in the voice uttered by the user, it triggers the function of executing the corresponding voice command.

[0034] Step S102: when there is missing information in the voice command, obtain auxiliary information; the auxiliary information includes visual information and / or vehicle information.

[0035] In this embodiment, since the detected voice command may be a fuzzy command, that is, there is missing information in the acquired voice command, so that the terminal device cannot respond to the voice command, it is necessary to determine whether there is missing information in the voice command after acquiring the voice command, for example, the voice command is "open" without the object to be operated, or the voice command is "air conditioning" without a specific operation method. Therefore, the terminal device responds to the voice command with missing information by acquiring auxiliary information, and the auxiliary information includes visual information and / or vehicle information, so as to use the information acquired by different sensors through different scenarios to achieve targeted auxiliary processing of voice commands, and assist in processing voice commands in a multimodal manner when necessary, thereby improving the accuracy of voice commands and improving the processing speed on the basis of ensuring the accuracy of voice commands. Among them, the above-mentioned vehicle information includes but is not limited to the running state of the vehicle, the vehicle speed, and the vehicle environment. The above-mentioned vehicle environment includes but is not limited to the weather conditions around the vehicle, the vehicle conditions around the vehicle, etc. The above-mentioned visual information includes but is not limited to the behavioral information of the passengers, the visual gaze area of ​​the passengers, etc. The above-mentioned behavioral information of the passengers includes but is not limited to the emotional information and action information of the passengers.

[0036] Specifically, a camera device may be provided in the vehicle to obtain image information of at least one passenger in the vehicle, and then the image information is processed using a neural network to obtain visual information of the passenger. The terminal device may periodically obtain vehicle information and visual information and record them.

[0037] In one embodiment, after acquiring image information of at least one passenger in the vehicle, behavior prediction of at least one passenger in the vehicle may be performed based on the image information to obtain behavior information corresponding to the at least one passenger.

[0038] Step S103: Perform information compensation on the voice command according to the auxiliary information to obtain a compensated voice command.

[0039] In this embodiment, the terminal device analyzes the environmental needs of the current passengers through auxiliary information, thereby determining the missing information in the voice command based on the analyzed environmental needs, and then uses the missing information to complete the voice command, that is, to compensate the voice command to obtain the compensated voice command, thereby improving the accuracy of the voice command. The missing information includes information such as the object to be operated, the location of the object to be operated, and the specific operation method.

[0040] In one embodiment, Figure 2 As shown, the above step S103 may include:

[0041] Step S201: determine the passenger's gaze area and behavior information based on visual information.

[0042] In this embodiment, since the passenger will pay attention to the object to be operated corresponding to the voice command when issuing a voice command, when the missing information in the voice command is the object to be operated, the terminal device can determine the passenger's sight area and behavior information through visual information, so that the object to be operated corresponding to the voice command can be determined from the passenger's sight area through behavior information. The above-mentioned behavior information includes the passenger's emotions, actions, etc. The above-mentioned passenger's emotions include but are not limited to happiness, anger, sadness, fear, fatigue, etc., and the above-mentioned passenger's actions include but are not limited to smoking, making phone calls, playing games, etc.

[0043] In one embodiment, before the above step S201, the following may be included: the terminal device may divide the vehicle into regions according to the locations of the objects to be operated in the vehicle that can be operated based on the voice commands of the passengers, and the divided regions are mutually exclusive, that is, the passengers will only focus on one region at the same time, so as to determine the passenger's sight area from the divided regions. The above vehicle region division may be displayed on the user guidance interface of the terminal device, so that the passengers can determine the simplest voice command that meets their needs through the region division displayed on the user guidance interface, thereby improving portability.

[0044] For example, Figure 3 As shown, Figure 3The middle terminal device is divided into four areas according to the positions of the windows, rearview mirrors, air conditioners and windshield on the vehicle, namely area A, area B, area C and area D. Among them, area A has a left window and a left rearview mirror; area B has the front glass; area C has the air conditioner; area D has the right window and a right rearview mirror.

[0045] In one embodiment, the above step S201 may include: the terminal device can determine the number of times the passenger gazes at at least two preset areas within the first preset time based on visual information, and the preset areas are the areas after the above-mentioned area division; finally, the sight fixation area is determined from the at least two preset areas based on the number of gazes, that is, the area with the largest number of gazes is selected as the sight fixation area.

[0046] In one embodiment, the above step S201 may include: the terminal device may determine the area where the passenger is looking at when issuing a voice command based on visual information, and determine the area as the visual attention area.

[0047] Step S202: determining at least one object to be operated from the gaze area.

[0048] In this embodiment, since there are many objects in the gaze area, the terminal device can determine the objects to be operated based on the voice command of the passenger from the gaze area, thereby speeding up the speed of determining the objects to be operated corresponding to the voice command. For example, the left window and the left rearview mirror are determined from the gaze area A.

[0049] Step S203: determining a target object from at least one object to be operated according to the vehicle information and the behavior information.

[0050] In this embodiment, to ensure the safety of passengers, the terminal device may first exclude the objects to be operated according to the vehicle information, and then determine the target object from the excluded objects to be operated according to the behavior information. It is understandable that for safety reasons, the terminal device may exclude some target objects that may affect safety in the current environment according to the vehicle information. For example, when the vehicle's operating state is determined to be the driving state according to the vehicle information, the seat, rearview mirror, and HUT are not adjusted to improve safety, but the windows and air conditioners waiting for operation can be operated in the driving state.

[0051] Exemplarily, based on the above example, when the current voice command is "open", since the voice command has no clear object and position, the above-mentioned gaze area is determined to be area A, and the objects to be operated located in area A are obtained, namely, the left window and the left rearview mirror, and the determination is continued based on the vehicle information. If it is determined based on the vehicle information that the vehicle is in a parked state, then the left window and the left rearview mirror can both be target objects. Based on the behavior information, it is determined that the current emotion of the passenger is anger or fatigue, and the action is smoking, then the current environmental requirement is determined to be that the passenger needs to calm down, thereby determining that the target object is the left window. This example determines the gaze area to narrow the range of objects to be operated that may be the target object, and further excludes objects to be operated that may pose safety hazards based on the vehicle information to ensure vehicle safety, and then finally determines the target object based on the behavior information to ensure the accuracy of the target object.

[0052] In one embodiment, the terminal device can also exclude objects to be operated based only on the vehicle information to determine the target object. For example, based on the above example, if it is determined based on the vehicle information that the vehicle is in a driving state, and the left rearview mirror is excluded because the vehicle cannot adjust the rearview mirror during driving for safety reasons, then the target object is determined to be the left window. This example determines the gaze area to narrow the range of objects to be operated that may be the target object, and then quickly determines the target object based on the vehicle information, thereby further ensuring vehicle safety while ensuring accuracy.

[0053] In one embodiment, the terminal device may also exclude the objects to be operated based only on the behavior information, thereby determining the target object. For example, based on the above example, if the current voice command is "open" and the above-mentioned line of sight area is area D, then the above-mentioned objects to be operated are determined to be the right window and the right rearview mirror. If it is determined based on the behavior information that the current emotion of the person in the front passenger seat is angry and is on the phone, then it is determined that the current environmental requirement is that the person in the front passenger seat needs to calm down, and the target object is determined to be the right window. This example determines the line of sight area to narrow the range of objects to be operated that may be the target object, and then finally determines the target object through the behavior information, thereby ensuring the accuracy of the target object.

[0054] In one embodiment, the terminal device may further determine at least one target object based on the determined environmental demand, and the at least one target object must meet preset rules such as vehicle safety and weather conditions. For example, when the environmental demand is that the passengers need to be calm, the air conditioner can be turned on while the car window is opened. Alternatively, when the passengers look at the air-conditioned area while smoking, it means that the current environmental demand is to purify the air in the car. Therefore, when it is determined based on the vehicle information that the weather condition does not belong to the preset abnormal weather, the window can be opened while the air conditioner is turned on. The abnormal weather includes but is not limited to rainy weather, haze weather, etc.

[0055] Step S204: performing information compensation on the voice command according to the target object.

[0056] In this embodiment, after determining the target object, the terminal device performs information compensation on the voice command according to the target object, thereby improving the voice command.

[0057] In one embodiment, Figure 4 As shown, the above step S103 may include:

[0058] Step S401: Determine at least one object to be operated according to a voice command.

[0059] In this embodiment, when the passenger issues a voice command, there may be a situation where the object to be operated is not clearly stated. For example, the voice command only includes the word "window", but does not specify whether it is a left window or a right window, that is, there is no clear position of the object to be operated. Therefore, the terminal device may first determine at least one object to be operated according to the voice command, and then make subsequent judgments based on the determined object to be operated to determine whether there is an object to be operated that meets the environmental requirements.

[0060] Step S402: determine the passenger's gaze area based on the visual information.

[0061] In this embodiment, since the passenger will pay attention to the object to be operated corresponding to the voice command when issuing a voice command, the terminal device can determine the passenger's line of sight through visual information, thereby achieving the purpose of excluding the object to be operated. For example, different windows correspond to different areas. Based on the above example, the windows are respectively located in area A and area D, so the window to be operated can be determined by determining the line of sight area.

[0062] Step S403: determining a target object from at least one object to be operated according to the vehicle information and the sight area.

[0063] In this embodiment, in order to ensure the safety of passengers, the terminal device may first exclude objects to be operated based on vehicle information, and then determine the target object from the excluded objects to be operated based on the line of sight area.

[0064] Exemplarily, based on the above example, if the current voice command is "raise it a little", since the objects to be operated that can be adjusted can be selected according to the voice command, for example, in this example, the objects to be operated that can be adjusted are the air conditioner and the seat. After determining the objects to be operated, continue to determine according to the vehicle information. If it is determined according to the vehicle information that the vehicle is in a parked state, it means that the seat and the air conditioner can be adjusted, and continue to determine the corresponding target object according to the line of sight area. For example, if the line of sight area is area B, it can be determined that the seat currently needs to be raised, that is, the target object is the seat. If the line of sight area is area C, it can be determined that the air conditioner temperature currently needs to be raised, that is, the target object is the air conditioner. In this example, the range of objects to be operated that may be the target object is determined by determining the voice command, and the vehicle information further excludes objects to be operated that may have safety hazards to ensure vehicle safety, and then finally determines the target object through the line of sight area, thereby ensuring the accuracy of the target object.

[0065] In one embodiment, the terminal device can also exclude the objects to be operated based only on the vehicle information, thereby determining the target object. For example, based on the above example, if it is determined based on the vehicle information that the vehicle is in a driving state, the seat can be excluded because the seat cannot be adjusted during driving for safety reasons, thereby determining that the target object is an air conditioner. Alternatively, the terminal device can make a judgment based on other information in the vehicle information. If it is determined based on the vehicle information that the temperature inside the vehicle environment is lower than a preset threshold, the target object can be determined to be an air conditioner. After determining the range of the objects to be operated that may be the target object by determining the voice command, this example can quickly determine the target object only through the vehicle information, and can further ensure vehicle safety on the basis of ensuring accuracy.

[0066] In one embodiment, the terminal device can also exclude the objects to be operated based only on the line of sight area, thereby determining the target object. For example, based on the above example, if the current voice command is "open the window", since the windows are respectively located in area A and area D, the window to be operated can be clearly identified by determining the line of sight area. If the line of sight area is area A, the target object is the left window; if the line of sight area is area D, the target object is the right window. After determining the range of the objects to be operated that may be the target object by determining the voice command, this example can quickly determine the target object only by the line of sight area, thereby ensuring the accuracy of the target object.

[0067] Step S404: perform information compensation on the voice command according to the target object.

[0068] In this embodiment, after determining the target object, the terminal device performs information compensation on the voice command according to the target object, thereby improving the voice command.

[0069] In one embodiment, step S103 may include: the terminal device may first determine at least one object to be operated according to the voice command, and then make a subsequent judgment based on the determined object to be operated to determine whether there is an object to be operated that meets the environmental requirements. Specifically, since there will be certain behavioral information when the passenger issues a voice command, such as emotional changes, movement changes, etc., the terminal device can determine the behavioral information of the passenger through visual information to determine the target object from the objects to be operated based on the behavioral information. In addition, in order to ensure the safety of the passengers, the terminal device can further exclude the objects to be operated based on the vehicle information, and then determine the target object from the excluded objects to be operated based on the behavioral information. After determining the target object, the voice command is compensated for information based on the target object, thereby improving the voice command.

[0070] Exemplarily, based on the above example, if the current voice command is "open the window", the terminal device can determine the position of the window to be opened through the behavioral information of the passengers in the vehicle. If the passenger in the front passenger seat is currently angry and is on the phone, the current environmental information is determined to be that the person in the front passenger seat needs to calm down, and the target object is determined to be the right window. After determining the range of objects to be operated that may be the target object by determining the voice command, this example can quickly determine the target object only through behavioral information, thereby ensuring the accuracy of the target object.

[0071] In one embodiment, the above step S103 may include: the terminal device may establish a compensation model in advance by fusing image features and voice features at the feature level of the neural network, thereby processing the auxiliary information according to the compensation model to obtain compensation information, and performing information compensation on the voice command according to the compensation information.

[0072] Step S104: Execute the compensated voice command.

[0073] In this embodiment, after receiving the compensated voice command, the terminal device needs to determine whether the compensated voice command can be executed. If it cannot be executed, a secondary poll is performed, that is, the passenger is reminded to issue the voice command again, or the voice command is discarded; if it can be executed, the compensated voice command is executed, thereby improving the voice commands issued by the passengers in the vehicle in a multimodal manner, so that the vehicle accurately responds to the voice commands issued by the passengers, avoiding the situation where the voice command is not responded to when the voice command is missing.

[0074] In one embodiment, after step S104, the method may further include: the terminal device detects the operation information of the passenger within the second preset time to analyze the passenger, and then adjusts the compensation model according to the obtained operation information. It is understandable that the terminal device can better understand the user's intention when the voice command is missing through learning, thereby improving the user experience.

[0075] For example, after the terminal device opens the window according to the compensated voice command, when it detects closing the window, adjusting the window opening ratio, or turning on additional music and other operation information, the detected operation information is recorded and used as input to train the model to better meet user needs.

[0076] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0077] Corresponding to the voice control method described above, Figure 5 FIG. 1 is a schematic diagram of the structure of a voice control device in an embodiment of the present application. Figure 5 As shown, the above-mentioned voice control device may include:

[0078] The command acquisition module 501 is used to acquire voice commands.

[0079] The information acquisition module 502 is used to acquire auxiliary information when there is missing information in the voice command; the auxiliary information includes visual information and / or vehicle information.

[0080] The information compensation module 503 is used to perform information compensation on the voice command according to the auxiliary information to obtain a compensated voice command.

[0081] The execution module 504 is used to execute the compensated voice command.

[0082] In one embodiment, the information compensation module 503 may include:

[0083] The first area determination unit is used to determine the passenger's gaze area and behavior information based on visual information.

[0084] The first object determination unit is used to determine at least one object to be operated from the sight gazing area.

[0085] The first target determination unit is used to determine a target object from at least one object to be operated according to vehicle information and behavior information.

[0086] The first information compensation unit is used to perform information compensation on the voice command according to the target object.

[0087] In one embodiment, the first area determining unit may include:

[0088] The number determination subunit is used to determine the number of times the passenger gazes at at least two preset areas within a first preset time based on the visual information.

[0089] The area determination subunit is used to determine a gaze fixation area from at least two preset areas according to the number of fixations.

[0090] In one embodiment, the information compensation module 503 may further include:

[0091] The second object determination unit is used to determine at least one object to be operated according to the voice command.

[0092] The second area determination unit is used to determine the passenger's sight area based on the visual information.

[0093] The second target determination unit is used to determine a target object from at least one object to be operated according to the vehicle information and the sight area.

[0094] The second information compensation unit is used to perform information compensation on the voice command according to the target object.

[0095] In one embodiment, the information compensation module 503 may further include:

[0096] The third object determination unit is used to determine at least one object to be operated according to the voice command.

[0097] The information determination unit is used to determine the behavior information of passengers based on the visual information.

[0098] The third target determination unit is used to determine a target object from at least one object to be operated according to the vehicle information and the behavior information.

[0099] The third information compensation unit is used to perform information compensation on the voice command according to the target object.

[0100] In one embodiment, the information compensation module 503 may further include:

[0101] The information processing unit is used to process the auxiliary information according to a preset compensation model to obtain compensation information.

[0102] The fourth information compensation unit is used to perform information compensation on the voice command according to the compensation information.

[0103] In one embodiment, the above-mentioned voice control device may further include:

[0104] The detection module is used to detect the operation information of the passengers within the second preset time.

[0105] The adjustment module is used to adjust the compensation model according to the operation information.

[0106] In the embodiment of the present application, a voice command is obtained and a corresponding judgment is performed. When there is missing information in the voice command, auxiliary information is obtained; the auxiliary information includes visual information and / or vehicle information, and the voice command is compensated according to the auxiliary information to obtain a compensated voice command, that is, a more accurate voice command, so as to execute the compensated voice command, so that the vehicle accurately responds to the voice command issued by the passengers.

[0107] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned system embodiments and method embodiments, and will not be repeated here.

[0108] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0109] like Figure 6 As shown, the terminal device 6 of this embodiment includes: at least one processor 600 ( Figure 6 Only one is shown in the figure), a memory 601 connected to the processor 600, and a computer program 602 stored in the memory 601 and executable on the at least one processor 600, such as a voice control program. When the processor 600 executes the computer program 602, the steps in the above-mentioned various voice control method embodiments are implemented, such as Figure 1 Alternatively, when the processor 600 executes the computer program 602, the functions of the modules in the above-mentioned device embodiments are realized, for example, Figure 5 Functions of modules 501 to 504 are shown.

[0110] Exemplarily, the computer program 602 may be divided into one or more modules, which are stored in the memory 601 and executed by the processor 600 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 602 in the terminal device 6. For example, the computer program 602 may be divided into a command acquisition module 501, an information acquisition module 502, an information compensation module 503, and an execution module 504, and the specific functions of each module are as follows:

[0111] A command acquisition module 501 is used to acquire a voice command;

[0112] An information acquisition module 502 is used to acquire auxiliary information when there is missing information in the voice command; the auxiliary information includes visual information and / or vehicle information;

[0113] An information compensation module 503, used to perform information compensation on the voice command according to the auxiliary information to obtain a compensated voice command;

[0114] The execution module 504 is used to execute the compensated voice command.

[0115] The terminal device 6 may include, but is not limited to, a processor 600 and a memory 601. Those skilled in the art will appreciate that Figure 6 It is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, buses, etc.

[0116] The processor 600 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0117] In some embodiments, the memory 601 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 601 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Further, the memory 601 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 601 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 601 may also be used to temporarily store data that has been output or is to be output.

[0118] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For the part that is not described or recorded in detail in a certain embodiment, refer to the relevant description of other embodiments.

[0119] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0120] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0121] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0123] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A voice control method, characterized in that: include: Get voice commands; When there is missing information in the voice command, obtaining auxiliary information; the auxiliary information includes visual information and vehicle information; The vehicle information includes the vehicle's operating status, vehicle speed, and vehicle environment; Performing information compensation on the voice command according to the auxiliary information to obtain a compensated voice command; executing the compensated voice command; The performing information compensation on the voice command according to the auxiliary information includes: determining the sight fixation area and behavior information of the passenger according to the visual information, the behavior information including the emotion and action of the passenger; determining at least one object to be operated from the sight fixation area; determining a target object from the at least one object to be operated according to the vehicle information and the behavior information; and performing information compensation on the voice command according to the target object; Alternatively, the performing information compensation on the voice command according to the auxiliary information includes: determining at least one object to be operated according to the voice command; determining a sight area of ​​the passenger according to the visual information; determining a target object from the at least one object to be operated according to the vehicle information and the sight area; and performing information compensation on the voice command according to the target object; Alternatively, the performing information compensation on the voice command according to the auxiliary information includes: determining at least one object to be operated according to the voice command; determining behavior information of a passenger according to the visual information; determining a target object from the at least one object to be operated according to the vehicle information and the behavior information; and performing information compensation on the voice command according to the target object; After executing the compensated voice command, the method further includes: Detecting the operation information of passengers within a second preset time; The compensation model is adjusted according to the operation information.

2. The voice control method according to claim 1, characterized in that: The determining of the passenger's gaze area and behavior information based on the visual information includes: Determining, based on the visual information, the number of times the passenger gazes at at least two preset areas within a first preset time; The line of sight fixation area is determined from the at least two preset areas according to the number of fixations.

3. The voice control method according to claim 1, characterized in that: The performing information compensation on the voice command according to the auxiliary information includes: Processing the auxiliary information according to a preset compensation model to obtain compensation information; The voice command is compensated for information according to the compensation information.

4. A voice control device, characterized in that: include: A command acquisition module, used to acquire voice commands; An information acquisition module, used for acquiring auxiliary information when there is missing information in the voice command; the auxiliary information includes visual information and vehicle information; the vehicle information includes the running state of the vehicle, the vehicle speed and the vehicle environment; An information compensation module, used to perform information compensation on the voice command according to the auxiliary information to obtain a compensated voice command; An execution module, used for executing the compensated voice command; The performing information compensation on the voice command according to the auxiliary information includes: determining the sight fixation area and behavior information of the passenger according to the visual information, the behavior information including the emotion and action of the passenger; determining at least one object to be operated from the sight fixation area; determining a target object from the at least one object to be operated according to the vehicle information and the behavior information; and performing information compensation on the voice command according to the target object; Alternatively, the performing information compensation on the voice command according to the auxiliary information includes: determining at least one object to be operated according to the voice command; determining a sight area of ​​the passenger according to the visual information; determining a target object from the at least one object to be operated according to the vehicle information and the sight area; and performing information compensation on the voice command according to the target object; Alternatively, the performing information compensation on the voice command according to the auxiliary information includes: determining at least one object to be operated according to the voice command; determining behavior information of a passenger according to the visual information; determining a target object from the at least one object to be operated according to the vehicle information and the behavior information; and performing information compensation on the voice command according to the target object; After executing the compensated voice command, the method further includes: Detecting the operation information of passengers within a second preset time; The compensation model is adjusted according to the operation information.

5. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a voice control method as described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a voice control method as claimed in any one of claims 1 to 3 are implemented.

Images