Device control method and apparatus

Abstract

This application provides a device control method and apparatus for reducing power consumption and resource occupation in multi-device wake-up scenarios and improving a seamless user experience. [Solution] The method includes the steps of: an in-vehicle device acquiring voice information of a first user, the voice information indicating that a first action should be performed; an in-vehicle device determining a target device from a plurality of candidate devices based on the voice information, the plurality of candidate devices including an in-vehicle device and at least one terminal device; when the target device belongs to at least one terminal device, the in-vehicle device transmitting instruction information to the target device, the instruction information indicating that the target device should perform a first action; or when the target device is an in-vehicle device, the in-vehicle device performing a first action.

Description

Technical Field

[0001] This application relates to the field of intelligent control, and in particular, to a device control method and apparatus.

Background Art

[0002] With the development of intelligent control technology, more intelligent devices exist in the same scenario. For example, mobile phones, smart watches, smart headsets, and tablet computers of the same user, or intelligent devices used by different users, all exist in the in-vehicle scenario or the smart home scenario. In this case, multiple intelligent devices may use the same wake-up word. When the user performs a wake-up, multiple devices may respond simultaneously. As a result, the user experience is degraded.

[0003] In a multi-device wake-up scenario, a common solution is to determine a response solution based on the distance between the device and the user, or based on a preset device priority. For example, in an in-vehicle scenario, the user uses voice to wake up the device. In this case, the wake-up engine of each device is in an active state, and each device can be woken up after receiving the voice. The woken-up device can use broadcast information to send wake-up information, and the wake-up information includes the type of the device and the sound pressure intensity. All devices may use the same wake-up decision logic. After obtaining the broadcast information of other devices, each device can determine whether to respond to the user's voice based on the priority information of the device and the received broadcast information. Finally, the device that determines to respond to the voice performs the response, and then performs identification and response.

[0004] However, the aforementioned solutions have many drawbacks. For example, selecting devices based solely on sound intensity fails to meet the intelligent selection requirements for multi-device coordination. In noisy environments, the proportional relationship between sound intensity and distance weakens, leading to inaccurate decisions. In addition, the simultaneous operation of wake-up engines for all devices wastes computing power, resulting in high power consumption and resource occupation for individual devices. Therefore, there is an urgent need in the industry for optimized intelligent control solutions for multi-device wake-up scenarios. [Overview of the Initiative]

[0005] This application provides a device control method and apparatus for reducing power consumption and resource occupation in multi-device wake-up scenarios and improving a seamless user experience. [Means for solving the problem]

[0006] According to a first embodiment, a device control method is provided. The method includes the steps of: an in-vehicle device acquiring voice information of a first user, the voice information indicating that a first operation should be performed; the in-vehicle device determining a target device from a plurality of candidate devices based on the voice information, the plurality of candidate devices including an in-vehicle device and at least one terminal device; and, when the target device belongs to at least one terminal device, the in-vehicle device transmitting instruction information to the target device, the instruction information indicating that the target device should perform a first operation, or, when the target device is an in-vehicle device, the in-vehicle device performing a first operation.

[0007] In a multi-device wake-up scenario, candidate devices host the wake-up engine in the in-vehicle device to perform wake-up decisions and wake-up responses. The in-vehicle device is responsible for wake-up decisions and wake-up responses for candidate devices, and after making a decision, the in-vehicle device either executes the decision or sends the decision result to the terminal device. In this solution, only the in-vehicle device needs to enable the wake-up engine, thereby reducing power consumption and resource usage of candidate devices and improving the seamless user experience. Terminal devices may be connected via wired or wireless connection to the in-vehicle device or the vehicle to which the in-vehicle device belongs, but it should be understood that terminal devices do not belong to the vehicle.

[0008] In relation to the first aspect, in one possible embodiment of the first aspect, at least one terminal device registers registration information with an in-vehicle device.

[0009] In a multi-device wake-up scenario, terminal devices can register information with the in-vehicle device in advance. As a result, the in-vehicle device performs wake-up decisions and wake-up responses in the multi-device scenario based on the registered information. After making a decision, the in-vehicle device either executes the decision or sends the decision result to the terminal device. This solution can reduce power consumption and resource occupation of candidate devices and improve the seamless user experience.

[0010] In relation to the first aspect, in one possible embodiment of the first aspect, the registration information includes account information, the account information includes at least one of the user's account identifier, the user's voiceprint feature information, and the user's facial image feature information.

[0011] The registration information transmitted from the terminal device to the in-vehicle device includes the user's account identifier, user's voiceprint characteristics, and user's facial image characteristics. As a result, the in-vehicle device makes decisions based on the account information when performing wake-up decisions and wake-up responses, thereby improving the accuracy of cooperative policies in multi-device scenarios.

[0012] In relation to the first aspect, in one possible embodiment of the first aspect, the registration information further includes device information, the device information includes at least one of the device type and media access control (MAC) address information.

[0013] In relation to the first aspect, in one possible embodiment of the first aspect, the method further includes the steps of: an in-vehicle device transmitting search information to a terminal device, wherein the search information indicates that the terminal device registers with the in-vehicle device; and the in-vehicle device receiving registration information from the terminal device.

[0014] In relation to the first aspect, in one possible embodiment of the first aspect, the voice information indicates reference information corresponding to a target device. The reference information includes at least one of intent information indicating the intent of the voice information, a wake-up word used to wake up the in-vehicle device, the wake-up word belonging to a plurality of wake-up words, and a pre-configured correspondence between the plurality of wake-up words and a plurality of candidate devices, and user identification information indicating the identity of a first user. The step of the in-vehicle device determining a target device from a plurality of candidate devices based on the voice information includes the step of determining a target device from a plurality of candidate devices based on the reference information.

[0015] The in-vehicle device may function as the sole responding device and make a decision based on a wake-up word, intent information corresponding to the voice information, and user identification information to determine the target device to perform the first action indicated by the voice information. This solution can improve the seamless user experience and enhance the accuracy of the collaborative policy in multi-device scenarios.

[0016] In relation to the first aspect, in one possible embodiment of the first aspect, the step of an in-vehicle device determining a target device from a plurality of candidate devices based on reference information includes the step of the in-vehicle device determining a first subset of candidate devices from a plurality of candidate devices based on intent information, wherein the types of candidate devices in the first subset of candidate devices correspond to the intent information, and the step of the in-vehicle device determining a target device from the first subset of candidate devices based on user identification information.

[0017] In-vehicle devices can improve the intelligence used to identify target devices, enhance the seamless user experience, and improve the overall user experience by narrowing down the list of candidate devices corresponding to voice information based on intent information, and then further determining the target device based on user identification information.

[0018] In relation to the first aspect, in one possible embodiment of the first aspect, the step of an in-vehicle device determining a target device from a plurality of candidate devices based on reference information includes the step of the in-vehicle device determining a second subset of candidate devices corresponding to a wake-up word, wherein there is a pre-configured correspondence between the wake-up word and the candidate devices in the second subset of candidate devices; and the step of the in-vehicle device determining a target device from the second subset of candidate devices based on intent information and user identification information.

[0019] In relation to the first aspect, in one possible embodiment of the first aspect, the step of an in-vehicle device acquiring reference information based on voice information includes the step of the in-vehicle device acquiring at least one user feature information from among first user sound source positioning information, first user voiceprint feature information, and first user facial image feature information based on voice information, and the step of the in-vehicle device determining user identification information based on at least one user feature information.

[0020] In relation to the first aspect, in one possible embodiment of the first aspect, a wake-up engine in an in-vehicle device is enabled, and a wake-up engine in at least one terminal device is disabled.

[0021] In a multi-device wake-up scenario, the wake-up engine of the terminal device may be disabled, leaving only the wake-up engine of the in-vehicle device, thereby reducing resource usage and power consumption of the terminal device.

[0022] A device control method is provided according to a second embodiment. This method includes the steps of: a terminal device transmitting registration information to an in-vehicle device, the registration information being used by the terminal device to register with the in-vehicle device; and the terminal device receiving instruction information from the in-vehicle device, the instruction information including an operation indicating that the terminal device should perform a first operation, the first operation being indicated to be performed by a first user's voice information.

[0023] In a multi-device wake-up scenario, the terminal device may register information with the in-vehicle device in advance. As a result, the in-vehicle device executes wake-up determination and wake-up response in a multi-device scenario based on the registered information. After making the determination, the in-vehicle device sends the determination result to the terminal device. In this solution, the power consumption and resource occupancy of candidate devices can be reduced, and the seamless user experience is improved. The terminal device may be connected to the in-vehicle device or the vehicle by wire or wirelessly, but the terminal device does not belong to the vehicle.

[0024] In relation to the second aspect, in one possible implementation of the second aspect, the registered information includes account information, and the account information includes at least one of a user's account identifier, the user's voiceprint feature information, and the user's face image feature information.

[0025] In relation to the second aspect, in one possible implementation of the second aspect, the registered information further includes device information, and the device information includes at least one of a device type and MAC address information.

[0026] In relation to the second aspect, in one possible implementation of the second aspect, the method further includes a step of disabling the wake-up engine of the terminal device after the terminal device sends the registered information, where the wake-up engine is configured to respond to the user's voice information.

[0027] In relation to the second aspect, in one possible implementation of the second aspect, the method further includes a step of the terminal device receiving search information sent by the in-vehicle device, where the search information indicates that the terminal device registers with the in-vehicle device.

[0028] According to the third aspect, an in-vehicle device is provided. The in-vehicle device includes a module configured to execute the method according to the first aspect or any possible implementation of the first aspect.

[0029] According to a fourth aspect, a terminal device is provided. The terminal device includes a module configured to execute a method according to the second aspect or any possible implementation of the second aspect.

[0030] According to a fifth aspect, an in-vehicle device is provided. The device includes a communication interface and a processor. The processor is configured to call a computer program from a memory. When the computer program is executed, the processor is configured to execute a method according to the first aspect or any possible implementation of the first aspect.

[0031] According to a sixth aspect, a terminal device is provided. The terminal device includes a communication interface and a processor. The processor is configured to call a computer program from a memory. When the computer program is executed, the processor is configured to execute a method according to the second aspect or any possible implementation of the second aspect.

[0032] According to a seventh aspect, a computer-readable storage medium configured to store a computer program is provided. The computer program includes code used to execute a method according to the first aspect or any possible implementation of the first aspect.

[0033] According to an eighth aspect, a computer-readable storage medium configured to store a computer program is provided. The computer program includes code used to execute a method according to the second aspect or any possible implementation of the second aspect.

[0034] According to a ninth aspect, a vehicle is provided. The vehicle includes an in-vehicle device according to the third aspect or any possible implementation of the third aspect and an in-vehicle device according to the fifth aspect or any possible implementation of the fifth aspect.

[0035] According to the tenth aspect, a computer program product is provided which includes a computer program. The computer program includes code used to perform a method according to the first aspect or any possible embodiment of the first aspect and a method according to the second aspect or any possible embodiment of the second aspect. [Brief explanation of the drawing]

[0036] [Figure 1] This is a schematic diagram of an application scenario according to one embodiment of this application. [Figure 2] This is a schematic diagram of a smart home scenario according to one embodiment of the present application. [Figure 3] This is a schematic diagram of a vehicle scenario according to one embodiment of this application. [Figure 4] This is a schematic flowchart of a device control method according to one embodiment of this application. [Figure 5] This is a schematic flowchart illustrating how a terminal device registers with an in-vehicle device, according to one embodiment of this application. [Figure 6] This is a specific flowchart of a device control method according to one embodiment of this application. [Figure 7] This is a schematic block diagram of apparatus 700 according to another embodiment of the present application. [Figure 8] This is a schematic block diagram of apparatus 800 according to another embodiment of the present application. [Figure 9] This is a schematic block diagram of apparatus 900 according to another embodiment of the present application. [Modes for carrying out the invention]

[0037] The technical solution of this application will be described below with reference to the attached drawings.

[0038] In embodiments of this application, a terminal device may be a user device, access terminal, subscriber unit, subscriber station, mobile station, mobile console, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user equipment. Alternatively, a terminal device may be a cellular phone, cordless phone, Session Initiation Protocol (SIP) phone, Wireless Local Loop (WLL) station, Personal Digital Assistant (PDA), handheld device with wireless communication capabilities, computing device, another processing device connected to a wireless modem, wearable device, vehicle, terminal device in a future 5G network, terminal device in a future advanced Public Land Mobile Network (PLMN), etc. This is not limited to embodiments of this application.

[0039] The solutions in the embodiments of this application are applicable to multi-device wake-up scenarios, such as in-vehicle scenarios or smart home scenarios.

[0040] Figure 1 is a schematic diagram of an application scenario according to one embodiment of the present application. As shown in Figure 1, the application scenario includes a central device and at least one terminal device. The central device and the terminal device can communicate with each other. For example, the central device may be an in-vehicle device or a device in a smart home scenario, and may include, for example, a head unit in an in-vehicle scenario or a set-top box in a smart home scenario, or any terminal device having communication capabilities. The head unit may be an in-vehicle domain controller and a control system for the in-vehicle domain controller mounted in a vehicle, and may enable communication between a person and a vehicle, and between the vehicle and the outside.

[0041] Terminal devices include, but are not limited to, mobile phones, tablet computers, wearable devices, smartwatches, smart headsets, home appliances, or other devices with communication capabilities.

[0042] The wake-up engine may be located in both the central device and the terminal device. For example, when the wake-up engine is activated, the user may use voice to wake up the central device or terminal device to respond to the user's request. The wake-up engine may also be a program located in the device and used to respond to the user's voice information. The wake-up engine may be configured to detect user voice keywords and respond accordingly based on those keywords. When the wake-up engine is enabled, it must continuously monitor the user's voice information, resulting in high resource utilization and power consumption, especially in multi-microphone scenarios.

[0043] In some examples, the difference between a central device and a terminal device lies in the fact that the central device is responsible for responding to user voice information, analyzing the voice information, and then either performing the action indicated by the voice information or indicating that the terminal device should perform the action. Terminal devices do not need to respond to user voice information and only need to perform actions based on the instructions of the central device. Therefore, the central device can enable its wake-up engine, and terminal devices can disable their wake-up engine to reduce power consumption and computing power.

[0044] In some cases, there is no essential hardware difference between a central device and terminal devices. For example, one terminal device may be selected as the central device from among several terminal devices, and the other terminal devices may send registration information to the central device.

[0045] Optionally, the central device and terminal devices may belong to the same user or to different users.

[0046] Figure 2 is a schematic diagram of a smart home scenario according to one embodiment of the present application. As shown in Figure 2, in the smart home scenario, a set-top box may be set up as the central device. Of course, another device (e.g., a smart screen) may be selected as an alternative central device. Terminal devices may include, but are not limited to, electronic devices with wireless communication capabilities such as air conditioners, printers, refrigerators, electric irons, vacuum cleaners, game consoles, microwave ovens, access control systems, lighting systems, and shower systems. Each device may transmit registration information to the set-top box. The set-top box then plays the role of responding to and analyzing the user's voice information and indicating the corresponding device to perform the action indicated by the voice information.

[0047] Figure 3 is a schematic diagram of a vehicle scenario according to one embodiment of the present application. As shown in Figure 3, in the vehicle, the central device may include an in-vehicle device, which may be a head unit or another device located in the vehicle. Terminal devices may include, but are not limited to, mobile phones, personal computers, tablet computers, game consoles, smartwatches, and smart headsets. After detecting that a terminal device has entered the vehicle, the terminal device may transmit registration information to the in-vehicle device and disable the terminal device's wake-up engine. The in-vehicle device is responsible for responding to user voice information and performing unified assignments to enhance a seamless user experience.

[0048] The following describes the method in the embodiments of this application using an example where the central device is an in-vehicle device. It should be understood that the in-vehicle device described below may be replaced by another type of central device, such as a set-top box or smart screen in a smart home scenario.

[0049] Figure 4 is a schematic flowchart of a device control method according to one embodiment of the present application. The in-vehicle device in Figure 4 may be a head unit or other device located in a vehicle, as long as it has communication and processing functions. As shown in Figure 4, the method includes the following:

[0050] S401: The in-vehicle device acquires voice information from a first user, and the voice information indicates that a first action should be performed.

[0051] Optionally, the voice information may include a wake-up word, which may be used to wake up the in-vehicle device. After capturing the wake-up word, the in-vehicle device may respond to the wake-up word. Optionally, the in-vehicle device may further perform semantic analysis of the voice information to obtain intent information. The intent information indicates the intent of the first user, i.e., the execution of a first action.

[0052] The specific content of the first action is not limited in this application. The first action may be any instruction transmitted by the first user. For example, the first action may be playing music, making a call, adjusting the temperature of an air conditioner, etc.

[0053] S402: The in-vehicle device determines a target device from a plurality of candidate devices based on voice information, and the plurality of candidate devices include the in-vehicle device and at least one terminal device.

[0054] Optionally, at least one terminal device registers registration information with the in-vehicle device. The registration information is used by each terminal device to register with the in-vehicle device. For example, the registration information may include the terminal device's device information and / or account information. As an example, Table 1 shows the information registered by a terminal device with the in-vehicle device. As shown in Table 1, the device information may include the device type, MAC address information, etc. The account information may include the user's account identifier (ID), the user's voiceprint feature information, the user's facial image feature information, etc. It should be understood that the appropriate addition or deletion of information necessary for registration is within the scope of protection of the embodiments of this application. The specific registration process of the terminal device will continue to be described below with reference to the attached drawings.

[0055] [Table 1]

[0056] Optionally, when the in-vehicle device detects that a terminal device is nearby, for example, when it detects that the distance between the terminal device and the in-vehicle device is less than a preset threshold, the in-vehicle device may send search information to the terminal device, which indicates that the terminal device is registering with the in-vehicle device. After receiving the search information, the terminal device may send registration information to the in-vehicle device. The registration information may include the aforementioned device information and / or account information so that the in-vehicle device registers related information about the terminal device.

[0057] S403: When the target device belongs to at least one terminal device, the in-vehicle device sends instruction information to the target device, indicating that the target device will perform a first operation.

[0058] Optionally, the in-vehicle device may be configured to enable the wake-up engine, while at least one terminal device may be configured to disable the wake-up engine. In other words, to reduce power consumption, only the in-vehicle device is responsible for responding to user voice information, while all other devices have the wake-up engine disabled.

[0059] In some cases, after registering with an in-vehicle device, a terminal device may be able to disable the terminal device's wake-up engine.

[0060] In some other examples, when the terminal device detects that the distance between it and the in-vehicle device is greater than or equal to a preset threshold, i.e., when the terminal device leaves the multi-device environment covered by the in-vehicle device, the terminal device may enable its wake-up engine to respond independently to the wake-up word again. The preset threshold described above may be set on an implementation basis and is not particularly limited in this application.

[0061] In this embodiment of the present application, in a multi-device scenario, the wake-up engine of the terminal device may be disabled, and only the wake-up engine of the in-vehicle device may be maintained, thereby reducing the resource occupation and power consumption of the terminal device.

[0062] S404: When the target device is an in-vehicle device, the in-vehicle device performs the first operation.

[0063] In this embodiment of the present application, in a multi-device wake-up scenario, the terminal device hosts the wake-up engine in the in-vehicle device to perform wake-up decisions and wake-up responses. The terminal device may pre-register information with the in-vehicle device. The in-vehicle device is responsible for wake-up decisions and wake-up responses for the terminal device, and after making a decision, the in-vehicle device transmits the decision result to the target device. In this solution, only the in-vehicle device needs to enable the wake-up engine, thereby reducing power consumption and resource occupation of the terminal device.

[0064] In some examples, the in-vehicle device and at least one terminal device are located within the same local area network. The local area network may be a wired local area network, a wireless local area network, or a combination of a wired local area network and a wireless local area network.

[0065] In S402, the in-vehicle device may determine a target device from multiple candidate devices in multiple embodiments. For example, the audio information may indicate reference information corresponding to the target device, and the reference information is Intentional information that indicates the intent of speech information, intent information is obtained by performing semantic analysis of speech information, A wake-up word used to wake up an in-vehicle device, wherein the wake-up word belongs to a plurality of wake-up words, and there is a pre-configured correspondence between the plurality of wake-up words and a plurality of candidate devices, and User identification information that indicates the user's identifying information, It includes at least one of the following.

[0066] The in-vehicle device may determine the target device from multiple candidate devices based on reference information.

[0067] In this embodiment of the present application, the in-vehicle device may function as the sole responding device and make a decision based on a wake-up word, intent information corresponding to the voice information, and user identification information to determine the target device to perform a first action indicated by voice information. This solution can improve the seamless user experience and enhance the accuracy of the coordination policy in multi-device wake-up scenarios.

[0068] In some examples, an in-vehicle device may perform semantic analysis of voice information to obtain corresponding intent information, and then determine a target device from several candidate devices based on that intent information. Intent information may be understood as information indicating the user's intent. For example, the user's voice information may be music playback. However, the in-vehicle device needs to perform semantic analysis of the voice information to understand the user's intent.

[0069] In some examples, the step of an in-vehicle device determining a target device from multiple candidate devices based on voice information includes the step of the in-vehicle device performing semantic analysis of the voice information to obtain intent information corresponding to the voice information, and the step of the in-vehicle device determining a target device from multiple candidate devices based on the intent information.

[0070] A device may determine a user's action intent based on intent information. Intent information may explicitly or implicitly indicate the type of device the user wants to interact with. For example, when intent information indicates "control airglow," the indicated device type is an in-car device. When intent information indicates "view health information," the indicated device type is a mobile phone. When intent information indicates "make a call," the indicated device may be either a mobile phone or an in-car device. An in-car device may further determine whether the indicated device is a mobile phone or an in-car device based on past calls made by the user. In addition, if there are clear keywords indicating the device type in the intent information, the target device or the type of target device may be determined directly. For example, intent information may indicate "make a call using a mobile phone" or "check the weather using a tablet computer."

[0071] Optionally, the in-vehicle device may further determine a target device from multiple candidate devices based on intent information and user identification information in order to improve the efficiency of determining the target device.

[0072] In some examples, the in-vehicle device determines a first subset of candidate devices from multiple candidate devices based on intent information, the type of candidate device within the first subset corresponds to the intent information, and the in-vehicle device determines a target device from the first subset of candidate devices based on user identification information.

[0073] For example, an in-vehicle device may determine the type of target device based on intent information, and then, in order to further determine the target device, it may be associated with a device corresponding to the user identification information based on the user identification information.

[0074] In this embodiment of the present application, the in-vehicle device can improve the intelligence for identifying the target device, enhance the seamless user experience, and improve the user experience by narrowing down the list range of candidate devices corresponding to voice information based on intent information, and then further determining the target device based on user identification information.

[0075] Optionally, the in-vehicle device may acquire user identification information in the following manner: Based on the audio information, the in-vehicle device acquires at least one of the following user feature information: the sound source positioning information of the first user, the voiceprint feature information of the first user, and the facial image feature information of the first user. Based on the at least one user feature information, the in-vehicle device determines user identification information.

[0076] For example, user sound source positioning information may indicate the user's location in a scenario, and the scenario may be an in-vehicle scenario or an indoor scenario. User facial image feature information may be acquired based on the corresponding location. Then, the identity of the user sitting at that location may be determined by comparing the user's facial image feature information with the facial features in the registration information stored in the in-vehicle device. Correspondingly, whether the user at the corresponding location is in a speaking state may be further verified based on the open / closed state of the user's lips to determine whether the user feature information is correct.

[0077] In another example, the user's identity may be determined by comparing the voiceprint features of the spoken speech with the voiceprint features of the registration information stored in the in-vehicle device.

[0078] Optionally, the aforementioned method for obtaining user identification information may be completed independently by the in-vehicle device, or it may be completed by the in-vehicle device in cooperation with another device. For example, the in-vehicle device may use a camera to obtain the user's facial image feature information and a microphone to obtain the user's sound source positioning information and voiceprint feature information.

[0079] In this embodiment of the present application, the in-vehicle device may acquire user characteristic information by making full use of the sound source features and voiceprint features of the audio information in order to determine user identification information based on user characteristic information and to determine a target device based on the user identification information. This improves intelligence in multi-device wake-up scenarios and improves the user experience.

[0080] Optionally, the audio information may further include a wake-up word. The wake-up word may be a specific word or sound clip within the audio information, and the word or sound clip may be used to wake up a specific device. In this embodiment of the application, the wake-up word is used to wake up an in-vehicle device. In addition to serving the function of waking up an in-vehicle device, the wake-up word may be used to distinguish between different devices. For example, multiple wake-up words may be set, and there may be a pre-set correspondence between each wake-up word and one or more candidate devices.

[0081] For example, there may be a correspondence between a wake-up word and a device of the same type. When an in-vehicle device captures a wake-up word in audio information, it may assume that the wake-up word was used to wake up the corresponding device. For example, the wake-up keyword might be "Hey Celia," and this wake-up keyword might be used to wake up a mobile phone.

[0082] Optionally, the correspondence between the wake-up word and candidate devices may be pre-configured before the in-vehicle device leaves the factory, and the correspondence may be stored in a memory unit within the in-vehicle device. Alternatively, the correspondence between the wake-up word and candidate devices may be pre-configured by the user and stored in a memory unit within the in-vehicle device.

[0083] Optionally, multiple candidate devices may be divided into multiple candidate device subsets based on the wake-up word. For example, Table 2 shows an example of a correspondence between a wake-up word and a candidate device. As shown in Table 2, wake-up word A may be set to correspond to mobile phones, tablet computers, and head units, and wake-up word B may be set to correspond to smart headsets and smartwatches.

[0084] [Table 2]

[0085] In one example, there may be crossover sets between candidate device subsets corresponding to different wake-up words, or there may not be crossover sets between candidate device subsets corresponding to different wake-up words.

[0086] When there is only one candidate device corresponding to a wake-up word, the in-vehicle device may determine the candidate device as the target device based on the wake-up word in the voice information. When there are multiple candidate devices corresponding to a wake-up word, the in-vehicle device may determine a subset of candidate devices corresponding to the wake-up word, and then determine the target device within that subset of candidate devices.

[0087] For example, an in-vehicle device determines a wake-up word in voice information, and based on the wake-up word, the in-vehicle device determines a second subset of candidate devices corresponding to the wake-up word. Multiple candidate devices belong to multiple subsets of candidate devices, and there is a pre-configured correspondence between multiple wake-up words and multiple subsets of candidate devices. Based on intent information and user identification information, the in-vehicle device determines a target device from the second subset of candidate devices.

[0088] For example, when voice information includes wake-up word A, according to Table 2, candidate devices corresponding to wake-up word A include mobile phones, tablet computers, and head units. However, the intent information indicates "turn on the air conditioner," and only the head unit can perform this action. Therefore, the head unit may be determined to be the target device, and the action of turning on the air conditioner may be performed using the head unit.

[0089] As another example, suppose the in-vehicle device is a head unit, the in-vehicle scenario includes user 1 and user 2, user 1 has mobile phone 1, and user 2 has mobile phone 2. When the voice information includes wake-up word A, according to Table 2, wake-up word A corresponds to mobile phone 1, mobile phone 2, and the head unit. When the content indicated by the intent information is "View today's steps," the type of target device can be determined to be a mobile phone. Next, user characteristic information can be obtained using the voice information, and it is determined that the source of the voice information is user 2. Therefore, the target device is determined to be user 2's mobile phone 2. Thus, the head unit can respond to the command in the voice information and instruct mobile phone 2 to open the health information interface on the mobile phone.

[0090] Figure 5 is a schematic flowchart illustrating a terminal device registering with an in-vehicle device according to one embodiment of this application. As shown in Figure 5, the solution includes, but is not limited to, the following.

[0091] S501: The in-vehicle device transmits search information, the terminal device receives the search information in response, and the terminal device registers the search information with the in-vehicle device.

[0092] Optionally, under certain conditions, the in-vehicle device can be triggered to transmit search information. For example, the in-vehicle device may detect that the distance between the in-vehicle device and the terminal device is below a preset threshold, or that a near-field communication (NFC) touch has occurred between the in-vehicle device and the terminal device. NFC is a short-range high-frequency wireless communication technology that enables contactless point-to-point data transmission and exchange between electronic devices. An NFC touch refers to a situation where electronic devices use NFC to detect each other's presence and communicate.

[0093] In addition, for example, certain conditions may further include the following scenarios: a scenario in which an action such as unlocking and powering on or restoring the vehicle from hibernation is performed; a scenario in which an action such as opening the vehicle doors is performed; and a scenario in which the vehicle owner account is switched.

[0094] Optionally, search information may be transmitted by broadcast instead. For example, search information may include searching for Bluetooth Low Energy (BLE) broadcasts. Searching for BLE broadcasts may refer to broadcast information transmitted using BLE technology.

[0095] S502: The terminal device transmits registration information to the in-vehicle device, and in response, the in-vehicle device receives the registration information and uses the registration information to register with the in-vehicle device by the terminal device.

[0096] Optionally, if a terminal device is registered with an in-vehicle device, the in-vehicle device may ignore the registration information, or it may update the registration information.

[0097] Optionally, registration information may include device information and account information for the terminal device. Device information may include device type and MAC address. Account information may include the user's account identifier, user's voiceprint feature information, or user's facial image feature information.

[0098] Optionally, registration information may be transmitted via broadcast.

[0099] S503: The in-vehicle device registers terminal devices based on the registration information.

[0100] For example, an in-vehicle device may establish and store a local registry and add the registration information of the terminal device to the entries.

[0101] In some cases, the process of a terminal device registering with an in-vehicle device may be completed within the networking process between the terminal device and the in-vehicle device. For example, after receiving registration information, the in-vehicle device may send its Wi-Fi name to the terminal device. In this case, the terminal device may establish a network connection with the in-vehicle device based on the received Wi-Fi name and communicate with the in-vehicle device via Wi-Fi in subsequent processes.

[0102] Figure 6 is a specific flowchart of a device control method according to one embodiment of this application. As shown in Figure 6, this method includes the following:

[0103] S601: The in-vehicle device registers the registration information of the terminal device.

[0104] For example, a user may have a terminal device in the vehicle, which may include a mobile phone, tablet computer, etc. When the presence of a terminal device is detected nearby, the in-vehicle device is triggered to transmit search information via Bluetooth®. After receiving the search information, each terminal device transmits registration information to the in-vehicle device in response to the search information. After receiving the registration information, the in-vehicle device transmits Wi-Fi information to each terminal device, and a communication connection is established between each terminal device and the in-vehicle device.

[0105] After registration, each terminal device may disable its wake-up engine to reduce resource usage. The in-vehicle device then takes over the wake-up response.

[0106] Optionally, after registration, each terminal device may display an alternative interface to the user on its interface indicating whether to disable the wake-up engine. The user then decides whether to disable the wake-up engine.

[0107] Optionally, after receiving registration information for the terminal device, the in-vehicle device may send instruction information to the terminal device to instruct it to disable the terminal device's engine.

[0108] S602: The in-vehicle device receives voice information and is woken up by the voice information.

[0109] For example, the voice information may include a specific wake-up word, and the in-vehicle device will wake up after detecting the wake-up word. Optionally, the in-vehicle device may respond to the voice information using voice.

[0110] S603: The in-vehicle device determines, based on voice information, a target device that is configured to perform the action indicated by the voice information.

[0111] For example, an in-vehicle device obtains at least one of the following pieces of information based on voice information: intent information, wake-up word, and user identification information. Based on the aforementioned information, the in-vehicle device determines the target device.

[0112] S604: The in-vehicle device indicates that the target device will perform the action indicated by the voice information, or the in-vehicle device performs the action.

[0113] For example, if the target device is a terminal device other than an in-vehicle device, the in-vehicle device may send instruction information to the target device to indicate that the target device should perform an action. If the target device is an in-vehicle device, the in-vehicle device will perform the action.

[0114] Figure 7 is a schematic block diagram of an apparatus 700 according to one embodiment of the present application. The apparatus 700 includes an acquisition module 710, a determination module 720, and an execution module 730. The apparatus 700 may be an in-vehicle device in the embodiments described above, or a component (such as a chip) of an in-vehicle device. The apparatus 700 may perform steps or processes corresponding to the in-vehicle device in the embodiments of the methods described above.

[0115] The acquisition module 710 is configured to acquire voice information of a first user, which indicates that a first action should be performed. The decision module 720 is configured to determine a target device from a plurality of candidate devices based on the voice information, which include the device 700 and at least one terminal device. The execution module 730 is configured to send instruction information to the target device when the target device belongs to at least one of the terminal devices, which indicates that the target device should perform a first action, or, if the target device is the device 700, to perform a first action.

[0116] In some cases, at least one terminal device registers registration information with device 700.

[0117] In some examples, registration information includes account information, which includes at least one of the following: the user's account identifier, the user's voiceprint feature information, and the user's facial image feature information.

[0118] In some examples, registration information further includes device information, which includes at least one of the device type and MAC address information.

[0119] In some examples, the acquisition module 710 is further configured to send search information to a terminal device, the search information to indicate that the terminal device is registering with the device 700, and to receive registration information from the terminal device.

[0120] In some examples, the voice information indicates reference information corresponding to the target device. The reference information includes at least one of intent information indicating the intent of the voice information, a wake-up word used to wake up the in-vehicle device, the wake-up word belonging to a plurality of wake-up words, and a pre-configured correspondence between the plurality of wake-up words and a plurality of candidate devices, and user identification information indicating the identity of a first user. The decision module 720 is specifically configured to determine the target device from a plurality of candidate devices based on the reference information.

[0121] In some examples, the decision module 720 is specifically configured to determine a first candidate device subset from a plurality of candidate devices based on intent information, the type of candidate device in the first candidate device subset corresponds to the intent information, and to determine a target device from the first candidate device subset based on user identification information.

[0122] In some examples, the decision module 720 is specifically configured to determine a second subset of candidate devices corresponding to a wake-up word, where there is a pre-configured correspondence between the wake-up word and the candidate devices in the second subset, and to determine a target device from the second subset of candidate devices based on at least one of intent information and user identification information.

[0123] In some examples, the decision module 720 is specifically configured to acquire at least one user feature information from among the first user's sound source positioning information, the first user's voiceprint feature information, and the first user's facial image feature information, based on the audio information, and to determine user identification information based on at least one user feature information.

[0124] In some cases, the wake-up engine in device 700 is enabled, and the wake-up engine in at least one terminal device is disabled.

[0125] Figure 8 is a schematic block diagram of a device 800 according to one embodiment of the present application. The device 800 includes a transmitting module 810 and a receiving module 820. Optionally, the device 800 further includes a processing module 830. The device 800 may be a terminal device in the above-described embodiment, or a component (such as a chip) of a terminal device.

[0126] The transmitting module 810 is configured to transmit registration information to an in-vehicle device, which is used by the device 800 to register with the in-vehicle device. The receiving module 820 is configured to receive instruction information from the in-vehicle device, which indicates that the device 800 should perform a first action, the first action including an action indicated to be performed in first user voice information.

[0127] In some examples, registration information includes account information, which includes at least one of the following: the user's account identifier, the user's voiceprint feature information, and the user's facial image feature information.

[0128] In some examples, registration information further includes device information, which includes at least one of the device type and MAC address information.

[0129] In some examples, the device 800 further includes a processing module 830, which is configured to disable its wake-up engine after registration information has been transmitted, and the wake-up engine is configured to respond to user voice information.

[0130] In some examples, the receiving module 820 is further configured to receive search information transmitted by the in-vehicle device, where the search information indicates that the device 800 is registering with the in-vehicle device.

[0131] Figure 9 is a schematic block diagram of apparatus 900 according to another embodiment of the present application. Apparatus 900 includes a processor 910. The processor 910 is configured to execute computer programs or instructions stored in memory 920, or to read data stored in memory 920, in order to perform the method in the embodiments of the method described above. Optionally, there may be one or more processors 910.

[0132] Optionally, the device 900 further includes a memory 920, as shown in Figure 9. The memory 920 is configured to store computer programs or instructions and / or data. The memory 920 may be integrated into the processor 910 or it may be located separately. Optionally, one or more memories 920 may be present.

[0133] Optionally, the device 900 further includes a communication interface 930, as shown in Figure 9. The communication interface 930 is configured to receive and / or transmit signals. For example, the processor 910 is configured to control the communication interface 930 to transmit and / or receive signals.

[0134] In the first solution, the device 900 is configured to perform the operations performed by the in-vehicle device in the embodiments of the method described above.

[0135] For example, the processor 910 is configured to execute a computer program or instruction stored in the memory 920 in order to perform the relevant operations of the in-vehicle device in the embodiments of the method described above. For example, the method is implemented by the in-vehicle device of the embodiments shown in Figures 4 to 6.

[0136] In the second solution, the device 900 is configured to perform the operations performed by the terminal device in the embodiments of the method described above.

[0137] For example, the processor 910 is configured to execute a computer program or instruction stored in the memory 920 in order to perform the associated operations of the terminal device in the embodiments of the method described above. For example, the method performed by the terminal device is implemented in the embodiments shown in Figures 4 to 6.

[0138] It should be noted that the device 900 in Figure 9 may be an in-vehicle device or terminal device in the embodiments described above, or it may be a chip or chip system, such as a system on a chip (SoC). This is not limited to the foregoing.

[0139] Those skilled in the art will recognize that the units and algorithmic steps in the examples described with reference to the embodiments disclosed herein may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the function is performed by hardware or software will depend on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described function for specific applications, but the embodiments should not be considered to exceed the scope of this application.

[0140] For the sake of simplicity, it will be readily apparent to those skilled in the art that the detailed operating processes of the aforementioned systems, devices, and units should be referred to in the corresponding processes in the embodiments of the methods described above. Details will not be repeated here.

[0141] It should be understood that in some embodiments provided in this application, the disclosed systems, apparatus, and methods may be implemented in other ways. For example, the embodiments of the described apparatus are merely examples. For example, the division into units is merely a division of logical functions, and other divisions may be possible in actual embodiments. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the presented or described mutual coupling or direct coupling or communication connection may be implemented through some interfaces. Indirect coupling or communication connection between apparatus or units may be implemented electronically, mechanically, or in other forms.

[0142] Units described as separate parts may or may not be physically separate, and parts presented as units may or may not be physical units, that is, they may be located in one location or distributed across multiple network units. Some or all of the units may be selected based on the actual requirements in order to achieve the objectives of the solutions of the embodiments.

[0143] In addition, the functional units of the embodiments of this application may be integrated into a single processing unit, or each unit may exist physically independently, or two or more units may be integrated into a single unit.

[0144] When functions are implemented in the form of software function units and sold or used as independent products, those functions may be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of this application, or parts thereof, as essential, may be implemented in the form of a software product. A computer software product is stored on a storage medium and includes several instructions for instructing a computer device (which may be a personal computer, server, or network device) to perform all or part of the steps of the methods described in embodiments of this application. The aforementioned storage medium includes any medium capable of storing program code, such as a USB flash drive, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0145] The foregoing description is merely a specific embodiment of the present application and is not intended to limit the scope of protection of this application. Any modifications or substitutions that are readily conceivable by a person skilled in the art within the scope of the art disclosed herein shall also fall within the scope of protection of this application. Accordingly, the scope of protection of this application shall be subject to the scope of protection of the claims. [Explanation of Symbols]

[0146] 700 equipment 710 Acquisition Module 720 Decision Module 730 Executable Modules 800 equipment 810 Transmitter Module 820 Receiver Module 830 Processing Module 900 equipment 910 Processor 920 memory 930 Communication Interface

Claims

1. A device control method, A step of obtaining voice information of a first user using an in-vehicle device, wherein the voice information indicates that a first operation is to be performed. The steps include: determining a target device from a plurality of candidate devices based on the voice information using the in-vehicle device, wherein the plurality of candidate devices include the in-vehicle device and at least one terminal device; When the target device belongs to the at least one terminal device, the in-vehicle device transmits instruction information to the target device, wherein the instruction information indicates that the target device will perform the first operation, or When the target device is the in-vehicle device, the in-vehicle device performs the first operation. Methods that include...

2. The method according to claim 1, wherein the at least one terminal device registers registration information with the in-vehicle device.

3. The aforementioned registration information includes account information, and the aforementioned account information is User account identifier, user voiceprint feature information, and user facial image feature information The method according to claim 2, comprising at least one of the following.

4. The method according to claim 2 or 3, wherein the registration information further includes device information, and the device information includes at least one of device type, media access control, MAC, and address information.

5. The method described above is The steps include: transmitting search information to the terminal device using the in-vehicle device, wherein the search information indicates that the terminal device will register it with the in-vehicle device; The in-vehicle device receives the registration information from the terminal device. The method according to any one of claims 2 to 4, further comprising:

6. The aforementioned audio information indicates reference information corresponding to the target device, and the reference information is, Intentional information indicating the intent of the aforementioned audio information, A wake-up word used to wake up the in-vehicle device, wherein the wake-up word belongs to a plurality of wake-up words, and there is a pre-set correspondence between the plurality of wake-up words and the plurality of candidate devices, and User identification information indicating the identity of the first user, The method according to any one of claims 1 to 5, comprising at least one of the following:

7. The in-vehicle device performs the step of determining a target device from a plurality of candidate devices based on the voice information, The steps include: determining a first candidate device subset from the plurality of candidate devices based on the intent information using the in-vehicle device, wherein the type of candidate device within the first candidate device subset corresponds to the intent information; The in-vehicle device performs the steps of determining the target device from the first candidate device subset based on the user identification information. The method according to claim 6, including the method described in claim 6.

8. The in-vehicle device performs the step of determining a target device from a plurality of candidate devices based on the voice information, A step of determining a second candidate device subset corresponding to the wake-up word using the in-vehicle device, the step including having a pre-defined correspondence between the wake-up word and the candidate devices in the second candidate device subset, The in-vehicle device determines the target device from the second candidate device subset based on at least one of the intent information and the user identification information. The method according to claim 6, including the method described in claim 6.

9. The method described above is The steps include: obtaining at least one user characteristic information from among the first user's sound source positioning information, the first user's voiceprint characteristic information, and the first user's facial image characteristic information using the in-vehicle device based on the aforementioned audio information; The in-vehicle device performs the steps of determining the user identification information based on the at least one user characteristic information and The method according to any one of claims 6 to 8, further comprising:

10. The method according to any one of claims 1 to 9, wherein the wake-up engine in the in-vehicle device is enabled and the wake-up engine in at least one terminal device is disabled.

11. A step of transmitting registration information to an in-vehicle device by a terminal device, wherein the registration information is used by the terminal device to register with the in-vehicle device; The steps include: receiving instruction information from the in-vehicle device via the terminal device, wherein the instruction information indicates that the terminal device should perform a first operation, and the first operation is indicated to be performed by voice information of a first user; A device control method including

12. The aforementioned registration information includes account information, and the aforementioned account information is User account identifier, user voiceprint feature information, and user facial image feature information The method according to claim 11, comprising at least one of the following.

13. The method according to claim 11 or 12, wherein the registration information further includes device information, and the device information includes at least one of device type and media access control MAC address information.

14. The method described above is A step of disabling the wake-up engine of the terminal device after it has transmitted the registration information, wherein the wake-up engine is configured to respond to the user's voice information. The method according to any one of claims 11 to 13, further comprising:

15. The method described above is A step of receiving search information transmitted by the in-vehicle device using the terminal device, wherein the search information indicates that the terminal device will register it with the in-vehicle device. The method according to any one of claims 11 to 14, further comprising:

16. It is an in-car device, An acquisition module configured to acquire voice information of a first user, wherein the voice information indicates that a first operation is performed, A decision module configured to determine a target device from a plurality of candidate devices based on the aforementioned voice information, wherein the plurality of candidate devices include the in-vehicle device and at least one terminal device, and the decision module comprises An execution module configured to send instruction information to the target device when the target device belongs to the at least one terminal device, wherein the instruction information indicates that the target device will perform the first operation, or an execution module that performs the first operation when the target device is the in-vehicle device An in-vehicle device equipped with the following features.

17. The in-vehicle device according to claim 16, wherein at least one terminal device registers registration information with the in-vehicle device.

18. The aforementioned registration information includes account information, and the aforementioned account information is User account identifier, user voiceprint feature information, and user facial image feature information The in-vehicle device according to claim 17, comprising at least one of the following.

19. The in-vehicle device according to claim 17 or 18, wherein the registration information further includes device information, the device information includes at least one of device type, media access control, MAC, and address information.

20. The in-vehicle device according to any one of claims 17 to 19, wherein the acquisition module is further configured to transmit search information to the terminal device, the search information to indicate that the terminal device registers with the in-vehicle device, and the in-vehicle device receives the registration information from the terminal device.

21. The aforementioned audio information indicates reference information corresponding to the target device, and the reference information is, Intentional information indicating the intent of the aforementioned audio information, A wake-up word used to wake up the in-vehicle device, wherein the wake-up word belongs to a plurality of wake-up words, and there is a pre-set correspondence between the plurality of wake-up words and the plurality of candidate devices, and User identification information indicating the identity of the first user, An in-vehicle device according to any one of claims 16 to 20, comprising at least one of the following:

22. The in-vehicle device according to claim 21, wherein the decision module is particularly configured to determine a first candidate device subset from the plurality of candidate devices based on intent information, the type of candidate device in the first candidate device subset corresponds to the intent information, and the target device is determined from the first candidate device subset based on user identification information.

23. The in-vehicle device according to claim 21, wherein the decision module is particularly configured to determine a second candidate device subset corresponding to the wake-up word, wherein there is a pre-configured correspondence between the wake-up word and the candidate devices in the second candidate device subset, and to determine the target device from the second candidate device subset based on at least one of the intent information and the user identification information.

24. The in-vehicle device according to any one of claims 21 to 23, wherein the determination module is particularly configured to acquire at least one user feature information from among the first user's sound source positioning information, the first user's voiceprint feature information, and the first user's face image feature information based on the audio information, and to determine the user identification information based on the at least one user feature information.

25. The in-vehicle device according to any one of claims 16 to 24, wherein the wake-up engine in the in-vehicle device is enabled and the wake-up engine in at least one terminal device is disabled.

26. Terminal device, A transmission module configured to transmit registration information to an in-vehicle device, wherein the registration information is used by the terminal device to register with the in-vehicle device; A receiving module configured to receive instruction information from the in-vehicle device, wherein the instruction information includes an operation that indicates the terminal device should perform a first operation, and the first operation is indicated to be performed by voice information of a first user. A terminal device equipped with the following features.

27. The aforementioned registration information includes account information, and the aforementioned account information is User account identifier, user voiceprint feature information, and user facial image feature information The terminal device according to claim 26, comprising at least one of the following.

28. The terminal device according to claim 26 or 27, wherein the registration information further includes device information, and the device information includes at least one of device type, media access control, MAC, and address information.

29. The terminal device according to any one of claims 26 to 28, further comprising a processing module, the processing module configured to disable the wake-up engine of the terminal device after the registration information has been transmitted, and the wake-up engine configured to respond to the user's voice information.

30. The terminal device according to any one of claims 26 to 29, wherein the receiving module is further configured to receive search information transmitted by the in-vehicle device, and the search information indicates that the terminal device registers with the in-vehicle device.

31. A processor configured to call a computer program from memory, wherein when the computer program is executed, the processor is capable of performing the method described in any one of claims 1 to 10. An in-vehicle device equipped with the following features.

32. A processor configured to call a computer program from memory, wherein when the computer program is executed, the processor is capable of performing the method described in any one of claims 11 to 15. A terminal device equipped with the following features.

33. A vehicle comprising the in-vehicle device according to any one of claims 16 to 25.

34. A computer-readable storage medium configured to store a computer program, wherein the computer program includes code used to perform the method described in any one of claims 1 to 10, or code used to perform the method described in any one of claims 11 to 15.

35. A computer program product comprising code used to perform the method described in any one of claims 1 to 10, or code used to perform the method described in any one of claims 11 to 15.

Images