Voice command processing method and apparatus for electronic device

Abstract

A voice command processing method and apparatus of an electronic device are provided. The voice command processing method includes receiving a voice input, wherein the voice input includes a task to be performed and an object to perform the task to be performed, and in response to the object being a non-specific object, transmitting a first message to a target device to cause the target device to output information for requesting performance of the task to be performed according to the first message.

Description

Technical Field

[0001] This disclosure generally relates to the field of electronic technology, and more specifically, to a voice command processing method and apparatus for an electronic device. Background Technology

[0002] With the widespread use of smart speakers, they are becoming increasingly integrated into people's lives.

[0003] Smart speakers can not only play songs and audiobooks, but also shop online, provide weather forecasts, and interact with users via voice to answer various questions, such as chatting, providing translation services, and mimicking animal sounds. Furthermore, smart speakers can control home IoT devices via voice commands, such as turning on the TV, turning off the air conditioner, and adjusting light brightness. They can also automatically turn on and adjust IoT devices based on the current scene.

[0004] The widespread adoption of smart devices has brought unprecedented convenience to people. However, smart devices are still unable to accurately parse and execute some user voice commands to meet user needs. Summary of the Invention

[0005] Exemplary embodiments of this disclosure provide a voice command processing method and apparatus for an electronic device, which can accurately parse and execute user voice commands, thereby meeting user needs and improving user experience.

[0006] According to a first aspect of the present disclosure, a voice command processing method for an electronic device is provided, comprising: receiving voice input, wherein the voice input includes a task to be performed and an object to perform the task; and in response to the object being a non-specific object, sending a first message to a target device, such that the target device outputs information for requesting the execution of the task based on the first message.

[0007] Optionally, the voice command processing method further includes: obtaining keywords representing an object from the content of the voice input; and determining that the object is a non-specific object when the keywords contain specific words.

[0008] Optionally, the step of sending a first message to a target device in response to the object being a non-specific object includes: in response to the object being a non-specific object, determining that there are other electronic devices in the surrounding environment of the cluster where the electronic device is located, which are other electronic devices that perform tasks; identifying the other electronic devices as the target device, and sending a first message to the target device.

[0009] Optionally, the step of sending a first message to a target device in response to the object being a non-specific object includes: in response to the object being a non-specific object, determining a target object currently suitable for performing the task to be performed; determining electronic devices around the target object as target devices, and sending a first message to the target device.

[0010] Optionally, the step of determining a target object suitable for performing the task to be performed includes: identifying task execution entities in the environment surrounding other electronic devices in the cluster where the electronic device is located as candidate objects; and determining a target object from the candidate objects based on the content of the task to be performed and the characteristic information of the candidate objects.

[0011] Optionally, the voice command processing method further includes: receiving feature information of candidate objects in the surrounding environment determined by the other electronic device; and / or receiving feature data of candidate objects in the surrounding environment collected by the other electronic device, and determining feature information of candidate objects based on the received feature data of candidate objects.

[0012] Optionally, the feature data of the candidate object includes at least one of the following: an image of the candidate object, audio of the candidate object, and identification information of the candidate object's wearable electronic device.

[0013] Optionally, the feature information of the candidate object includes: the attribute information and / or status information of the candidate object.

[0014] Optionally, the state information of the candidate object is used to describe the current state of the candidate object.

[0015] Optionally, the attribute information of the candidate includes at least one of the following: identity information, age group, ability level, and physical strength level.

[0016] Optionally, the step of determining the feature information of the candidate object based on the feature data of the received candidate object includes: inputting the image of the received candidate object into a face recognition model and / or inputting the audio of the received candidate object into a voiceprint recognition model to obtain the attribute information of the candidate object; and / or inputting the image of the received candidate object into an image content recognition model to obtain text used to describe the state of the person in the image as the state information of the candidate object; and / or determining the attribute information of the candidate object based on the identification information of the wearable electronic device of the candidate object.

[0017] Optionally, the step of determining the target object from the candidate objects based on the content of the task to be executed and the characteristic information of the candidate objects includes: filtering out a first candidate object capable of executing the task to be executed from the candidate objects based on the content of the task to be executed and the attribute information of the candidate objects; filtering out a second candidate object currently having time to execute the task to be executed from the first candidate objects based on the status information of the first candidate object; and determining the target object based on the filtered second candidate object.

[0018] Optionally, the step of selecting a first candidate object capable of performing the task to be performed from the candidate objects based on the content of the task to be performed and the attribute information of the candidate objects includes: inputting the content of the task to be performed into at least one information classification model to obtain at least one attribute category to which the task to be performed belongs; determining the candidate objects whose attribute information matches the obtained at least one attribute category as the first candidate object capable of performing the task to be performed, wherein the at least one attribute category includes a category for characterizing danger and / or a category for characterizing difficulty.

[0019] Optionally, the step of selecting a second candidate object that currently has time to execute the task to be executed from the first candidate objects based on the status information of the first candidate objects includes: inputting the status information of the first candidate objects into a status classification model to obtain the status category to which the first candidate objects belong; determining whether the first candidate objects are in a leaveable state based on the status category to which the first candidate objects belong; and determining the first candidate objects in the leaveable state as the second candidate objects that currently have time to execute the task to be executed.

[0020] Optionally, the step of determining the target object based on the selected second candidate objects includes: determining all second candidate objects as the target object; or, based on the identity information of the second candidate objects, obtaining the priority of the initiator of the voice input requesting each second candidate object to perform a task; and determining the target object from the second candidate objects based on the obtained priority.

[0021] Optionally, the voice command processing method further includes: generating a first message for requesting the target object to execute the task based on the content of the task to be executed, the identity information and current location of the initiator of the voice input, and the identity information of the target object.

[0022] Optionally, the voice command processing method further includes: when a second message indicating that no target object has responded is received from the target device, outputting information indicating that no target object has responded, and querying the initiator of the voice input whether to send the first message to the target device again or to re-identify other target objects currently suitable for performing the task to be performed.

[0023] According to a second aspect of the present disclosure, a voice command processing apparatus for an electronic device is provided, comprising: a voice input receiving unit configured to receive voice input, wherein the voice input includes a task to be executed and an object to execute the task; and a message sending unit configured to send a first message to a target device in response to the object being a non-specific object, such that the target device outputs information for requesting the execution of the task based on the first message.

[0024] Optionally, the voice command processing device further includes: a non-specific object determination unit, configured to obtain keywords representing an object from the content of the voice input; and to determine the object as a non-specific object when the keywords contain specific words.

[0025] Optionally, the message sending unit is configured to: in response to the object being a non-specific object, determine that there are other electronic devices in the surrounding environment of the cluster where the electronic device is located, and identify the other electronic devices as the target device, and send a first message to the target device.

[0026] Optionally, the message sending unit includes: a target object determination unit, configured to determine a target object suitable for performing the task to be performed in response to the object being a non-specific object; and a sending unit, configured to determine electronic devices around the target object as target devices and send a first message to the target devices.

[0027] Optionally, the target object determination unit is configured to: determine the task execution entities in the surrounding environment of other electronic devices in the cluster where the electronic device is located as candidate objects; and determine the target object from the candidate objects based on the content of the task to be executed and the characteristic information of the candidate objects.

[0028] Optionally, the voice command processing device further includes: a feature information acquisition unit, configured to receive feature information of candidate objects in the surrounding environment determined by the other electronic device; and / or to receive feature data of candidate objects in the surrounding environment collected by the other electronic device, and determine the feature information of the candidate objects based on the received feature data of the candidate objects.

[0029] Optionally, the feature data of the candidate object includes at least one of the following: an image of the candidate object, audio of the candidate object, and identification information of the candidate object's wearable electronic device.

[0030] Optionally, the feature information of the candidate object includes: the attribute information and / or status information of the candidate object.

[0031] Optionally, the state information of the candidate object is used to describe the current state of the candidate object.

[0032] Optionally, the attribute information of the candidate includes at least one of the following: identity information, age group, ability level, and physical strength level.

[0033] Optionally, the feature information acquisition unit is configured to: input the received image of the candidate object into a face recognition model and / or input the received audio of the candidate object into a voiceprint recognition model to obtain the attribute information of the candidate object; and / or input the received image of the candidate object into an image content recognition model to obtain text describing the state of the person in the image as the state information of the candidate object; and / or determine the attribute information of the candidate object based on the identification information of the wearable electronic device of the candidate object.

[0034] Optionally, the target object determination unit is configured to: filter out a first candidate object capable of executing the task to be executed from the candidate objects based on the content of the task to be executed and the attribute information of the candidate objects; filter out a second candidate object currently having time to execute the task to be executed from the first candidate objects based on the status information of the first candidate object; and determine the target object based on the filtered second candidate object.

[0035] Optionally, the target object determination unit is configured to: input the content of the task to be executed into at least one information classification model to obtain at least one attribute category to which the task to be executed belongs; determine the candidate object that matches the attribute information with the obtained at least one attribute category as the first candidate object capable of executing the task to be executed, wherein the at least one attribute category includes a category for characterizing danger and / or a category for characterizing difficulty.

[0036] Optionally, the target object determination unit is configured to: input the state information of the first candidate object into the state classification model to obtain the state category to which the first candidate object belongs; determine whether the first candidate object is in a leaveable state based on the state category to which the first candidate object belongs; and determine the first candidate object in the leaveable state as the second candidate object that currently has time to execute the task to be executed.

[0037] Optionally, the target object determination unit is configured to: determine all second candidate objects as target objects; or, based on the identity information of the second candidate objects, obtain the priority of the initiator of the voice input requesting each second candidate object to perform a task; and determine the target object from the second candidate objects based on the obtained priority.

[0038] Optionally, the voice command processing device further includes a message generation unit configured to generate a first message for requesting the target object to execute the task based on the content of the task to be executed, the identity information and current location of the initiator of the voice input, and the identity information of the target object.

[0039] Optionally, the voice command processing device further includes: an interrogation unit configured to, when receiving a second message from the target device indicating that no target object has responded, output information indicating that no target object has responded, and interrogate the initiator of the voice input whether to send the first message to the target device again or to re-identify other target objects currently suitable for performing the task to be performed.

[0040] According to a third aspect of the present disclosure, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, it implements the voice command processing method of the electronic device as described above.

[0041] According to a fourth aspect of the present disclosure, an electronic device is provided, wherein the electronic device includes: a processor; and a memory storing a computer program, wherein when the computer program is executed by the processor, it implements the voice command processing method of the electronic device as described above.

[0042] The voice command processing method and apparatus for an electronic device according to exemplary embodiments of the present disclosure are capable of accurately understanding and responding to voice commands that do not specify an object to be executed, and sending messages to the target device so that the target device outputs information for requesting the execution of the task to be executed.

[0043] Further aspects and / or advantages of the general concept of this disclosure will be set forth in part in the description which follows, and in part will be clear from the description or may be learned by practice of the general concept of this disclosure. Attached Figure Description

[0044] The above and other objects and features of exemplary embodiments of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, which exemplarily illustrate the embodiments, wherein:

[0045] Figure 1 A flowchart illustrating a voice command processing method for an electronic device according to an exemplary embodiment of the present disclosure;

[0046] Figure 2 A flowchart illustrating a method for sending a first message to a target device according to an exemplary embodiment of the present disclosure;

[0047] Figure 3 A flowchart illustrating a method for sending a first message to a target device according to another exemplary embodiment of the present disclosure;

[0048] Figure 4 An example of an image content recognition model according to an exemplary embodiment of the present disclosure is shown;

[0049] Figure 5 A flowchart illustrating a method for determining a target object from candidate objects according to an exemplary embodiment of the present disclosure;

[0050] Figure 6 A structural block diagram of a voice command processing apparatus for an electronic device according to an exemplary embodiment of the present disclosure is shown. Detailed Implementation

[0051] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, examples of which are illustrated in the drawings, wherein the same reference numerals always refer to the same parts. The embodiments will now be described with reference to the accompanying drawings in order to explain this disclosure.

[0052] Figure 1 A flowchart illustrating a voice command processing method for an electronic device (hereinafter also referred to as a first electronic device for ease of description) according to an exemplary embodiment of the present disclosure is shown. The method can be implemented by a computer program. For example, the method can be executed by a voice processing application installed in the first electronic terminal, or by a functional program implemented in the operating system of the first electronic device. As an example, the first electronic device may be a mobile communication terminal (e.g., a smartphone), a smart wearable device (e.g., a smartwatch), a tablet computer, a personal computer, a digital multimedia player, a smart speaker, a smart home device, or other electronic devices.

[0053] Reference Figure 1 In step S101, voice input is received.

[0054] The voice input includes the task to be performed and the object to perform the task.

[0055] In step S102, in response to the object being a non-specific object, a first message is sent to the target device so that the target device outputs information for requesting the execution of the task to be executed based on the first message.

[0056] Here, "non-specific object" means a non-specific object, not referring to a particular object. In other words, the content of the voice input involves requesting other objects to perform a task, but the content of the voice input does not explicitly specify which specific object is being requested to perform the task.

[0057] For example, when the voice input includes the sentence, "Who can get me the scissors?", the task to be performed in the voice input is "get the scissors", and the object to perform the task is "who". Similarly, when the voice input includes the sentence, "Can someone bring the eye drops to the room?", the task to be performed in the voice input is "bring the eye drops to the room", and the object to perform the task is "someone".

[0058] As an example, the voice command processing method of an electronic device according to an exemplary embodiment of the present disclosure may further include: after step S101, obtaining keywords representing an object from the content of the voice input; and when the keywords contain specific words, determining that the object is a non-specific object.

[0059] As an example, Natural Language Processing (NLP) technology can be used to process the content of the speech input to obtain keywords representing objects in the content of the speech input. In addition, keywords representing the task to be performed (i.e., the content of the task to be performed) can also be obtained from the content of the speech input.

[0060] As an example, NLP techniques may include, but are not limited to, at least one of the following: Named Entity Recognition (NER), regular expressions, Part-of-Speech Tagging (POS), and Dependency Parsing (DP).

[0061] In the first embodiment, NER technology or regular expressions can be used to identify the subject information in the voice input content. For example, the subject keywords in "Who can help me get the scissors?" can be identified as "who", "me", and "scissors"; the subject keywords in "Can someone bring the eye drops to the room?" can be identified as "someone", "eye drops", and "room".

[0062] In the second embodiment, part-of-speech tagging technology can be used to identify keywords representing actions in the content of the voice input. For example, the keyword representing the action in "Who can help me get the scissors?" can be identified as "get"; and the keyword representing the action in "Can someone bring the eye drops to the room?" can be identified as "bring".

[0063] In the third embodiment, DP technology can be used to identify the grammatical dependency relationships of each word in the speech input content, that is, to determine the subject-verb-object information in the speech input content. For example, to determine which word in the speech input content is the subject (e.g., "who" is the subject), which is the verb (e.g., "take" is the verb), and which is the object (e.g., "scissors" is the object).

[0064] It should be understood that at least one of the first, second, and third embodiments can be used to process the content of the voice input to obtain a comprehensive processing result.

[0065] As an example, the specific words may include, but are not limited to, at least one of the following: who, someone.

[0066] It should be understood that the number of target devices can be one or more.

[0067] As an example, the target device may output information for requesting the execution of the task to be executed or display information for requesting the execution of the task to be executed based on the first message voice output.

[0068] As an example, the target device may output information requesting a task execution entity to perform the task to be performed based on the first message. For example, the task execution entity can be understood as an object capable of performing the task, and the type of the task execution entity can be preset. For example, the task execution entity can be a natural person and / or an intelligent robot.

[0069] The following will combine Figures 2 to 5 An exemplary embodiment of step S102 will be described below.

[0070] Figure 2 A flowchart illustrating a method for sending a first message to a target device according to an exemplary embodiment of the present disclosure is provided, wherein step S102 may include steps S201 and S202.

[0071] Reference Figure 2 In step S201, in response to the object being a non-specific object, it is determined that there are other electronic devices in the surrounding environment of the cluster where the first electronic device is located, which are other electronic devices that perform the task.

[0072] As an example, the electronic devices included in the cluster can be specified in advance; that is, a cluster list can be pre-set. As another example, the cluster containing the first electronic device can include electronic devices located on the same local area network as the first electronic device.

[0073] In one embodiment, other electronic devices (e.g., other remote devices) within the cluster where the first electronic device resides can determine whether a task execution entity exists in the surrounding environment. These remote electronic devices, where a task execution entity exists, report this information to the first electronic device. Thus, the first electronic device can identify these remote electronic devices as remote electronic devices within its cluster whose surrounding environment contains a task execution entity. For example, a remote device can refer to a device not located in the same space as the first electronic device (e.g., a device not in the same room).

[0074] In another embodiment, the first electronic device may receive environmental data of the surrounding environment collected by other electronic devices in the cluster where the first electronic device is located, and determine the presence of other electronic devices in the surrounding environment that are task execution subjects based on the received environmental data.

[0075] As an example, environmental data of the surrounding environment may include, but is not limited to, at least one of the following: images of the surrounding environment captured, audio of the surrounding environment collected, and identification information of mobile devices appearing in the surrounding environment. For example, the mobile device may be a wearable device. For example, the identification information of the mobile device may be the short-range communication identification information of the mobile device, such as Bluetooth identification information, UWB tag identification information, or Wi-Fi identification information.

[0076] For example, if a task performer (e.g., a natural person) is identified from an image of the surrounding environment captured by an electronic device, then it is determined that a task performer exists in the surrounding environment of the electronic device. Similarly, if the sound of a task performer (e.g., a human voice) is identified from audio of the surrounding environment captured by the electronic device, then it is determined that a task performer exists in the surrounding environment of the electronic device. Furthermore, if the electronic device detects a mobile device within a certain radius (e.g., by detecting its presence through a short-range communication identifier of the mobile device), then it is determined that a task performer exists in the surrounding environment of the electronic device.

[0077] In step S202, other electronic devices in the surrounding environment that contain the task execution entity, as determined in step S201, are identified as target devices, and a first message is sent to the target devices. In other words, in addition to the first electronic device, all electronic devices in the cluster that contain the task execution entity are considered target devices, and a first message is sent to them uniformly.

[0078] Figure 3 A flowchart illustrating a method for sending a first message to a target device according to another exemplary embodiment of the present disclosure is shown, wherein step S102 may include steps S301 and S302.

[0079] Reference Figure 3In step S301, in response to the object being a non-specific object, a target object suitable for performing the task to be performed is determined.

[0080] As an example, the task execution entities in the surrounding environment of other electronic devices in the cluster where the first electronic device is located can be identified as candidate objects; and the target object can be determined from the candidate objects based on the content of the task to be executed and the characteristic information of the candidate objects.

[0081] As an example, the voice command processing method of an electronic device according to an exemplary embodiment of this disclosure may further include: acquiring feature information of candidate objects. In one embodiment, feature information of candidate objects in the surrounding environment determined by the other electronic device may be received from the other electronic device; that is, the other electronic device may acquire feature information of candidate objects in the surrounding environment itself and provide it to the first electronic device. In another embodiment, feature data of candidate objects in the surrounding environment collected by the other electronic device may be received from the other electronic device, and feature information of candidate objects may be determined based on the received feature data of candidate objects.

[0082] As an example, the feature data of the candidate object may include, but is not limited to, at least one of the following: an image of the candidate object, audio of the candidate object, and identification information of the candidate object's wearable electronic device.

[0083] As an example, the feature information of a candidate object may include, but is not limited to, the candidate object's attribute information and / or status information.

[0084] As an example, the state information of a candidate object can be used to describe its current state. For example, the current state could be the current action or what it is currently doing.

[0085] As an example, the attribute information of a candidate may include, but is not limited to, at least one of the following: identity information, age group, ability level, and physical strength level. For example, identity information may be a name or membership status (e.g., family member status). For example, the age group type may include, but is not limited to, at least one of the following: child, adult, senior. For example, the ability level may include, but is not limited to, at least one of the following: strong, medium, weak. For example, the physical strength level may include, but is not limited to, at least one of the following: strong, medium, weak.

[0086] As an example, the step of determining the feature information of a candidate object based on the feature data of the received candidate object may include at least one of steps (a), (b), and (c).

[0087] Step (a) involves inputting the received candidate object image into the face recognition model and / or inputting the received candidate object audio into the voiceprint recognition model to obtain the candidate object attribute information.

[0088] As an example, a face recognition model may include, but is not limited to, at least one of the following machine learning models: MTCNN (Multi-task Cascaded Convolutional Networks) model, CascadeCNN (Cascade Convolutional Networks) model, and DenseBox model.

[0089] As an example, a voiceprint recognition model may include, but is not limited to, at least one of the following machine learning models: GMM-UBM (Gaussian Mixture Model-Universal Background Model) or Deep Neural Network (DNN) model.

[0090] As an example, the received candidate object's image can be input into a face recognition model and / or the received candidate object's audio can be input into a voiceprint recognition model to obtain the candidate object's identity information. Then, based on the candidate object's identity information, a pre-set attribute information data table can be queried to obtain other attribute information of the candidate object. As an example, the attribute information data table can be shown in Table 1.

[0091] Table 1 Attribute Information Data Table

[0092]

[0093]

[0094] In addition, some data in the table above is in the "NA" state, which means the following: Grandma is not wearing a mobile device, so there is no corresponding information.

[0095] As an example, the first electronic device and other electronic devices can share the aforementioned attribute information data table. As an example, the aforementioned attribute information data table can be stored in the cloud. As an example, the attribute information data table can be updated in real time. It should be understood that the identities in the above embodiments are not limited to family members and may include other identities.

[0096] Step (b): Determine the attribute information of the candidate object based on the identification information of the wearable electronic device of the candidate object.

[0097] As an example, the candidate's identity information can be obtained based on the identification information of the candidate's wearable electronic device, and then other attribute information of the candidate can be obtained by querying a pre-set attribute information data table based on the candidate's identity information.

[0098] Step (c) involves inputting the received candidate object image into the image content recognition model to obtain text describing the state of the person in the image as the candidate object's state information.

[0099] Figure 4 An example of an image content recognition model according to an exemplary embodiment of this disclosure is shown. The image content recognition model can be used to determine what a candidate object in an image is doing.

[0100] As an example, the encoder in an image content recognition model can use a CNN (Convolutional Neural Network) model, and the decoder can use an RNN (Recurrent Neural Network) model. The CNN model is used to extract features from the image and generate a context vector, while the RNN model uses the context vector to output text related to the image content.

[0101] For example, for an image containing candidate objects, the image content recognition model identifies the image description information "There is a person using a computer to surf the Internet", and the face recognition model identifies the person as "Dad". Then, NLP analysis can be performed on the description information (e.g., using DP to identify the subject, verb and object of the sentence). Then, the identity information identified by the face recognition model is used to replace the subject of the description information to obtain a comprehensive result (e.g., "Dad is using a computer to surf the Internet").

[0102] As an example, the feature information of the candidate object obtained through the above exemplary embodiments can be shown in Table 2. The location information of the candidate object is used to indicate the electronic devices around the candidate object, that is, the electronic devices that provide the first electronic device with relevant information (e.g., feature information or feature data) of the candidate object.

[0103] Table 2 Feature Information Table

[0104] Candidate 1 Baby "Playing games" Equipment C Candidate 2 dad "Internet" Equipment B Candidate 3 Mother "Sewing clothes" Equipment A Candidate 4 grandfather "Reading the newspaper" Equipment E Candidate 5 grandmother "Cook" Device D

[0105] It should be understood that the number of target users can be one or more.

[0106] In one embodiment, the step of determining the target object from the candidate objects based on the content of the task to be executed and the characteristic information of the candidate objects may include: filtering out a first candidate object capable of executing the task to be executed from all candidate objects based on the content of the task to be executed and the attribute information of the candidate objects; then, filtering out a second candidate object currently having time to execute the task to be executed from all first candidate objects based on the status information of the first candidate object; and finally, determining the target object based on all the filtered second candidate objects. The following will combine... Figure 5 The exemplary embodiment will now be described in detail.

[0107] In another embodiment, the step of determining the target object from the candidate objects based on the content of the task to be executed and the characteristic information of the candidate objects may include: filtering out a third candidate object from all candidate objects that currently has time to execute the task to be executed based on the status information of the candidate objects; then, filtering out a fourth candidate object from all third candidate objects that is capable of executing the task to be executed based on the content of the task to be executed and the attribute information of the candidate objects; and finally, determining the target object based on all the filtered fourth candidate objects.

[0108] In step S302, electronic devices around the target object are identified as target devices, and a first message is sent to the target devices.

[0109] Figure 5 A flowchart illustrating a method for determining a target object from candidate objects according to an exemplary embodiment of the present disclosure is shown.

[0110] Reference Figure 5 In step S401, based on the content of the task to be executed and the attribute information of the candidate objects, a first candidate object capable of executing the task to be executed is selected from the candidate objects.

[0111] As an example, the content of the task to be executed can be input into at least one information classification model to obtain at least one attribute category to which the task to be executed belongs; and the candidate objects whose attribute information matches the obtained at least one attribute category are determined as the first candidate objects capable of executing the task to be executed.

[0112] As an example, the at least one attribute category may include, but is not limited to: categories for characterizing hazard and / or categories for characterizing ease of use.

[0113] As an example, a candidate object whose ability level and / or physical strength level in the attribute information matches at least one attribute category can be identified as the first candidate object capable of performing the task to be performed.

[0114] As an example, information classification models may include, but are not limited to, at least one of the following: BERT (Bidirectional Encoder Representations from Transformers) model, RNN (Recurrent Neural Network) model, and CNN (Convolutional Neural Network) model.

[0115] For example, when the received voice input is "Can someone get me some scissors?" or "Can someone bring eye drops to my room?", the keywords "scissors" and "eye drops" in the voice input can be extracted using the exemplary embodiment above. Then, an information classification model can be used to classify the extracted keywords for the task. For example, the keyword "hazard level" can be classified into three categories (high / medium / low), and / or the keyword "weight" can also be classified into three categories (heavy / medium / light). For example, if the keyword for the task is "scissors", the resulting attribute category could be "high hazard level" or "light weight". As an example, more keyword hazard and weight categories are shown in Table 3.

[0116] Table 3 Attribute Category Table

[0117] Scissors high light dumbbel middle middle goods middle Heavy eye drops Low light fruit Low light book Low light

[0118] Next, based on the determined attribute category, the attribute category matching table (e.g., as shown in Table 4) can be queried to obtain information such as "suitable people" and / or "required abilities" that match the determined attribute category. The "suitable people" and "required abilities" correspond to the candidate's attribute information, namely "age group" and "physical strength level" or "ability level," respectively. Thus, candidate objects whose attribute information matches at least one obtained attribute category can be identified as the first candidate object capable of performing the task to be performed.

[0119] Table 4 Attribute Category Matching Table

[0120]

[0121]

[0122] For example, the attributes of "scissors" are "High Risk" and "Light Weight". Referring to Table 4, we know that "Suitable for: Adults / Elderly", not "Children", so the candidate "Baby" should be excluded from the first candidate list. For example, the attributes of "dumbbells" are "Medium Risk" and "Medium Weight". Referring to Table 4, we know that "Suitable for: Adults" and "Required Ability: Strong". Comparing this with Table 1, we determine that only the candidate "Father" meets the criteria, so only "Father" is selected as the first candidate. For example, the attributes of "eye drops" are "Low Risk" and "Light Weight". Referring to both Table 4 and Table 1, we determine that all candidates meet the criteria, so all candidates are selected as the first candidate.

[0123] In step S402, based on the status information of the first candidate object, a second candidate object that currently has time to execute the task to be executed is selected from the first candidate objects.

[0124] As an example, it can be determined whether the first candidate object is in a leaveable state based on the status information of the first candidate object; and the first candidate object in the leaveable state is determined as the second candidate object that currently has time to execute the task to be executed.

[0125] As an example, the state information of the first candidate object can be input into the state classification model to obtain the state category to which the first candidate object belongs; and based on the state category to which the first candidate object belongs, it can be determined whether the first candidate object is in a state where it can leave.

[0126] As an example, the status category may include, but is not limited to, at least one of the following: "Working", "Sleeping", and "Leisure". As an example, the status category to which different status information belongs can be determined using Table 5.

[0127] Table 5 Status Category Table

[0128] Cooking rice Work Doing homework Work sleep sleep Internet Leisure Play games Leisure Drinking tea Leisure

[0129] Then, you can query a custom rules table (e.g., Table 6) based on the "status category" to determine whether the candidate object is "in a leaveable state".

[0130] Table 6 Custom Rules Table

[0131] Work no sleep no Leisure yes

[0132] For example, Grandma is cooking in the kitchen; her state category is "Working," and she is in a "cannot leave" state, so she should be excluded, meaning she is not a second candidate. Dad is using the computer to browse the internet; his state category is "Leisure," and he is in a "can leave" state, so he has time to perform his task, meaning he is a second candidate.

[0133] In step S403, the target object is determined based on the second candidate object selected.

[0134] In one embodiment, all second candidate objects may be identified as the target object.

[0135] In another embodiment, the priority of the voice input initiator requesting each second candidate object to perform a task can be obtained based on the identity information of the second candidate objects; and the target object can be determined from the second candidate objects based on the obtained priority. It should be understood that the higher the priority, the more likely it is to be identified as the target object.

[0136] Priorities can be determined based on statistical information (e.g., historical records). For example, family members can be prioritized based on statistics of past task completion. For instance, the habits of the person initiating the voice input (i.e., the person giving the command) can be used to statistically determine who has historically prioritized (from highest to lowest) requesting a task. Furthermore, it's also possible to statistically determine who has historically prioritized (from highest to lowest) requesting each type of task. For example, when the command giver is the mother, the priority of family members performing the task might be: father > grandmother > grandfather > baby.

[0137] return Figure 1 As an example, the voice command processing method of an electronic device according to an exemplary embodiment of the present disclosure may further include: generating a first message for requesting the target object to execute the task to be performed based on the content of the task to be performed, the identity information and current location of the initiator of the voice input, and the identity information of the target object.

[0138] For example, when the first electronic device receives the mother's voice input as, "Can someone get me the scissors?", and determines the target to be "Dad", then the target device is device B, which is near Dad. The first electronic device (i.e., device A) can first convert the voice input according to a predetermined rule, and then instruct device B to play the converted message: "Dad, Mom is in room 2 (near device A) and says, please get her the scissors." After receiving the notification, device B plays the voice command according to the message.

[0139] As an example, the voice command processing method of an electronic device according to an exemplary embodiment of the present disclosure may further include: when receiving a second message from a target device indicating that no target object has responded, outputting information indicating that no target object has responded, and querying the initiator of the voice input whether to send the first message to the target device again or to re-determine other target objects currently suitable for performing the task to be performed.

[0140] Furthermore, as an example, the voice command processing method of an electronic device according to an exemplary embodiment of the present disclosure may further include: when a voice message fed back by a target user is received from a target device, outputting the received voice message.

[0141] Furthermore, as an example, the target device may respond to the received first message by outputting a corresponding voice command and begin collecting feedback from the target object. When the feedback is a question related to the content of the voice command (e.g., NLP technology can be used to make a judgment), a call connection is established between the target device and the first electronic device, so that the initiator of the voice input and the target object can interact by voice through the connection.

[0142] If a user wants to interact with others using nearby electronic devices but is unsure who is suitable for interaction in the current situation, current technologies cannot adequately address this user need. According to an exemplary embodiment of this disclosure, when a user wants to interact with others using an electronic device but is unsure who is suitable for interaction in the current situation, the system can determine suitable target objects for interaction and nearby suitable electronic devices for interaction. For example, when the first electronic device receives the mother's voice input: "Who can get me the scissors?", according to the above embodiment, candidate objects include: the father near device B, the baby near device C, and the grandmother near device D. Then, based on the candidate objects' attribute information, status information, and the content of the task to be performed, the final target object can be determined. For example, based on the candidate objects' attribute information and the content of the task to be performed, it is determined that the candidate object "baby" is unable to perform the task; based on the candidate objects' status information, it is determined that the candidate object "grandmother" currently does not have time to perform the task; finally, the father is determined as the target object. Then, device B near the father is the target device and will output the corresponding voice command: "Dad, Mom is in room 2 (near device A) and says, please get her the scissors."

[0143] Figure 6 A structural block diagram of a voice command processing apparatus for an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0144] like Figure 6 As shown, the voice command processing apparatus of an electronic device according to an exemplary embodiment of the present disclosure includes: a voice input receiving unit 101 and a message sending unit 102.

[0145] Specifically, the voice input receiving unit 101 is configured to receive voice input, wherein the voice input includes a task to be performed and an object to perform the task.

[0146] The message sending unit 102 is configured to send a first message to the target device in response to the object being a non-specific object, so that the target device outputs information for requesting the execution of the task to be executed based on the first message.

[0147] As an example, the voice command processing device may further include: a non-specific object determination unit (not shown), which is configured to obtain keywords representing an object from the content of the voice input; and determine that the object is a non-specific object when the keywords contain specific words.

[0148] As an example, the message sending unit 102 may be configured to: in response to the object being a non-specific object, determine that there are other electronic devices in the surrounding environment of the cluster where the electronic device is located, and identify the other electronic devices as the target device, and send a first message to the target device.

[0149] As an example, the message sending unit 102 may include: a target object determination unit (not shown) and a sending unit (not shown), wherein the target object determination unit is configured to determine a target object suitable for performing the task to be performed in response to the object being a non-specific object; and the sending unit is configured to determine electronic devices around the target object as target devices and send a first message to the target devices.

[0150] As an example, the target object determination unit can be configured to: determine the task execution entities in the surrounding environment of other electronic devices in the cluster where the electronic device is located as candidate objects; and determine the target object from the candidate objects based on the content of the task to be executed and the characteristic information of the candidate objects.

[0151] As an example, the voice command processing device may further include: a feature information acquisition unit (not shown), configured to receive feature information of candidate objects in the surrounding environment determined by the other electronic device; and / or, to receive feature data of candidate objects in the surrounding environment collected by the other electronic device, and to determine feature information of candidate objects based on the received feature data of candidate objects.

[0152] As an example, the feature data of a candidate object may include at least one of the following: an image of the candidate object, audio of the candidate object, and identification information of the candidate object's wearable electronic device.

[0153] As an example, the feature information of a candidate object may include: the candidate object's attribute information and / or status information.

[0154] As an example, the state information of a candidate object can be used to describe the current state of the candidate object.

[0155] As an example, the attribute information of the candidate may include at least one of the following: identity information, age group, ability level, and physical strength level.

[0156] As an example, the feature information acquisition unit can be configured to: input the received image of the candidate object into a face recognition model and / or input the received audio of the candidate object into a voiceprint recognition model to obtain the attribute information of the candidate object; and / or input the received image of the candidate object into an image content recognition model to obtain text used to describe the state of the person in the image as the state information of the candidate object; and / or determine the attribute information of the candidate object based on the identification information of the wearable electronic device of the candidate object.

[0157] As an example, the target object determination unit can be configured to: filter out a first candidate object capable of executing the task to be executed from the candidate objects based on the content of the task to be executed and the attribute information of the candidate objects; filter out a second candidate object currently having time to execute the task to be executed from the first candidate objects based on the status information of the first candidate object; and determine the target object based on the filtered second candidate object.

[0158] As an example, the target object determination unit may be configured to: input the content of the task to be performed into at least one information classification model to obtain at least one attribute category to which the task to be performed belongs; determine the candidate object that matches the attribute information with the obtained at least one attribute category as the first candidate object capable of performing the task to be performed, wherein the at least one attribute category includes a category for characterizing danger and / or a category for characterizing difficulty.

[0159] As an example, the target object determination unit can be configured to: input the state information of the first candidate object into the state classification model to obtain the state category to which the first candidate object belongs; determine whether the first candidate object is in a leaveable state based on the state category to which the first candidate object belongs; and determine the first candidate object in the leaveable state as the second candidate object that currently has time to execute the task to be executed.

[0160] As an example, the target object determination unit can be configured to: determine all second candidate objects as target objects; or, based on the identity information of the second candidate objects, obtain the priority of the initiator of the voice input requesting each second candidate object to perform a task; and determine the target object from the second candidate objects based on the obtained priority.

[0161] As an example, the voice command processing device may further include: a message generation unit (not shown), which is configured to generate a first message for requesting the target object to execute the task based on the content of the task to be executed, the identity information and current location of the initiator of the voice input, and the identity information of the target object.

[0162] As an example, the voice command processing device may further include: an interrogation unit (not shown), configured to, when receiving a second message from the target device indicating that no target object has responded, output information indicating that no target object has responded, and interrogate the initiator of the voice input whether to send the first message to the target device again or to re-identify other target objects currently suitable for performing the task to be performed.

[0163] It should be understood that the specific processing performed by the voice command processing device of the electronic device according to the exemplary embodiments of this disclosure has been referenced. Figures 1 to 5 A detailed description has been provided, and the relevant details will not be repeated here.

[0164] Furthermore, it should be understood that the various units in the voice command processing apparatus of the electronic device according to the exemplary embodiments of this disclosure may be implemented as hardware components and / or software components. Those skilled in the art, based on the processes performed by the defined various units, may implement the various units, for example, using a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0165] A computer-readable storage medium according to an exemplary embodiment of the present disclosure stores a computer program that, when executed by a processor, causes the processor to perform a voice command processing method of an electronic device as described in the exemplary embodiments above. The computer-readable storage medium can be any data storage device capable of storing data read by a computer system. Examples of computer-readable storage media may include: read-only memory, random access memory, read-only optical disk, magnetic tape, floppy disk, optical data storage device, and carrier waves (such as data transmission via the Internet through wired or wireless transmission paths).

[0166] An electronic device according to an exemplary embodiment of the present disclosure includes a processor (not shown) and a memory (not shown), wherein the memory stores a computer program that, when executed by the processor, implements the voice command processing method of the electronic device as described in the exemplary embodiment above.

[0167] While some exemplary embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A voice command processing method for an electronic device, wherein, include: Receive voice input, wherein the voice input includes a task to be performed and an object to perform the task; In response to the fact that the object is a non-specific object, a target object suitable for performing the task to be performed is determined; Electronic devices surrounding the target object are identified as target devices, and a first message is sent to the target device, causing the target device to output information requesting the execution of the task to be performed based on the first message. The step of determining the target object suitable for performing the task to be performed includes: The task execution entities in the surrounding environment of other electronic devices in the cluster where the electronic device is located are identified as candidate objects; Receive feature information of candidate objects in the surrounding environment as determined by the other electronic device, and / or receive feature data of candidate objects in the surrounding environment as collected by the other electronic device, and determine feature information of candidate objects based on the received feature data of candidate objects; Based on the content of the task to be executed and the characteristic information of the candidate objects, the target object is determined from the candidate objects.

2. The voice command processing method according to claim 1, wherein, Also includes: Extract keywords representing the object from the content of the voice input; When the keyword contains specific words, the object is determined to be a non-specific object.

3. The voice command processing method according to claim 1, wherein, The feature data of the candidate object includes at least one of the following: the image of the candidate object, the audio of the candidate object, and the identification information of the wearable electronic device of the candidate object; And / or, The characteristic information of the candidate object includes: the attribute information and / or status information of the candidate object.

4. The voice command processing method according to claim 3, wherein, The state information of a candidate object is used to describe the current state of the candidate object; And / or, The candidate's attribute information includes at least one of the following: identity information, age group, ability level, and physical strength level.

5. The voice command processing method according to claim 1, wherein, The steps for determining the feature information of candidate objects based on the received feature data of candidate objects include: The received candidate object image is input into the face recognition model and / or the received candidate object audio is input into the voiceprint recognition model to obtain the candidate object attribute information; And / or, input the received image of the candidate object into the image content recognition model to obtain text describing the state of the person in the image as the state information of the candidate object; And / or, determine the attribute information of the candidate object based on the identification information of the candidate object's wearable electronic device.

6. The voice command processing method according to claim 1, wherein, The steps for determining the target object from the candidate objects based on the content of the task to be executed and the feature information of the candidate objects include: Based on the content of the task to be executed and the attribute information of the candidate objects, a first candidate object capable of executing the task to be executed is selected from the candidate objects; Based on the status information of the first candidate object, a second candidate object that currently has time to execute the task to be executed is selected from the first candidate object; Based on the second candidate selected, the target object is determined.

7. The voice command processing method according to claim 6, wherein, The step of selecting a first candidate object capable of executing the task to be executed from the candidate objects, based on the content of the task to be executed and the attribute information of the candidate objects, includes: Input the content of the task to be executed into at least one information classification model to obtain at least one attribute category to which the task to be executed belongs; Candidate objects whose attribute information matches at least one obtained attribute category are identified as the first candidate objects capable of performing the task to be performed. The at least one attribute category includes categories for characterizing hazard and / or categories for characterizing ease of use.

8. The voice command processing method according to claim 6, wherein, The step of selecting a second candidate object that currently has time to execute the task to be executed from the first candidate object based on the status information of the first candidate object includes: Input the state information of the first candidate object into the state classification model to obtain the state category to which the first candidate object belongs; Based on the state category to which the first candidate object belongs, determine whether the first candidate object is in a state where it can leave. The first candidate object that is in a leaveable state is determined as the second candidate object that currently has time to execute the task to be executed.

9. The voice command processing method according to claim 6, wherein, The steps for determining the target object based on the selected second candidate objects include: All second-candidate objects are identified as target objects; Alternatively, based on the identity information of the second candidate object, the priority of the initiator of the voice input requesting each second candidate object to perform the task is obtained; based on the obtained priority, the target object is determined from the second candidate objects.

10. The voice command processing method according to claim 1, wherein, Also includes: Based on the content of the task to be executed, the identity information and current location of the initiator of the voice input, and the identity information of the target object, a first message is generated to request the target object to execute the task to be executed.

11. The voice command processing method according to claim 1, wherein, Also includes: When a second message indicating that no target object has responded is received from the target device, information indicating that no target object has responded is output, and the initiator of the voice input is asked whether to send the first message to the target device again or to re-identify other target objects that are currently suitable to perform the task to be performed.

12. A voice command processing device for an electronic device, wherein, include: A voice input receiving unit is configured to receive voice input, wherein the voice input includes a task to be performed and an object to perform the task; The message sending unit is configured to, in response to the object being a non-specific object, determine a target object suitable for performing the task to be performed, identify electronic devices surrounding the target object as target devices, and send a first message to the target device, so that the target device outputs information requesting the execution of the task to be performed based on the first message. The message sending unit is also configured as follows: The task execution entities in the surrounding environment of other electronic devices in the cluster where the electronic device is located are identified as candidate objects; Receive feature information of candidate objects in the surrounding environment as determined by the other electronic device, and / or receive feature data of candidate objects in the surrounding environment as collected by the other electronic device, and determine feature information of candidate objects based on the received feature data of candidate objects; Based on the content of the task to be executed and the characteristic information of the candidate objects, the target object is determined from the candidate objects.

13. A computer-readable storage medium storing a computer program, wherein, When the computer program is executed by a processor, it implements the voice command processing method of the electronic device as described in any one of claims 1 to 11.

14. An electronic device, wherein, The electronic device includes: processor; A memory storing a computer program that, when executed by a processor, implements the voice command processing method of the electronic device as described in any one of claims 1 to 11.

Images