Equipment control method and device and storage medium

Abstract

The invention relates to an equipment control method and device and a storage medium. The device control method comprises the steps that a first device collects a voice signal of a user in real time; in response to the existence of the target voice signal including the target semantic content, feeding back second equipment operation information matched with the target semantic content; wherein the target voice signal is determined on the basis of semantic recognition of all voice signals collected in real time and when target semantic content is recognized; the target semantic content represents a semantic attribute used for triggering a second device to execute an operation. According to the voice interaction method and the voice interaction device, the user only needs to express the state of the user under the condition that the device does not need to be awakened, and the device can actively control the operation related to and corresponding to the state, so that voice interaction based on device control is more intelligent and more convenient.

Description

Technical Field

[0001] The present disclosure relates to the field of human-computer interaction, and in particular to a device control method, device and storage medium. Background Art

[0002] Among the related technologies, voice, language and hearing are the most natural and convenient means for humans to communicate and obtain information. Verbal interaction has brought about a fundamental change in human-computer interaction and has become a common core technology that many industries such as smart mobile terminals, smart education, and smart home appliances rely on for development. It is the commanding height for the future development of the information technology industry in the era of big data and cognitive computing, and has broad application prospects. However, at present, voice interaction still remains at the level of rational and relatively clear information transmission. Emotional communication and non-verbal symbols are still absent in human-computer interaction, resulting in the mechanization of the human-computer interaction mechanism. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a device control method, apparatus and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a device control method is provided, comprising a first device collecting a user's voice signal in real time; in response to the presence of a target voice signal including target semantic content, feeding back second device operation information matching the target semantic content; wherein the target voice signal is based on semantic recognition of all voice signals collected in real time, and is determined when the target semantic content is recognized; the target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

[0005] In one embodiment, the semantic attribute used to trigger the second device to perform an operation includes at least one of the following: a control instruction used to trigger the second device to perform an operation; one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

[0006] In one implementation, the semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

[0007] In one embodiment, the method further includes: sending the voice signal to a cloud, so that the cloud sends a control instruction to the second device; wherein the control instruction is sent when the cloud recognizes the target voice signal.

[0008] According to a second aspect of an embodiment of the present disclosure, a device control method is provided, the method comprising: acquiring all voice signals collected and sent in real time by a first device; performing semantic recognition on all voice signals collected and sent by the device respectively; in response to recognizing target semantic content, determining that a target voice signal is recognized, sending a control instruction to a second device, and sending second device operation information matching the target semantic content to the first device; the target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

[0009] In one embodiment, the semantic attribute used to trigger the second device to perform an operation includes at least one of the following: a control instruction used to trigger the second device to perform an operation; one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

[0010] In one implementation, the semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

[0011] According to a third aspect of an embodiment of the present disclosure, a device for controlling a device is provided, the device comprising: a collection unit, configured to collect a user's voice signal in real time by a first device; a processing unit, configured to feedback, in response to the presence of a target voice signal including target semantic content, second device operation information matching the target semantic content; wherein the target voice signal is based on semantic recognition of all voice signals collected in real time, and is determined when the target semantic content is recognized; the target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

[0012] In one embodiment, the semantic attribute used to trigger the second device to perform an operation includes at least one of the following: a control instruction used to trigger the second device to perform an operation; one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

[0013] In one embodiment, the processing unit is further used to send the voice signal to the cloud, so that the cloud sends a control instruction to the second device; wherein the control instruction is sent when the cloud recognizes the target voice signal.

[0014] According to the fourth aspect of an embodiment of the present disclosure, a device for controlling a device is provided, the device comprising: an acquisition unit for acquiring a voice signal collected and sent by a first device in real time; a recognition unit for performing semantic recognition on all voice signals collected and sent by the device respectively; and a response unit for determining that a target voice signal is recognized in response to recognition of target semantic content, sending a control instruction to a second device, and sending second device operation information matching the target semantic content to the first device.

[0015] In one embodiment, the semantic attribute used to trigger the second device to perform an operation includes at least one of the following: a control instruction used to trigger the second device to perform an operation; one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

[0016] In one implementation, the semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

[0017] According to a fifth aspect of an embodiment of the present disclosure, a device for controlling a device is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to: execute the device control method described in the first aspect or any one of the first aspects.

[0018] According to the sixth aspect of an embodiment of the present disclosure, a device control apparatus is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to: execute the device control method described in the second aspect or any one of the second aspects.

[0019] According to the seventh aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a device, the device is enabled to execute the method for controlling the device described in the first aspect or any one of the first aspects.

[0020] According to an eighth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor in the cloud, the cloud can execute the method for controlling the device described in the second aspect or any one of the second aspects.

[0021] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: According to the embodiments of the present disclosure, the user's voice signal is collected in real time by the first device. In response to the existence of a target voice signal including target semantic content, the second device operation information matching the target semantic content is fed back. Among them, by performing semantic recognition on all voice signals collected in real time respectively, and when the target semantic content is recognized, the target voice signal can be determined. The target semantic content is a semantic attribute used to trigger the second device to perform an operation. Furthermore, in this way, the target voice signal expressed by the user can control the operation related to and corresponding to the target voice signal without waking up the device, thereby solving the problem that in human-computer interaction, the terminal keeps responding to the user in the absence of a wake-up word.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] Figure 1 A schematic diagram showing a traditional voice interaction method.

[0025] Figure 2 The present invention is a flowchart of a device control method according to an exemplary embodiment.

[0026] Figure 3 A flowchart of a device control method provided by an exemplary implementation example of the present disclosure is shown.

[0027] Figure 4 It is a schematic diagram of a voice interaction based on a device control method according to an exemplary implementation example.

[0028] Figure 5 It is a scene diagram implemented based on a device control method according to an exemplary implementation example.

[0029] Figure 6 It is a scene diagram implemented based on a device control method according to an exemplary implementation example.

[0030] Figure 7 The invention is a block diagram of a device controlling an apparatus according to an exemplary embodiment.

[0031] Figure 8 The invention is a block diagram of a device controlling an apparatus according to an exemplary embodiment.

[0032] Fig. 9The invention is a block diagram of an apparatus for controlling a device according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0034] The device control method provided by the embodiments of the present disclosure is applied to scenarios of human-computer interaction, for example, to scenarios of voice interaction of smart speakers.

[0035] At present, voice interaction starts with wake-up. For example, the user needs to say the wake-up word to wake up the speaker. After the speaker responds, the user can express his or her intention and conduct real voice interaction. This traditional voice interaction method makes the current human-computer interaction mechanized and always in the mode of wake-up + question and answer. Figure 1 A schematic diagram of a traditional voice interaction method is shown in FIG. Figure 1 As shown, step A is that the user says the wake-up word. Step B is that the front end recognizes the wake-up word through the voice recognition algorithm for wake-up. Step C is to start receiving sound to the cloud for voice recognition and natural language processing. Step D is to respond to the user, including answering sound and lighting effects. Step E is that the user continues to say "turn on the air conditioner". Step F is that the front end uploads the voice recognition audio. Step G is that the cloud recognizes "turn on the air conditioner" as a semantic attribute set through voice recognition. Step H is to send cloud control to the local.

[0036] Therefore, the traditional voice interaction method needs to rely on wake-up to start the interaction, which cannot improve the convenience and intelligence of voice interaction, making the interaction lack of initiative.

[0037] In view of this, the present disclosure provides a device control method, in which the user expresses his or her own status without waking up the device, and the device can actively control the operations related to and corresponding to the status, making the voice interaction based on device control more intelligent and convenient.

[0038] Figure 2 is a flow chart of a device control method according to an exemplary embodiment. Figure 2 As shown, the device control method includes the following steps.

[0039] In step S11, the first device collects the user's voice signal in real time.

[0040] In the embodiment of the present disclosure, the first device continuously performs a sound receiving operation for the user's speech. The first device may be a smart speaker, a terminal, etc.

[0041] In step S12, in response to the presence of the target speech signal including the target semantic content, second device operation information matching the target semantic content is fed back.

[0042] In the disclosed embodiment, semantic recognition can be performed on all voice signals collected in real time, and when the target semantic content is recognized, the target voice signal can be determined.

[0043] In the embodiment of the present disclosure, the target semantic content can be understood as a semantic attribute used to trigger the second device to perform an operation.

[0044] In an example of the present disclosure, the first device and the second device may be the same device or different devices, and are not limited thereto.

[0045] The device control method provided by the embodiment of the present disclosure collects the user's voice signal in real time through the first device. In response to the existence of a target voice signal including target semantic content, the second device operation information matching the target semantic content is fed back. Among them, by performing semantic recognition on all voice signals collected in real time respectively, and when the target semantic content is recognized, the target voice signal can be determined. The target semantic content is a semantic attribute used to trigger the second device to perform an operation. In this way, without waking up the device, the operation related to and corresponding to the target voice signal can be controlled according to the target voice signal expressed by the user, making the voice interaction based on the device control method more intelligent and convenient, thereby solving the problem of the device always responding to the user in the absence of a wake-up word in human-computer interaction.

[0046] Embodiments of the Disclosure The device control method involved in the embodiments of the disclosure is described below.

[0047] In the embodiment of the present disclosure, the first device collects the user's voice signal in real time, responds to the presence of a target voice signal including target semantic content, and feeds back a second device matching the target semantic content. According to the target semantic content, an operation corresponding to the semantic content is performed. The target semantic content represents a semantic attribute used to trigger the second device to perform the operation.

[0048] In the embodiment of the present disclosure, the semantic attribute used to trigger the second device to perform an operation includes: a control instruction used to trigger the second device to perform an operation.

[0049] In an example of the present disclosure, a specified control instruction is used to trigger the second device to perform an operation. For example, a user issues a specified control instruction such as "please turn on the air conditioner" or "turn on the TV".

[0050] In an embodiment of the present disclosure, the semantic attributes used to trigger the second device to perform an operation also include: one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instructions that trigger the second device to perform the operation.

[0051] For example, in the disclosed embodiment, the sentence expressed by the user includes "Ah, the drink spilled on the floor", and the semantics of the sentence is determined to belong to the pre-set semantic attribute set, i.e., the smart home semantic attribute set, based on the analysis. And it is determined that the sentence is associated with the second device operation information, i.e., the second device operation information is "the floor cleaning robot has been turned on to clean the floor where the drink spilled".

[0052] For example, in the embodiment of the present disclosure, the sentence expressed by the user includes "My father's birthday is the day after tomorrow, don't forget to remind me", and the semantics of the sentence is determined to belong to the preset semantic attribute set, that is, the calendar semantic attribute set. And it is determined that the sentence is associated with the second device operation information, that is, the second device operation information is "A reminder has been added, the day after tomorrow is your father's birthday."

[0053] In the disclosed embodiment, if the sentence expressed by the user includes "you are back", and it is determined based on the collection and analysis that the semantics of the sentence does not belong to the preset semantic attribute set, no response is made.

[0054] In an example of the present disclosure, the semantic attribute set is obtained by pre-training. Based on the segmentation of the semantic attribute set, the first device can know whether the user's speech is spoken to itself.

[0055] In the disclosed embodiment, the semantic attributes in the pre-trained semantic attribute set are pre-trained based on the semantic content, the second device operation information, and the control instruction.

[0056] In an example of the present disclosure, the semantic attributes in the semantic attribute set are used to divide a field into more detailed fields. For example, the smart home semantic attribute set includes air conditioners, curtains, etc., and air conditioners can be divided into on, off, temperature increase, temperature decrease, etc.

[0057] For example, in one example, a semantic attribute set can do something like a smart home semantic attribute set, which can understand the user's intention, whether to turn on or off the air conditioner, and can actively operate the smart home based on the user's expression and the judgment of the semantic attribute set without the need for a wake-up word.

[0058] For example, in one example of the present disclosure, a user is on the phone with the other party. At this time, in the absence of a wake-up word, the device receives the user's words, understands what the user means, and determines whether the user is speaking to himself, and then responds to the words spoken to himself.

[0059] The following is an explanation of the implementation process of training a preset semantic attribute set in the embodiments of the present disclosure.

[0060] In the disclosed embodiment, the semantic attribute set can be trained by the network model. For example, the segmentation of the semantic attribute set is determined based on the prompt of the ChatGenerativePre-trainedTransformer (ChatGPT). Among them, the Prompt model of ChatGPT is trained based on the training samples including the speech content corresponding to the speech signal.

[0061] In the disclosed embodiment, by subdividing the semantic attribute set through network model training, it is possible to achieve that without waking up the device, the user only needs to express his or her own status, and the device can actively control the operations related to and corresponding to the status.

[0062] In the embodiment of the present disclosure, the subdivision of the semantic attribute set can be trained through the network model. For example, the training can control the semantic attribute set of smart home, schedule, etc. by describing its own state, and the training is as follows:

[0063] {

[0064] "model":"gpt-3.5-turbo",

[0065] "messages":[

[0066] {"role":"system",

[0067] "content":"You are a smart furniture control center. You return json for every question I ask. First point: You can control the furniture according to my own state. If my statement is not related to furniture control or its own state, you return nothing. Second point: If my statement is related to the schedule, such as "timing" or "alarm clock", you return the corresponding content, otherwise return nothing"},

[0068] {"role":"user","content":"A bit hot"},

[0069] {"role":"assistant","content":"{res:"Have turned on the fan for you"}"},

[0070] {"role":"user","content":"I'm going out"},

[0071] {"role":"assistant","content":"{res:"Have helped you turn off all electrical appliances"}"},{"role":"user","content":"What time is it"},

[0072] {"role":"assistant","content":"{res:""}"},

[0073] {"role":"user","content":"I'm sleepy"},

[0074] {"role":"assistant","content":"{res:"I have turned off the bedroom lights for you and set the air conditioner to silent mode"}"},

[0075] {"role":"user","content":"Call me in half an hour"},

[0076] {"role":"assistant","content":"{res:"OK, the alarm has been set for you in half an hour"}"},],"temperature":0.6}

[0077] For example, in an example of the present disclosure, for a set of semantic attributes that have been trained, and the set of semantic attributes is home and schedule, only the semantics associated with the home will be responded to after the user expresses his or her own status, and the schedule will also be responded to, but all semantic attributes outside these two sets will be returned empty.

[0078] In the disclosed embodiment, by refining the semantic attribute set, the device can understand the meaning expressed by the user. According to whether the meaning expressed by the user matches the semantic attributes in the semantic attribute set, if it matches, the device controls the local device. If it does not match, it does not respond, which can improve the intelligence of device control.

[0079] In the disclosed embodiment, the first device sends a voice signal to the cloud, so that the cloud sends a control instruction to the second device based on the voice signal.

[0080] In one example of the present disclosure, the control instruction is sent when the cloud performs semantic recognition on the voice signal to obtain the target voice signal.

[0081] In the embodiment of the present disclosure, the first device is further used to receive second device operation information matching the target semantic content.

[0082] In one example of the present disclosure, the second device operation information matching the target semantic content is the operation to be performed corresponding to the target voice signal, or no response is made.

[0083] For example, in the embodiment of the present disclosure, the first device is always receiving sound and uploading the voice signal to the cloud, which performs voice recognition and natural language processing, and determines whether the category to which the semantic content corresponding to the voice signal belongs is within the preset semantic attribute set, and determines whether it matches the semantic attributes in the semantic attribute set. If they match, the operation information corresponding to the voice signal is fed back. If they do not match, the device does not make any response.

[0084] In the disclosed embodiment, the device controls the local device based on the control instruction.

[0085] For example, in the disclosed embodiment, the device receives the user saying "I am so hot", and after cloud analysis and understanding of "I am so hot", it is determined that "I am so hot" belongs to the smart home semantic attribute set, and the device controls the local device, such as turning on the air conditioner.

[0086] The device control method provided by the embodiment of the present disclosure responds to the target voice signal in the semantic attribute set through the voice signal collected in real time, and feeds back the operation information matching the semantic attribute in the semantic attribute set. In this way, the operation corresponding to the target voice signal can be controlled by judging whether the voice content belongs to the preset semantic attribute set without the need for a wake-up word, making the voice interaction based on the device control method more intelligent and convenient.

[0087] Figure 3 FIG. 1 is a flow chart of a device control method provided by an exemplary implementation example of the present disclosure. Figure 3 As shown, the following steps are included.

[0088] In step S21, a voice signal collected and sent in real time by the first device is obtained.

[0089] In the embodiment of the present disclosure, the acquired voice signal is a voice signal obtained by continuously receiving sound through the first device.

[0090] In step S22, semantic recognition is performed on all voice signals collected and sent by the device.

[0091] In the disclosed embodiment, semantic recognition is performed on the acquired speech signal to determine the target semantic content.

[0092] In an example of the present disclosure, semantic recognition is performed on a speech signal. The semantic recognition of the speech signal can be performed through speech recognition and natural language processing.

[0093] Among them, in the embodiments of the present disclosure, the goal of speech recognition (Automatic Speech Recognition, ASR) is to convert the vocabulary content in human speech into computer-readable input, such as keystrokes, binary codes or character sequences.

[0094] In an example of the present disclosure, Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life.

[0095] In step S23, in response to identifying the target semantic content, it is determined that the target voice signal is identified, a control instruction is sent to the second device, and second device operation information matching the target semantic content is sent to the first device.

[0096] In the embodiment of the present disclosure, the target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

[0097] In the embodiment of the present disclosure, the control instruction is used to control the second device to perform an operation corresponding to the target semantic content.

[0098] In an example of the present disclosure, in response to the identified target semantic content, the identified target voice signal is determined, and a control instruction is sent to the second device, and further, second device operation information matching the target semantic content is sent to the first device.

[0099] For example, in an example of the present disclosure, the first device has been collecting the user's voice content and sending it to the cloud. The cloud recognizes the voice content and determines whether the sentence belongs to a pre-stored set of semantic attributes. If it does, it controls the second device to perform the corresponding operation and sends the operation information, such as "to turn on the air conditioner" to the first device, and the first device responds to the user.

[0100] The device control method provided by the embodiment of the present disclosure performs semantic recognition of the target voice signal by acquiring the voice signal collected and sent in real time by the first device. Then, it is determined whether the target voice signal belongs to the semantic attribute set. If the voice content belongs to the semantic attribute set, and the semantic attribute is associated with the second device operation information, a corresponding response is made. Otherwise, no response is made. In this way, without the need for a wake-up word, the cloud can accurately determine whether the voice signal belongs to the semantic attribute set.

[0101] In the embodiment of the present disclosure, the semantic attribute used to trigger the second device to perform an operation includes: a control instruction used to trigger the second device to perform an operation.

[0102] In an embodiment of the present disclosure, the semantic attributes used to trigger the second device to perform an operation also include: one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instructions that trigger the second device to perform the operation.

[0103] In the embodiment of the present disclosure, the semantic attributes in the semantic attribute set are pre-trained based on the semantic content, the second device operation information, and the control instructions.

[0104] The embodiment of the present disclosure is described below before issuing a control instruction to the second device involved in the embodiment of the present disclosure.

[0105] In the disclosed embodiment, before the cloud sends a control instruction to the second device, it determines whether the semantic content belongs to a pre-trained semantic attribute set.

[0106] In an example of the present disclosure, if it belongs to a pre-trained semantic attribute set, the cloud sends a control instruction, the second device receives the control instruction, and executes the control instruction.

[0107] In the embodiment of the present disclosure, when the semantic content does not belong to the pre-trained semantic attribute set, no response is made and no interaction with the second device is performed.

[0108] In an example of the present disclosure, if it is determined that the semantic content does not belong to the pre-trained semantic attribute set, no response is made. It can be understood that the semantic content does not belong to the pre-set semantic attribute set. Furthermore, it can be ensured that in the absence of a wake-up word and the first device is always receiving sound, no response is made to the voice signal that does not belong to the preset semantic attribute set.

[0109] In the disclosed embodiment, the voice signal is recognized to determine the voice content, and whether the voice content belongs to the pre-trained semantic attribute set is judged, and then a corresponding response is made, so that the first device can be more intelligent without the wake-up word.

[0110] In the disclosed embodiment, after natural language processing is performed on the speech content, it is determined whether the speech content belongs to a preset semantic attribute set.

[0111] In an example of the present disclosure, the voice content is processed so that the voice content can be read and understood. Based on the understanding of the voice content, it is determined whether the voice content belongs to a semantic attribute set.

[0112] The disclosed embodiment performs speech recognition on a speech signal to determine a target speech signal corresponding to the speech signal, and performs natural language processing on the speech signal to determine whether the speech content belongs to a preset semantic attribute set based on the understanding of the semantic content, thereby making speech interaction more intelligent.

[0113] The following is an example of an implementation process of a device control method according to the present disclosure.

[0114] Figure 4 is a schematic diagram showing a voice interaction method based on a device control method according to an exemplary implementation example. Figure 4 As shown, step a is that the user speaks and there is ambient sound. Step b is that the sound is continuously collected to the cloud for speech recognition and natural language processing. Step c is that the speech recognition accuracy is improved. Step d is that the real-time speech recognition audio is continuously uploaded. Step e is that speech recognition and natural language processing are performed. In step f, f1 is that if the natural language processing is to control or specify a set of semantic attributes, the control is sent to the local device. In step f, f2 is that if the natural language processing is not to control or specify a set of semantic attributes, speech recognition and natural language processing are continued.

[0115] The disclosed embodiment provides a device method, which obtains a voice signal and determines whether the semantic content corresponding to the voice signal belongs to a preset semantic attribute set. If it does, the operation corresponding to the semantic content is performed. If it does not, no response is given. Through the disclosed embodiment, the user only needs to express his or her own state without waking up the device, and the device can actively control the operation related to and corresponding to the state, making the voice interaction method more intelligent and convenient.

[0116] The following embodiment of the present disclosure describes a scene graph implemented based on a device control method of the present disclosure in combination with the above-mentioned network model training example.

[0117] Figure 5 is a scene diagram based on a device control method according to an exemplary implementation example. When the speech content is analyzed and understood to belong to a preset semantic attribute set, the speaker Y makes a corresponding response. Figure 5As shown, when the user says "I'm up", and the speaker understands and analyzes that the sentence belongs to the home semantic attribute set, the speaker executes and responds "I have opened the curtains for you and turned on the bedroom lights." When the user says "Some music would be nice," and the speaker understands and analyzes that the sentence belongs to the home semantic attribute set, the speaker executes and responds "I have turned on the music player for you and started playing music." When the user says "Ah, the drink spilled on the floor," and the speaker understands and analyzes that the sentence belongs to the home semantic attribute set, the speaker executes and responds "I have turned on the floor cleaning robot for you to clean the floor where the drink spilled." When the user says "The day after tomorrow is my dad's birthday, don't forget to remind me," and the speaker understands and analyzes that the sentence belongs to the schedule semantic attribute set, the speaker executes and responds "A reminder has been added for you, the day after tomorrow is your dad's birthday."

[0118] Figure 6 is a scene diagram based on a device control method according to an exemplary implementation example. When the understood and analyzed voice content does not belong to the preset semantic attribute set, the speaker Y does not respond. Figure 6 As shown, when the user says "this is a difficult thing", and the understanding and analysis shows that the sentence does not belong to the home semantic attribute set or the schedule semantic attribute set, the speaker returns empty, that is, no reply. When the user says "the children are out of school", and the understanding and analysis shows that the sentence does not belong to the home semantic attribute set or the schedule semantic attribute set, the speaker returns empty. When the user says "I'm starving, let's order a takeaway", and the understanding and analysis shows that the sentence does not belong to the home semantic attribute set or the schedule semantic attribute set, the speaker returns empty. When the user says "you're back", and the understanding and analysis shows that the sentence does not belong to the home semantic attribute set or the schedule semantic attribute set, the speaker returns empty.

[0119] It can be understood that the present disclosure Figure 5 and Figure 6 This is just a schematic illustration. Figure 5 or Figure 6 Any one of the examples may also be any other example except the present disclosure, and the present disclosure is not limited to this.

[0120] Based on the same concept, an embodiment of the present disclosure also provides a device for controlling equipment.

[0121] It is understandable that the device control apparatus provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0122] Figure 7 is a block diagram of a device control apparatus according to an exemplary embodiment. Figure 7 The device 100 includes a collection unit 101 and a processing unit 102.

[0123] The collection unit 101 is used for the first device to collect the user's voice signal in real time.

[0124] The processing unit 102 is configured to feed back second device operation information matching the target semantic content in response to the presence of the target speech signal including the target semantic content.

[0125] In the disclosed embodiment, the target speech signal is based on semantic recognition of all speech signals collected in real time, and is determined when the target semantic content is recognized.

[0126] In an example of the present disclosure, the target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

[0127] In an embodiment of the present disclosure, the semantic attribute used to trigger the second device to perform an operation includes at least one of the following: a control instruction used to trigger the second device to perform an operation; one or more semantic attributes in a semantic attribute set, wherein the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

[0128] In the embodiment of the present disclosure, the semantic attributes in the semantic attribute set are pre-trained based on the semantic content, the second device operation information, and the control instructions.

[0129] In the disclosed embodiment, the processing unit 102 is further configured to send the voice signal to the cloud, so that the cloud sends a control instruction to the second device.

[0130] In one example of the present disclosure, the control command is sent when the cloud recognizes the target voice signal.

[0131] Figure 8 is a block diagram of a device control apparatus according to an exemplary embodiment. Figure 8 The device 200 includes an acquisition unit 201, an identification unit 202, and a response unit 203.

[0132] The acquisition unit 201 is used to acquire the voice signal collected and sent by the first device in real time.

[0133] The recognition unit 202 is used to perform semantic recognition on all voice signals collected and sent by the device.

[0134] The responding unit 203 is used to respond to the recognition of the target semantic content, determine that the target voice signal is recognized, send a control instruction to the second device, and send second device operation information matching the target semantic content to the first device.

[0135] In the embodiment of the present disclosure, the semantic attribute used to trigger the second device to perform an operation includes at least one of the following: a control instruction used to trigger the second device to perform an operation. One or more semantic attributes in a semantic attribute set, where the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

[0136] In the embodiment of the present disclosure, the semantic attributes in the semantic attribute set are pre-trained based on the semantic content, the second device operation information, and the control instructions.

[0137] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0138] Fig. 9 1 is a block diagram of an apparatus 300 for device control according to an exemplary embodiment. For example, the apparatus 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0139] Reference Fig. 9 , the device 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .

[0140] The processing component 302 generally controls the overall operation of the device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

[0141] The memory 304 is configured to store various types of data to support operations on the device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phone book data, messages, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0142] The power component 306 provides power to the various components of the device 300. The power component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 300.

[0143] The multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front camera and / or a rear camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0144] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the device 300 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

[0145] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0146] The sensor assembly 314 includes one or more sensors for providing various aspects of the status assessment of the device 300. For example, the sensor assembly 314 can detect the open / closed state of the device 300, the relative positioning of components, such as the display and keypad of the device 300, the sensor assembly 314 can also detect the position change of the device 300 or a component of the device 300, the presence or absence of user contact with the device 300, the orientation or acceleration / deceleration of the device 300, and the temperature change of the device 300. The sensor assembly 314 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0147] The communication component 316 is configured to facilitate wired or wireless communication between the device 300 and other devices. The device 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0148] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0149] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, and the instructions can be executed by the processor 320 of the device 300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0150] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0151] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0152] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0153] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0154] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0155] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A device control method, characterized in that: The method comprises: The first device collects the user's voice signal in real time; In response to the presence of a target speech signal including target semantic content, feeding back second device operation information matching the target semantic content; Wherein, the target speech signal is based on semantic recognition of all speech signals collected in real time, and is determined when the target semantic content is recognized; The target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

2. The method according to claim 1, characterized in that The semantic attribute used to trigger the second device to perform an operation includes at least one of the following: A control instruction for triggering the second device to perform an operation; One or more semantic attributes in the semantic attribute set, where the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

3. The method according to claim 2, characterized in that The semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Sending the voice signal to the cloud, so that the cloud sends a control instruction to the second device; Wherein, the control instruction is sent when the cloud recognizes the target voice signal.

5. A device control method, characterized in that: The method comprises: Acquire a voice signal collected and sent by the first device in real time; Performing semantic recognition on all voice signals collected and sent by the device; In response to identifying the target semantic content, determining that the target voice signal is identified, sending a control instruction to the second device, and sending second device operation information matching the target semantic content to the first device; The target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

6. The method according to claim 5, characterized in that The semantic attribute used to trigger the second device to perform an operation includes at least one of the following: A control instruction for triggering the second device to perform an operation; One or more semantic attributes in the semantic attribute set, where the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

7. The method according to claim 6, characterized in that The semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

8. A device for controlling equipment, characterized in that: The device comprises: A collection unit, used for the first device to collect the user's voice signal in real time; A processing unit, configured to, in response to the presence of a target speech signal including target semantic content, feed back second device operation information matching the target semantic content; Wherein, the target speech signal is based on semantic recognition of all speech signals collected in real time, and is determined when the target semantic content is recognized; The target semantic content represents a semantic attribute used to trigger the second device to perform an operation.

9. The device according to claim 8, characterized in that: The semantic attribute used to trigger the second device to perform an operation includes at least one of the following: A control instruction for triggering the second device to perform an operation; One or more semantic attributes in the semantic attribute set, where the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

10. The device according to claim 9, characterized in that: The semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

11. The device according to any one of claims 8 to 10, characterized in that The processing unit is further configured to send the voice signal to a cloud, so that the cloud sends a control instruction to the second device; Wherein, the control instruction is sent when the cloud recognizes the target voice signal.

12. A device for controlling equipment, characterized in that: The device comprises: An acquisition unit, used to acquire a voice signal collected and sent by the first device in real time; A recognition unit, used for performing semantic recognition on all voice signals collected and sent by the device; The response unit is used to respond to the recognition of the target semantic content, determine that the target voice signal is recognized, send a control instruction to the second device, and send second device operation information matching the target semantic content to the first device.

13. The device according to claim 12, characterized in that: The semantic attribute used to trigger the second device to perform an operation includes at least one of the following: A control instruction for triggering the second device to perform an operation; One or more semantic attributes in the semantic attribute set, where the semantic attributes in the semantic attribute set are associated with the second device operation information and correspond to the control instruction that triggers the second device to perform the operation.

14. The device according to claim 13, characterized in that: The semantic attributes in the semantic attribute set are pre-trained based on semantic content, second device operation information, and control instructions.

15. A device for controlling equipment, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the device control method described in any one of claims 1-4.

16. A device for controlling equipment, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the device control method described in any one of claims 5-7.

17. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by the processor of the device, the device is enabled to execute the device control method described in any one of claims 1 to 4.

18. A storage medium, characterized in that: The storage medium stores instructions. When the instructions in the storage medium are executed by a processor in the cloud, the cloud can execute the device control method described in any one of claims 5 to 7.

Images