Command word response method, control device and apparatus

Abstract

The application is suitable for the field of voice control, and provides a command word response method, a control device and an apparatus. In the method provided by the application, after obtaining a control command word, the control device can determine whether to perform an interactive confirmation operation according to a scene coefficient corresponding to the control command word. If the control device determines to perform the interactive confirmation operation according to the scene coefficient, the control device and / or other devices issue an interactive confirmation prompt information. Then, when the control device receives a confirmation response corresponding to the interactive confirmation prompt information, the control device responds to the control command word. In the command word response method of the application, the control device does not perform interactive confirmation on all control command words, but distinguishes the control command words according to the scene coefficients, performs interactive confirmation on the control command words corresponding to part of the scene coefficients, reduces the complexity of voice control, improves the fluency of voice control, and has strong ease of use and practicality.

Description

Technical Field

[0001] This application relates to the field of voice control, and in particular to a command word response method, control device, and apparatus. Background Technology

[0002] With the development of voice recognition technology, more and more smart home devices can be controlled by users through voice commands or can interact with users via voice. However, the voice signal recognized by voice recognition may be the user's own voice, or it may be the voice signal emitted by other electronic devices equipped with speakers.

[0003] Because the acoustic characteristics of voice signals emitted by users and voice signals emitted by electronic devices are quite similar, the possibility of voice recognition misidentification is higher when the latter scenario occurs, which may lead to operations that do not conform to the user's intention and affect the user experience.

[0004] To address this, some manufacturers have proposed multi-turn dialogue solutions. In these solutions, when voice recognition detects control-related command words, smart home devices can engage in multiple rounds of dialogue with the user to confirm the user's true intentions.

[0005] While the aforementioned multi-turn dialogue scheme can reduce the possibility of misidentification to some extent, the fact that users need to engage in multiple dialogues during voice control undoubtedly increases the complexity of voice control and affects its fluency. Summary of the Invention

[0006] This application provides a command word response method, control device, and apparatus that can solve the problems of high complexity and low fluency in existing multi-turn dialogue schemes.

[0007] In a first aspect, embodiments of this application provide a command word response method applied to a control device, the method comprising:

[0008] Acquire control command words corresponding to human voice signals; wherein, the human voice signals are collected by a sound pickup device;

[0009] Obtain the scene coefficients corresponding to the control command words;

[0010] Based on the scenario coefficients, determine whether to perform an interactive confirmation operation;

[0011] If the interactive confirmation operation is determined to be performed based on the scenario coefficient, then an interactive confirmation prompt message is issued and / or other devices are instructed to issue an interactive confirmation prompt message.

[0012] Upon receiving a confirmation response corresponding to the interactive confirmation prompt, the operation corresponding to the control command word is executed and / or the electronic device corresponding to the control command word is instructed to execute the operation corresponding to the control command word.

[0013] It should be noted that a sound pickup device refers to an electronic device equipped with a microphone. In some scenarios, the control device and the sound pickup device can be two independent electronic devices; in other scenarios, the control device and the sound pickup device can be the same electronic device.

[0014] When the sound pickup device collects a human voice signal, it can process the signal and send a control command word to the control device when it recognizes the command word.

[0015] Alternatively, the sound pickup device can send a human voice signal to the control device. When the control device receives the human voice signal, it can process the signal to obtain control commands.

[0016] After obtaining the control command word, the control device can obtain the scene coefficient corresponding to the control command word.

[0017] The scenario coefficient is used to characterize the scope of influence of control command words. The scope of influence refers to the degree of adverse impact on users when control command words are misidentified.

[0018] Therefore, the control device can determine whether to perform an interactive confirmation operation based on this scenario coefficient.

[0019] When the control device determines to perform an interactive confirmation operation, the control device can issue an interactive confirmation prompt message through itself, and / or, the control device can also instruct other devices to issue interactive confirmation prompt messages.

[0020] For example, assuming the control device is a home hub, when the home hub determines to perform an interactive confirmation operation, the home hub can display a prompt message on its display screen, and the home hub can instruct the smart speaker to play a prompt audio through its speaker.

[0021] When a user encounters the above interactive confirmation prompt, if the control command is the instruction the user wants to issue, the user can respond to the interactive confirmation prompt with a confirmation message.

[0022] When the control device receives the confirmation response corresponding to the above-mentioned interactive confirmation information, the control device may execute the operation corresponding to the control command word, and / or the control device may also instruct the electronic device corresponding to the above-mentioned control command word to execute the operation corresponding to the control command word.

[0023] For example, suppose the control device is a smart speaker, and the control command is "play TV series". When the smart speaker receives a confirmation response corresponding to the interaction confirmation information, the smart speaker can play the audio of the TV series through its own speaker and instruct the smart TV to play the TV series on screen.

[0024] In the command word response method provided in this application embodiment, the control device can determine the user's true intent through interactive confirmation, reducing the possibility of misidentification.

[0025] Furthermore, the control device does not perform interactive confirmation for all control command words. Instead, it distinguishes control command words based on scenario coefficients and performs interactive confirmation for control command words corresponding to certain scenario coefficients. This reduces the complexity of voice control, improves the fluency of voice control, and has strong usability and practicality.

[0026] In one possible implementation of the first aspect, if it is determined that the interactive confirmation operation will be performed, the method further includes:

[0027] If a negative response is received corresponding to the interactive confirmation prompt, or if no response is received corresponding to the interactive confirmation prompt, the operation corresponding to the control command word will not be executed, and the electronic device corresponding to the control command word will not be instructed to execute the operation corresponding to the control command word.

[0028] It should be noted that when the above control command words are not the instructions that the user wants to give, the user may respond negatively to the above interactive confirmation prompts, or the user may not respond to the above interactive confirmation prompts at all.

[0029] At this time, if the control device receives a negative response corresponding to the above-mentioned interactive confirmation prompt, or if the control device does not receive any response corresponding to the above-mentioned interactive confirmation prompt, the control device may not respond to the above-mentioned control command words.

[0030] The control device does not respond to the control command word, which means that the control device does not execute the operation corresponding to the control command word, and the control device does not instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word.

[0031] Using the above methods, the control device can determine the user's denial intent and will not respond to misidentified control command words, thus avoiding adverse effects on the user's personal and property safety.

[0032] In one possible implementation of the first aspect, the method further includes:

[0033] If, based on the scenario coefficient, it is determined that the interactive confirmation operation will not be performed, then the operation corresponding to the control command word will be performed and / or the electronic device corresponding to the control command word will be instructed to perform the operation corresponding to the control command word.

[0034] It should be noted that the control device can perform interactive confirmation operations for control command words corresponding to certain scenario coefficients. For other control command words, the control device can directly respond to the control command word without performing interactive confirmation operations, thereby reducing the complexity of voice control and improving the fluency of voice control.

[0035] In one possible implementation of the first aspect, determining whether to perform an interactive confirmation operation based on the scenario coefficient specifically includes:

[0036] If the scene coefficient meets the preset scene coefficient threshold condition, the interactive confirmation operation is executed; otherwise, the interactive confirmation operation is not executed.

[0037] It should be noted that the scenario coefficient is used to represent the scope of influence of a control command word. When the scenario coefficient meets a pre-set threshold condition, it indicates that the scope of influence of the control command word is relatively large. If the control command word is misidentified, it could easily have an adverse impact on the user's personal safety and property. In this case, for these control command words, the control device needs to perform an interactive confirmation operation to identify the user's true intention and reduce the occurrence of misidentification.

[0038] When the scene coefficient does not meet the preset scene coefficient threshold, it indicates that the influence range of the control command word is relatively small. Even if the control command word is misrecognized, it will not cause serious adverse effects on the user's personal safety and property. In this case, for these control command words, the control device can directly respond to the control command word without performing an interactive confirmation operation, reducing the complexity of voice control and improving the fluency of voice control.

[0039] The scene coefficient threshold condition can be set according to actual needs. For example, when the scene coefficient and the influence range of the control command word are positively correlated, the scene coefficient threshold condition can be set to the scene coefficient being greater than the scene coefficient threshold; when the scene coefficient and the influence range of the control command word are negatively correlated, the scene coefficient threshold condition can be set to the scene coefficient being less than or equal to the scene coefficient threshold; or, in other scenarios, the scene coefficient threshold condition can be set to other values. This application does not limit the specific content of the scene coefficient threshold condition in its embodiments.

[0040] In one possible implementation of the first aspect, the scene coefficient is a pre-set initial scene coefficient, or the scene coefficient is a target scene coefficient obtained by adjusting the initial scene coefficient according to scene adjustment parameters, which are determined based on the historical usage data of the control command words and / or other historical usage data of control command words.

[0041] It should be noted that the control device can use the initial scene coefficients preset by the manufacturer as the aforementioned scene coefficients. Alternatively, the control device can obtain scene adjustment parameters, adjust the initial scene coefficients according to the scene adjustment parameters to obtain the target scene coefficients, and use the target scene coefficients as the aforementioned scene coefficients.

[0042] The above-mentioned scenario adjustment parameters are determined based on the historical usage data of control command words and / or other historical data of control command words.

[0043] The aforementioned historical usage data may include one or more of the following: historical usage time period, historical usage count within a specified time period, and historical usage frequency within a specified time period.

[0044] For example, suppose the scene adjustment parameters include usage frequency, time period coefficient, and usage coefficient. In this case, the control device can use the historical usage frequency within the specified time period as the usage frequency, calculate the time period coefficient of the control command word based on the historical usage time period, and calculate the usage coefficient of the control command word based on the historical usage count of the control command word within the specified time period and the historical usage count of other control command words within the specified time period.

[0045] Since the aforementioned scenario adjustment parameters represent the user's usage of the control command word, the control device can dynamically adjust the initial scenario coefficients based on the scenario adjustment parameters to obtain more accurate initial scenario coefficients. This allows the control device to more reasonably classify the application scenarios corresponding to each control command word.

[0046] In one possible implementation of the first aspect, the issuance of the interactive confirmation prompt message includes one or more of the following: playing a prompt audio through a speaker, displaying the prompt message on a screen, flashing an indicator light, or emitting vibration through a vibration motor.

[0047] It should be noted that the way electronic devices send interactive confirmation prompts can be set according to the actual scenario.

[0048] Specifically, electronic devices can issue interactive confirmation prompts in one or more ways, such as playing a prompt audio through a speaker, displaying a prompt message on a screen, flashing an indicator light, or emitting vibrations through a vibration motor.

[0049] For example, assuming the aforementioned electronic device is a smart TV, when the smart TV issues an interactive confirmation prompt, it can display the prompt message "Please confirm whether to turn on the air conditioner" on the screen and broadcast it through the speaker.

[0050] This application does not limit the method by which electronic devices issue interactive confirmation prompts.

[0051] In one possible implementation of the first aspect, the determining response includes one or more of the following: a human voice signal containing a confirmation command word with a confirmation meaning, a touch operation on a confirmation control displayed on a screen, or a pressing operation on a confirmation button.

[0052] It should be noted that the specific form of the above confirmation response can be determined according to the actual scenario.

[0053] For example, in some scenarios, the aforementioned confirmation response can be a human voice signal containing a confirmation command word, such as a user saying, "I confirm turning on the air conditioner."

[0054] In other scenarios, the aforementioned confirmation response can be an electronic device detecting a touch operation on a confirmation control displayed on the screen. For example, assuming a home control system has a touch-sensitive display screen, the system can display the text "Please confirm whether to turn on the air conditioner," and provide "Yes" and "No" controls. When the user touches the "Yes" control, the home control system detects the confirmation response corresponding to the interactive confirmation prompt.

[0055] In other scenarios, the aforementioned confirmation response can also be a confirmation response detected by the electronic device upon detecting a press of the confirmation button. For example, assuming a smart speaker has a physical button, when the user presses the physical button, the smart speaker detects a confirmation response corresponding to the interactive confirmation prompt.

[0056] In other scenarios, the aforementioned confirmation response may take other forms. This application does not limit the specific form of the confirmation response.

[0057] In one possible implementation of the first aspect, before obtaining the scene coefficients corresponding to the control proposition, the method further includes:

[0058] Obtain the lexical confidence score and confidence threshold corresponding to the control command word;

[0059] If the confidence level of the word is determined to be greater than the confidence threshold, proceed with the next steps.

[0060] It should be noted that if the control device / pickup device detects that the word confidence of a certain control command word is greater than the confidence threshold, the control device considers that the control command word has been detected.

[0061] In one possible implementation of the first aspect, before performing subsequent steps if the confidence level of the word is determined to be greater than the confidence threshold, the method further includes:

[0062] Adjust the confidence level of the vocabulary and / or the confidence threshold according to the confidence level adjustment parameter corresponding to the control command word.

[0063] It should be noted that the control device can adjust the confidence level of the word and / or the confidence threshold according to the confidence level adjustment parameter, and then compare the word confidence level with the confidence threshold to verify the control command word.

[0064] When the confidence level of a word is greater than the confidence threshold, it means that the control command word has passed the verification.

[0065] When the confidence level of a word is less than or equal to the confidence threshold, it indicates that the validation of the control command word has failed.

[0066] Since the aforementioned confidence threshold represents the degree to which a user has previously used the control command word, or the scope of influence of the control command word, the control device can improve the recognition accuracy of the control command word and reduce the possibility of false recognition by adjusting the lexical confidence or confidence threshold of the control command word using confidence adjustment parameters.

[0067] In one possible implementation of the first aspect, the confidence adjustment parameter includes one or more of the following: initial scenario coefficient, historical usage frequency within a specified time period, time period coefficient, and usage coefficient.

[0068] It should be noted that the above usage frequency refers to the number of times a user uses control command words within a certain period of time, which can be understood as the historical usage frequency within a specified time period.

[0069] The above time period coefficient refers to the coefficient corresponding to the time period in which the above control command word is detected.

[0070] The usage coefficient mentioned above refers to the proportion of the number of times the above control command words are used to the total number of times all control command words are used.

[0071] The initial scenario coefficients mentioned above are used to represent the scope of influence of control command words. The scope of influence refers to the degree of adverse impact on the user when a control command word is misidentified.

[0072] Since the aforementioned confidence adjustment parameters represent the degree to which users have previously used control command words, or the scope of influence of control command words, using these parameters to adjust the lexical confidence or confidence threshold of control command words can improve the recognition accuracy of control command words and reduce the possibility of misidentification.

[0073] Secondly, embodiments of this application provide a command word response device applied to a control device, the device comprising:

[0074] The command word acquisition module is used to acquire control command words corresponding to human voice signals; wherein, the human voice signals are collected by a sound pickup device;

[0075] The scene coefficient module is used to obtain the scene coefficients corresponding to the control command words;

[0076] An interaction recognition module is used to determine whether to perform an interaction confirmation operation based on the scene coefficients.

[0077] An interaction execution module is used to issue an interaction confirmation prompt message and / or instruct other devices to issue an interaction confirmation prompt message if it is determined to perform the interaction confirmation operation based on the scenario coefficient.

[0078] The confirmation execution module is used to execute the operation corresponding to the control command word and / or instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word when receiving a confirmation response corresponding to the interactive confirmation prompt information.

[0079] In one possible implementation of the second aspect, the apparatus further includes:

[0080] The denial stop module is used to not execute the operation corresponding to the control command word and not instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word when a negative response corresponding to the interactive confirmation prompt is received, or when no response corresponding to the interactive confirmation prompt is received.

[0081] In one possible implementation of the second aspect, the apparatus further includes:

[0082] The direct response module is configured to, if determined based on the scenario coefficient that the interactive confirmation operation should not be performed, execute the operation corresponding to the control command word and / or instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word.

[0083] In one possible implementation of the second aspect, the interaction recognition module is specifically used to determine whether to perform the interaction confirmation operation if the scene coefficient meets the preset scene coefficient threshold condition; otherwise, it determines not to perform the interaction confirmation operation.

[0084] In one possible implementation of the second aspect, the scene coefficient is a pre-set initial scene coefficient, or the scene coefficient is a target scene coefficient obtained by adjusting the initial scene coefficient according to scene adjustment parameters, which are determined based on the historical usage data of the control command words and / or other historical usage data of control command words.

[0085] In one possible implementation of the second aspect, the historical usage data includes one or more of the following: historical usage time period, historical usage count within a specified time period, and historical usage frequency within a specified time period.

[0086] In one possible implementation of the second aspect, the issuance of interactive confirmation prompt information includes one or more of the following: playing prompt audio through a speaker, displaying prompt information on a screen, flashing an indicator light, or emitting vibration through a vibration motor.

[0087] In one possible implementation of the second aspect, the determining response includes one or more of the following: a voice signal containing a confirmation command word with a confirmation meaning, a touch operation on a confirmation control displayed on a screen, or a pressing operation on a confirmation button.

[0088] In one possible implementation of the second aspect, the apparatus further includes:

[0089] The confidence level acquisition module is used to acquire the lexical confidence level and confidence threshold corresponding to the control command word;

[0090] The confidence verification module is used to perform subsequent steps if it is determined that the confidence of the word is greater than the confidence threshold.

[0091] In one possible implementation of the second aspect, the apparatus further includes:

[0092] The confidence adjustment module is used to adjust the confidence of the vocabulary and / or the confidence threshold according to the confidence adjustment parameters corresponding to the control command words.

[0093] In one possible implementation of the second aspect, the confidence adjustment parameter includes one or more of the following: initial scenario coefficient, historical usage frequency within a specified time period, time period coefficient, and usage coefficient.

[0094] Thirdly, embodiments of this application provide a control device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the computer program to implement the method as described in the first aspect and any of the possible implementations of the first aspect.

[0095] Fourthly, embodiments of this application provide a computer-readable storage medium configured to store a computer program, characterized in that, when executed by a processor, the computer program implements the method as described in the first aspect and any of the possible implementations of the first aspect.

[0096] Fifthly, embodiments of this application provide a computer program product, characterized in that the computer program product is configured to run on a control device, causing the control device to perform the method as described in the first aspect and any of the possible implementations of the first aspect.

[0097] In a sixth aspect, embodiments of this application provide a chip system, characterized in that the chip system includes a memory and a processor, the processor being configured to execute a computer program stored in the memory to implement the method as described in the first aspect and any of the possible implementations of the first aspect.

[0098] The beneficial effects of the embodiments in this application compared with the prior art are:

[0099] In the command word response method of this application, after obtaining a control command word, the control device can determine whether to perform an interactive confirmation operation based on the scenario coefficient corresponding to the control command word. If the control device determines to perform interactive confirmation based on the scenario coefficient, it sends an interactive confirmation prompt message through itself and / or other devices. Subsequently, when the control device receives the confirmation response corresponding to the aforementioned interactive confirmation prompt message, the control device responds to the control command word.

[0100] In the command word response method of this application, the control device can determine the user's true intent through interactive confirmation, reducing the possibility of misidentification. Furthermore, the control device does not perform interactive confirmation for all control command words, but distinguishes control command words based on scenario coefficients, performing interactive confirmation only for control command words corresponding to certain scenario coefficients. This reduces the complexity of voice control, improves the fluency of voice control, and has strong usability and practicality. Attached Figure Description

[0101] Figure 1 A system architecture diagram of a smart home system provided in this application embodiment;

[0102] Figure 2This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0103] Figure 3 A scenario diagram provided for an embodiment of this application;

[0104] Figure 4 This is another scenario illustration provided for an embodiment of this application;

[0105] Figure 5 This is another scenario illustration provided for an embodiment of this application;

[0106] Figure 6 This is another scenario illustration provided for an embodiment of this application;

[0107] Figure 7 This is another scenario illustration provided for an embodiment of this application;

[0108] Figure 8 This is another scenario illustration provided for an embodiment of this application;

[0109] Figure 9 This is another scenario illustration provided for an embodiment of this application;

[0110] Figure 10 This is another scenario illustration provided for an embodiment of this application;

[0111] Figure 11 This is another scenario illustration provided for an embodiment of this application;

[0112] Figure 12 This is another scenario illustration provided for an embodiment of this application;

[0113] Figure 13 A flowchart illustrating a command word response method provided in an embodiment of this application;

[0114] Figure 14 A flowchart illustrating another command word response method provided in an embodiment of this application;

[0115] Figure 15 This is a schematic diagram of the structure of a command word response device provided in an embodiment of this application. Detailed Implementation

[0116] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0117] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0118] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0119] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0120] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0121] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0122] The term "multiple" as used in this application specification can be understood as "two or more", or as "at least two".

[0123] With the development of voice recognition technology, more and more smart home devices can be controlled by users through voice commands, or can interact with users via voice. For example, users can wake up smart home devices with wake-up commands such as "Hey Celia," and control smart home devices to perform corresponding operations with control commands such as "play music" or "stop playing."

[0124] However, when performing speech recognition, the voice signal being recognized may be the voice signal emitted by the user, or it may be the voice signal emitted by other electronic devices equipped with speakers.

[0125] For example, when a smart speaker detects a human voice signal "turn on music," the human voice signal could be a command given by the user to the smart speaker, or it could be a human voice signal emitted by the TV's speakers while a program is playing.

[0126] Because the acoustic characteristics of voice signals emitted by users and voice signals emitted by electronic devices are quite similar, the possibility of voice recognition misidentification is higher when the latter scenario occurs, which may lead to operations that do not conform to the user's intention and affect the user experience.

[0127] Furthermore, even if the aforementioned voice signal is a voice signal emitted by the user, it may be emitted unintentionally by the user and is not a command that the user intended to issue.

[0128] For example, when a smart speaker detects a voice signal saying "listen to rock music," the voice signal could be a command given to the smart speaker by the user; or, the voice signal could be something the user unintentionally made while chatting with someone else, but the user doesn't actually want the smart speaker to play rock music.

[0129] To address this, some manufacturers have proposed multi-turn dialogue solutions. In these solutions, when voice recognition detects control-related command words, smart home devices can engage in multiple rounds of dialogue with the user to confirm the user's true intentions.

[0130] For example, when a smart speaker detects a voice signal saying "Turn on music," it can announce, "Please confirm whether to turn on music." When the user answers "Confirm to turn on music," the smart speaker performs the "Turn on music" operation; when the user answers "Do not turn on music," the smart speaker does not respond to the "Turn on music" command.

[0131] While the aforementioned multi-turn dialogue scheme can reduce the possibility of misidentification to some extent, the fact that users need to engage in multiple dialogues during voice control undoubtedly increases the complexity of voice control and affects its fluency.

[0132] In view of this, the embodiments of this application provide a command word response method, which can perform interactive confirmation for some control scenarios with a large impact range through scenario classification, and not perform interactive confirmation for some scenarios with a small impact range, thereby reducing the complexity of voice control to a certain extent, enhancing the fluency of voice control, and having strong ease of use and practicality.

[0133] First, please check Figure 1 . Figure 1 An exemplary smart home system to which embodiments of this application are applicable is shown.

[0134] like Figure 1 As shown, the smart home system may include: one or more smart home devices 101 ( Figure 1 (Three are shown in the image), and one or more control devices 102 ( Figure 1 One is shown in the image.

[0135] The aforementioned smart home device 101 may include one or more of the following types of electronic devices: computer, mobile phone, tablet computer, smart TV, smart large screen, smart speaker, smart air conditioner, robot vacuum cleaner, dishwasher, smart lamp, smart door lock, smart curtain, router, home hub, etc.

[0136] In this context, a home hub can refer to a dedicated home control electronic device for controlling all smart home devices in the house. Alternatively, a home hub can refer to software that can be installed on existing smart home devices (such as routers, smart screens, etc.) in a user's home, enabling these existing smart home devices to control and make intelligent decisions for all smart home devices in the house, based on their existing functions.

[0137] Furthermore, the aforementioned smart home devices 101 can be located on the same local area network (LAN), and each smart home device 101 can interact with data via the LAN. And / or, each smart home device 101 can also interact with data via one or more communication methods such as Bluetooth, Wi-Fi, or Universal Serial Bus (USB).

[0138] Among the aforementioned smart home devices 101, some smart home devices 101 (such as smart TVs, smart speakers, etc.) are equipped with speakers and can play audio signals to the outside world. In the following description, these electronic devices equipped with speakers are defined as speaker devices.

[0139] Among the aforementioned smart home devices 101, some smart home devices 101 (such as smart speakers, tablets, etc.) are equipped with microphones, which can collect external audio signals. In the following description, these electronic devices equipped with microphones are defined as audio pickup devices.

[0140] Control device 102 refers to an electronic device that can control the aforementioned smart home device 101. Control device 102 can be the aforementioned smart home device 101, and / or, control device 102 can also be a remote server.

[0141] For example, in some scenarios, a user can set a smart TV as a control device 102 to control other smart home devices 101; in other scenarios, a user can set a smart speaker as a control device 102 to control other smart home devices 101; in still other scenarios, a user can set a home hub as a control device 102 to control other smart home devices 101; and in yet another scenario, a user can set a remote server as a control device 102 to control smart home devices 101 within a local area network via the remote server and the wide area network.

[0142] refer to Figure 2 , Figure 2 An exemplary schematic diagram of the structure of the electronic device 200 provided in the embodiments of this application is shown. The electronic device 200 may be the smart home device 101 described above, and / or the electronic device 200 may also be the control device 102 described above.

[0143] Electronic device 200 may include processor 210, external memory interface 220, internal memory 221, universal serial bus (USB) interface 230, charging management module 240, power management module 241, battery 242, antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, sensor module 280, button 290, motor 291, indicator 292, camera 293, display screen 294, and subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer sensor 280E, a distance sensor 280F, a proximity sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0144] Processor 210 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0145] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0146] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0147] In some embodiments, the processor 210 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0148] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 210 may include multiple I2C buses. The processor 210 can couple to the touch sensor 280K, charger, flash, camera 293, etc., through different I2C bus interfaces. For example, the processor 210 can couple to the touch sensor 280K through the I2C interface, enabling the processor 210 and the touch sensor 280K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 200.

[0149] The I2S interface can be used for audio communication. In some embodiments, the processor 210 may include multiple I2S buses. The processor 210 can be coupled to the audio module 270 via the I2S bus to enable communication between the processor 210 and the audio module 270. In some embodiments, the audio module 270 can transmit audio signals to the wireless communication module 260 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0150] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 270 and the wireless communication module 260 can be coupled via the PCM bus interface. In some embodiments, the audio module 270 can also transmit audio signals to the wireless communication module 260 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0151] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 210 and the wireless communication module 260. For example, the processor 210 communicates with the Bluetooth module in the wireless communication module 260 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 270 can transmit audio signals to the wireless communication module 260 via the UART interface to enable music playback through Bluetooth headphones.

[0152] The MIPI interface can be used to connect the processor 210 to peripheral devices such as the display screen 294 and the camera 293. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 210 and the camera 293 communicate via the CSI interface to enable the electronic device 200 to capture images. The processor 210 and the display screen 294 communicate via the DSI interface to enable the electronic device 200 to display images.

[0153] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 210 to a camera 293, a display screen 294, a wireless communication module 260, an audio module 270, a sensor module 280, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0154] USB port 230 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, or USB Type-C port. USB port 230 can be used to connect a charger to charge electronic device 200, and can also be used for data transfer between electronic device 200 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0155] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0156] The charging management module 240 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 receives charging input from the wired charger via a USB interface 230. In some wireless charging embodiments, the charging management module 240 receives wireless charging input via the wireless charging coil of the electronic device 200. While charging the battery 242, the charging management module 240 can also supply power to the electronic device via the power management module 241.

[0157] The power management module 241 connects the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, providing power to the processor 210, internal memory 221, display screen 294, camera 293, and wireless communication module 260, etc. The power management module 241 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 241 may also be located within the processor 210. In other embodiments, the power management module 241 and the charging management module 240 may be located in the same device.

[0158] The wireless communication function of electronic device 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor.

[0159] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0160] The mobile communication module 250 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 200. The mobile communication module 250 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 250 may be housed in the processor 210. In some embodiments, at least some functional modules of the mobile communication module 250 and at least some modules of the processor 210 may be housed in the same device.

[0161] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 270A, receiver 270B, etc.) or displays images or videos through the display screen 294. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 210 and may be housed in the same device as the mobile communication module 250 or other functional modules.

[0162] The wireless communication module 260 can provide solutions for wireless communication applications on the electronic device 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 210. The wireless communication module 260 can also receive signals to be transmitted from processor 210, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0163] In some embodiments, antenna 1 of electronic device 200 is coupled to mobile communication module 250, and antenna 2 is coupled to wireless communication module 260, enabling electronic device 200 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0164] Electronic device 200 implements display functions through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0165] Display screen 294 is used to display images, videos, etc. Display screen 294 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 200 may include one or N displays 294, where N is a positive integer greater than 1.

[0166] Electronic device 200 can perform shooting functions through ISP, camera 293, video codec, GPU, display screen 294 and application processor.

[0167] The ISP (Image Signal Processor) is used to process data fed back from the camera 293. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 293.

[0168] Camera 293 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 200 may include one or N cameras 293, where N is a positive integer greater than 1.

[0169] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 200 selects a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0170] Video codecs are used to compress or decompress digital video. Electronic device 200 may support one or more video codecs. Thus, electronic device 200 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0171] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0172] The external storage interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0173] Internal memory 221 can be used to store computer executable program code, which includes instructions. Internal memory 221 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 200 (such as audio data, phonebook, etc.). Furthermore, internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 210 executes various functional applications and data processing of electronic device 200 by running instructions stored in internal memory 221 and / or instructions stored in memory disposed in the processor.

[0174] Electronic device 200 can implement audio functions such as music playback and recording through audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, and application processor.

[0175] The audio module 270 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 270 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 270 may be located in the processor 210, or some functional modules of the audio module 270 may be located in the processor 210.

[0176] The speaker 270A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 200 can listen to music or make hands-free calls through the speaker 270A.

[0177] The receiver 270B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 200 answers a telephone call or voice message, the receiver 270B can be brought close to the ear to listen to the voice.

[0178] Microphone 270C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 270C, inputting the sound signal into microphone 270C. Electronic device 200 may have at least one microphone 270C. In some embodiments, electronic device 200 may have two microphones 270C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 200 may also have three, four, or more microphones 270C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0179] The headphone jack 270D is used to connect wired headphones. The headphone jack 270D can be a USB 230 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0180] Pressure sensor 280A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 280A can be disposed on display screen 294. There are many types of pressure sensors 280A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 280A, the capacitance between the electrodes changes. Electronic device 200 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 294, electronic device 200 detects the intensity of the touch operation based on pressure sensor 280A. Electronic device 200 can also calculate the touch position based on the detection signal from pressure sensor 280A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0181] The gyroscope sensor 280B can be used to determine the motion attitude of the electronic device 200. In some embodiments, the gyroscope sensor 280B can determine the angular velocity of the electronic device 200 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 280B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 280B detects the angle of the electronic device 200's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 200 through reverse movement, thus achieving image stabilization. The gyroscope sensor 280B can also be used in navigation and motion-sensing game scenarios.

[0182] The barometric pressure sensor 280C is used to measure air pressure. In some embodiments, the electronic device 200 calculates altitude using the air pressure value measured by the barometric pressure sensor 280C to assist in positioning and navigation.

[0183] The magnetic sensor 280D includes a Hall sensor. The electronic device 200 can use the magnetic sensor 280D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 200 is a flip phone, the electronic device 200 can detect the opening and closing of the flip cover using the magnetic sensor 280D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0184] The accelerometer 280E can detect the magnitude of acceleration of electronic device 200 in various directions (typically three axes). When electronic device 200 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic device, and can be applied to applications such as screen orientation switching and pedometers.

[0185] A distance sensor 280F is used to measure distance. Electronic device 200 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 200 can utilize the distance sensor 280F to measure distance for rapid focusing.

[0186] The proximity sensor 280G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 200 emits infrared light outward through the LED. The electronic device 200 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that an object is near the electronic device 200. When insufficient reflected light is detected, the electronic device 200 can determine that no object is near the electronic device 200. The electronic device 200 may use the proximity sensor 280G to detect when a user holds the electronic device 200 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 280G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0187] The ambient light sensor 280L is used to sense the brightness of ambient light. The electronic device 200 can adaptively adjust the brightness of its display screen 294 based on the sensed ambient light level. The ambient light sensor 280L can also be used to automatically adjust the white balance when taking photos. The ambient light sensor 280L can also work in conjunction with the proximity sensor 280G to detect whether the electronic device 200 is in a pocket, preventing accidental touches.

[0188] The fingerprint sensor 280H is used to collect fingerprints. The electronic device 200 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0189] Temperature sensor 280J is used to detect temperature. In some embodiments, electronic device 200 uses the temperature detected by temperature sensor 280J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 280J exceeds a threshold, electronic device 200 reduces the performance of a processor located near temperature sensor 280J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, electronic device 200 heats battery 242 to prevent abnormal shutdown of electronic device 200 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 200 boosts the output voltage of battery 242 to prevent abnormal shutdown due to low temperature.

[0190] Touch sensor 280K, also known as a "touch device," can be located on display screen 294. The touch sensor 280K and display screen 294 together form a touchscreen, also known as a "touchscreen." Touch sensor 280K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 294. In other embodiments, touch sensor 280K may also be located on the surface of electronic device 200, in a different position than display screen 294.

[0191] The bone conduction sensor 280M can acquire vibration signals. In some embodiments, the bone conduction sensor 280M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 280M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 280M can also be incorporated into headphones to form bone conduction headphones. The audio module 270 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 280M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 280M to realize heart rate detection functionality.

[0192] Buttons 290 include a power button, volume buttons, etc. Buttons 290 can be mechanical buttons or touch-sensitive buttons. Electronic device 200 can receive button input and generate key signal inputs related to user settings and function control of electronic device 200.

[0193] Motor 291 can generate vibration alerts. Motor 291 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations applied to different applications (such as taking photos, playing audio, etc.). Motor 291 can also correspond to different vibration feedback effects for touch operations applied to different areas of the display screen 294. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0194] Indicator 292 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0195] The SIM card interface 295 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 295 to make contact with and separate from the electronic device 200. The electronic device 200 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 295 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 295 is also compatible with different types of SIM cards. The SIM card interface 295 is also compatible with external memory cards. The electronic device 200 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 200 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 200 and cannot be separated from the electronic device 200.

[0196] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0197] The following will be based on Figure 1 The smart home system shown, and Figure 2 The structure of electronic devices in the smart home system shown is illustrated, and the command word response method provided in this application embodiment is described in detail in conjunction with specific application scenarios.

[0198] 1. Train the command word model.

[0199] During the initialization phase, the control device / pickup device can train the command word model locally, or it can receive a trained command word model (hereinafter referred to as the target command word model) sent by other electronic devices.

[0200] When the control device / pickup device trains the command word model locally, it can first acquire a training sample set. This training sample set includes at least one training sample, which contains sample audio and sample labels.

[0201] The above sample labels are used to indicate the command word type corresponding to the above sample audio. The command word type may include control command words and confirmation command words.

[0202] Control command words are used to indicate the operations that smart home devices need to perform. For example, control command words can include words such as "turn on music," "turn on air conditioner," "turn on TV," and "turn off music," which indicate the operations that smart home devices need to perform.

[0203] Confirmation commands are used to express a user's intent. For example, confirmation commands can include words that express user intent, such as "yes," "I confirm," "yes," "no," and "no." Alternatively, confirmation commands can be a combination of the above-mentioned words expressing intent plus a specific action, such as "I confirm turning on the air conditioner" or "I don't want to turn on the music."

[0204] The format of the sample labels can be set according to actual needs. For example, in some embodiments, the sample labels can be represented by numbers, such as "01" for "turn on the air conditioner", "02" for "turn on the music", "11" for "confirm", "12" for "do not", etc.; in other embodiments, the sample labels can be represented by letters, such as "a" for "turn on the air conditioner", "b" for "turn on the music", "c" for "confirm", "d" for "do not", etc.; in other embodiments, the sample labels can be represented by both letters and numbers, such as "a1" for "turn on the air conditioner", "a2" for "turn on the music", "b1" for "confirm", "b2" for "do not", etc.; in other embodiments, the sample labels can also be represented in other forms. This application does not limit the specific form of the sample labels.

[0205] After acquiring the training sample set, the control / pickup device can use the training sample set to train the initial command word model and obtain the target command word model.

[0206] The type of command word model can be set according to actual needs. For example, the command word model can be any one of the following: template matching-based command word model, Gaussian Mixture Model-Hidden Markov Model (GMM-HMM)-based command word model, neural network-based command word model, etc.

[0207] After acquiring the target command word model, the control / pickup device completes the initialization phase.

[0208] 2. Identify control command words.

[0209] When the first human voice signal is detected in the environment, the sound pickup device can capture it. The first human voice signal may be a voice signal emitted by a user, or it may be a voice signal emitted by a speaker device.

[0210] When the sound pickup device acquires the first human voice signal, it can use the target command word model locally to perform command word detection on the first human voice signal.

[0211] When the microphone detects a control command word, it can send the control command word and its corresponding lexical confidence level to the control device. The lexical confidence level indicates the degree of confidence in recognizing the control command word.

[0212] Alternatively, when the pickup device detects the first human voice signal, it can also directly send the first human voice signal to the control device.

[0213] When the control device receives the first human voice signal, it can use the target command word model to detect command words in the first human voice signal.

[0214] When the control device detects a control command word, it can obtain the lexical confidence of that control command word.

[0215] After obtaining the control command word, the control device can directly execute subsequent operations based on the control command word, or the control device can verify the control command word.

[0216] During the verification of control command words, the control device can obtain the vocabulary confidence level and the confidence level adjustment parameters corresponding to the control command words.

[0217] Confidence adjustment parameters may include one or more of the following: frequency of use, time period coefficient, usage coefficient, and initial scenario coefficient.

[0218] After obtaining the confidence adjustment parameters, the control device can adjust the vocabulary confidence and / or confidence threshold of the control command words according to the confidence adjustment parameters.

[0219] After adjusting the lexical confidence and / or confidence threshold of the control command word, the control device can compare the lexical confidence of the control command word with the confidence threshold.

[0220] When the confidence level of a word is less than or equal to the confidence threshold, it indicates that the control command word verification has failed, and the control device may not respond to the above control command word.

[0221] When the confidence level of a word is greater than the confidence threshold, it indicates that the control command word has been successfully verified, and the control device can perform subsequent operations on the aforementioned control command word.

[0222] Since the confidence adjustment parameter represents the degree to which a user has previously used the control command word, or the scope of influence of the control command word, the control device can improve the recognition accuracy of the control command word and reduce the possibility of misrecognition by adjusting the lexical confidence and / or confidence threshold of the control command word using the confidence adjustment parameter.

[0223] The following will provide a detailed explanation of each confidence level adjustment parameter mentioned above.

[0224] (1) Frequency of use.

[0225] The aforementioned usage frequency refers to the number of times a user uses control command words within a certain period of time, which can be understood as the historical usage frequency within a specified time period. For example, usage frequency can be expressed as 20 times / week, 5 times / day, 1 time / hour, etc. This application does not limit the specific form of usage frequency in its embodiments.

[0226] The frequency of use can be calculated using formula (1):

[0227] F=d1 / t1 (1)

[0228] Where F represents the frequency of use of the aforementioned control command word, d1 represents the number of times the user uses the control command word within the first specified time period, and t1 represents the length of the first specified time period.

[0229] For example, assuming the control device performs statistics based on data from the last 30 days, the first specified time period can be the last 30 days; d1 represents the number of times the user uses the control command word in the last 30 days; t1 is 30 days, that is, the length of the last 30 days.

[0230] (2) Time period coefficient.

[0231] The above time period coefficient refers to the coefficient corresponding to the time period in which the above control command word is detected.

[0232] The method of dividing usage time periods can be set according to actual needs. For example, in some scenarios, usage time periods can be divided by hour; in others, by day; in still others, by morning (e.g., 6:00 to 12:00), afternoon (e.g., 12:00 to 18:00), and evening (e.g., 18:00 to 6:00); in yet another scenario, by daytime (e.g., 6:00 to 18:00) and nighttime (e.g., 18:00 to 6:00); and in still others, usage time periods can be divided in other ways. This application does not limit the specific form of the usage time periods.

[0233] The time period coefficient can be positively correlated with the number of times the user uses the aforementioned control command words within the usage time period. For example, if the user frequently uses the control command word "turn on music" during the day, the time period coefficient for "turn on music" during the day can be higher; if the user almost never uses the control command word "turn on music" at night, the time period coefficient for "turn on music" at night can be lower.

[0234] The calculation method for the time period coefficient can be set according to actual needs. For example, in some embodiments, the time period coefficient can be calculated using formula (2).

[0235] T=A*d2 (2)

[0236] Where T represents the time period coefficient, A represents the first preset coefficient, and d2 represents the number of times the user uses the control command word within the second specified time period.

[0237] For example, suppose the control device performs statistics based on data from the last 30 days, with the usage period being nighttime. In this case, the second specified time period can be the last 30 days; assuming the user used the control command word 100 times in the last 30 days, with 70 of those times being at night, then d2 would be 70.

[0238] Alternatively, in other embodiments, the time period coefficient can also be calculated using formula (3).

[0239] T=B*d2 / d3 (3)

[0240] Where d3 represents the number of times the user uses the control command word during the non-use period of the second specified time period, and B represents the second preset coefficient.

[0241] For example, suppose the control device is based on data from the last 30 days, and the usage period is nighttime. If a user uses this control command 100 times in the last 30 days, and 70 of those times are at night, then d2 is 70 and d3 is 30.

[0242] Alternatively, in other embodiments, the time period coefficient can also be calculated using formula (4).

[0243] T=C*t2 / t3 (4)

[0244] Where t2 represents the number of target time periods within the second specified time period, and the target time period is the time period during which the control command word is used; t3 represents the number of non-target time periods within the second specified time period; and C is the third preset coefficient.

[0245] For example, suppose the control device performs statistics based on data from the last 30 days, with the usage period being nighttime. If the user used the control command word on 27 nights and did not use it on 3 nights within the last 30 days, then t2 is 27 and t3 is 3.

[0246] Alternatively, in other embodiments, the time period coefficient can also be calculated using other formulas. This application does not limit the specific calculation method for the time period coefficient.

[0247] (3) Use coefficient.

[0248] The usage factor mentioned above refers to the proportion of the number of times the above control command words are used to the total number of times all control command words are used. The usage factor can be expressed by formula (5):

[0249] U=d4 / n (5)

[0250] Where d4 represents the number of times the user uses the above control command words within the third specified time period; n represents the total number of times the user uses all control command words within the third specified time period.

[0251] For example, suppose the control device performs statistical analysis based on data from the last 30 days. The user used control command terms a total of 213 times in the last 30 days. Of these, the user used the command "turn on music" 52 times. In this case, d4 is 52, n is 213, and the usage coefficient for "turn on music" is 52 / 213 ≈ 0.244.

[0252] (4) Initial scene coefficients.

[0253] The initial scenario coefficients mentioned above are used to represent the scope of influence of control command words. The scope of influence refers to the degree of adverse impact on the user when a control command word is misidentified.

[0254] For example, suppose a user is taking a shower and a program on a smart TV issues a voice command to "adjust the water temperature to 80 degrees Celsius." If the water heater actually adjusts the water temperature to 80 degrees Celsius, it will scald the user, posing a significant threat to their safety. Therefore, the control command "adjust the water heater temperature" has a relatively wide impact.

[0255] Similarly, control commands such as "turn down the volume" and "play music" will not cause substantial adverse effects on the user's personal safety or property even if misidentified. Therefore, the scope of influence of control commands such as "turn down the volume" and "play music" is relatively small.

[0256] The initial scenario coefficient of a control command word can be positively correlated with its influence range. The larger the influence range of the control command word, the larger the initial scenario coefficient; conversely, the smaller the influence range of the control command word, the smaller the initial scenario coefficient.

[0257] For example, assuming that the impact range of "turning off the air conditioner" is larger than that of "turning off the TV", and the impact range of "turning off the TV" is larger than that of "turning down the volume", then the manufacturer can set the initial scene coefficient of "turning off the air conditioner" to 3, the initial scene coefficient of "turning off the TV" to 2, and the initial scene coefficient of "turning down the volume" to 1.

[0258] Furthermore, the initial scene coefficient values ​​corresponding to each control command word can be set according to actual needs. For example, in some embodiments, the manufacturer can set the initial scene coefficient for "turn off the air conditioner" to 3; in other embodiments, the manufacturer can set the initial scene coefficient for "turn off the air conditioner" to 3.1; in still other embodiments, the manufacturer can set the initial scene coefficient for "turn off the air conditioner" to 2.56; and in yet another embodiment, the manufacturer can set the initial scene coefficient for "turn off the air conditioner" to other values. This application does not limit the specific values ​​of the initial scene coefficients corresponding to each control command word.

[0259] When the control device adjusts the confidence level of the control command word based on the confidence level adjustment parameter, if the confidence level adjustment parameter indicates that the control command word is highly likely to be issued by the user, or that the consequences of misidentification of the control command word are small, the control device can appropriately increase the confidence level of the word to reduce the recognition difficulty of the control command word.

[0260] For example, suppose the reliability adjustment parameters include a usage coefficient. When the usage coefficient corresponding to a control command word is relatively large, it indicates that the user frequently uses the control command word, and that the control command word is more likely to have been issued by the user. In this case, the control device can appropriately increase the word confidence.

[0261] Assuming the reliability adjustment parameters include the initial scenario coefficient, a relatively small initial scenario coefficient for the control command word indicates that even if misidentified, the control command word will not cause substantial adverse effects on the user's personal safety or property. In this case, the control device can appropriately increase the word confidence level.

[0262] If the confidence adjustment parameter indicates that the above control command word is highly likely to be emitted by the speaker device, or that the consequences of misidentifying the above control command word are serious, then the control device can appropriately lower the confidence of the word to increase the recognition difficulty of the control command word.

[0263] For example, suppose the reliability adjustment parameters include a usage coefficient. When the usage coefficient corresponding to a control command word is relatively small, it indicates that the user does not frequently use the control command word, and the control command word is more likely to be emitted by the speaker device. In this case, the control device can appropriately increase the word confidence.

[0264] Assuming the reliability adjustment parameters include the initial scenario coefficient, a large initial scenario coefficient for a control command word indicates that misidentification of that command word could potentially have a substantial adverse impact on the user's personal safety and property. In this case, the control device can appropriately increase the word confidence level.

[0265] Specifically, consider usage frequency, time period coefficient, usage coefficient, and initial scenario coefficient as examples.

[0266] The confidence level of words is positively correlated with the frequency of use, the time period coefficient, and the usage coefficient, and negatively correlated with the initial scenario coefficient mentioned above.

[0267] The higher the frequency of use, the longer the time period coefficient, and the higher the usage coefficient, the more frequently users prefer to use the control command word, and the more likely that the control command word is a user-issued instruction. Therefore, the control device can increase the confidence level of the word to reduce the difficulty of recognizing the control command word and reduce the possibility of missed recognition.

[0268] The lower the frequency of use, the shorter the time period coefficient, and the lower the usage coefficient, the less the user likes to use the control command word, and the more likely that the control command word is emitted by the speaker device. Therefore, the control device can lower the confidence level of the word, increase the recognition difficulty of the control command word, and reduce the possibility of misrecognition.

[0269] The higher the initial scenario coefficient, the greater the influence range of the aforementioned control command words, and the more severe the consequences of misidentification. In this case, to reduce the impact of misidentification on users, the control device can lower the word confidence level, increase the recognition difficulty of the control command words, and reduce the possibility of misidentification.

[0270] The lower the initial scenario coefficient, the smaller the influence range of the aforementioned control command words, and the less severe the consequences of misidentification. In this case, the control device can increase the word confidence level to reduce the recognition difficulty of the control command words and decrease the possibility of missed recognition.

[0271] When the control device adjusts the confidence threshold of the control command word according to the confidence adjustment parameter, if the confidence adjustment parameter indicates that the above control command word is more likely to be issued by the user, or indicates that the consequences of misidentification of the above control command word are small, the control device can appropriately lower the confidence threshold to reduce the recognition difficulty of the control command word.

[0272] If the confidence adjustment parameter indicates that the above control command word is highly likely to be emitted by the speaker device, or that the consequences of misidentifying the above control command word are serious, then the control device can appropriately increase the confidence threshold to increase the recognition difficulty of the control command word.

[0273] Specifically, consider usage frequency, time period coefficient, usage coefficient, and initial scenario coefficient as examples.

[0274] The confidence threshold is negatively correlated with the frequency of use, the time period coefficient, and the usage coefficient, and positively correlated with the initial scenario coefficient mentioned above.

[0275] The higher the frequency of use, time period coefficient, and usage coefficient, the more likely the control command word is issued by the user. The control device can lower the confidence threshold to reduce the recognition difficulty of the control command word. Conversely, the lower the frequency of use, time period coefficient, and usage coefficient, the more likely the control command word is issued by the speaker device. The control device can raise the confidence threshold to increase the recognition difficulty of the control command word.

[0276] The higher the initial scenario coefficient, the greater the influence range of the control command word and the more serious the consequences of misidentification. The control device can increase the confidence threshold to increase the recognition difficulty of the control command word. The lower the initial scenario coefficient, the smaller the influence range of the control command word and the less serious the consequences of misidentification. The control device can decrease the confidence threshold to reduce the recognition difficulty of the control command word.

[0277] For example, assuming the confidence adjustment parameters include time period coefficients and initial scenario coefficients, and the confidence threshold for each control command word is 0.95.

[0278] Assume that the formula for calculating the confidence level of a word is formula (6), and the formula for calculating the confidence threshold is formula (7).

[0279] The expression for formula (6) is:

[0280] E1=e1+0.1*(T-0.5)-0.1*(k-2) (6)

[0281] The expression for formula (7) is:

[0282] E2=e2-0.1*(T-0.5)+0.1*(k-2) (7)

[0283] Where T is the time period coefficient; E1 is the word confidence after adjustment of the above control command words; e1 is the word confidence before adjustment; k is the initial scene coefficient of the above control command words; E2 is the confidence threshold after adjustment; e2 is the confidence threshold before adjustment.

[0284] like Figure 3 As shown, in one example, the smart home system is assumed to include a home hub 31 and a smart speaker 32. The home hub 31 is the control device, and the smart speaker 32 is the sound pickup device.

[0285] Suppose a user says to smart speaker 32, "Hey Celia, turn off the music," when leaving home in the morning. At this moment, smart speaker 32 detects the control command "turn off music" and its corresponding word confidence score of 0.97. Then, smart speaker 32 sends "turn off music" and its corresponding word confidence score to the home hub 31.

[0286] After receiving the message "Turn off music", the home hub 31 obtains the time period coefficient corresponding to "Turn off music" in the current time period and the initial scene coefficient corresponding to "Turn off music".

[0287] Assuming the coefficient for "turn off music" is 0.9 during the daytime period, it indicates that users frequently use this control command during the day. The initial scenario coefficient for "turn off music" is 1, indicating that the influence range of this control command is relatively small.

[0288] The family central hub 31 substitutes the word confidence, time period coefficient and initial scene coefficient of "turn off music" into formula (6) to obtain the updated word confidence as 0.97+0.1*(0.9-0.5)-0.1*(1-2)=1.11.

[0289] Since the updated word confidence score is 1.11, which is greater than the confidence threshold of 0.95, the Home Hub 31 determines that the control command word "turn off music" has been successfully verified and continues to execute subsequent operations.

[0290] like Figure 4 As shown, in another example, assume that the smart home system includes a home hub 41, a smart speaker 42, and a smart TV 43. The home hub 41 is the control device, the smart speaker 42 is the sound pickup device, and the smart TV 43 is the speaker device.

[0291] Suppose that while a user is watching a program on smart TV 43 at night, the program is playing the dialogue, "Why are you still turning on the air conditioner in such cold weather? Turn it off now!" At this moment, smart speaker 42 detects the control command "turn off the air conditioner" and its corresponding word confidence score of 0.96. Then, smart speaker 42 sends "turn off the air conditioner" and its corresponding word confidence score to the home hub 41.

[0292] After receiving the command "Turn off the air conditioner", the home hub 41 obtains the time period coefficient corresponding to "Turn off the air conditioner" in the current time period and the initial scene coefficient corresponding to "Turn off the air conditioner".

[0293] Assuming the coefficient for "turn off the air conditioner" is 0.1 during the nighttime period, it means users almost never use this command at night. The initial scenario coefficient for "turn off the air conditioner" is 3, indicating that this command has a wide range of influence.

[0294] The home hub 41 substitutes the confidence threshold, time period coefficient and initial scenario coefficient of "turn off the air conditioner" into formula (7) to obtain the updated confidence threshold as 0.95-0.1*(0.1-0.5)+0.1*(3-2)=1.09.

[0295] Since the confidence score of the phrase "turn off the air conditioner" is 0.96, which is less than the updated confidence score threshold of 1.09, the home hub 41 determines that the verification of the control command phrase "turn off the air conditioner" has failed, and the home hub 41 does not respond to the control command phrase "turn off the air conditioner".

[0296] As can be seen from the above example, the control device can adjust the vocabulary confidence and / or confidence threshold of the control command words through the confidence adjustment parameter, and use the updated vocabulary confidence and / or updated confidence threshold for verification.

[0297] Since the confidence adjustment parameter represents the degree to which a user has previously used the control command word, or the scope of influence of the control command word, the control device can improve the recognition accuracy of the control command word and reduce the possibility of misrecognition by adjusting the lexical confidence and / or confidence threshold of the control command word using the confidence adjustment parameter.

[0298] 3. Scene classification.

[0299] After recognizing the control command words, the control device can classify the scenario of the control command words according to the initial scenario coefficient.

[0300] When the initial scene coefficient is less than or equal to the scene coefficient threshold, the control device can directly execute the operation corresponding to the above control command word without interacting with the user for confirmation. Alternatively, the control device can directly send an operation instruction to the smart home device corresponding to the above control command word to instruct the smart home device to execute the operation corresponding to the above control command word.

[0301] When the initial scene coefficient is greater than the scene coefficient threshold, the control device can interact with the user for confirmation.

[0302] Understandably, the initial scenario coefficient of a control command word symbolizes its scope of influence. If control command words are not categorized by scenario based on the initial scenario coefficient, the control device may either treat all control command words the same and require interactive confirmation, or fail to require interactive confirmation at all.

[0303] If the control device recognizes some control command words with a relatively small impact and requires interactive confirmation every time, it will annoy the user and affect the user experience.

[0304] For example, in daily life, users may frequently use control commands with a small impact, such as "turn on music," "turn up the volume," "turn down the volume," and "turn off the lights." If the control device has to interact with the user for confirmation every time the user uses such a control command, the user will inevitably find it inconvenient and even annoying.

[0305] Alternatively, if the control device identifies some control command words with a wide impact but does not conduct a dialogue confirmation, misidentification may occur, which may adversely affect the personal and property safety of users.

[0306] For example, while a user is taking a shower, a program on a smart TV might issue a voice prompt to "adjust the water temperature to 80 degrees Celsius." If the control device does not interact with the user for confirmation and instead adjusts the water temperature to 80 degrees Celsius, it could scald the user and pose a significant threat to their personal safety.

[0307] Therefore, in this embodiment of the application, the control device can classify the scenarios of control command words by initial scenario coefficients, perform interactive confirmation for control command words with a large impact range, and not perform interactive confirmation for control command words with a small impact range, thereby improving the fluency of user voice control and providing users with a better user experience while ensuring the safety of users' personal and property safety.

[0308] Furthermore, when performing scene classification, the control device can directly use the initial scene coefficients or, alternatively, use the adjusted initial scene coefficients (hereinafter referred to as the target scene coefficients) for scene classification.

[0309] When the control device classifies the scene according to the target scene coefficient, the control device can obtain the scene adjustment parameters of the control command word after recognizing the control command word.

[0310] Scene adjustment parameters may include one or more of the following: usage frequency, time period coefficient, usage coefficient, etc.

[0311] After obtaining the scene adjustment parameters, the control device can adjust the initial scene coefficients of the control command words according to the scene adjustment parameters to obtain the target scene coefficients.

[0312] Then, the control device can compare the target scene coefficient with the scene coefficient threshold.

[0313] When the target scene coefficient is less than or equal to the scene coefficient threshold, the control device can directly execute the operation corresponding to the above control command word without interacting with the user for confirmation, and / or the control device can also directly send operation instructions to the smart home device corresponding to the above control command word to instruct the smart home device to execute the operation corresponding to the above control command word.

[0314] When the target scene coefficient is greater than the scene coefficient threshold, the control device can interact with the user for confirmation.

[0315] The method by which the control equipment adjusts the initial scene coefficients according to the scene can be set according to actual needs.

[0316] Specifically, when the control device adjusts the initial scene coefficient according to the scene adjustment parameters, if the scene adjustment parameters indicate that the user frequently uses the control command word, the control device can appropriately lower the initial scene coefficient to reduce the number of interactions between the control device and the user and improve the smoothness of the user's operation.

[0317] If the scene adjustment parameters indicate that the user does not frequently use the control command word, the control device can appropriately increase the initial scene coefficient to increase the likelihood of interaction and confirmation between the control device and the user, enabling the control device to more accurately understand the user's intent and reduce the probability of misidentification.

[0318] Taking usage frequency, time period coefficient, and usage coefficient as examples, the target scenario coefficient is negatively correlated with usage frequency, time period coefficient, and usage coefficient.

[0319] A higher frequency of use, time period coefficient, and usage coefficient indicate that users prefer to use the control command. In this case, frequent interaction between the control device and the user can impair the smoothness of user operation and negatively impact the user experience. Therefore, for higher frequency of use, time period coefficient, and usage coefficient, the initial scene coefficient of the control device can be appropriately lowered.

[0320] The lower the frequency of use, the shorter the time period coefficient, and the lower the usage coefficient, the less the user likes to use the control command word. In this case, the control command word is more likely to be emitted by the speaker device. To avoid operations that contradict the user's intentions, the control device can appropriately increase the initial scene coefficient to increase the probability of interaction and confirmation between the control device and the user. Through interactive confirmation, the control device can more accurately understand the user's intentions, reduce the possibility of misoperation, and minimize the impact of misoperation on the user.

[0321] For example, assuming the scene adjustment parameters include a time period coefficient and the scene coefficient threshold is 2, the initial scene coefficient adjustment formula is formula (8).

[0322] The expression for formula (8) is:

[0323] K = k * (1.5 - T) (8)

[0324] Where K is the target scene coefficient, k is the initial scene coefficient, and T is the time period coefficient.

[0325] like Figure 5 As shown, in one example, it is assumed that the smart home system includes a home hub 51, a smart speaker 52, and a smart air conditioner 53.

[0326] At night, when the user returns home, he says to the smart speaker 52, "Xiaoyi, Xiaoyi, turn on the air conditioner."

[0327] At this moment, the smart speaker 52 detects the control command "turn on the air conditioner" and transmits the command "turn on the air conditioner" to the home hub 51.

[0328] After receiving the control command "turn on the air conditioner", the home hub 51 obtains the initial scene coefficient corresponding to "turn on the air conditioner" and the time period coefficient of "turn on the air conditioner" at night.

[0329] Assuming the initial scenario coefficient for "turn on the air conditioner" is 3, it indicates that the influence range of the control command "turn on the air conditioner" is relatively large; assuming the time period coefficient for "turn on the air conditioner" at night is 0.9, it indicates that users frequently use the command "turn on the air conditioner" at night.

[0330] At this point, the home central hub 51 can substitute the initial scene coefficient and time period coefficient of "turn on the air conditioner" into formula (8) to obtain the target scene coefficient as 3*(1.5-0.9)=1.8.

[0331] Since the target scenario coefficient of 1.8 is less than the scenario coefficient threshold of 2, the home hub 51 does not need to interact with the user for confirmation. The home hub 51 can directly send the operation command "turn on the air conditioner" to the smart air conditioner 53.

[0332] After receiving the above operation command, the smart air conditioner 53 enters working mode and completes this voice control process.

[0333] In other words, in this example, although "turn on the air conditioner" is a control command with a wide impact, since users often use this control command at night, in order to improve the user's control fluency, the home hub 51 can lower the initial scene coefficient of this control command, skip the interactive confirmation process, and directly instruct the smart air conditioner 53 to execute the operation corresponding to the control command "turn on the air conditioner".

[0334] like Figure 6 As shown, in another example, it is assumed that the smart home system includes a home hub 61, a smart speaker 62, and a smart TV 63.

[0335] At night, the user is watching a program on Smart TV 63 at home. The program is playing a dialogue that says, "I want to listen to rock music."

[0336] At this time, the smart speaker 62 detects the control command "listen to rock music" and sends the control command "listen to rock music" to the home hub 61.

[0337] After receiving the control command "listen to rock music", the home hub 61 obtains the initial scene coefficient corresponding to "listen to rock music" and the time period coefficient of "listen to rock music" at night.

[0338] Assuming the initial scenario coefficient for "listen to rock music" is 1.5, it indicates that the influence range of the control command "listen to rock music" is moderate; assuming the coefficient for "listen to rock music" during the nighttime period is 0.1, it indicates that users almost never use the command "listen to rock music" at night.

[0339] At this point, the family central hub 61 can substitute the initial scene coefficient and time period coefficient of "listening to rock music" into formula (8) to obtain the target scene coefficient as 1.5*(1.5-0.1)=2.1.

[0340] Since the target scene coefficient 2.1 is greater than the scene coefficient threshold 2, the home hub 61 needs to interact with the user to confirm based on the control command word "listen to rock music".

[0341] In other words, in this example, although the control command "listen to rock music" has a moderate range of influence and usually does not require interactive confirmation, since users almost never use this control command at night, the home hub 61 can increase the initial scenario coefficient of this control command and interact with the user to reduce the inconvenience caused by accidental operation, thereby more accurately understanding the user's intention and improving the accuracy of recognizing the control command.

[0342] As can be seen from the above examples, in the command word recognition method provided in this application embodiment, the control device can classify the application scenarios corresponding to each control command word according to the target scenario coefficient.

[0343] When the target scene coefficient is less than or equal to the scene coefficient threshold, the control device can directly execute the operation corresponding to the control command word without interacting with the user for confirmation, and / or the control device can also send operation instructions to the smart home device corresponding to the above control command word to instruct the smart home device to execute the operation corresponding to the above control command word, simplifying the control process and improving the fluency of user voice control.

[0344] When the target scenario coefficient is greater than the scenario coefficient threshold, the control device can interact with the user for confirmation. Through interactive confirmation, the device can understand the user's true control intention and reduce the possibility of misoperation.

[0345] Furthermore, the control device can dynamically adjust the initial scene coefficients based on the scene adjustment parameters. Since the scene adjustment parameters represent the user's usage of the control command, dynamically adjusting the initial scene coefficients based on these parameters yields more accurate initial scene coefficients, allowing the control device to more rationally classify different application scenarios.

[0346] 4. Interactive confirmation.

[0347] When the control device interacts with the user for confirmation, the control device can perform a prompting operation to prompt the user for secondary confirmation.

[0348] The aforementioned prompting action can be the control device issuing an interactive confirmation prompt, and / or the control device instructing other smart home devices to issue interactive confirmation prompts. The interactive confirmation prompts can take one or more forms, including sound, text, light, and vibration.

[0349] For example, such as Figure 7 In some scenarios, the control device is assumed to be a home hub 71, and the speaker device is a smart speaker 72. When the home hub 71 needs to interactively confirm the control command "turn on the air conditioner," it can send a broadcast command to the smart speaker 72. After receiving the broadcast command, the smart speaker 72 plays "Please confirm whether to turn on the air conditioner" through its speaker, prompting the user to confirm again via sound.

[0350] like Figure 8 As shown, in other scenarios, assuming the control device is a smart speaker 81, when the smart speaker 81 needs to interact and confirm the control command "turn on the air conditioner," it can send a display command to the smart TV 82. Upon receiving the display command, the smart TV 82 displays "Please confirm whether to turn on the air conditioner" on its screen, prompting the user for secondary confirmation.

[0351] like Figure 9 As shown, in other scenarios, assuming the control device is a smart speaker 91, when the smart speaker 91 needs to confirm a control command, it can send a light command to a smart home device equipped with a light source (i.e., a light source device 92, such as a light bulb). Upon receiving the light command, the light source device 92 prompts the user for secondary confirmation by flashing its light or emitting a specific color of light.

[0352] like Figure 10 As shown, in other scenarios, assuming the control device is a home hub 1001, when the home hub 1001 needs to interactively confirm a control command, it can send a vibration command to a smart home device equipped with a vibration motor (i.e., vibration device 1002, such as a remote control). Upon receiving the vibration command, vibration device 1002 prompts the user for secondary confirmation through vibration.

[0353] When a user notices a prompt from the control device, the user can perform a confirmation action.

[0354] At this point, the control device can respond to the user's confirmation operation and determine the user's operating intention.

[0355] If the above confirmation operation expresses the intent of confirmation (i.e. the above confirmation response), the control device may perform the operation corresponding to the above control command word, and / or the control device may also send an operation instruction to the smart home device corresponding to the above control command word to instruct the smart home device to perform the operation corresponding to the above control command word.

[0356] If the above confirmation operation expresses a denial intention (i.e., the above denial response), the control device may not respond to the above control command word.

[0357] The format of the above confirmation operation can be set according to actual needs.

[0358] In some possible implementations, the confirmation action described above can be a specific button touched by the user. This button can be a physical button or a virtual button.

[0359] When the user touches the confirmation button, the control device can respond to the user's operation and execute the operation corresponding to the above control command word, and / or, the control device can also send an operation instruction to the smart home device corresponding to the above control command word to instruct the smart home device to execute the operation corresponding to the above control command word.

[0360] When the user presses the deny button, the control device can respond to the user's operation and not respond to the aforementioned control command.

[0361] For example, such as Figure 11 As shown, assume that the home control unit 1100 displays "Please confirm whether to turn on the air conditioner" on its screen and provides virtual buttons 1101 and 1102. Virtual button 1101 is the button corresponding to "Yes", and virtual button 1102 is the button corresponding to "No".

[0362] When a user clicks the virtual button 1101, it indicates that the user wants to turn on the air conditioner. At this time, the home control unit 1100 executes the operation corresponding to the command "turn on the air conditioner".

[0363] When a user clicks virtual button 1102, it indicates that the user does not want to turn on the air conditioner. At this time, the home control center 1100 does not respond to the command "turn on the air conditioner".

[0364] In some other possible implementations, the above confirmation operation can issue a second voice signal to the user containing a confirmation command word.

[0365] After the control device performs the prompting operation, the pickup device can listen to the surrounding sound signals.

[0366] When the sound pickup device acquires a second human voice signal, it can use the target command word model locally to perform command word detection on the second human voice signal.

[0367] When the pickup device detects an acknowledgment command word, it can send that acknowledgment command word to the control device.

[0368] Alternatively, when the pickup device detects a second human voice signal, it can also directly send the second human voice signal to the control device.

[0369] After receiving the second human voice signal, the control device can use the target command word model to detect command words in the second human voice signal.

[0370] When the control device detects a confirmation command word, the control device can obtain that confirmation command word.

[0371] When the aforementioned confirmation command word is a command word with a confirmation meaning, the control device can execute the operation corresponding to the aforementioned control command word, and / or, the control device can also send an operation instruction to the smart home device corresponding to the aforementioned control command word to instruct the smart home device to execute the operation corresponding to the aforementioned control command word.

[0372] When the aforementioned confirmation command word is a command word with a negative meaning, the control device may not respond to the aforementioned control command word.

[0373] For example, such as Figure 12 As shown, the smart home system is assumed to include a home hub 121, a smart speaker 122, and a smart air conditioner 123.

[0374] When the home hub 121 needs to interact with the user to confirm the control command "turn on the air conditioner", the home hub 121 can send a broadcast command to the smart speaker 122.

[0375] When the smart speaker 122 receives the broadcast command, it broadcasts "Please confirm whether to turn on the air conditioner" through the microphone array.

[0376] When the user heard the voice announcement, they said "Yes" to the smart speaker 122.

[0377] At this time, the smart speaker 122 collects the user's second voice signal, uses the above-mentioned target command word model to identify the second voice signal, and obtains the confirmation command word "yes".

[0378] Then, the smart speaker 122 sends the confirmation command "Yes" to the home hub 121.

[0379] After receiving the confirmation command "Yes", the home control center 121 determines that the confirmation command expresses a confirmation meaning. Therefore, the home control center 121 can send an operation command to the smart air conditioner 123.

[0380] After receiving the above operation command, the smart air conditioner 123 enters working mode and completes this voice control process.

[0381] It's important to note that in traditional multi-turn dialogue solutions, electronic devices typically require models such as Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Natural-language Understanding (NLU) to engage in conversation with the user. However, these models require significant storage space and consume substantial computing resources. Therefore, these models are usually hosted in the cloud, with local electronic devices leveraging cloud computing resources to achieve multi-turn dialogue.

[0382] However, when local electronic devices are in a weak network environment, they struggle to interact with the motion data, causing their conversational functions to malfunction. Furthermore, because local electronic devices need to interact with the cloud, traditional multi-turn conversation solutions suffer from high latency and a poor user experience.

[0383] However, in this embodiment, the control device / voice pickup device can engage in multi-turn dialogues with the user through the target command word model. Compared to conventional command word models, the target command word model provided in this embodiment, while adding confirmation command words during training, may increase the difficulty of the training process, such as the difficulty of constructing the training sample set and training the command word model itself.

[0384] However, after training, the target command word model is comparable in size to a regular command word model, occupies little storage space, and can be directly deployed on the control device / pickup device.

[0385] Furthermore, because confirmation command words are added during training, the target command word model can achieve multi-turn dialogues with users by detecting confirmation command words, thus handling confirmation mechanisms in various scenarios. Since the target command word model only requires minimal computing resources to detect confirmation command words, the control device / voice pickup device can also conduct multi-turn dialogues with users without relying on cloud computing resources.

[0386] In summary, in the command word response method of this embodiment, the control device / pickup device can directly use the target command word model locally to conduct multi-turn dialogues with the user without relying on cloud computing resources or data interaction with the cloud. It can be used in environments with weak or no network, and has low latency, greatly improving the user experience.

[0387] In other possible implementations, the confirmation operation described above can also take other forms. For example, the confirmation operation can also be a gesture operation, etc. This application does not limit the specific form of the confirmation operation described above.

[0388] Furthermore, when the control device detects a confirmation operation indicating a denial intention, the control device can also provide feedback adjustment to the aforementioned control command words, lowering the confidence level of the next word of the aforementioned control command words, and / or raising the confidence level threshold of the aforementioned control command words.

[0389] For example, suppose the voice pickup device recognizes the control command "turn on the air conditioner" in this voice control process. However, during the interactive confirmation process, the control device detects a confirmation operation that expresses denial, indicating that the recognized control command "turn on the air conditioner" is a misrecognized control command.

[0390] Therefore, the control device can increase the confidence threshold for "turn on the air conditioner," for example, by increasing the confidence threshold from 0.95 to 0.98. Alternatively, the control device can also decrease the confidence of the next time the word "turn on the air conditioner" is recognized. For example, if the confidence of the next time the word "turn on the air conditioner" is recognized is 0.96, the control device can decrease the confidence of the word "turn on the air conditioner" to 0.93.

[0391] By using feedback adjustment, the control device can increase the difficulty of recognizing control command words, reduce the probability of subsequent misrecognition of the control command words, avoid the control device repeatedly recognizing incorrect control command words, and improve the recognition accuracy of control command words.

[0392] The following section will describe the command word response method in detail with reference to specific flowcharts.

[0393] Please see Figure 13 When the sound pickup device collects a human voice signal, it can process the signal locally and send the control command word to the control device when the control command word is recognized.

[0394] Alternatively, the microphone can send the voice signal to the control device. Upon receiving the voice signal, the control device processes it and identifies the control command words.

[0395] It should be noted that the control command words identified above can be understood as the vocabulary confidence of the sound pickup / control device in recognizing the control command word being greater than the confidence threshold corresponding to the control command word.

[0396] After obtaining the control command words, the control device can directly execute the steps in the scenario classification without verifying the control command words.

[0397] Alternatively, the control device can also verify the control command words to improve the accuracy of recognizing them.

[0398] During the verification process, the control device can adjust the word confidence and / or confidence threshold based on the confidence adjustment parameters.

[0399] Then, the control device compares the word confidence level with the confidence threshold.

[0400] When the confidence level of a word is less than or equal to the confidence threshold, it indicates that the above control command word verification has failed, and the control device may not respond to the control command word.

[0401] When the confidence level of a word is greater than the confidence threshold, it indicates that the above control command word verification is successful, and the control device can execute the steps in the subsequent scenario classification.

[0402] In the scene classification step, the control device can directly use the initial scene coefficient of the control command word as the target scene coefficient, or the control command word can adjust the initial scene coefficient according to the scene adjustment parameters to obtain the target scene coefficient.

[0403] Then, the control device can determine whether the target scene coefficient is greater than the scene coefficient threshold.

[0404] When the target scene coefficient is less than or equal to the scene coefficient threshold, it means that the influence range of the above control command words is small, the control device does not need to interact with the user for confirmation, the control device can directly execute the operation corresponding to the above control command words, and / or, the control device can instruct the electronic device corresponding to the above control command words to execute the operation corresponding to the above control command words.

[0405] When the target scenario coefficient is greater than the scenario coefficient threshold, it indicates that the influence range of the above control command words is large, and the control device needs to interact with the user for confirmation. At this time, the control device can perform a prompt operation.

[0406] Subsequently, if the control device detects a confirmation operation indicating the user's intention to confirm, it indicates that the aforementioned control command word is the instruction the user wants to issue, and the control device can execute the operation corresponding to the aforementioned control command word, and / or, the control device can instruct the electronic device corresponding to the aforementioned control command word to execute the operation corresponding to the aforementioned control command word.

[0407] If the control device detects a confirmation operation by the user indicating a denial intention, it means that the above control command word is not the instruction the user wants to issue, and the control device may not respond to the above control command word.

[0408] In addition, when the control device detects a user's confirmation operation expressing denial intent, it can also lower the confidence level of the next word for the control command word, and / or raise the confidence threshold of the control command word, thereby increasing the recognition difficulty of the control command word and reducing the probability of subsequent misrecognition of the control command word.

[0409] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic.

[0410] Furthermore, during the implementation of the embodiments of this application, the control device can implement all the processes described above, or the control device can also implement some of the processes described above.

[0411] For example, in some application scenarios, the control device may not need to perform the process of verifying the control command words in Section 2; in other application scenarios, the control device may not need to perform the process of adjusting the initial scenario coefficients in Section 3.

[0412] Therefore, when implementing the command word response method provided in the embodiments of this application, the control device can implement some or all of the processes described above according to the needs of the actual scenario. The content described above should not constitute any limitation on the implementation process of the embodiments of this application.

[0413] The following will describe in detail another command word response method provided in the embodiments of this application from the perspective of the control device. Please refer to... Figure 14 The command word response method provided in this embodiment includes:

[0414] S141. Obtain the control command word corresponding to the human voice signal; wherein, the human voice signal is collected by the sound pickup device.

[0415] A sound pickup device refers to an electronic device equipped with a microphone. In the embodiments of this application, the control device and the sound pickup device can be two independent electronic devices, or the control device and the sound pickup device can be the same electronic device.

[0416] When the sound pickup device collects a human voice signal, it can process the signal and send a control command word to the control device when it recognizes the command word.

[0417] Alternatively, the sound pickup device can send a human voice signal to the control device. When the control device receives the human voice signal, it processes it to obtain control command words.

[0418] S142. Obtain the scene coefficients corresponding to the control command words.

[0419] After obtaining the control command word, the control device can obtain the scene coefficient corresponding to the control command word.

[0420] The scenario coefficient is used to characterize the scope of influence of control command words. The scope of influence refers to the degree of adverse impact on users when control command words are misidentified.

[0421] S143. Based on the scenario coefficient, determine whether to perform an interactive confirmation operation.

[0422] After obtaining the scene coefficients, the control device can determine whether to perform an interactive confirmation operation based on the scene coefficients.

[0423] Specifically, if the scene coefficient meets the preset scene coefficient threshold condition, the control device can determine to perform the interactive confirmation operation; otherwise, the control device can determine not to perform the interactive confirmation operation.

[0424] When the scene coefficient meets the pre-set scene coefficient threshold, it indicates that the influence range of the control command word is relatively large. If the control command word is misidentified, it may adversely affect the personal safety and property of the user. At this time, the control device needs to perform an interactive confirmation operation to identify the user's true intention and reduce the occurrence of misidentification.

[0425] When the scene coefficient does not meet the preset scene coefficient threshold, it indicates that the influence range of the control command word is relatively small. Even if the control command word is misrecognized, it will not have a substantial adverse impact on the user's personal safety or property. In this case, the control device can directly respond to the control command word without performing an interactive confirmation operation, reducing the complexity of voice control and improving the fluency of voice control.

[0426] The scene coefficient threshold condition can be set according to actual needs. For example, when the scene coefficient and the influence range of the control command word are positively correlated, the scene coefficient threshold condition can be set to the scene coefficient being greater than the scene coefficient threshold; when the scene coefficient and the influence range of the control command word are negatively correlated, the scene coefficient threshold condition can be set to the scene coefficient being less than or equal to the scene coefficient threshold; or, in other scenarios, the scene coefficient threshold condition can be set to other values. This application does not limit the specific content of the scene coefficient threshold condition in its embodiments.

[0427] S144. If it is determined that an interactive confirmation operation should be performed based on the scenario coefficient, then an interactive confirmation prompt message shall be issued and / or other devices shall be instructed to issue an interactive confirmation prompt message.

[0428] When the control device determines to perform an interactive confirmation operation, the control device can issue an interactive confirmation prompt message through itself, and / or, the control device can also instruct other devices to issue interactive confirmation prompt messages.

[0429] For example, assuming the control device is a home hub, when the home hub determines to perform an interactive confirmation operation, the home hub can display an interactive confirmation prompt on its display screen, and the home hub can instruct the smart speaker to broadcast the interactive confirmation prompt via its speaker.

[0430] S145. Upon receiving a confirmation response corresponding to the interactive confirmation prompt, execute the operation corresponding to the control command word and / or instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word.

[0431] When a user encounters the above interactive confirmation prompt, if the control command is the instruction the user wants to issue, the user can respond to the interactive confirmation prompt with a confirmation message.

[0432] When the control device receives the confirmation response corresponding to the above-mentioned interactive confirmation information, the control device may execute the operation corresponding to the control command word, and / or, the control device may instruct the electronic device corresponding to the above-mentioned control command word to execute the operation corresponding to the control command word.

[0433] For example, suppose the control device is a smart speaker, and the control command is "play TV series". When the smart speaker receives a confirmation response corresponding to the interaction confirmation information, the smart speaker can play the audio of the TV series through its own speaker and instruct the smart TV to play the TV series on screen.

[0434] In the command word response method provided in this application embodiment, the control device can determine the user's true intent through interactive confirmation, reducing the possibility of misidentification.

[0435] Furthermore, the control device does not perform interactive confirmation for all control command words. Instead, it distinguishes control command words based on scenario coefficients and performs interactive confirmation for control command words corresponding to certain scenario coefficients. This reduces the complexity of voice control, improves the fluency of voice control, and has strong usability and practicality.

[0436] Optionally, if it is determined that an interactive confirmation operation will be performed, the above method further includes:

[0437] If a negative response corresponding to the interactive confirmation prompt is received, or if no response corresponding to the interactive confirmation prompt is received, the operation corresponding to the control command word will not be executed, and the electronic device corresponding to the control command word will not be instructed to execute the operation corresponding to the control command word.

[0438] It should be noted that when the above control command words are not the instructions that the user wants to give, the user may respond negatively to the above interactive confirmation prompts, or the user may not respond to the above interactive confirmation prompts.

[0439] At this time, if the control device receives a negative response corresponding to the above-mentioned interactive confirmation prompt, or if the control device does not receive any response corresponding to the above-mentioned interactive confirmation prompt, the control device may not respond to the above-mentioned control command words.

[0440] The control device does not respond to the control command word, which means that the control device does not execute the operation corresponding to the control command word, and the control device does not instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word.

[0441] By using the above method, when a user expresses a denial, the control device can choose not to respond to the control command words, thus avoiding any substantial adverse impact on the user's personal safety and property.

[0442] Optionally, the above method further includes:

[0443] If, based on the scenario coefficient, it is determined that no interactive confirmation operation should be performed, then the operation corresponding to the control command word will be executed and / or the electronic device corresponding to the control command word will be instructed to execute the operation corresponding to the control command word.

[0444] It should be noted that the control device can perform interactive confirmation operations for control command words corresponding to certain scenario coefficients. For other control command words, the control device can directly respond to the control command word without performing interactive confirmation operations, thereby reducing the complexity of voice control and improving the fluency of voice control.

[0445] Optionally, the scene coefficient is a pre-set initial scene coefficient, or the scene coefficient is a target scene coefficient obtained by adjusting the initial scene coefficient according to scene adjustment parameters, which are determined based on historical usage data of control command words and / or other historical usage data of control command words.

[0446] It should be noted that the control device can use the initial scene coefficients preset by the manufacturer as the aforementioned scene coefficients. Alternatively, the control device can obtain scene adjustment parameters, adjust the initial scene coefficients according to the scene adjustment parameters to obtain the target scene coefficients, and use the target scene coefficients as the aforementioned scene coefficients.

[0447] The above-mentioned scenario adjustment parameters are determined based on the historical usage data of control command words and / or other historical data of control command words.

[0448] The aforementioned historical usage data includes one or more of the following: historical usage time period, historical usage frequency within a specified time period, and historical usage frequency within a specified time period.

[0449] For example, suppose the scene adjustment parameters include usage frequency, time period coefficient, and usage coefficient. The control device can use the historical usage frequency within a specified time period as the usage frequency, calculate the time period coefficient of the control command word based on the historical usage time period, and calculate the usage coefficient of the control command word based on the historical usage count of the control command word within the specified time period and the historical usage count of other control command words within the specified time period.

[0450] Since the aforementioned scenario adjustment parameters represent the user's usage of the control command word, the control device can dynamically adjust the initial scenario coefficients based on the scenario adjustment parameters to obtain more accurate initial scenario coefficients. This allows the control device to more reasonably classify the application scenarios corresponding to each control command word.

[0451] Optionally, the interactive confirmation prompt message may include one or more of the following: playing a prompt audio through a speaker, displaying a prompt message on a screen, flashing an indicator light, or emitting vibration through a vibration motor.

[0452] It should be noted that the way electronic devices send interactive confirmation prompts can be set according to the actual scenario.

[0453] Specifically, electronic devices can issue interactive confirmation prompts in one or more ways, such as playing a prompt audio through a speaker, displaying a prompt message on a screen, flashing an indicator light, or emitting vibrations through a vibration motor.

[0454] For example, assuming the aforementioned electronic device is a smart TV, when the smart TV issues an interactive confirmation prompt, it can display the text "Please confirm whether to turn on the air conditioner" on the screen and broadcast it through the speaker.

[0455] This application does not limit the method by which electronic devices issue interactive confirmation prompts.

[0456] Optionally, the response may include one or more of the following: a voice signal containing a confirmation command word, a touch operation on a confirmation control displayed on the screen, or a press operation on a confirmation button.

[0457] It should be noted that the specific form of the above confirmation response can be determined according to the actual scenario.

[0458] For example, in some scenarios, the aforementioned confirmation response can be a human voice signal containing a confirmation command word, such as a user saying, "I confirm turning on the air conditioner."

[0459] In other scenarios, the aforementioned confirmation response can be an electronic device detecting a touch operation confirming a control on a display screen. For example, suppose a home control system has a touch-sensitive display screen that can display the text "Please confirm whether to turn on the air conditioner," along with "Yes" and "No" controls. When the user touches the "Yes" control, the home control system detects the user's confirmation response.

[0460] In other scenarios, the aforementioned confirmation response can also be the electronic device detecting a press of the confirmation button. For example, suppose a smart speaker has a physical button; when the user presses the physical button, the smart speaker detects the user's confirmation response.

[0461] In other scenarios, the aforementioned confirmation response may take other forms. This application does not limit the specific form of the confirmation response.

[0462] Optionally, before obtaining the scenario coefficients corresponding to the control propositions, the above method further includes:

[0463] Obtain the lexical confidence score and confidence threshold corresponding to the control command words;

[0464] If the confidence level of a word is determined to be greater than the confidence threshold, proceed with the next steps.

[0465] It should be noted that if the control device / pickup device detects that the word confidence of a certain control command word is greater than the confidence threshold, the control device considers that the control command word has been detected.

[0466] Optionally, if the word confidence is determined to be greater than a confidence threshold, the above method further includes the following steps before performing subsequent steps:

[0467] Adjust the vocabulary confidence and / or confidence threshold based on the confidence adjustment parameters corresponding to the control command words.

[0468] It should be noted that the control device can adjust the confidence level of the word and / or the confidence threshold according to the confidence level adjustment parameter, and then compare the word confidence level with the confidence threshold to verify the control command word.

[0469] When the confidence level of a word is greater than the confidence threshold, it means that the control command word has passed the verification.

[0470] When the confidence level of a word is less than or equal to the confidence threshold, it indicates that the validation of the control command word has failed.

[0471] Since the aforementioned confidence threshold represents the degree to which a user has previously used the control command word, or the scope of influence of the control command word, the control device can improve the recognition accuracy of the control command word and reduce the possibility of false recognition by adjusting the lexical confidence or confidence threshold of the control command word using confidence adjustment parameters.

[0472] Optionally, the confidence adjustment parameters include one or more of the following: initial scenario coefficient, historical usage frequency within a specified time period, time period coefficient, and usage coefficient.

[0473] It should be noted that the above usage frequency refers to the number of times a user uses control command words within a certain period of time, which can be understood as the historical usage frequency within a specified time period.

[0474] The above time period coefficient refers to the coefficient corresponding to the time period in which the above control command word is detected.

[0475] The usage coefficient mentioned above refers to the proportion of the number of times the above control command words are used to the total number of times all control command words are used.

[0476] The initial scenario coefficients mentioned above are used to represent the scope of influence of control command words. The scope of influence refers to the degree of adverse impact on the user when a control command word is misidentified.

[0477] Since the aforementioned confidence adjustment parameters represent the degree to which users have previously used control command words, or the scope of influence of control command words, using these parameters to adjust the lexical confidence or confidence threshold of control command words can improve the recognition accuracy of control command words and reduce the possibility of misidentification.

[0478] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0479] Corresponding to the command word response method described in the above embodiments, Figure 15 A structural block diagram of the command word response device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0480] This device can be applied to control equipment, see reference. Figure 15 The device includes:

[0481] The command word acquisition module 151 is used to acquire the control command words corresponding to the human voice signal; wherein, the human voice signal is collected by the sound pickup device;

[0482] Scene coefficient module 152 is used to obtain the scene coefficients corresponding to the control command words;

[0483] The interaction recognition module 153 is used to determine whether to perform an interaction confirmation operation based on the scene coefficient;

[0484] The interaction execution module 154 is used to issue an interaction confirmation prompt message and / or instruct other devices to issue an interaction confirmation prompt message if it is determined to perform an interaction confirmation operation based on the scenario coefficient.

[0485] The confirmation execution module 155 is used to execute the operation corresponding to the control command word and / or instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word when receiving a confirmation response corresponding to the interactive confirmation prompt information.

[0486] Optionally, the above-mentioned device further includes:

[0487] The denial stop module is used to not execute the operation corresponding to the control command word and not instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word when a negative response corresponding to the interactive confirmation prompt is received, or when no response corresponding to the interactive confirmation prompt is received.

[0488] Optionally, the above-mentioned device further includes:

[0489] The direct response module is used to execute the operation corresponding to the control command word and / or instruct the electronic device corresponding to the control command word to execute the operation corresponding to the control command word if it is determined, based on the scenario coefficient, that no interactive confirmation operation should be performed.

[0490] Optionally, the interaction recognition module 153 is specifically used to determine whether to perform an interaction confirmation operation if the scene coefficient meets the preset scene coefficient threshold condition; otherwise, it determines not to perform an interaction confirmation operation.

[0491] Optionally, the above-mentioned scene coefficients are preset initial scene coefficients, or the scene coefficients are target scene coefficients obtained by adjusting the initial scene coefficients according to scene adjustment parameters, which are determined based on historical usage data of control command words and / or other historical usage data of control command words.

[0492] Optionally, the aforementioned historical usage data includes one or more of the following: historical usage time period, historical usage frequency within a specified time period, and historical usage frequency within a specified time period.

[0493] Optionally, the aforementioned interactive confirmation prompt message includes one or more of the following: playing a prompt audio through a speaker, displaying a prompt message on a screen, flashing an indicator light, or emitting vibration through a vibration motor.

[0494] Optionally, the above-mentioned confirmation response includes one or more of the following: a voice signal containing a confirmation command word with a confirmation meaning, a touch operation on a confirmation control displayed on the screen, or a press operation on a confirmation button.

[0495] Optionally, the above-mentioned device further includes:

[0496] The confidence score acquisition module is used to acquire the lexical confidence score and confidence threshold corresponding to the control command words;

[0497] The confidence verification module is used to perform subsequent steps if the confidence of a word is determined to be greater than the confidence threshold.

[0498] Optionally, the above-mentioned device further includes:

[0499] The confidence adjustment module is used to adjust the confidence level and / or confidence threshold of words according to the confidence adjustment parameters corresponding to the control command words.

[0500] Optionally, the confidence adjustment parameters mentioned above include one or more of the following: initial scenario coefficient, historical usage frequency within a specified time period, time period coefficient, and usage coefficient.

[0501] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0502] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0503] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0504] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0505] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0506] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A command word response method, applied to a control device, characterized in that, The method includes: Acquire control command words corresponding to human voice signals; wherein, the human voice signals are collected by a sound pickup device; Obtain the initial scene coefficients corresponding to the control command words; The initial scene coefficient is adjusted according to the scene adjustment parameters to obtain the target scene coefficient. The scene adjustment parameters are determined based on the historical usage data of the control command words and / or other historical usage data of control command words. The historical usage data includes one or more of the following: historical usage time period, historical usage count within a specified time period, and historical usage frequency within a specified time period. The target scene coefficient is negatively correlated with the usage frequency, time period coefficient, and usage coefficient. Based on the target scene coefficient, determine whether to perform an interactive confirmation operation. When the target scene coefficient is less than or equal to the scene coefficient threshold, the control device does not perform interactive confirmation with the user, while when the target scene coefficient is greater than the scene coefficient threshold, the control device performs interactive confirmation with the user. If the interactive confirmation operation is determined to be performed based on the target scenario coefficient, then an interactive confirmation prompt message is issued and / or other devices are instructed to issue an interactive confirmation prompt message. Upon receiving a confirmation response corresponding to the interactive confirmation prompt, the operation corresponding to the control command word is executed and / or the electronic device corresponding to the control command word is instructed to execute the operation corresponding to the control command word.

2. The method according to claim 1, characterized in that, If it is determined that the interactive confirmation operation will be performed, the method further includes: If a negative response is received corresponding to the interactive confirmation prompt, or if no response is received corresponding to the interactive confirmation prompt, the operation corresponding to the control command word will not be executed, and the electronic device corresponding to the control command word will not be instructed to execute the operation corresponding to the control command word.

3. The method according to claim 2, characterized in that, The method further includes: If, based on the target scenario coefficient, it is determined that the interactive confirmation operation will not be performed, then the operation corresponding to the control command word will be performed and / or the electronic device corresponding to the control command word will be instructed to perform the operation corresponding to the control command word.

4. The method according to any one of claims 1-3, characterized in that, The interactive confirmation prompt message includes one or more of the following: playing a prompt audio through a speaker, displaying a prompt message on a screen, flashing an indicator light, or emitting vibration through a vibration motor.

5. The method according to any one of claims 1-3, characterized in that, The confirmation response includes one or more of the following: a human voice signal containing a confirmation command word, a touch operation on a confirmation control displayed on the screen, or a press operation on a confirmation button.

6. The method according to any one of claims 1-3, characterized in that, Before obtaining the initial scene coefficients corresponding to the control command words, the method further includes: Obtain the lexical confidence score and confidence threshold corresponding to the control command word; If the confidence level of the word is determined to be greater than the confidence threshold, proceed with the next steps.

7. The method according to claim 6, characterized in that, Before performing subsequent steps when the confidence level of the word is determined to be greater than the confidence threshold, the method further includes: Adjust the confidence level of the vocabulary and / or the confidence threshold according to the confidence level adjustment parameter corresponding to the control command word.

8. The method according to claim 7, characterized in that, The confidence adjustment parameters include one or more of the following: initial scenario coefficient, historical usage frequency within a specified time period, time period coefficient, and usage coefficient.

9. A control device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor is configured to implement the method as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium configured to store a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

11. A computer program product, characterized in that, The computer program product is configured to run on a control device, causing the control device to perform the method as described in any one of claims 1 to 8.

12. A chip system, characterized in that, The chip system includes a memory and a processor, the processor being configured to execute a computer program stored in the memory to implement the method as described in any one of claims 1 to 8.

Images