Smart home equipment control method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a smart home equipment control method and device, electronic equipment and a readable storage medium, smart home equipment is connected with front-end voice panels and a home host, and the front-end voice panels are distributed in different areas of a family. The method comprises the following steps: acquiring voice audio data in a corresponding area monitored by a front-end voice panel; processing the voice audio data to obtain a processing result; determining feedback content for the smart home equipment and / or the front-end voice panel according to the processing result; and sending the feedback content to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel execute a corresponding action according to the feedback content. According to the embodiment of the invention, a home host or other equipment with relatively high computing power does not need to be deployed in each region of the home, so that the cost of realizing the whole-house intelligent voice in the home is reduced, and the popularization rate of the whole-house intelligent voice is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of Internet technology, and in particular, to a smart home device control method, a smart home device control device, an electronic device, and a computer-readable storage medium. Background Art

[0002] At present, whole-house smart voice is becoming popular, and accordingly, the use of home voice host panels for realizing whole-house smart voice is becoming popular. The screen sizes of most home voice host panels are 7 inches, 10 inches, 12 inches or even larger, which occupy a large space and have strong hardware capabilities. They support the voice of the entire full link and all categories of equipment (electrical appliances). Most voice functions can be realized on the home voice host panel, and after processing, commands can be sent to control the corresponding smart home devices to realize device control.

[0003] However, since the hardware of the home voice host panel needs to support voice for all links and all categories of devices, the computing power of the device is relatively high and the hardware cost is very high. It is difficult to install one in every room of the home, and the penetration rate is not high. Summary of the invention

[0004] In view of the above problems, a method and apparatus for controlling smart home devices is proposed to overcome the above problems or at least partially solve the above problems. The specific technical solution is as follows:

[0005] The embodiment of the present invention discloses a method for controlling a smart home device, wherein the smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The method includes:

[0006] Acquire voice audio data in the corresponding area monitored by the front-end voice panel;

[0007] Processing the speech audio data to obtain a processing result;

[0008] Determining feedback content for the smart home device and / or the front-end voice panel according to the processing result;

[0009] The feedback content is sent to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel performs corresponding actions according to the feedback content.

[0010] In one embodiment of the present invention, processing the speech audio data to obtain a processing result includes:

[0011] Extracting audio feature information from the speech audio data;

[0012] Determining a processing priority of the voice audio data according to the audio feature information;

[0013] The voice and audio data are processed according to the processing priority corresponding to the voice and audio data to obtain a processing result.

[0014] In one embodiment of the present invention, the audio feature information includes at least a front-end voice panel identifier, a task submission time, sound pressure data, a time weight coefficient, and a sound pressure weight coefficient; determining the processing priority of the voice audio data according to the audio feature information includes:

[0015] Inputting the audio feature information into a preset priority calculation formula to obtain a priority score for each of the voice audio data;

[0016] The processing priority of each of the voice audio data is determined according to the priority score.

[0017] In an embodiment of the present invention, the home host includes a preset instruction set, the instruction set includes control instructions, and the voice audio data is processed to obtain a processing result, including:

[0018] Converting the speech audio data into text data;

[0019] identifying user intent from the text data;

[0020] When there is a control instruction matching the user intention in the preset instruction set, taking the matching control instruction as the target control instruction;

[0021] When there is no control instruction matching the user's intention in the preset instruction set, generating error prompt information for the voice audio data;

[0022] The target control instruction and the error prompt information are used as the processing result of the voice audio data.

[0023] In one embodiment of the present invention, determining the feedback content for the smart home device and / or the front-end voice panel according to the processing result includes:

[0024] When the processing result is the target control instruction, the target control instruction is sent to a corresponding smart home device; the smart home device is used to perform a corresponding action according to the target control instruction;

[0025] Generate correct prompt information for the target control instruction, and send the correct prompt information to the front-end voice panel; the front-end voice panel is used to display or broadcast the correct prompt information.

[0026] In one embodiment of the present invention, determining the feedback content for the smart home device and / or the front-end voice panel according to the processing result includes:

[0027] When the processing result is the error prompt information, the error prompt information is sent to the front-end voice panel; the front-end voice panel is used to display or broadcast the error prompt information.

[0028] In one embodiment of the present invention, the front-end voice panel is used to monitor the environmental audio data of the corresponding area, and when it is detected that the environmental audio data includes wake-up audio data, the front-end voice panel is woken up.

[0029] In one embodiment of the present invention, the front-end voice panel is also used to display or broadcast a prompt message of successful wake-up upon successful wake-up, or to display or broadcast a prompt message of failed wake-up upon failed wake-up.

[0030] In one embodiment of the present invention, when the front-end voice panel is successfully awakened, the front-end voice panel is used to send the voice audio data to the home host when the voice audio data is detected within a preset time period, or display or broadcast a prompt message of monitoring failure when the voice audio data is not detected within the preset time period.

[0031] The embodiment of the present invention further discloses a smart home device control device, wherein the smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The device comprises:

[0032] A monitoring module, used to obtain voice audio data in a corresponding area monitored by the front-end voice panel;

[0033] A processing module, used for processing the voice audio data to obtain a processing result;

[0034] A feedback module, used to determine feedback content for the smart home device and / or the front-end voice panel according to the processing result;

[0035] An execution module is used to send the feedback content to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel performs corresponding actions according to the feedback content.

[0036] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0037] The memory is used to store computer programs;

[0038] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0039] The embodiment of the present invention further discloses a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the embodiment of the present invention.

[0040] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0041] The embodiments of the present invention include the following advantages:

[0042] In an embodiment of the present invention, a smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The front-end voice panel monitors the voice and audio data in the corresponding area and submits the voice and audio data to the home host. The home host processes the voice and audio data to obtain a processing result, and determines and sends the feedback content for the smart home device and / or the front-end voice panel according to the processing result. After receiving the feedback content, the smart home device and / or the front-end voice panel perform corresponding actions according to the feedback content, thereby realizing the control of the smart home device. The front-end voice panel of the embodiment of the present invention can be distributed in different areas of the home, so as to monitor the voice and audio data of the corresponding area and submit it to the home host. The computing power of the device of the home host is higher than that of the front-end voice panel, so the voice and audio data can be quickly processed to obtain the processing result, and then the feedback information corresponding to the processing result is sent to the smart home device and / or the front-end voice panel to perform the corresponding action. Since it is not necessary to deploy a home host or other devices with higher computing power in each area of ​​the home, the cost of realizing whole-house smart voice in the home is reduced, and the popularity of whole-house smart voice is increased. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of a method for controlling a smart home device provided in an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of a deployment structure provided in an embodiment of the present invention;

[0045] Figure 3 is a schematic diagram of a hardware structure provided in an embodiment of the present invention;

[0046] Figure 4 It is a schematic diagram of an operation flow provided in an embodiment of the present invention;

[0047] Figure 5 is a structural block diagram of a smart home device control device provided in an embodiment of the present invention;

[0048] Figure 6 It is a schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Reference Figure 1 , shows a flowchart of a method for controlling a smart home device provided in an embodiment of the present invention, wherein the smart home device is connected to a front-end voice panel and a home host, and the front-end voice panels are distributed in different areas of the home. The method may specifically include the following steps:

[0051] Step 11, obtaining voice audio data in the corresponding area monitored by the front-end voice panel;

[0052] Reference Figure 2 , is a schematic diagram of a deployment structure provided in an embodiment of the present invention. Smart home devices, front-end voice panels (panel 1, panel 2, panel 3, panel 4, panel 5 and panel 6) and a home host (central host / host) can be deployed in a home. The home host is a device used to centrally control the smart home devices in the home. The device computing power of the home host is greater than the device computing power of the smart home devices and the front-end voice panel. Therefore, the home host can support voice of full-link and full-category devices.

[0053] Specifically, smart home devices may include but are not limited to lighting equipment, dehumidifiers, switches, sockets, sensor devices, smoke alarms, smart door locks, and security cameras, etc., and can communicate data with the home host and the front-end voice panel through Wi-Fi (Wireless Fidelity) communication protocol, Zigbee (a wireless communication protocol for low-speed short-distance transmission) communication protocol, Bluetooth communication protocol, NFC (Near Field Communication) communication protocol and other communication protocols. The front-end voice panel can be deployed in various areas of the home, such as small voice panels (such as 86 boxes or mobile phones, tablet computers and other terminal devices) in areas such as the hall, dining room, kitchen, children's room, study, bedroom and corridor. The front-end voice panel can process less voice and audio data, support simple instruction sets, and do not support semantic understanding. The broadcast content generated according to the voice and audio data is very simple. If the voice and audio data of the user collected by the front-end voice panel is processed only, the user's experience is poor. Therefore, the front-end voice panel of the embodiment of the present invention can submit the voice and audio data in the corresponding area obtained by monitoring to the home host for processing.

[0054] Reference Figure 3 , is a schematic diagram of a hardware structure provided in an embodiment of the present invention. The home host may include a core processor, a communication module and a memory. The core processor may implement ASR (Automatic Speech Recognition), NLP (Natural Language Processing), TTS (Text-To-Speech), control and other processing of voice and audio data. The communication module may be used to communicate with the front-end voice panel and smart home devices. The memory may store a preset instruction set. The front-end voice panel may include components such as a light / screen, a microphone and a speaker, as well as a communication module and an MCU (Microcontroller Unit) processor. The light / screen may be used to display or prompt the user. The microphone may collect sound data such as voice and audio data. The speaker may play audio.

[0055] Step 12: Process the speech audio data to obtain a processing result.

[0056] Step 13, determining feedback content for the smart home device and / or the front-end voice panel according to the processing result;

[0057] In the embodiment of the present invention, the home host is a device for centrally controlling smart home devices in the home, which can connect, manage, coordinate and communicate with the connected smart home devices and the front-end voice panel, and can also realize the visual control service of the smart home devices. The home host can use the powerful device computing power to perform ASR, NLP, TTS and other processing on the voice and audio data submitted by the front-end voice panel to obtain the processing results, and can also generate feedback content for the front-end voice panel or smart home devices according to the processing results. Among them, the processing results may include finding control instructions that match the voice and audio data, not finding control instructions that match the voice and audio data, etc., and the feedback content may include but is not limited to control instructions for smart home devices, correct prompt information and error prompt information, etc.

[0058] Step 14: Send the feedback content to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel perform corresponding actions according to the feedback content.

[0059] In an embodiment of the present invention, the home host obtains a processing result based on processing the voice audio data, and determines the feedback content for the smart home device and / or the front-end voice panel based on the processing result, and then sends the feedback content to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel can perform corresponding actions according to the feedback content, such as broadcasting corresponding prompt information or executing corresponding control instructions.

[0060] In an embodiment of the present invention, a smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The front-end voice panel monitors the voice and audio data in the corresponding area and submits the voice and audio data to the home host. The home host processes the voice and audio data to obtain a processing result, and determines and sends the feedback content for the smart home device and / or the front-end voice panel according to the processing result. After receiving the feedback content, the smart home device and / or the front-end voice panel perform corresponding actions according to the feedback content, thereby realizing the control of the smart home device. The front-end voice panel of the embodiment of the present invention can be distributed in different areas of the home, so as to monitor the voice and audio data of the corresponding area and submit it to the home host. The computing power of the device of the home host is higher than that of the front-end voice panel, so the voice and audio data can be quickly processed to obtain the processing result, and then the feedback information corresponding to the processing result is sent to the smart home device and / or the front-end voice panel to perform the corresponding action. Since it is not necessary to deploy a home host or other devices with higher computing power in each area of ​​the home, the cost of realizing whole-house smart voice in the home is reduced, and the popularity of whole-house smart voice is increased.

[0061] In an embodiment of the present invention, processing the speech audio data to obtain a processing result may include:

[0062] Extracting audio feature information from the speech audio data;

[0063] Determining a processing priority of the voice audio data according to the audio feature information;

[0064] The voice and audio data are processed according to the processing priority corresponding to the voice and audio data to obtain a processing result.

[0065] The audio feature information includes at least information such as the front-end voice panel identification, task submission time, sound pressure data, time weight coefficient and sound pressure weight coefficient.

[0066] In an embodiment of the present invention, the home host can extract audio feature information from the voice audio data, and then calculate the processing priority corresponding to each voice audio data based on the extracted audio feature information, and then queue the voice audio data for processing based on the processing priority, ensuring that voice audio data with a long submission time and a higher user intention can be processed first, and obtaining the processing results corresponding to the voice audio data. By dynamically adjusting the processing order according to the characteristics of the voice audio data, it is ensured that important voice audio data can be processed in a timely manner, ensuring the user's control experience of smart home devices.

[0067] In an embodiment of the present invention, determining the processing priority of the speech audio data according to the audio feature information may include:

[0068] Inputting the audio feature information into a preset priority calculation formula to obtain a priority score for each of the voice audio data;

[0069] The processing priority of each of the voice audio data is determined according to the priority score.

[0070] In an embodiment of the present invention, when the home host receives multiple voice audio data, it needs to respond quickly considering the user's requirement for real-time voice reply. Therefore, the home host queues all the voice audio data submitted by the front-end voice panel and processes each one according to the processing priority and task characteristics corresponding to the voice audio data.

[0071] Specifically, the home host can input the audio feature information extracted from the voice audio data into a preset priority calculation formula to obtain the priority score of each voice audio data, and then determine the processing priority of each voice audio data according to the size of the priority score. For example, voice audio data with a high priority score is processed first, or the range of the priority score determines the priority range interval of the voice audio data, and then the processing priority within the priority range interval can be further determined according to the task submission time or the user's personal setting preferences.

[0072] In an example of the present invention, the processing priority calculation formula may include:

[0073] Priority(i)=W time ·(-Ti)+W volume Volume i

[0074] Among them, Priority(i) represents the processing priority; i represents the front-end voice panel identifier (ID); Ti represents the task submission time (unit: second); Volumei represents the sound pressure data (unit: decibel, dB); Wtime represents the time weight coefficient (default value: 1); Wvolume represents the sound pressure weight coefficient (default value: 0.5, indicating that the impact of sound pressure is less than time).

[0075] In an embodiment of the present invention, the home host includes a preset instruction set, the instruction set includes control instructions, and the voice audio data is processed to obtain a processing result, including:

[0076] Converting the speech audio data into text data;

[0077] identifying user intent from the text data;

[0078] When there is a control instruction matching the user intention in the preset instruction set, taking the matching control instruction as the target control instruction;

[0079] When there is no control instruction matching the user's intention in the preset instruction set, generating error prompt information for the voice audio data;

[0080] The target control instruction and the error prompt information are used as the processing result of the voice audio data.

[0081] In a specific implementation, the home host is preset with an instruction set, which includes a variety of control instructions supported by smart home devices. The control instructions can be used to control smart home devices to turn on, off, set parameters, etc. For example, the control instruction can be to instruct the air conditioner to control the indoor temperature at around 20 degrees.

[0082] ASR (Automatic Speech Recognition), NLP (Natural Language Processing), TTS (Text-To-Speech)

[0083] In an embodiment of the present invention, first, the home host may use ASR (speech recognition) to convert voice audio data into text data, and then use NLP (natural language processing) to identify user intent from the text data. Next, it may search in the preset instruction set whether there is a control instruction that matches the user intent. For example, assuming that the user intends to open the curtains, if the preset instruction set includes a control instruction for opening the curtains, the control instruction for opening the curtains may be used as a target control instruction that matches the user intent. If the preset instruction set does not include a control instruction for opening the curtains, it may be determined that the voice audio data is incorrect (for example, the smart home device does not include curtains, the user's pronunciation is incorrect, or the curtains are restricted to an action that cannot be opened, etc.), and an error prompt message may be generated for the voice audio data. The target control instruction and the error prompt message may be used as the processing result of the voice audio data by the home host. Based on the processing result, feedback content may be further generated and submitted to the corresponding smart home device or the front-end voice panel.

[0084] In an embodiment of the present invention, determining the feedback content for the smart home device and / or the front-end voice panel according to the processing result may include:

[0085] When the processing result is the target control instruction, the target control instruction is sent to a corresponding smart home device; the smart home device is used to perform a corresponding action according to the target control instruction;

[0086] Generate correct prompt information for the target control instruction, and send the correct prompt information to the front-end voice panel; the front-end voice panel is used to display or broadcast the correct prompt information.

[0087] In an embodiment of the present invention, if the result of the home host processing the voice audio data is a target control instruction, the target control instruction can be sent to the corresponding smart home device, so that the corresponding smart home device can perform the corresponding action according to the target control instruction, thereby realizing the user's precise control of the smart home device. Furthermore, the home host can also generate correct prompt information for the target control instruction, and send the correct prompt information to the front-end voice panel, which is used to display the correct prompt information through the screen, or broadcast the correct prompt information through the speaker. In addition, the correct prompt information can also be displayed through a light. For example, if it is a correct prompt information, a green light can be turned on to remind the user that the control of the smart home device has been achieved.

[0088] In an embodiment of the present invention, determining the feedback content for the smart home device and / or the front-end voice panel according to the processing result may include:

[0089] When the processing result is the error prompt information, the error prompt information is sent to the front-end voice panel; the front-end voice panel is used to display or broadcast the error prompt information.

[0090] In an embodiment of the present invention, if the processing result of the home host on the voice audio data is an error prompt message, the error prompt message can be sent to the front-end voice panel, and the front-end voice panel is used to display the error prompt message through the screen, or to broadcast the error prompt message through a speaker. In addition, the error prompt message can also be displayed through a light. For example, if it is an error prompt message, a red light can be turned on to remind the user that the corresponding voice needs to be re-issued to collect the corresponding voice audio data for controlling the smart home device.

[0091] In one embodiment of the present invention, the front-end voice panel is used to monitor the environmental audio data of the corresponding area, and when it is detected that the environmental audio data includes wake-up audio data, the front-end voice panel is woken up.

[0092] In an embodiment of the present invention, multiple front-end voice panels (terminal devices) start voice services and can monitor the environmental audio data of the corresponding areas. When it is detected that the user sends relevant wake-up audio data, such as "GL", it can be determined that the wake-up audio data is detected. At this time, the front-end voice panel can be woken up, and the user's voice audio data can be monitored and then submitted to the home host for processing.

[0093] In one embodiment of the present invention, the front-end voice panel is also used to display or broadcast a prompt message of successful wake-up upon successful wake-up, or to display or broadcast a prompt message of failed wake-up upon failed wake-up.

[0094] Furthermore, in an embodiment of the present invention, when the MCU processor of the front-end voice panel detects wake-up audio data and successfully wakes up the front-end voice panel, it can remind the user through the prompt light, screen, speaker, etc. of the front-end voice panel, so that the user can confirm that the front-end voice panel has been successfully awakened. Of course, when the MCU processor of the front-end voice panel detects ambient audio data but does not monitor wake-up audio data, it can remind the user through the prompt light, screen, speaker, etc. of the front-end voice panel, so that the user can confirm that the front-end voice panel has failed to wake up.

[0095] In one embodiment of the present invention, when the front-end voice panel is successfully awakened, the front-end voice panel is used to send the voice audio data to the home host when the voice audio data is detected within a preset time period, or display or broadcast a prompt message of monitoring failure when the voice audio data is not detected within the preset time period.

[0096] In an embodiment of the present invention, when the front-end voice panel is successfully awakened, it can monitor whether there is voice audio data issued by the user within a preset time period. For example, it can monitor whether there is voice audio data issued by the user within 1 minute. If the voice audio data issued by the user is monitored within 1 minute, the voice audio data can be submitted to the home host for processing. If the voice audio data issued by the user is not monitored within 1 minute, the user can be reminded through the prompt light, screen, speaker, etc. of the front-end voice panel, and a prompt message of monitoring failure can be displayed to remind the user to re-wake up the front-end voice panel or re-send the voice audio data.

[0097] The embodiment of the present invention allows the MCU processor of the terminal device to process simple wake-up functions and feedback information functions (lights, interface displays, voice broadcasts). The terminal device is installed on a small panel with low cost and multiple panels can be installed. The MCU processor of the terminal device sends the collected voice and audio data to the local home host to process the full-link voice. The home host then sends the feedback content generated according to the voice and audio data to the terminal device, or sends it to the smart home device for device control.

[0098] In order to enable those skilled in the art to better understand the embodiments of the present invention, a specific example is used below for illustration. Figure 4 , is a schematic diagram of an operation flow provided in an embodiment of the present invention, and the specific process may include:

[0099] Step S101, multiple front-end voice panels (terminal devices / voice panels) start voice services and monitor environmental audio data. When the environmental audio data includes relevant wake-up audio data sent by the user, the MCU processor of the front-end voice panel determines that it is wake-up audio data, and feeds back to the user through the prompt light, screen, speaker broadcast voice of the front-end voice panel, etc., so that the user can confirm that the front-end voice panel wakes up successfully. Of course, if the front-end voice panel fails to wake up, a prompt message of wake-up failure is returned;

[0100] Step S102: After the front-end voice panel is successfully awakened, the microphone of the front-end voice panel will record the voice audio data (voice command) for a preset time. If the voice audio data is not collected, a prompt message of failure can be fed back through the light, screen, and speaker; if the voice audio data is successfully collected, the collected voice audio data is processed into digital audio data (i.e., voice audio data that can be quickly understood by the home host) through the terminal device MCU, and sent to the central host (home host) through the communication module;

[0101] Step S103, the home host connects to the voice and audio data of all front-end voice panels. Due to the real-time requirement of voice reply, the home host needs to respond quickly. The home host needs to queue the voice and audio data of all front-end voice panels and process them in sequence according to processing priority and task characteristics.

[0102] Step S104: The processor of the home host performs automatic speech recognition and natural language processing on the voice audio data to obtain a processing result, and then makes a judgment. If it is judged that the voice audio data has a corresponding correct control instruction, the corresponding control instruction can be sent to the smart home device through the communication module to realize the control of the smart home device.

[0103] Step S105: If it is determined that the voice audio data has the corresponding correct control instruction, the correct prompt information can be sent back to the terminal device (front-end voice panel), or the correct prompt information can be synthesized into a broadcast content by the processor, and then the broadcast content is sent to the terminal device through the communication module. If it is determined that the voice audio data does not have the corresponding control instruction, the operation can be stopped, and the error prompt information can be sent back to the terminal device, or the error prompt information can be synthesized into a broadcast content by the processor, and then sent to the terminal device through the communication module;

[0104] In addition, the Ajitai host may also need to perform a waiting timeout check. If the queue of voice audio data is too long, it should provide feedback to the user for more than a certain time (such as 2 seconds). The feedback may be a voice report, such as "Please wait" or a screen display or prompt light information;

[0105] Step S106: After receiving the success prompt information and the failure prompt information, the terminal device feeds back the success prompt information and the failure prompt information to the user through the light, screen, and speaker.

[0106] The embodiment of the present invention splits the entire voice link into a front-end voice panel and a home host. The front-end voice panel mainly processes wake-up. The successful wake-up can collect voice audio data and send it to the home host for processing. The powerful device computing power of the home host performs ASR (speech recognition), NLP (natural language processing) and other processing to obtain feedback content, wherein the feedback content can be synthesized into synthesized speech through TTS (speech synthesis), and then the feedback content is returned to the front-end voice panel for display or broadcast, and the matching control instructions are sent to the corresponding smart home devices to realize device control.

[0107] The application of the embodiment of the present invention enables all front-end voice panels in the same network in the home to have full-link voice functions, complete voice instruction sets, better experience, and low cost for installing multiple front-end voice panels. In addition, the front-end voice panel and the home host in the embodiment of the present invention are devices located in the same home network, so the network is stable, and the user's voice can be collected and processed throughout the process in the same home, ensuring personal privacy and security.

[0108] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0109] Reference Figure 5 , shows a structural block diagram of a smart home device control device provided in an embodiment of the present invention, wherein the smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The device may specifically include the following modules:

[0110] Monitoring module 501, used to obtain voice audio data in the corresponding area monitored by the front-end voice panel;

[0111] The processing module 502 is used to process the speech audio data to obtain a processing result;

[0112] A feedback module 503 is used to determine feedback content for the smart home device and / or the front-end voice panel according to the processing result;

[0113] The execution module 504 is used to send the feedback content to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel performs corresponding actions according to the feedback content.

[0114] In one embodiment of the present invention, the processing module 502 is used to:

[0115] Extracting audio feature information from the speech audio data;

[0116] Determining a processing priority of the voice audio data according to the audio feature information;

[0117] The voice and audio data are processed according to the processing priority corresponding to the voice and audio data to obtain a processing result.

[0118] In one embodiment of the present invention, the audio feature information includes at least a front-end voice panel identifier, a task submission time, sound pressure data, a time weight coefficient, and a sound pressure weight coefficient; the processing module 502 is used to:

[0119] Inputting the audio feature information into a preset priority calculation formula to obtain a priority score for each of the voice audio data;

[0120] The processing priority of each of the voice audio data is determined according to the priority score.

[0121] In an embodiment of the present invention, the home host includes a preset instruction set, the instruction set includes control instructions, and the processing module 502 is used to:

[0122] Converting the speech audio data into text data;

[0123] identifying user intent from the text data;

[0124] When there is a control instruction matching the user intention in the preset instruction set, taking the matching control instruction as the target control instruction;

[0125] When there is no control instruction matching the user's intention in the preset instruction set, generating error prompt information for the voice audio data;

[0126] The target control instruction and the error prompt information are used as the processing result of the voice audio data.

[0127] In one embodiment of the present invention, the feedback module 503 is used to:

[0128] When the processing result is the target control instruction, the target control instruction is sent to a corresponding smart home device; the smart home device is used to perform a corresponding action according to the target control instruction;

[0129] Generate correct prompt information for the target control instruction, and send the correct prompt information to the front-end voice panel; the front-end voice panel is used to display or broadcast the correct prompt information.

[0130] In one embodiment of the present invention, the feedback module 503 is used to:

[0131] When the processing result is the error prompt information, the error prompt information is sent to the front-end voice panel; the front-end voice panel is used to display or broadcast the error prompt information.

[0132] In one embodiment of the present invention, the front-end voice panel is used to monitor the environmental audio data of the corresponding area, and when it is detected that the environmental audio data includes wake-up audio data, the front-end voice panel is woken up.

[0133] In one embodiment of the present invention, the front-end voice panel is also used to display or broadcast a prompt message of successful wake-up upon successful wake-up, or to display or broadcast a prompt message of failed wake-up upon failed wake-up.

[0134] In one embodiment of the present invention, when the front-end voice panel is successfully awakened, the front-end voice panel is used to send the voice audio data to the home host when the voice audio data is detected within a preset time period, or display or broadcast a prompt message of monitoring failure when the voice audio data is not detected within the preset time period.

[0135] In an embodiment of the present invention, a smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The front-end voice panel monitors the voice and audio data in the corresponding area and submits the voice and audio data to the home host. The home host processes the voice and audio data to obtain a processing result, and determines and sends the feedback content for the smart home device and / or the front-end voice panel according to the processing result. After receiving the feedback content, the smart home device and / or the front-end voice panel perform corresponding actions according to the feedback content, thereby realizing the control of the smart home device. The front-end voice panel of the embodiment of the present invention can be distributed in different areas of the home, so as to monitor the voice and audio data of the corresponding area and submit it to the home host. The computing power of the device of the home host is higher than that of the front-end voice panel, so the voice and audio data can be quickly processed to obtain the processing result, and then the feedback information corresponding to the processing result is sent to the smart home device and / or the front-end voice panel to perform the corresponding action. Since it is not necessary to deploy a home host or other devices with higher computing power in each area of ​​the home, the cost of realizing whole-house smart voice in the home is reduced, and the popularity of whole-house smart voice is increased.

[0136] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0137] It should be noted that the embodiments of the present invention may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0138] In addition, an embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned smart home device control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0139] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned smart home device control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0140] An embodiment of the present invention also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned smart home device control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0141] Figure 6 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0142] The electronic device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, a processor 610, and a power supply 611. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0143] It should be understood that in the embodiment of the present invention, the radio frequency unit 601 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 610 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 601 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 601 can also communicate with the network and other devices through a wireless communication system.

[0144] The electronic device provides users with wireless broadband Internet access through the network module 602, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0145] The audio output unit 603 can convert the audio data received by the RF unit 601 or the network module 602 or stored in the memory 609 into an audio signal and output it as sound. Moreover, the audio output unit 603 can also provide audio output related to a specific function performed by the electronic device 600 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 603 includes a speaker, a buzzer, a receiver, etc.

[0146] The input unit 604 is used to receive audio or video signals. The input unit 604 may include a graphics processor (GPU) 6041 and a microphone 6042, and the graphics processor 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 606. The image frame processed by the graphics processor 6041 can be stored in the memory 609 (or other storage medium) or sent via the radio frequency unit 601 or the network module 602. The microphone 6042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 601 in the case of a telephone call mode.

[0147] The electronic device 600 also includes at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 6061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 6061 and / or the backlight when the electronic device 600 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 605 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0148] The display unit 606 is used to display information input by the user or information provided to the user. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0149] The user input unit 607 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 607 includes a touch panel 6071 and other input devices 6072. The touch panel 6071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 6071 or near the touch panel 6071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 6071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 610, receives the command sent by the processor 610 and executes it. In addition, the touch panel 6071 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch panel 6071, the user input unit 607 may also include other input devices 6072. Specifically, other input devices 6072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0150] Furthermore, the touch panel 6071 may be overlaid on the display panel 6061. When the touch panel 6071 detects a touch operation on or near it, it is transmitted to the processor 610 to determine the type of touch event. Then, the processor 610 provides corresponding visual output on the display panel 6061 according to the type of touch event. Figure 6 In the figure, the touch panel 6071 and the display panel 6061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 6071 and the display panel 6061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0151] The interface unit 608 is an interface for connecting an external device to the electronic device 600. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 608 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 600 or may be used to transmit data between the electronic device 600 and an external device.

[0152] The memory 609 can be used to store software programs and various data. The memory 609 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 609 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0153] The processor 610 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 609 and calling data stored in the memory 609, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 610.

[0154] The electronic device 600 may also include a power supply 611 (such as a battery) for supplying power to each component. Preferably, the power supply 611 may be logically connected to the processor 610 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0155] In addition, the electronic device 600 includes some functional modules not shown, which will not be described in detail here.

[0156] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0158] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0159] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0160] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0161] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0164] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0165] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A smart home device control method, characterized in that: The smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The method includes: Acquire voice audio data in the corresponding area monitored by the front-end voice panel; Processing the speech audio data to obtain a processing result; Determining feedback content for the smart home device and / or the front-end voice panel according to the processing result; The feedback content is sent to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel performs corresponding actions according to the feedback content.

2. The method according to claim 1, characterized in that Processing the speech audio data to obtain a processing result includes: Extracting audio feature information from the speech audio data; Determining a processing priority of the voice audio data according to the audio feature information; The voice and audio data are processed according to the processing priority corresponding to the voice and audio data to obtain a processing result.

3. The method according to claim 2, characterized in that The audio feature information includes at least the front-end voice panel identification, task submission time, sound pressure data, time weight coefficient and sound pressure weight coefficient; Determining the processing priority of the voice audio data according to the audio feature information includes: Inputting the audio feature information into a preset priority calculation formula to obtain a priority score for each of the voice audio data; The processing priority of each of the voice audio data is determined according to the priority score.

4. The method according to claim 1, characterized in that: The home host includes a preset instruction set, the instruction set includes control instructions, and the voice audio data is processed to obtain a processing result, including: Converting the speech audio data into text data; identifying user intent from the text data; When there is a control instruction matching the user intention in the preset instruction set, taking the matching control instruction as the target control instruction; When there is no control instruction matching the user's intention in the preset instruction set, generating error prompt information for the voice audio data; The target control instruction and the error prompt information are used as the processing result of the voice audio data.

5. The method according to claim 4, characterized in that Determining feedback content for the smart home device and / or the front-end voice panel according to the processing result includes: When the processing result is the target control instruction, the target control instruction is sent to a corresponding smart home device; the smart home device is used to perform a corresponding action according to the target control instruction; Generate correct prompt information for the target control instruction, and send the correct prompt information to the front-end voice panel; the front-end voice panel is used to display or broadcast the correct prompt information.

6. The method according to claim 5, characterized in that Determining feedback content for the smart home device and / or the front-end voice panel according to the processing result includes: When the processing result is the error prompt information, the error prompt information is sent to the front-end voice panel; the front-end voice panel is used to display or broadcast the error prompt information.

7. The method according to claim 1, characterized in that The front-end voice panel is used to monitor the environmental audio data of the corresponding area, and when it is detected that the environmental audio data includes wake-up audio data, the front-end voice panel is woken up.

8. The method according to claim 7, characterized in that The front-end voice panel is also used to display or broadcast a prompt message of successful wake-up when the wake-up is successful, or to display or broadcast a prompt message of failed wake-up when the wake-up fails.

9. The method according to claim 8, characterized in that When the front-end voice panel is successfully awakened, the front-end voice panel is used to send the voice audio data to the home host when the voice audio data is detected within a preset time period, or display or broadcast a prompt message of monitoring failure when the voice audio data is not detected within the preset time period.

10. A smart home equipment control device, characterized in that: The smart home device is connected to a front-end voice panel and a home host, and the front-end voice panel is distributed in different areas of the home. The device includes: A monitoring module, used to obtain voice audio data in a corresponding area monitored by the front-end voice panel; A processing module, used for processing the voice audio data to obtain a processing result; A feedback module, used to determine feedback content for the smart home device and / or the front-end voice panel according to the processing result; An execution module is used to send the feedback content to the smart home device and / or the front-end voice panel, so that the smart home device and / or the front-end voice panel performs corresponding actions according to the feedback content.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 9 when executing the program stored in the memory.

12. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 9.

Images