Smart home voice interaction control method and system

Abstract

The invention discloses a smart home voice interaction control method and system, and relates to the technical field of smart home. The problems that in the prior art, the intention of a user cannot be accurately understood through non-voice information such as emotion and expression of the user, a control strategy is solidified and lacks individuation, and user requirements and emotion cannot be met in real time are solved. By fusing voice and non-voice information, accurate recognition of user intentions is enhanced, recognition accuracy of voice instructions is improved, emotional states and requirements of users are understood more comprehensively, particularly in a noisy environment or a complex situation, recognition and response effects of the system are obviously improved, and user experience is improved. By utilizing the emotion analysis and context understanding technology of the user, the interaction mode is dynamically adjusted according to the emotion state of the user, the emotion demands of different users can be flexibly adapted, more humanized services are provided, the control strategy is continuously optimized according to the feedback of the user, the user satisfaction is improved, and the user experience is improved. Therefore, the intelligent degree and the user experience of the smart home system are improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a smart home voice interaction control method and system. Background Art

[0002] Existing smart home voice interaction systems often have problems such as response delay, low voice recognition accuracy, and inability to fully understand the user's emotional state, especially in noisy environments, where voice recognition accuracy is easily reduced. For example, a Chinese patent application with publication number CN116758914A discloses a smart home voice interaction control method, which is applied to a central control gateway and is characterized in that it includes: receiving text information sent by a voice panel, the text information is obtained by converting the voice information collected by the voice panel; performing semantic analysis on the received text information to determine the semantic content; controlling the smart home device to perform corresponding operations according to the semantic content, and replying the corresponding text information to the voice panel, and playing the corresponding voice content through the voice panel.

[0003] Although the above patent uses a combination of a central control gateway and a voice panel to replace multiple gateways with screens, saving costs, the voice interaction method often cannot accurately understand the user's intentions through non-voice information such as emotions and expressions. In addition, the device control strategy of the existing system is relatively fixed, lacks personalized adjustment and optimization, and cannot make effective feedback and adjustments based on the user's real-time needs and emotional state. Summary of the invention

[0004] The purpose of the present invention is to provide a smart home voice interaction control method and system, which improves the recognition accuracy of voice commands by integrating voice and non-voice information, dynamically adjusts the interaction mode according to the user's emotional state, flexibly adapts to the emotional needs of different users, provides more humane services, and continuously optimizes control strategies based on user feedback to improve user satisfaction, so as to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: The smart home voice interaction control method comprises the following steps: Step 1: Voice input collection and preprocessing: The user's voice input signal is collected through an intelligent voice collection device and the voice input signal is preprocessed; Step 2: Recognition and analysis of voice commands: Perform text recognition on the pre-processed voice input signal, convert the recognition result into text commands, and perform semantic analysis on the text commands to determine the user's intention and generate user commands; Step 3: Generation of device control strategy: Match the corresponding device control strategy according to the generated user instructions, determine the user needs and the operation logic of the target device, and adjust the device control strategy according to the user's historical operation habits and preferences; Step 4: Execution and real-time monitoring of equipment operations: Establish a communication link with the target equipment according to the equipment control strategy, send operation instructions to the target equipment, and monitor the response of the equipment in real time; Step 5: Feedback and continuous optimization of operation results: After the device operation is completed, the execution results will be fed back to the user through voice, and the user operation data will be recorded.

[0006] Furthermore, the step 1: collecting and preprocessing the voice input also includes: Based on the matching of the acquired voice input signal with the preset wake-up words and phrases, triggering the wake-up monitoring data collection device; When collecting voice input signals, the user's non-voice information is collected based on the monitoring data collection device to obtain user monitoring data; Identify the user's facial expressions and hand movements based on the acquired user monitoring data, analyze the data features of non-voice information, and understand the user's emotional state and intentions; At the same time, the voice input signal and non-voice information are synchronized based on the timestamp, an association relationship between the voice input signal and the non-voice information is established, and the semantics of the voice input signal is supplemented based on the non-voice information.

[0007] Furthermore, the step 2: recognition and analysis of voice commands specifically includes: Effective speech extraction: After collecting the speech input signal, the endpoint detection technology is used to remove the invalid silent parts before and after the speech, retain the effective speech information, and split the long speech input signal into multiple logical segments; Voice quality assessment: monitor and evaluate the quality of voice input signals in real time. If the voice quality does not meet the standards, optimize the parameters or prompt the user to re-enter the voice. At the same time, dynamically adjust the microphone acquisition parameters according to environmental factors. Text instruction generation: Perform speech recognition processing on qualified speech input signals, convert the recognition results into text information, and correct the noise, accent or ambiguous pronunciation in the speech input signals; Semantic parsing: Perform word segmentation on the converted text information, split the text instructions into words or phrases, identify the part of speech of each word based on the semantics of the supplementary voice input signal, determine the grammatical relationship between words, determine the user's operation intention based on the parsing results, and identify key entities in the text; Contextual understanding: After each round of conversation, the system records the relevant information and status of the user's input, understands the context of continuous voice commands, maintains the continuity of the conversation, and infers the user's further needs based on the previous commands and operation results; User instruction generation: Generate structured user instructions based on the recognition results of semantic parsing and the inference results of context understanding.

[0008] Furthermore, the quality of the voice input signal is monitored and evaluated in real time, including: Real-time monitoring of signal characteristic parameters of the voice input signal, wherein the signal characteristic parameters include a signal-to-noise ratio, a signal difference ratio corresponding to a distortion, a jitter parameter, and a drift parameter; Obtaining a first speech quality assessment coefficient using a signal difference ratio corresponding to the signal-to-noise ratio and the distortion; The first speech quality assessment coefficient is obtained by the following formula:

[0009] Among them, Q 01 represents the first speech quality assessment coefficient; n represents the number of unit times contained in the speech input signal; S i+1 represents the signal-to-noise ratio of the speech input signal corresponding to the i+1th unit time; S i represents the signal-to-noise ratio of the speech input signal corresponding to the i-th unit time; P i represents the signal difference ratio corresponding to the distortion of the speech input signal corresponding to the i-th unit time; P b S represents the standard deviation of the signal difference ratio corresponding to the distortion corresponding to n unit time; b Represents the standard deviation of the signal-to-noise ratio of the speech input signal corresponding to n unit time; Acquire a second voice quality assessment coefficient using the jitter parameter and the drift parameter; The second voice quality assessment coefficient is obtained by the following formula:

[0010] Among them, Q 02 represents the second speech quality assessment coefficient; n represents the number of unit times contained in the speech input signal; J i and Y i J represents the jitter parameter and drift parameter of the speech input signal corresponding to the i-th unit time; i+1 and Y i+1 represents the jitter parameter and drift parameter of the speech input signal corresponding to the i+1th unit time; J b and Y bIndicates the jitter parameter standard deviation and drift parameter standard deviation of the speech input signal corresponding to n unit time; The quality of the speech input signal is determined by using the first speech quality assessment coefficient and the second speech quality assessment coefficient.

[0011] Further, using the first speech quality assessment coefficient and the second speech quality assessment coefficient to determine the quality of the speech input signal includes: Using a preset first evaluation coefficient reference value and a preset second evaluation coefficient reference value to respectively perform normalization processing on the first speech quality evaluation coefficient and the second speech quality evaluation coefficient, and obtain the normalized first speech quality evaluation coefficient and the second speech quality evaluation coefficient; Comparing the normalized first speech quality assessment coefficient with the second speech quality assessment coefficient; When the first speech quality assessment coefficient is lower than the second speech quality assessment coefficient, the first comprehensive assessment model is retrieved; when the first speech quality assessment coefficient is higher than or equal to the second speech quality assessment coefficient, the second comprehensive assessment model is retrieved; Using the first comprehensive evaluation model in combination with the first speech quality evaluation coefficient and the second speech quality evaluation coefficient to obtain a first comprehensive evaluation coefficient, and comparing the first comprehensive evaluation coefficient with a preset first comprehensive coefficient threshold; when the first comprehensive evaluation coefficient exceeds the preset first comprehensive coefficient threshold, determining that the speech quality does not meet the standard; The first comprehensive evaluation coefficient is obtained by the following formula:

[0012] Among them, Z 01 Indicates the first comprehensive evaluation coefficient; Q x01 represents the first speech quality assessment coefficient after normalization; Q x02 represents the second speech quality assessment coefficient after normalization; Using the second comprehensive evaluation model in combination with the first speech quality evaluation coefficient and the second speech quality evaluation coefficient to obtain a second comprehensive evaluation coefficient, and comparing the second comprehensive evaluation coefficient with a preset second comprehensive coefficient threshold; when the second comprehensive evaluation coefficient exceeds the preset second comprehensive coefficient threshold, determining that the speech quality does not meet the standard; The second comprehensive evaluation coefficient is obtained by the following formula:

[0013] Among them, Z 02 Indicates the second comprehensive evaluation coefficient; Q x01 represents the first speech quality assessment coefficient after normalization; Q x02It represents the second speech quality assessment coefficient after normalization.

[0014] Furthermore, the step 2: recognition and analysis of voice commands also includes emotional perception and contextual understanding of text information, identifying the user's emotional state by analyzing the intonation and speed characteristics of the user's voice based on multimodal data, combined with the data characteristics of non-voice information obtained simultaneously, and adjusting the device control strategy according to the user's emotional state.

[0015] Furthermore, the step three, before executing the control strategy, also includes confirming interaction with the user through voice to ensure the accuracy of the control operation. The specific steps are as follows: Voice feedback and confirmation: After the control strategy is generated, the expected operation will be fed back to the user through voice. The feedback content will include a summary of the understood user needs to help the user quickly understand the operation to be performed; User confirmation and adjustment: Receive the user's confirmation or adjustment instructions through voice. If the user's confirmation content is unclear, it will automatically prompt and ask for clarification; Multi-round interaction and confirmation: If the user's instructions involve multiple devices and operation steps, ensure that each operation is in line with the user's intention through step-by-step confirmation, and optimize the user's operation experience based on the confirmation feedback after each step is completed; Personalized confirmation: Dynamically adjust the confirmation process based on the user’s emotional state; Confirmation information feedback: Once the control strategy is confirmed by the user, the user will be clearly informed of the specific content and operation results to be performed by voice before the operation is performed. After the operation is completed, the final result will continue to be confirmed by voice and an operation review will be provided.

[0016] Furthermore, the personalized confirmation specifically includes: Acquire the intonation and speech speed features of the user's voice, respond with the intonation and tone that matches the user's emotional state, and respond by matching the corresponding template sentence in the pre-stored voice database according to the user's emotional state; Classify the user's emotional state, determine the user's emotional category, match the corresponding interaction style to users in different emotional states, and adjust the execution priority of the task according to the user's emotional state.

[0017] Furthermore, the step 5: feedback and continuous optimization of operation results also includes: Regularly ask users about their satisfaction with the smart home system through voice, summarize satisfaction evaluation reports, and help users understand their usage habits and improvement progress; Conduct correlation analysis based on user feedback data, user specific operations and system response data, evaluate key indicators of each specific operation, and generate user satisfaction optimization strategies based on user usage habits and preferences; Identify the shortcomings of certain functions based on users' long-term feedback and needs, and promote iterative updates of the functions.

[0018] The present invention provides another technical solution, a smart home voice interaction control system, comprising: The data acquisition module is used to collect the user's voice input signal and monitoring video data, and pre-process the collected voice input signal and monitoring video data; The speech recognition and analysis module is used to convert the pre-processed speech input signal into text, perform semantic analysis on the text based on natural language processing technology, determine the operation instructions, and verify the recognition results of the speech input signal based on the monitoring video data, and generate prompts or clarification statements for ambiguous or incomplete instructions based on the recognition results; The control strategy generation module is used to match the operation logic of the target device according to the analysis results, generate the basic control strategy, and adjust the control strategy according to the user's historical operation data; The device control module is used to establish a communication link with the target device, send operation instructions through the home IoT protocol, and monitor the response status of the device in real time. If the device is abnormal, the exception handling mechanism is triggered and feedback is given to the user; The feedback optimization module is used to obtain user instructions and device response data, build a user behavior data set, generate shortcut operation logic based on the user's high-frequency usage instructions, collect evaluation feedback from the user at preset time intervals, and optimize the control instructions based on the evaluation feedback.

[0019] Compared with the prior art, the present invention has the following beneficial effects: By integrating voice and non-voice information, the accurate recognition of user intentions is enhanced, which not only improves the recognition accuracy of voice commands, but also can more comprehensively understand the user's emotional state and needs, especially in noisy environments or complex situations. The recognition and response effects of the system are significantly improved. By utilizing the user's emotion analysis and context understanding technology, the interaction method is dynamically adjusted according to the user's emotional state. It can flexibly adapt to the emotional needs of different users and provide more humane services. It can also continuously optimize the control strategy based on user feedback to improve user satisfaction, thereby improving the intelligence of the smart home system and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the smart home voice interaction control method of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In order to solve the technical problems that the existing technology cannot accurately understand the user's intention through non-voice information such as emotions and expressions, the control strategy is rigid and lacks personalization, and cannot adapt to user needs and emotions in real time, please refer to Figure 1 , this embodiment provides the following technical solutions: The smart home voice interaction control method comprises the following steps: Step 1: Voice input collection and preprocessing: The user's voice input signal is collected through intelligent voice collection equipment (such as microphone array or intelligent voice terminal), and the voice input signal is preprocessed, including noise suppression, voice signal enhancement, echo cancellation and reverberation suppression, to ensure the quality of voice data; Step 2: Recognition and analysis of voice commands: Perform text recognition on the pre-processed voice input signal, convert the recognition result into a text command, and perform semantic analysis on the text command to determine the user's intention, generate user commands, and perform emotional perception and contextual understanding on the text information. Based on multimodal data, the user's voice intonation and speed characteristics are analyzed, combined with the data characteristics of the non-voice information obtained simultaneously, the user's emotional state, such as anxiety, pleasure, tension, etc., is identified, and the device control strategy is adjusted according to the user's emotional state. For example, when the user says "turn off the lights" with a tired expression, the system can infer that the user is in a resting state and automatically adjust the status of other home appliances (such as air conditioners and curtains) to provide a more comfortable rest environment and provide more humane services; Step 3: Generation of device control strategy: Match the corresponding device control strategy according to the generated user instructions, determine the user needs and the operation logic of the target device, and adjust the device control strategy according to the user's historical operation habits and preferences to achieve personalized control; Step 4: Execution and real-time monitoring of equipment operations: Establish a communication link with the target equipment according to the equipment control strategy, send operation instructions to the target equipment, and monitor the response of the equipment in real time; Step 5: Feedback and continuous optimization of operation results: After the device operation is completed, the execution results will be fed back to the user through voice, and the user operation data will be recorded to optimize the subsequent experience.

[0023] In this embodiment, by integrating voice and non-voice information, the accurate recognition of user intentions is enhanced, which not only improves the recognition accuracy of voice commands, but also can more comprehensively understand the user's emotional state and needs, especially in noisy environments or complex situations. The recognition and response effects of the system are significantly improved. By utilizing the user's emotion analysis and context understanding technology, the interaction mode is dynamically adjusted according to the user's emotional state, which can flexibly adapt to the emotional needs of different users and provide more humane services. It can also continuously optimize the control strategy based on user feedback to improve user satisfaction, thereby improving the intelligence level of the smart home system and user experience.

[0024] In this embodiment, the step 1: collecting and preprocessing the voice input also includes: Based on the obtained voice input signal, the preset wake-up words and phrases are matched to trigger the awakening of the monitoring data collection device, such as "Hello, smart home" or "Turn on home assistant"; When collecting voice input signals, the monitoring data collection device collects non-voice information such as facial expressions and gestures of the user to obtain user monitoring data; Identify the user's facial expressions and hand movements based on the acquired user monitoring data, analyze the data characteristics of non-voice information, understand the user's emotional state and intention, and adjust the subsequent interaction strategy. For example, if the user shows a nervous or anxious expression, the system will speed up the response and give priority to emergency operation requests; At the same time, the voice input signal and non-voice information are synchronized based on the timestamp, the association between the voice input signal and the non-voice information is established, and the semantics of the voice input signal is supplemented based on the non-voice information. For example, when the user says "open the curtains" and makes a gesture pointing to the curtains, the system will prioritize recognizing the gesture and performing related operations; In this embodiment, the accuracy of speech recognition and the comprehensiveness of understanding of user intentions are improved by fusing multimodal data, and multimodal data are processed synchronously. With the assistance of non-voice information such as facial expressions and gestures, the accuracy of speech recognition is further improved, and the depth of understanding of user intentions is increased, providing a reliable data foundation for subsequent voice command recognition and analysis, and ensuring that the system can run smoothly in complex environments and multiple interaction scenarios.

[0025] In this embodiment, the step 2: recognition and analysis of voice commands specifically includes: Effective speech extraction: After collecting the speech input signal, the endpoint detection technology is used to remove the invalid silent parts before and after the speech, retain the effective speech information, and split the long speech input signal into multiple logical segments; Voice quality assessment: monitors and evaluates the quality of voice input signals in real time. If the voice quality does not meet the standards (such as signal distortion, excessive noise, etc.), it will actively optimize parameters or prompt users to re-enter to ensure that the voice data meets the requirements of subsequent processing. At the same time, it will dynamically adjust the microphone gain, sampling frequency and other acquisition parameters according to environmental factors such as environmental noise level, user distance and device performance to optimize the voice acquisition effect. Text instruction generation: Perform speech recognition processing on qualified speech input signals, convert the recognition results into text information, and correct the noise, accent or ambiguous pronunciation in the speech input signals to improve recognition accuracy; Semantic parsing: Perform word segmentation on the converted text information, divide the text instructions into words or phrases, identify the part of speech of each word, such as noun, verb, adjective, etc., based on the semantics of the supplementary voice input signal, determine the grammatical relationship between words, determine the user's operation intention based on the parsing results, such as turning on and off the lights, adjusting the temperature, etc., identify key entities in the text, such as device type, device name, location, time, operation instructions and their values, etc. For example, if the user says "raise the temperature in the living room by two degrees", the system will recognize "raise" as the operation instruction, "living room" as the target device, and "temperature" as the adjustment attribute, and finally extract the "raise temperature" instruction and determine the target temperature control device; Contextual understanding: After each round of conversation, the system records the relevant information and status of the user's input, understands the context of continuous voice commands, maintains the continuity of the conversation, and infers the user's further needs based on the previous commands and operation results. The system can recognize and process multiple rounds of conversations, realize logical associations across commands, and avoid repeated recognition or invalid operations. For example, when the user says "lower the living room lights" and "increase the air conditioning temperature" in succession, the system not only recognizes the two independent commands, but also understands the time and space relationship between the two commands, and coordinates them according to the current status of the living room equipment; User instruction generation: Generate structured user instructions based on the recognition results of semantic parsing and the inference results of context understanding.

[0026] In this embodiment, through effective voice extraction, voice quality assessment, text command generation, semantic parsing, context understanding and user command generation, the interactive capability and user experience of the smart home system are significantly improved, and the accuracy of voice data and the comprehensiveness of command parsing are ensured, so that the system can understand user intentions more accurately and generate operation instructions accordingly, thereby improving the accuracy and efficiency of voice recognition, enhancing the interaction consistency between users and the system, and achieving more natural and intelligent voice interaction, thereby greatly improving user satisfaction and the practicality of the smart home system.

[0027] Specifically, the quality of the voice input signal is monitored and evaluated in real time, including: Real-time monitoring of signal characteristic parameters of the voice input signal, wherein the signal characteristic parameters include a signal-to-noise ratio, a signal difference ratio corresponding to a distortion, a jitter parameter, and a drift parameter; Obtaining a first speech quality assessment coefficient using a signal difference ratio corresponding to the signal-to-noise ratio and the distortion; The first speech quality assessment coefficient is obtained by the following formula:

[0028] Among them, Q 01 represents the first speech quality assessment coefficient; n represents the number of unit times contained in the speech input signal; S i+1 represents the signal-to-noise ratio of the speech input signal corresponding to the i+1th unit time; S i represents the signal-to-noise ratio of the speech input signal corresponding to the i-th unit time; P i represents the signal difference ratio corresponding to the distortion of the speech input signal corresponding to the i-th unit time; P b S represents the standard deviation of the signal difference ratio corresponding to the distortion corresponding to n unit time; b Represents the standard deviation of the signal-to-noise ratio of the speech input signal corresponding to n unit time; Acquire a second voice quality assessment coefficient using the jitter parameter and the drift parameter; The second voice quality assessment coefficient is obtained by the following formula:

[0029] Among them, Q 02 represents the second speech quality assessment coefficient; n represents the number of unit times contained in the speech input signal; J i and Y i J represents the jitter parameter and drift parameter of the speech input signal corresponding to the i-th unit time; i+1 and Y i+1 represents the jitter parameter and drift parameter of the speech input signal corresponding to the i+1th unit time; J b and Y b Indicates the jitter parameter standard deviation and drift parameter standard deviation of the speech input signal corresponding to n unit time; The quality of the speech input signal is determined by using the first speech quality assessment coefficient and the second speech quality assessment coefficient.

[0030] The technical effect of the above technical solution is: by real-time monitoring of multiple signal characteristic parameters of the voice input signal (signal-to-noise ratio, signal difference ratio corresponding to distortion, jitter parameter and drift parameter), the technical solution can comprehensively and meticulously evaluate the quality of the voice signal. These parameters cover the main quality characteristics of the voice signal, thereby ensuring the accuracy and reliability of the evaluation results. The technical solution uses unit time (such as seconds, milliseconds, etc.) as the basic unit of evaluation. By monitoring and calculating the signal characteristic parameters in each unit time, it can reflect the changes in the quality of the voice signal in real time. This dynamic and adaptive evaluation method helps to promptly discover and deal with quality problems in the voice signal. The first voice quality assessment coefficient (Q 01 ) and the second speech quality assessment coefficient (Q 02 ), this technical solution converts the quality of speech signals into specific numerical indicators. This quantitative evaluation standard not only facilitates the understanding and comparison of the quality differences of different speech signals, but also provides a clear reference for subsequent speech signal processing. This technical solution uses statistics such as standard deviation to reflect the distribution of signal characteristic parameters, thereby simplifying the calculation process and improving the evaluation efficiency. At the same time, through a formulaic calculation method, this technical solution can quickly derive the voice quality assessment coefficient, providing strong support for application scenarios such as real-time voice communication. The formulas and parameters in this technical solution can be adjusted and optimized according to actual application requirements. For example, different weight coefficients or thresholds can be set according to different speech coding standards, transmission conditions or application scenarios to meet different evaluation requirements. This technical solution mainly relies on the monitoring and calculation of signal characteristic parameters, without the need for complex hardware support or additional signal processing steps. Therefore, it is relatively simple and low-cost to implement, and is suitable for voice communication systems of various sizes.

[0031] In summary, this technical solution can comprehensively and accurately evaluate the quality of voice signals by real-time monitoring of multiple signal characteristic parameters of voice input signals and calculating the corresponding voice quality assessment coefficients, and has the advantages of high dynamic adaptability, quantitative evaluation standards, high efficiency, flexibility and easy implementation. These technical effects provide strong technical support and guarantee for application scenarios such as real-time voice communication.

[0032] Specifically, determining the quality of the speech input signal by using the first speech quality assessment coefficient and the second speech quality assessment coefficient includes: Using a preset first evaluation coefficient reference value and a preset second evaluation coefficient reference value to respectively perform normalization processing on the first speech quality evaluation coefficient and the second speech quality evaluation coefficient, and obtain the normalized first speech quality evaluation coefficient and the second speech quality evaluation coefficient; Comparing the normalized first speech quality assessment coefficient with the second speech quality assessment coefficient; When the first speech quality assessment coefficient is lower than the second speech quality assessment coefficient, the first comprehensive assessment model is retrieved; when the first speech quality assessment coefficient is higher than or equal to the second speech quality assessment coefficient, the second comprehensive assessment model is retrieved; Using the first comprehensive evaluation model in combination with the first speech quality evaluation coefficient and the second speech quality evaluation coefficient to obtain a first comprehensive evaluation coefficient, and comparing the first comprehensive evaluation coefficient with a preset first comprehensive coefficient threshold; when the first comprehensive evaluation coefficient exceeds the preset first comprehensive coefficient threshold, determining that the speech quality does not meet the standard; The first comprehensive evaluation coefficient is obtained by the following formula:

[0033] Among them, Z 01 Indicates the first comprehensive evaluation coefficient; Q x01 represents the first speech quality assessment coefficient after normalization; Q x02 represents the second speech quality assessment coefficient after normalization; Using the second comprehensive evaluation model in combination with the first speech quality evaluation coefficient and the second speech quality evaluation coefficient to obtain a second comprehensive evaluation coefficient, and comparing the second comprehensive evaluation coefficient with a preset second comprehensive coefficient threshold; when the second comprehensive evaluation coefficient exceeds the preset second comprehensive coefficient threshold, determining that the speech quality does not meet the standard; The second comprehensive evaluation coefficient is obtained by the following formula:

[0034] Among them, Z 02 Indicates the second comprehensive evaluation coefficient; Q x01 represents the first speech quality assessment coefficient after normalization; Q x02 It represents the second speech quality assessment coefficient after normalization.

[0035] The technical effect of the above technical solution is: the first speech quality assessment coefficient and the second speech quality assessment coefficient are normalized by the preset first assessment coefficient reference value and the second assessment coefficient reference value, so that the two coefficients can be compared on the same scale. This normalization process helps to eliminate the comparison difficulties caused by different dimensions or value ranges between different assessment coefficients, thereby improving the accuracy and effectiveness of the comparison. The technical solution introduces a first comprehensive assessment model and a second comprehensive assessment model, which can combine the first speech quality assessment coefficient and the second speech quality assessment coefficient after normalization to calculate the first comprehensive assessment coefficient and the second comprehensive assessment coefficient respectively. This comprehensive assessment method not only considers the information of a single assessment coefficient, but also integrates the comprehensive effects of multiple assessment coefficients, thereby enhancing the comprehensiveness and accuracy of the evaluation. By comparing the first comprehensive assessment coefficient and the second comprehensive assessment coefficient with the preset first comprehensive coefficient threshold and the second comprehensive coefficient threshold respectively, the technical solution can standardize whether the quality of the speech input signal meets the standard. This threshold comparison method not only simplifies the determination process, but also improves the objectivity and consistency of the determination. The technical solution can comprehensively evaluate the speech input signal according to different quality characteristics (such as signal-to-noise ratio, distortion, jitter parameter, drift parameter, etc.). By adjusting the reference value in the normalization process, the parameters and thresholds in the comprehensive evaluation model, the technical solution can flexibly adapt to different application scenarios and evaluation requirements. The technical solution improves the efficiency and accuracy of speech quality evaluation through a formulaic calculation method and a standardized judgment process. At the same time, since the comprehensive evaluation model can integrate the information of multiple evaluation coefficients, the technical solution has higher accuracy and reliability when evaluating the quality of complex speech signals. The technical solution mainly relies on formulaic calculations and standardized judgment processes, without complex hardware support or additional signal processing steps. Therefore, it is relatively simple to implement and has low cost. At the same time, since the technical solution has good scalability, more evaluation coefficients or comprehensive evaluation models can be added according to actual needs to further improve the accuracy and comprehensiveness of the evaluation.

[0036] In summary, this technical solution achieves a comprehensive, accurate and efficient evaluation of the quality of voice input signals through normalization processing, comprehensive evaluation model, threshold comparison, etc. These technical effects provide strong technical support and guarantee for application fields such as voice communication and voice recognition.

[0037] In this embodiment, the step three, before executing the control strategy, also includes confirming interaction with the user through voice to ensure the accuracy of the control operation. The specific steps are as follows: Voice feedback and confirmation: After the control strategy is generated, the expected operation will be fed back to the user through voice, including key parameters such as the type of equipment, operation instructions, and the location of the target equipment. For example, "I am about to increase the temperature of the air conditioner in the living room by two degrees. Are you sure?" The feedback content will include a summary of the user's understood needs to help the user quickly understand the operation to be performed; User confirmation and adjustment: Receive user confirmation or adjustment instructions through voice. For example, users can confirm with a simple "yes" or "no". If the user wants to adjust the setting, they can further indicate the modification content, such as "lower it a little" or "increase the temperature by five degrees". If the user's confirmation content is unclear, it will automatically prompt and ask for clarification, for example: "How much do you want to increase the air conditioning temperature?"; Multi-round interaction and confirmation: If the user's instructions involve multiple devices and operation steps, ensure that each operation is in line with the user's intention through step-by-step confirmation. For example, "The 'turn off the living room lights' instruction is being executed, and the next step is the 'adjust the temperature' operation. Do you confirm?" The user's operation experience is optimized based on the confirmation feedback after each step is completed, ensuring that each instruction is clear and accurate. Personalized confirmation: Dynamically adjust the confirmation process based on the user's emotional state. When the user is identified as nervous or anxious, a faster and simpler confirmation interaction method is used to speed up the response. Conversely, when the user is relaxed or happy, a gentler and slower interaction method is used to enhance the personalized user experience. For example, if the user's voice is accompanied by anxiety, the system may simplify the confirmation process and quickly execute the command: "I have raised the air conditioning temperature by two degrees to ensure your comfort." Specifically: Acquire the intonation and speech speed features of the user's voice, respond with the intonation and tone that matches the user's emotional state, and respond by matching the corresponding template sentence in the pre-stored voice database according to the user's emotional state, such as providing encouragement or comfort when the user is depressed; Classify the user's emotional state, determine the user's emotional category, and match the corresponding interaction style to users in different emotional states, such as more lively, formal or warm; and adjust the execution priority of tasks according to the user's emotional state, such as giving priority to related tasks when the user is urgent or anxious; Confirmation information feedback: Once the control strategy is confirmed by the user, the user will be informed of the specific content and results of the operation to be performed by voice before the operation is performed, further increasing the user's sense of transparency and control. For example, "Your air conditioning temperature has been raised by two degrees, and the system will adjust the light brightness next." After the operation is completed, the final result is confirmed by voice and the operation review is provided, for example, "The air conditioning temperature in the living room has been adjusted to the target value, and the operation is completed."

[0038] In this embodiment, confirmation interaction with the user through voice ensures accurate understanding of the user's intention and reduces misoperation. At the same time, the user's emotional state is taken into consideration, providing more humane services, enhancing the user's trust and satisfaction with the smart home system, and improving the transparency of operations, so that users can control the home environment more confidently and conveniently, thereby improving the overall quality of life and smart living experience.

[0039] In this embodiment, the step 5: feedback and continuous optimization of operation results also includes: Regularly ask users about their satisfaction with the smart home system through voice, such as "Are you satisfied with your recent smart home operation experience?" and "Which functions do you hope to improve?", and summarize the satisfaction evaluation report to help users understand their usage habits and improvement progress. For example, the monthly report generated by the system can include content such as "In the past month, your satisfaction with the home temperature control system has increased by 15%", showing users the effect of continuous improvement; Conduct correlation analysis based on user feedback data, specific user operations, and system response data, and evaluate key indicators such as the success rate, accuracy, and response time of each specific operation, and generate user satisfaction optimization strategies based on user usage habits and preferences; Identify the deficiencies of certain functions based on long-term user feedback and needs, and promote iterative updates of the function. For example, if users frequently report "inaccurate voice recognition" or "too long control delay", enhance the accuracy and response speed of voice recognition through software updates.

[0040] In this embodiment, the synchronization between the system and user needs is ensured through a continuous optimization mechanism, which effectively improves the user experience and performance of the smart home system, enhances user satisfaction and loyalty, and promotes the continuous improvement and development of smart home technology.

[0041] In order to better implement a smart home voice interaction control method, the present invention provides a smart home voice interaction control system, comprising: The data acquisition module is used to collect the user's voice input signal and monitoring video data, and pre-process the collected voice input signal and monitoring video data; The speech recognition and analysis module is used to convert the pre-processed speech input signal into text, perform semantic analysis on the text based on natural language processing technology, determine the operation instructions, and verify the recognition results of the speech input signal based on the surveillance video data, and generate prompts or clarification statements for ambiguous or incomplete instructions based on the recognition results, such as "Please specify which room's equipment is controlled?"; The control strategy generation module is used to match the operation logic of the target device according to the analysis results, generate the basic control strategy, and adjust the control strategy to meet the user's habitual preferences according to the user's historical operation data; The device control module is used to establish a communication link with the target device, send operation instructions through the home IoT protocol, and monitor the response status of the device in real time to ensure that the device operates as expected. If the device is abnormal (such as unable to respond to instructions), the exception handling mechanism is triggered and feedback is given to the user, such as "the air conditioner is not responding, please check the connection"; The feedback optimization module is used to obtain user command and device response data, build a user behavior data set, generate shortcut operation logic based on the user's high-frequency usage commands, such as "one-click activation of away mode", and collect evaluation feedback (such as operation satisfaction) from users at preset time intervals. The control commands are optimized based on the evaluation feedback, ensuring that the system continues to learn and adapt to user needs, thereby significantly improving the user experience, enhancing the intelligence of home automation and the quality of user interaction, and creating a more comfortable, convenient and intelligent living environment for users.

[0042] In this embodiment, the accuracy and reliability of instructions are improved through precise voice recognition and video data verification, while taking user habits and preferences into consideration, the personalization and convenience of operations are improved.

[0043] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A smart home voice interaction control method, characterized in that: The following steps are involved: Step 1: Voice input acquisition and preprocessing: The user's voice input signal is acquired through an intelligent voice acquisition device, and the voice input signal is preprocessed, which also includes: Based on the matching of the acquired voice input signal with the preset wake-up words and phrases, triggering the wake-up monitoring data collection device; When collecting voice input signals, the user's non-voice information is collected based on the monitoring data collection device to obtain user monitoring data; Identify the user's facial expressions and hand movements based on the acquired user monitoring data, analyze the data features of non-voice information, and understand the user's emotional state and intentions; At the same time, the voice input signal and the non-voice information are synchronized based on the timestamp, the association relationship between the voice input signal and the non-voice information is established, and the semantics of the voice input signal is supplemented based on the non-voice information; Step 2: Recognition and analysis of voice commands: Perform text recognition on the pre-processed voice input signal, convert the recognition result into text commands, and perform semantic analysis on the text commands to determine the user's intention and generate user commands; Step 3: Generation of device control strategy: Match the corresponding device control strategy according to the generated user instructions, determine the user needs and the operation logic of the target device, and adjust the device control strategy according to the user's historical operation habits and preferences; Step 4: Execution and real-time monitoring of equipment operations: Establish a communication link with the target equipment according to the equipment control strategy, send operation instructions to the target equipment, and monitor the response of the equipment in real time; Step 5: Feedback and continuous optimization of operation results: After the device operation is completed, the execution results will be fed back to the user through voice, and the user operation data will be recorded.

2. The smart home voice interaction control method according to claim 1, characterized in that: The step 2: recognition and analysis of voice commands, specifically includes: Effective speech extraction: After collecting the speech input signal, the endpoint detection technology is used to remove the invalid silent parts before and after the speech, retain the effective speech information, and split the long speech input signal into multiple logical segments; Voice quality assessment: monitor and evaluate the quality of voice input signals in real time. If the voice quality does not meet the standards, optimize the parameters or prompt the user to re-enter the voice. At the same time, dynamically adjust the microphone acquisition parameters according to environmental factors. Text instruction generation: Perform speech recognition processing on qualified speech input signals, convert the recognition results into text information, and correct the noise, accent or ambiguous pronunciation in the speech input signals; Semantic parsing: Perform word segmentation on the converted text information, split the text instructions into words or phrases, identify the part of speech of each word based on the semantics of the supplementary voice input signal, determine the grammatical relationship between words, determine the user's operation intention based on the parsing results, and identify key entities in the text; Contextual understanding: After each round of conversation, the system records the relevant information and status of the user's input, understands the context of continuous voice commands, maintains the continuity of the conversation, and infers the user's further needs based on the previous commands and operation results; User instruction generation: Generate structured user instructions based on the recognition results of semantic parsing and the inference results of context understanding.

3. The smart home voice interaction control method according to claim 2, characterized in that: Monitor and evaluate the quality of voice input signals in real time, including: Real-time monitoring of signal characteristic parameters of the voice input signal, wherein the signal characteristic parameters include a signal-to-noise ratio, a signal difference ratio corresponding to a distortion, a jitter parameter, and a drift parameter; Obtaining a first speech quality assessment coefficient using a signal difference ratio corresponding to the signal-to-noise ratio and the distortion; The first speech quality assessment coefficient is obtained by the following formula: Among them, Q 01 represents the first speech quality assessment coefficient; n represents the number of unit times contained in the speech input signal; S i+1 represents the signal-to-noise ratio of the speech input signal corresponding to the i+1th unit time; S i represents the signal-to-noise ratio of the speech input signal corresponding to the i-th unit time; P i represents the signal difference ratio corresponding to the distortion of the speech input signal corresponding to the i-th unit time; P b S represents the standard deviation of the signal difference ratio corresponding to the distortion corresponding to n unit time; b Represents the standard deviation of the signal-to-noise ratio of the speech input signal corresponding to n unit time; Acquire a second voice quality assessment coefficient using the jitter parameter and the drift parameter; The second voice quality assessment coefficient is obtained by the following formula: Among them, Q 02 represents the second speech quality assessment coefficient; n represents the number of unit times contained in the speech input signal; J i and Y i J represents the jitter parameter and drift parameter of the speech input signal corresponding to the i-th unit time; i+1 and Y i+1 represents the jitter parameter and drift parameter of the speech input signal corresponding to the i+1th unit time; J b and Y b Indicates the jitter parameter standard deviation and drift parameter standard deviation of the speech input signal corresponding to n unit time; The quality of the speech input signal is determined by using the first speech quality assessment coefficient and the second speech quality assessment coefficient.

4. The smart home voice interaction control method according to claim 3, characterized in that: Determining the quality of a speech input signal by using the first speech quality assessment coefficient and the second speech quality assessment coefficient comprises: Using a preset first evaluation coefficient reference value and a preset second evaluation coefficient reference value to respectively perform normalization processing on the first speech quality evaluation coefficient and the second speech quality evaluation coefficient, and obtain the normalized first speech quality evaluation coefficient and the second speech quality evaluation coefficient; Comparing the normalized first speech quality assessment coefficient with the second speech quality assessment coefficient; When the first speech quality assessment coefficient is lower than the second speech quality assessment coefficient, the first comprehensive assessment model is retrieved; when the first speech quality assessment coefficient is higher than or equal to the second speech quality assessment coefficient, the second comprehensive assessment model is retrieved; Using the first comprehensive evaluation model in combination with the first speech quality evaluation coefficient and the second speech quality evaluation coefficient to obtain a first comprehensive evaluation coefficient, and comparing the first comprehensive evaluation coefficient with a preset first comprehensive coefficient threshold; when the first comprehensive evaluation coefficient exceeds the preset first comprehensive coefficient threshold, determining that the speech quality does not meet the standard; The first comprehensive evaluation coefficient is obtained by the following formula: Among them, Z 01 Indicates the first comprehensive evaluation coefficient; Q x01 represents the first speech quality assessment coefficient after normalization; Q x02 represents the second speech quality assessment coefficient after normalization; Using the second comprehensive evaluation model in combination with the first speech quality evaluation coefficient and the second speech quality evaluation coefficient to obtain a second comprehensive evaluation coefficient, and comparing the second comprehensive evaluation coefficient with a preset second comprehensive coefficient threshold; when the second comprehensive evaluation coefficient exceeds the preset second comprehensive coefficient threshold, determining that the speech quality does not meet the standard; The second comprehensive evaluation coefficient is obtained by the following formula: Among them, Z 02 Indicates the second comprehensive evaluation coefficient; Q x01 represents the first speech quality assessment coefficient after normalization; Q x02 It represents the second speech quality assessment coefficient after normalization.

5. The smart home voice interaction control method according to claim 4, characterized in that: The step 2: recognition and analysis of voice commands also includes emotional perception and contextual understanding of text information, identifying the user's emotional state by analyzing the intonation and speed characteristics of the user's voice based on multimodal data, combining the data characteristics of non-voice information obtained simultaneously, and adjusting the device control strategy according to the user's emotional state.

6. The smart home voice interaction control method according to claim 5, characterized in that: The step three, before executing the control strategy, also includes confirming interaction with the user through voice to ensure the accuracy of the control operation. The specific steps are as follows: Voice feedback and confirmation: After the control strategy is generated, the expected operation will be fed back to the user through voice. The feedback content will include a summary of the understood user needs to help the user quickly understand the operation to be performed; User confirmation and adjustment: Receive the user's confirmation or adjustment instructions through voice. If the user's confirmation content is unclear, it will automatically prompt and ask for clarification; Multi-round interaction and confirmation: If the user's instructions involve multiple devices and operation steps, ensure that each operation is in line with the user's intention through step-by-step confirmation, and optimize the user's operation experience based on the confirmation feedback after each step is completed; Personalized confirmation: Dynamically adjust the confirmation process based on the user’s emotional state; Confirmation information feedback: Once the control strategy is confirmed by the user, the user will be clearly informed of the specific content and operation results to be performed by voice before the operation is performed. After the operation is completed, the final result will continue to be confirmed by voice and an operation review will be provided.

7. The smart home voice interaction control method according to claim 6, characterized in that: The personalized confirmation specifically includes: Acquire the intonation and speech speed features of the user's voice, respond with the intonation and tone that matches the user's emotional state, and respond by matching the corresponding template sentence in the pre-stored voice database according to the user's emotional state; Classify the user's emotional state, determine the user's emotional category, match the corresponding interaction style to users in different emotional states, and adjust the execution priority of the task according to the user's emotional state.

8. The smart home voice interaction control method according to claim 7, characterized in that: The step 5: feedback and continuous optimization of operation results also includes: Regularly ask users about their satisfaction with the smart home system through voice, summarize satisfaction evaluation reports, and help users understand their usage habits and improvement progress; Conduct correlation analysis based on user feedback data, user specific operations and system response data, evaluate key indicators of each specific operation, and generate user satisfaction optimization strategies based on user usage habits and preferences; Identify the shortcomings of certain functions based on users' long-term feedback and needs, and promote iterative updates of the functions.

9. A smart home voice interaction control system, applied in the smart home voice interaction control method according to claim 8, characterized in that: include: The data acquisition module is used to collect the user's voice input signal and monitoring video data, and pre-process the collected voice input signal and monitoring video data; The speech recognition and analysis module is used to convert the pre-processed speech input signal into text, perform semantic analysis on the text based on natural language processing technology, determine the operation instructions, and verify the recognition results of the speech input signal based on the monitoring video data, and generate prompts or clarification statements for ambiguous or incomplete instructions according to the recognition results; The control strategy generation module is used to match the operation logic of the target device according to the analysis results, generate the basic control strategy, and adjust the control strategy according to the user's historical operation data; The device control module is used to establish a communication link with the target device, send operation instructions through the home IoT protocol, and monitor the response status of the device in real time. If the device is abnormal, the exception handling mechanism is triggered and feedback is given to the user; The feedback optimization module is used to obtain user instructions and device response data, build a user behavior data set, generate shortcut operation logic based on the user's high-frequency usage instructions, collect evaluation feedback from the user at preset time intervals, and optimize the control instructions based on the evaluation feedback.

Images