Intelligent audio control method, system and storage medium

Abstract

The present invention belongs to the technical field of audio control, and specifically relates to an intelligent audio control method, system and storage medium. The present invention focuses the interactive control of the intelligent audio on the home appliance on the control effect, and can screen out the interactive control records with abnormal control effects based on the historical interactive control records, and then conduct a correlation analysis between the abnormal source and the complexity of the voice control instructions, thereby deriving the targeted adjustment direction of the audio interactive control, realizing the feedback closed-loop adjustment of the audio interactive control, effectively filling the gap in the feedback control of the existing audio interactive control, and accelerating the continuous optimization and improvement of the audio interactive control to a certain extent, so that the user experience is continuously improved, which is conducive to the sustainable development of the intelligent audio.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio control, and in particular relates to an intelligent audio control method, system and storage medium. Background Art

[0002] With the continuous development of society, people's focus of life has gradually shifted from material pursuits to spiritual and leisure and entertainment pursuits. Music, as a common form of leisure and entertainment, has become an indispensable part of people's daily lives. With the rise of streaming services, people can listen to music through the Internet anytime and anywhere, and high-quality audio equipment has gradually become a must-have for homes. At the same time, the development of the Internet of Things and artificial intelligence technology has made audio not only limited to music playback, but has become the center of interactive control of smart homes. This interactive control, with its convenience and interactivity, not only facilitates people's daily lives, but also meets modern people's pursuit of efficient, intelligent and personalized life.

[0003] There are also technical solutions for interactively controlling home appliances using smart speakers in the prior art. For example, a smart home control system based on smart speakers is disclosed in the Chinese invention patent CN107290970A. By using smart speakers with multiple different interactive modes as the central control, the smart speakers can give full play to their artificial intelligence functions of listening and speaking, receive user instructions and issue corresponding operation instructions. During use, users can use methods including voice control to control home appliances. The operation process is highly intelligent and can well realize the interaction between multiple home appliances and users. Although this solution provides smart control of multiple home appliances by speakers, its interactive control is mainly focused on the user interactive experience at the front end. For example, the device compatibility reflected in this solution ensures that the speakers can be more effectively compatible with as many home appliances as possible, and lacks attention and feedback on the control effect. Specifically, when some control instructions are not responded to in time and cannot be accurately executed, the system fails to identify, record, and trace the source in time, making it difficult to provide reliable data and feedback for interactive control adjustment, making it difficult to continuously optimize and improve the system. In the long run, the optimization process of the system may be very slow and unable to respond quickly to changes in user needs, thereby reducing the user experience and being unfavorable to the sustainable development of smart speakers. Summary of the invention

[0004] To this end, one purpose of the embodiments of the present application is to provide a smart speaker control method, system and storage medium. By focusing the interactive control of the smart speaker on home appliances on the control effect, it is possible to strengthen the monitoring and feedback of the control effect based on historical interactive control data and establish a complete feedback mechanism, thereby effectively filling the gap in feedback control of existing audio interactive control.

[0005] The objective of the present invention can be achieved through the following technical solutions: The first aspect of the present invention provides a smart speaker control method, comprising a smart speaker, a router, a home appliance and a cloud platform, wherein the smart speaker obtains the user's current command through voice collection and controls the home appliance to perform corresponding actions through the transmission of control commands through the router, the router is connected to the cloud platform through a home network, and the interactive control information is uploaded and stored in the cloud platform to form an interactive control record, comprising the following steps: (1) retrieving the historical interactive control record of the smart speaker on the home appliance from the cloud platform, and extracting control state parameters therefrom, specifically including control response time, control execution consistency and control execution time, thereby evaluating the control effect of the historical interactive control record.

[0006] (2) Filter out abnormal interactive control records based on the control effect evaluation of historical interactive control records.

[0007] (3) The abnormal interaction control records are traced for abnormal control, and then the abnormal control traceability results are analyzed for tendency to obtain the tendency of abnormal interaction control.

[0008] (4) Extract the voice control commands of the abnormal interactive control records to identify the command complexity, and then correlate and analyze the control effects of the abnormal interactive control records with the command complexity.

[0009] (5) Determine the adjustment direction of the audio interaction control based on the tendency of abnormal interaction control and the correlation between the control effect and instruction complexity of the abnormal interaction control records.

[0010] (6) Perform audio interaction monitoring and control according to the adjustment direction of the audio interaction control.

[0011] A second aspect of the present invention provides an intelligent audio control system, comprising the following modules:

[0012] The historical control effect evaluation module is used to retrieve the historical interactive control records of smart speakers on home appliances from the cloud platform and extract control state parameters from them, including control response time, control execution consistency and control execution time, so as to evaluate the control effect of historical interactive control records.

[0013] The abnormal interaction control record screening module is used to screen out abnormal interaction control records based on the control effect evaluation of historical interaction control records.

[0014] The abnormal control tracing and analysis module is used to perform abnormal control tracing on abnormal interactive control records, and then perform tendency analysis on the abnormal control tracing results to obtain the tendency direction of abnormal interactive control.

[0015] The abnormal control association analysis module is used to extract the voice control instructions of the abnormal interactive control records to identify the complexity of the instructions, and then associate the control effect of the abnormal interactive control records with the complexity of the instructions for analysis.

[0016] The interactive control adjustment determination module is used to determine the adjustment direction of the audio interactive control based on the tendency of the abnormal interactive control and the correlation between the control effect and the instruction complexity of the abnormal interactive control record.

[0017] The interactive control adjustment implementation module is used to perform audio interactive monitoring and control according to the adjustment direction of the audio interactive control.

[0018] The instruction module library is used to store template contents of different control instructions.

[0019] A third aspect of the present invention provides a storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the steps in an intelligent audio control method as described in the present invention.

[0020] Compared with the prior art, the advantages of the present invention are as follows: by focusing the interactive control of smart speakers on home appliances on the control effect, the present invention can screen out interactive control records with abnormal control effects based on historical interactive control records, and then conduct a correlation analysis between the abnormality tracing and the complexity of voice control instructions, thereby deriving a targeted adjustment direction for the interactive control of the speakers, realizing feedback closed-loop adjustment of the interactive control of the speakers, and to a certain extent accelerating the continuous optimization and improvement of the interactive control of the speakers, so that the user experience is continuously improved, which is conducive to the sustainable development of smart speakers. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0022] Figure 1 The present invention is a flowchart of the steps for implementing the method.

[0023] Figure 2 This is a system module connection diagram of the present invention.

[0024] Figure 3 This is a block diagram of the device connections involved in the application of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Example 1

[0027] The present invention provides a smart speaker control method, comprising a smart speaker, a router, home appliances and a cloud platform, wherein the smart speaker obtains the user's current instructions through voice collection and transmits control instructions through the router to control the home appliances to perform corresponding actions, and the router is connected to the cloud platform through a home network, and uploads interactive control information to the cloud platform to form interactive control records.

[0028] In the implementation of the above solution, the router is the core device of the home LAN. It is responsible for connecting multiple devices to the same network. Generally, a wifi connection is used. Specifically, the router is responsible for forwarding data packets within the home network and between the home network and the Internet. When the user issues instructions through the smart speaker, these instructions need to be sent to the corresponding home devices through the router. Similarly, when home devices need to respond to instructions or upload status information, these data also need to be transmitted to the smart speaker or other control devices through the router. Therefore, smart speakers and home devices usually need to be connected to the home network to enable communication between devices. For the connection between the above devices, see Figure 3 .

[0029] See also Figure 1 As shown, the specific implementation of the present invention includes the following steps: (1) retrieving the historical interactive control records of the smart speaker on the home appliance from the cloud platform, and extracting control state parameters therefrom, specifically including control response time, control execution consistency and control execution time, thereby evaluating the control effect of the historical interactive control records.

[0030] It is important to know that the control response time refers to the time between the smart speaker collecting the voice control command and the home appliance receiving the voice control command. The control execution consistency refers to the consistency between the control result and the voice control command, that is, whether the result of the execution of the home appliance meets the user's expectations. The control execution time refers to the time between the home appliance receiving the voice control command and the end of the execution.

[0031] The control effect evaluation of the historical interactive control records is as follows: the control state parameters of each historical interactive control record are imported into the evaluation formula Get the control effect value of historical interactive control records , where Indicates the control response time. represents the ideal control response time, Indicates the control execution time. Indicates the reference control execution time, represents the control execution consistency, and The value is 0 or 1.

[0032] It should be noted that the ideal control response time is the control response time obtained when the smart speaker and home appliances are ideally connected through a router, that is, the control response time when there is no network delay or network interference. It can be estimated based on the bandwidth of the home network. For example, the ideal control response time is 1s.

[0033] It is further explained that the control execution consistency value is 1 when the control result is consistent with the voice control instruction, and the control execution consistency value is 0 when the control result is inconsistent with the voice control instruction.

[0034] Once again, it is important to note that The acquisition process is as follows: extract voice control instructions from each interactive control record, and classify the same voice control instructions to obtain several interactive control records involved in each control instruction.

[0035] In the example of the above solution, assuming that the home appliance is a lamp, the voice control commands of the smart speaker to the lamp include turning on the light, turning off the light, brightening the light, dimming the light, etc., thereby classifying the interactive control records corresponding to the same voice control commands.

[0036] The shortest control execution duration is extracted from a plurality of interactive control records involved in each voice control instruction as a reference control execution duration of the corresponding interactive control record.

[0037] It should be pointed out that the shortest control execution time usually represents the optimal execution path that home appliances can achieve and can be used as a benchmark for measuring other execution times.

[0038] The present invention can analyze the entire control process from multiple angles by performing control effect analysis based on control response time, control execution consistency and control execution time of historical interactive control records, which is beneficial to improving the accuracy of the analysis.

[0039] (2) Filter out abnormal interactive control records based on the control effect evaluation of historical interactive control records. The specific screening is as follows: compare the control effect value of each historical interactive control record with the set target control effect value, and filter out historical interactive control records with a value less than the target control effect value as abnormal interactive control records. The target control effect value is initially set. The purpose of setting is to assist in the screening of abnormal interactive control records. It can be set according to the optimization requirements. When the optimization requirements are high, a lower target control effect value needs to be set so that more abnormal interactive control records can be screened out. When the optimization requirements are not high, a higher target control effect value can be set to screen out a small number of abnormal interactive control records. By way of example, the target control effect value is 70%.

[0040] (3) The abnormal interaction control records are traced for abnormal control, and then the abnormal control traceability results are analyzed for tendency to obtain the tendency of abnormal interaction control.

[0041] In the optimized implementation of the above scheme, the abnormal control tracing of the abnormal interaction control record is implemented as follows: extracting the voice control instruction collection time, the voice control instruction transmission time and the voice control instruction content from the abnormal interaction control record.

[0042] The voice control instructions recorded in the abnormal interactive control are divided into voice segments and non-voice segments using a voice activity detection algorithm, wherein the non-voice segment is the noise segment.

[0043] The content of the voice control command is converted into the frequency domain to obtain a spectrogram, where the spectrogram of the voice signal is constructed with frequency as the horizontal axis and amplitude (or energy) as the vertical axis. The peak area in the spectrogram indicates that the signal has a strong component at a certain frequency, that is, the main frequency component. The main frequency component of the voice segment and its corresponding energy distribution characteristics are extracted from the spectrogram, and the signal energy of the voice segment is calculated. At the same time, the main frequency distribution of the non-voice segment and its corresponding energy distribution characteristics are extracted from the spectrogram, and the signal energy of the non-voice segment is calculated in the same way.

[0044] In the example implementation of the above solution, the signal energy calculation of the speech segment can adopt the short-time energy calculation method, that is, summing the square of the amplitude of the signal in the speech segment. The signal energy of the non-speech segment is calculated similarly with reference to the signal energy of the speech segment.

[0045] The signal energy of all voice segments and the signal energy of all non-voice segments are averaged, and the calculation results are used to calculate the signal-to-noise ratio of the voice control command. The specific signal-to-noise ratio calculation formula is: , where , They represent the average signal energy of speech segments and the average signal energy of non-speech segments respectively.

[0046] The present invention performs spectrum analysis on the voice control command and then extracts the signal-to-noise ratio. Compared with directly extracting the signal-to-noise ratio, it can more accurately separate the signal and noise components, improve the reliability and robustness of the signal-to-noise ratio, and thus improve the accuracy of the speech recognition system.

[0047] The voice control instructions are converted into text to obtain text content, and the text content is used to identify the instruction intent and entity using natural language processing software. The template content of the corresponding instruction is extracted from the instruction module library of the smart speaker, and the text content of the voice control instruction is overlapped with the template content of the corresponding instruction for semantic quality scoring.

[0048] It should be added that the intent in the instruction refers to the goal or task that the user wants to achieve through an instruction. In the field of natural language processing, an "intention" can be understood as a specific action that the user wants to perform or a type of information to obtain. The entity in the instruction refers to the specific object or parameter contained in the instruction issued by the user. Entities usually refer to specific things or concepts mentioned in the instruction, which can be devices, rooms, time, values, etc. In the example where the voice control instruction is "turn off the lights in the living room", the entities are "living room" and "lights", and the intention is: "turn off".

[0049] It should be further added that the command control library of the smart speaker usually stores template contents of different commands so that the system can correctly understand and execute the user's voice commands. The command library can be stored on the cloud platform for easy updating and synchronization.

[0050] In the optimization operation of the above scheme, the text content of the voice control instruction is overlapped with the template content of the corresponding instruction, and then the semantic quality score is performed. Specifically, the number of overlapping phrases can be counted and divided by the number of phrases divided by the text content of the voice control instruction to obtain the semantic quality score.

[0051] When judging the semantic quality of a voice control instruction, the present invention uses the result of the overlap between the text content converted from the voice control instruction and the template content of the corresponding instruction as a basis for judgment. On the one hand, the template content provides a standard for judgment, so that there is a unified scale for judging the semantic quality of different voice instructions, thereby enhancing the credibility of the semantic quality judgment. On the other hand, the judgment process is repeatable and easy to operate, thereby improving the judgment efficiency to a certain extent.

[0052] The voice control command acquisition utility coefficient is calculated by taking the weighted average of the signal-to-noise ratio and the semantic quality score of the voice control command, where both the signal-to-noise ratio and the semantic quality score have a positive impact on the voice control command acquisition utility coefficient.

[0053] In the specific implementation of the above solution, the weight values ​​of the signal-to-noise ratio and the semantic quality score can be assigned weights according to actual needs and importance. For example, if the signal-to-noise ratio is more important, the weight values ​​corresponding to the signal-to-noise ratio and the semantic quality score can be set to 0.7 and 0.3, and vice versa, if the semantic quality score is more important, the weight values ​​corresponding to the signal-to-noise ratio and the semantic quality score can be set to 0.3 and 0.7.

[0054] During the time period formed by the voice control command collection time and the voice control command transmission time, the operation indicators of the smart speaker and home appliances and the transmission status indicators of the home network are retrieved respectively, and then the operation indicators of the smart speaker are compared with the corresponding normal operation indicators, and the operation compliance of the smart speaker is calculated. The operation indicators of the home appliances are compared with their normal operation indicators, and the operation compliance of the home appliances is calculated. At the same time, the transmission status indicators of the home network are compared with the ideal transmission status indicators, and the transmission compliance of the home network is calculated.

[0055] In the example of the above scheme, the operating indicators of the smart speaker include but are not limited to the color, current, voltage, etc. of the status indicator light. The operation compliance of the smart speaker can be calculated by subtracting the operating indicator of the smart speaker from the normal operating indicator, taking the absolute value and dividing the result by the normal operating indicator, and then taking three quarters as the base and using the result as the exponent to construct an exponential function to obtain the operation compliance of the smart speaker.

[0056] In another example of the above scheme, when the household appliance is a lamp, its operating indicators include but are not limited to brightness, color temperature, current, voltage, etc., wherein the calculation of the operating compliance of the household appliance is similarly referred to the calculation of the operating compliance of the smart speaker.

[0057] In another example of the above solution, the transmission status indicators of the home network include but are not limited to transmission speed, transmission delay duration, transmission packet loss rate, etc. The calculation of the home network transmission compliance is as follows: , where , , They represent the transmission speed, transmission delay, and transmission packet loss rate of the home network respectively. , , They represent the ideal transmission speed, ideal transmission delay time, and ideal transmission packet loss rate of the home network respectively.

[0058] It is important to know that the normal operating indicators of smart speakers and home appliances can be extracted from their user manuals based on the device specifications and models, and the ideal transmission status indicators of the home network can be obtained based on the home network bandwidth.

[0059] The voice control command collection utility coefficient, smart speaker operation compliance, home appliance operation compliance and home network transmission compliance corresponding to each abnormal interactive control record are compared with the corresponding set thresholds respectively. If some parameters are less than the set threshold, the parameter is used as the abnormal direction of the abnormal interactive control record. Specifically, when the voice control command collection utility coefficient is less than the set threshold, the abnormal direction is determined to be poor voice control command recognition; when the smart speaker operation compliance is less than the set threshold, the abnormal direction is determined to be abnormal smart speaker operation; when the home appliance operation compliance is less than the set threshold, the abnormal direction is determined to be abnormal home appliance operation; when the home network transmission compliance is less than the set threshold, the abnormal direction is determined to be network transmission abnormality.

[0060] It should be added that there is more than one situation where the abnormal interaction control record corresponds to the abnormal pointing.

[0061] The present invention can help accurately locate the root cause of the abnormal interaction problem by tracing the abnormal interaction control records in multiple dimensions from the recognition utility of voice control instructions, the operating status of home appliances, the operating status of smart speakers, and the transmission status of home networks. At the same time, by systematically analyzing the abnormal records, the scope of troubleshooting can be quickly narrowed, unnecessary inspection steps can be reduced, and the efficiency of troubleshooting can be improved. In addition, a more refined monitoring mechanism can be established through the analysis of abnormal records to achieve real-time monitoring and early warning of the system.

[0062] In the further implementation of the above scheme, the abnormal control tracing results are subjected to the following tendency analysis process: the abnormal direction of each abnormal interaction control record is classified according to the same abnormal direction, and the number of abnormal interaction control records classified by each abnormal direction is summarized, and then the abnormal direction corresponding to the largest number of abnormal interaction control records is taken as the tendency direction of the abnormal interaction control.

[0063] By summarizing the abnormality directions, the present invention can clearly identify the most common abnormality directions, perform priority processing, optimize resource allocation and fault repair strategies, and at the same time, by solving the most common abnormal problems, it can significantly improve the user experience and satisfaction and reduce the user's frustration and distress.

[0064] (4) Extract the voice control commands of the abnormal interactive control records to identify the command complexity, and then correlate and analyze the control effects of the abnormal interactive control records with the command complexity.

[0065] As a specific implementation of the above solution, the instruction complexity identification process is as follows: the voice control instructions of each abnormal interactive control record are processed by natural language processing software, and the number of entities and intentions extracted are counted.

[0066] It is important to understand that the number of intents in a command refers to the number of goals or tasks that the user wants to achieve through a command. When a voice control command issued by a user contains multiple goals, we say that the command has multiple intents.

[0067] In the specific first example, “turn off the lights in the living room” has only one intention: turn off the lights.

[0068] In the specific second example, "Turn off the lights in the living room and adjust the air-conditioning temperature to 25 degrees" has two intentions: turning off the lights and adjusting the air-conditioning temperature.

[0069] Generally speaking, multi-intent commands are usually more complex than single-intent commands because they require the system to understand and process multiple requests at the same time. Therefore, the number of intents in a command can be used as a reflection of the complexity of the command.

[0070] It is important to further understand that the number of entities in a command refers to the number of specific objects or parameters contained in the command issued by the user. Entities usually refer to specific things or concepts mentioned in the command, which can be devices, rooms, time, values, etc. Identifying the number of entities in a command helps the system better understand the user's intention and perform corresponding operations.

[0071] As an embodiment, the entity of “turn on the light in the living room” is “the light in the living room”.

[0072] The entity in “Turn off the air conditioner in the bedroom” is “bedroom”.

[0073] The entity of "adjust the temperature to 25 degrees" is "25 degrees".

[0074] The entity of "Wake me up at 7 am" is "7 am".

[0075] Generally speaking, the more entities there are in an instruction, the greater the difficulty of semantic parsing and the processing complexity of the instruction. Therefore, the number of entities in an instruction can be used as a factor reflecting the complexity of the instruction.

[0076] After the voice control instructions of each abnormal interactive control record are converted into text, the text length is counted, and the grammatical structure analysis is performed to obtain the grammatical structure type, specifically including simple sentences and complex sentences.

[0077] Once again, it is important to understand that the length of the text of the voice control command does reflect the complexity of the command to some extent. Generally speaking, longer commands tend to contain more information and are therefore likely to be more complex.

[0078] Finally, it is important to understand that the grammatical structure also plays a role in the complexity of voice control commands. Generally speaking, the more complex the grammatical structure, the higher the complexity. For example, complex sentences and sentences containing clauses are usually more complex than simple sentences.

[0079] It should be added that the grammatical structure type of the voice control instruction can be distinguished according to the number of sentences that the voice control instruction text can be divided into. Generally speaking, the more sentences there are, the more complex the grammatical structure type is.

[0080] In the above supplementary example, "turn on the light in the living room" can only be divided into a single clause, and the grammatical structure type is a simple sentence.

[0081] In another example described above, "at eight o'clock in the evening, adjust the lights in the living room to 50% brightness, turn on the TV, and switch to the movie channel" can be divided into multiple clauses, and the grammatical structure type is a complex sentence.

[0082] The number of entities, number of intentions, text length and grammatical structure type of the voice control instructions corresponding to each abnormal interaction control record are standardized to obtain the standard value of the entity, standard value of the intention, standard value of the text length and standard value of the grammatical structure of the voice control instructions corresponding to each abnormal interaction control record, and the weighted average is calculated to obtain the instruction complexity corresponding to each abnormal interaction control record.

[0083] The above-mentioned standardization of the number of entities, the number of intentions, the text length and the grammatical structure type is to ensure the comparability between different features. For example, each feature can be converted into a value between 0 and 1. Specifically, the conversion can set an upper limit reference value for the number of entities, the number of intentions, and the text length, and compare the number of entities, the number of intentions, and the text length with the upper limit reference value to perform the conversion. The grammatical structure type can convert simple sentences into 0.5 and complex sentences into 1.

[0084] In the above specific embodiment, assume that the voice control command is: "At 8 o'clock in the evening, adjust the living room light to 50% brightness, turn on the TV, and switch to the movie channel. The number of entities extracted: 4 (8 o'clock in the evening, living room light, 50%, movie channel); the number of intents: 3 (adjust brightness, turn on TV, switch channels); text length: 24 Chinese characters, grammatical structure type: complex sentence.

[0085] The process of associating and analyzing the control effect and instruction complexity of the abnormal interactive control record is as follows: a two-dimensional coordinate system is constructed with the instruction complexity as the horizontal coordinate and the control effect value as the vertical coordinate, and a correlation curve between the control effect and instruction complexity is formed in the constructed two-dimensional coordinate system for the control effect value and instruction complexity corresponding to each abnormal interactive control record.

[0086] Based on the correlation curve between control effect and instruction complexity, the correlation coefficient is used to calculate the correlation between control effect and instruction complexity.

[0087] It should be noted that the correlation coefficient can be calculated using the Pearson correlation coefficient or the Spearman rank correlation coefficient, where the Pearson correlation coefficient is applicable to the linear relationship between two variables, and the Spearman rank correlation coefficient is applicable to the nonlinear relationship between two variables. The specific selection can be based on the specific relationship type formed in the complex correlation curve between the control effect and the instruction. The value range of the Pearson correlation coefficient or the Spearman rank correlation coefficient is -1 to 1, where +1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation.

[0088] Comparing the correlation between control effect and instruction complexity with the preset effective correlation, it should be understood that the present invention studies the negative correlation between control effect and instruction complexity, that is, the greater the instruction complexity, the smaller the control effect. If the correlation between instruction complexity reaches the effective correlation, it is determined that there is a correlation between control effect and instruction complexity. Otherwise, it is determined that there is no correlation between control effect and instruction complexity.

[0089] (5) Based on the tendency of abnormal interactive control and the correlation between the control effect and the complexity of the instructions recorded by the abnormal interactive control, the adjustment direction of the audio interactive control is determined as follows: when using smart speakers to interactively control home appliances, the monitoring frequency of the tendency is increased.

[0090] Create a standard style of voice control commands, and use the voice assistant to collect the user's voice control commands in real time to identify whether they are complex commands. If they are complex commands, transform the collected voice control commands according to the standard style of voice control commands, and then feed back the transformed voice control commands to the user, and guide the user to repeat the corresponding commands. This method can improve the recognition accuracy of voice control commands and reduce recognition errors caused by complex commands.

[0091] (6) Perform audio interaction monitoring and control according to the adjustment direction of the audio interaction control.

[0092] Example 2

[0093] Reference Figure 2 As shown, the present invention proposes an intelligent audio control system, comprising the following modules: a historical control effect evaluation module, which is used to retrieve the historical interactive control records of the intelligent audio on the home appliances from the cloud platform, and extract the control state parameters therefrom, specifically including the control response time, the control execution consistency and the control execution time, thereby evaluating the control effect of the historical interactive control records;

[0094] The abnormal interaction control record screening module is connected to the historical control effect evaluation module and is used to screen out abnormal interaction control records based on the control effect evaluation of historical interaction control records;

[0095] The abnormal control tracing and parsing module is connected to the abnormal interaction control record screening module, and is used to perform abnormal control tracing on the abnormal interaction control records, and then perform tendency analysis on the abnormal control tracing results to obtain the tendency direction of abnormal interaction control;

[0096] The abnormal control association analysis module is connected to the abnormal interaction control record screening module, and is used to extract the voice control instructions of the abnormal interaction control records to identify the complexity of the instructions, and then associate and analyze the control effect of the abnormal interaction control records with the complexity of the instructions;

[0097] The interactive control adjustment determination module is connected to the abnormal control source tracing analysis module and the abnormal control association analysis module respectively, and is used to determine the adjustment direction of the audio interactive control based on the tendency of the abnormal interactive control and the correlation between the control effect and the command complexity of the abnormal interactive control record;

[0098] An interactive control adjustment implementation module, connected to the interactive control adjustment determination module, is used to perform audio interaction monitoring and control according to the adjustment direction of the audio interaction control;

[0099] The instruction module library is connected to the abnormal control tracing and parsing module and is used to store the template content of different control instructions.

[0100] Example 3

[0101] The present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in an intelligent audio control method as described in the present invention.

[0102] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A smart speaker control method, comprising a smart speaker, a router, a home appliance and a cloud platform, wherein the smart speaker obtains the user's current command through voice collection and transmits the control command through the router to control the home appliance to perform the corresponding action, the router is connected to the cloud platform through a home network, and the interactive control information is uploaded and stored in the cloud platform to form an interactive control record, characterized in that: The following steps are involved: (1) Retrieve the historical interactive control records of smart speakers on home devices from the cloud platform and extract control state parameters from them, including control response time, control execution consistency, and control execution time, so as to evaluate the control effect of the historical interactive control records; (2) Screen out abnormal interactive control records based on the control effect evaluation of historical interactive control records; (3) Trace the abnormal control source of the abnormal interaction control records, and then analyze the abnormal control source tracing results to obtain the tendency of abnormal interaction control; (4) Extract the voice control commands of the abnormal interactive control records to identify the command complexity, and then correlate and analyze the control effect of the abnormal interactive control records with the command complexity; (5) Determine the adjustment direction of the audio interaction control based on the tendency of abnormal interaction control and the correlation between the control effect and the complexity of the command recorded by the abnormal interaction control; (6) Performing audio interaction monitoring and control according to the adjustment direction of the audio interaction control; The control effect evaluation of the historical interactive control record is as follows: Import the control state parameters of each historical interactive control record into the evaluation formula Get the control effect value of historical interactive control records , where Indicates the control response time. represents the ideal control response time, Indicates the control execution time. Indicates the reference control execution time, represents the control execution consistency, and The value is 0 or 1. The acquisition process is as follows: Extracting voice control instructions from each interactive control record, and classifying the same voice control instructions to obtain a number of interactive control records involved in each control instruction; Extracting the shortest control execution time from a number of interactive control records involved in each voice control instruction as a reference control execution time of the corresponding interactive control record; The abnormal control tracing of abnormal interaction control records is implemented as follows: Extracting the voice control command collection time, voice control command transmission time and voice control command content from the abnormal interactive control record; The signal-to-noise ratio and semantic quality score of the voice control command are analyzed according to the content of the voice control command in the abnormal interactive control record, and the voice control command acquisition utility coefficient is obtained by combining the two evaluations; During the period of voice control command collection and transmission, the smart speaker operation compliance, home appliance operation compliance and home network transmission compliance are analyzed, and the abnormal direction of the abnormal interactive control record is determined based on the threshold comparison results of the voice control command collection utility coefficient, smart speaker operation compliance, home appliance operation compliance and home network transmission compliance.

2. The intelligent audio control method according to claim 1, characterized in that: The abnormal interaction control records are screened as follows: The control effect value of each historical interactive control record is compared with the set target control effect value, and the historical interactive control records with a control effect value less than the target control effect value are selected as abnormal interactive control records.

3. The intelligent audio control method according to claim 1, characterized in that: The specific implementation of abnormal control tracing for abnormal interaction control records is as follows: The voice control instruction content of the abnormal interactive control record is divided into voice segments and non-voice segments using a voice activity detection algorithm; The voice control instruction content is converted into the frequency domain to obtain a spectrogram, and the main frequency components of the voice segment and its corresponding energy distribution characteristics are extracted from the spectrogram, thereby calculating the signal energy of the voice segment. At the same time, the main frequency distribution of the non-voice segment and its corresponding energy distribution characteristics are extracted from the spectrogram, and the signal energy of the non-voice segment is calculated in the same way; The signal energies of all speech segments and the signal energies of all non-speech segments are averaged, and the calculation results are used to calculate the signal-to-noise ratio of the voice control command; The voice control command content is converted into text content, and the text content is used to identify the command intent and entity using natural language processing software, thereby extracting the template content of the corresponding command from the command module library of the smart speaker, and then the text content of the voice control command is overlapped with the template content of the corresponding command to perform semantic quality scoring; The signal-to-noise ratio and semantic quality score of the voice control command are weighted averaged to obtain the voice control command acquisition utility coefficient; In the period formed by the voice control command collection time and the voice control command transmission time, the operation indicators of the smart speakers and home appliances and the transmission status indicators of the home network are retrieved respectively, and then the operation indicators of the smart speakers are compared with the corresponding normal operation indicators, and the operation compliance of the smart speakers is calculated. The operation indicators of the home appliances are compared with their normal operation indicators, and the operation compliance of the home appliances is calculated. At the same time, the transmission status indicators of the home network are compared with the ideal transmission status indicators, and the transmission compliance of the home network is calculated. The voice control command collection utility coefficient, smart speaker operation compliance, home appliance operation compliance and home network transmission compliance corresponding to each abnormal interactive control record are compared with the corresponding set thresholds. If some parameters are less than the set threshold, the parameter will be used as the abnormal point of the abnormal interactive control record.

4. The intelligent audio control method according to claim 3, characterized in that: The process of analyzing the tendency of abnormal control tracing results is as follows: The abnormal directions of each abnormal interaction control record are classified according to the same abnormal direction, and the number of abnormal interaction control records classified by each abnormal direction is summarized, and then the abnormal direction corresponding to the largest number of abnormal interaction control records is taken as the tendency direction of abnormal interaction control.

5. The intelligent audio control method according to claim 1, characterized in that: The instruction complexity identification process is as follows: The voice control instructions of each abnormal interactive control record are processed by natural language processing software, and the number of entities and intentions extracted are counted; The voice control instructions of each abnormal interactive control record are converted into text and the text length is counted. At the same time, the grammatical structure analysis is performed to obtain the grammatical structure type, specifically including simple sentences and complex sentences; The number of entities, number of intentions, text length and grammatical structure type of the voice control instructions corresponding to each abnormal interaction control record are standardized to obtain the standard value of the entity, standard value of the intention, standard value of the text length and standard value of the grammatical structure of the voice control instructions corresponding to each abnormal interaction control record, and the weighted average is calculated to obtain the instruction complexity corresponding to each abnormal interaction control record.

6. The intelligent audio control method according to claim 5, characterized in that: The process of associating and analyzing the control effect of the abnormal interaction control record with the instruction complexity is as follows: A two-dimensional coordinate system is constructed with the instruction complexity as the horizontal coordinate and the control effect value as the vertical coordinate, and a correlation curve between the control effect value and the instruction complexity corresponding to each abnormal interactive control record is formed in the constructed two-dimensional coordinate system; Based on the correlation curve between control effect and instruction complexity, the correlation coefficient is used to calculate the correlation between control effect and instruction complexity; The correlation between the control effect and the instruction complexity is compared with the preset effective correlation. If the correlation between the control effect and the instruction complexity reaches the effective correlation, it is recognized that there is a correlation between the control effect and the instruction complexity. Otherwise, it is recognized that there is no correlation between the control effect and the instruction complexity.

7. The intelligent audio control method according to claim 1, characterized in that: The operation of determining the adjustment direction of the audio interactive control is as follows: Increase the frequency of monitoring for orientation when using smart speakers to interactively control home devices; When there is a correlation between the recognition control effect and the complexity of the command, a standard style of voice control commands is created, and the user's voice control commands are collected in real time with the help of a voice assistant to identify whether they are complex commands. If they are identified as complex commands, the collected voice control commands are transformed according to the standard style of voice control commands, and the transformed voice control commands are then fed back to the user, guiding the user to restate the corresponding commands.

8. An intelligent sound control system, used to execute the method according to any one of claims 1 to 7, characterized in that: Includes the following modules: The historical control effect evaluation module is used to retrieve the historical interactive control records of the smart speaker on the home device from the cloud platform, and extract the control state parameters from it, including the control response time, control execution consistency and control execution time, so as to evaluate the control effect of the historical interactive control records; An abnormal interaction control record screening module is used to screen out abnormal interaction control records based on the control effect evaluation of historical interaction control records; The abnormal control tracing and analysis module is used to trace the abnormal control of the abnormal interaction control records, and then analyze the abnormal control tracing results to obtain the tendency of abnormal interaction control; The abnormal control correlation analysis module is used to extract the voice control instructions of the abnormal interactive control records to identify the complexity of the instructions, and then correlate and analyze the control effect of the abnormal interactive control records with the complexity of the instructions; An interactive control adjustment determination module, used to determine the adjustment direction of the audio interactive control based on the tendency of abnormal interactive control and the correlation between the control effect and the command complexity of the abnormal interactive control record; An interactive control adjustment implementation module is used to perform audio interaction monitoring and control according to the adjustment direction of the audio interaction control; The instruction module library is used to store template contents of different control instructions.

9. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the intelligent audio control method as described in any one of claims 1-7.

Images