A voice recognition noise reduction optimization method and system

Abstract

The embodiment of the application provides a speech recognition noise reduction optimization method and system, and relates to the technical field of speech recognition. The method comprises the following steps: receiving an audio signal containing residual noise; performing noise type identification on the audio signal to obtain a residual noise type and intensity; extracting speech features from the audio signal according to the residual noise type and intensity; generating a preliminary recognition result according to the speech features, and calculating an initial confidence degree of the preliminary recognition result; and performing a recognition decision according to the preliminary recognition result and the initial confidence degree. The application can improve the accuracy and robustness of speech recognition.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a speech recognition noise reduction optimization method and system. Background Technology

[0002] In related technologies, voice recognition devices are widely used in smart home environments, typically with built-in noise reduction functions to combat background noise. These systems are often optimized for common, relatively stable noise sources in the home (such as air conditioner hum and television background noise) from the outset. By learning the characteristics of these stable noises, they suppress them without affecting the voice signal. This noise reduction method based on the assumption of stationary noise performs well in most everyday scenarios, ensuring clear recognition of voice commands. However, the increasing dynamism and complexity of the home environment pose significant challenges to traditional noise reduction methods. For example, cooking in the kitchen generates a series of non-stationary, transient, wide-frequency-distribution, and drastically varying noises, such as frying sounds, boiling water sounds, and the broadband noise of the range hood. The acoustic characteristics of these noises are drastically different from the stationary noise assumed by traditional models, leading to a significant decrease in the effectiveness of the original noise reduction model. A large amount of residual noise is incorrectly fed into the voice recognition engine, severely impacting recognition accuracy. Furthermore, when family members multitask in the same space, such as watching TV or using a tablet while cooking, children playing with toys that make sounds, or pets suddenly barking, competing speech interference, intermittent high-frequency noise, and sudden high-intensity impact noise are introduced. These noise sources of different natures combine to form a complex, composite noise background. Traditional noise reduction methods struggle to effectively distinguish between target user speech and non-target speech, and also find it difficult to simultaneously suppress multiple types of noise with vastly different characteristics. In such a harsh acoustic environment, the effective components of the speech signal are submerged and distorted by various noises, causing a sharp drop in the signal-to-noise ratio, resulting in a significant reduction in the performance of the speech recognition system, or even rendering it unable to provide effective service. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a speech recognition noise reduction optimization method and system, aiming to improve the accuracy and robustness of speech recognition.

[0004] In a first aspect, embodiments of this application provide a speech recognition noise reduction optimization method, including:

[0005] Receive audio signals containing residual noise;

[0006] The audio signal is subjected to noise type identification to obtain the residual noise type and intensity;

[0007] Based on the type and intensity of the residual noise, speech features are extracted from the audio signal;

[0008] Based on the speech features, a preliminary recognition result is generated, and the initial confidence level of the preliminary recognition result is calculated;

[0009] An identification decision is made based on the preliminary identification results and the initial confidence level.

[0010] According to some embodiments of this application, the step of performing an identification decision based on the preliminary identification result and the initial confidence level includes:

[0011] When the initial confidence level is lower than a preset threshold, obtain the status information and time information of active devices in the smart home system;

[0012] Based on the active device status information and the time information, the preliminary identification result is subjected to scene context-assisted judgment to obtain the scene matching factor;

[0013] The final confidence level is obtained by weighted fusion based on the initial confidence level and the scene matching factor.

[0014] Based on the final confidence level, an identification decision is made.

[0015] According to some embodiments of this application, the step of performing scene context-assisted judgment on the preliminary identification result based on the active device status information and the time information to obtain the scene matching factor includes:

[0016] Collect user presence sensor data, smart device interaction records, and active device status;

[0017] By analyzing the user presence sensor data, the smart device interaction records, the active device status, and the time information through Bayesian inference, the probability distribution of the user in the preset activity area can be inferred.

[0018] Based on the probability distribution and the content of the preliminary identification results, the degree of correlation between user intent and activity area is calculated to obtain the scene matching factor.

[0019] According to some embodiments of this application, the step of analyzing the user presence sensor data, the smart device interaction records, the active device status, and the time information through a Bayesian inference mechanism to infer the probability distribution of the user in a preset activity area includes:

[0020] Calculate the reliability weight of the user presence sensor data based on the historical accuracy and current signal strength.

[0021] Adjust the timeliness decay factor of the smart device interaction record based on the interval between the interaction time recorded by the smart device and the current time.

[0022] Based on the interaction type and duration of the smart device interaction records, assess the intent intensity of the smart device interaction records;

[0023] In Bayesian inference, the reliability weight is applied to the likelihood calculation corresponding to the user presence perceptron data, the timeliness decay factor and the intent intensity are applied to the likelihood calculation corresponding to the smart device interaction record, and the posterior probability of the user in the preset activity area is calculated by combining the active device status and the time information, so as to obtain the probability distribution of the user in the preset activity area.

[0024] According to some embodiments of this application, the step of calculating the posterior probability of a user in a preset activity area includes:

[0025] Maintain an activity pattern history record for each user, which records the duration of the user's stay and interaction frequency in various preset activity areas at different time periods;

[0026] When the smart home system detects changes in the environmental layout, or when the system discovers, through user presence sensor data belonging to the user and smart device interaction records belonging to the user, that a user's actual activity pattern in a specific area has a persistent deviation from the current prior probability, it triggers the prior probability update mechanism to obtain an updated prior probability.

[0027] In Bayesian inference, the updated prior probability is used to calculate the posterior probability of a user in a preset activity area.

[0028] According to some embodiments of this application, the step of obtaining the updated prior probability includes:

[0029] Retrieve user activity pattern history and current time period information;

[0030] Based on the user activity pattern history and the current time period information, calculate the short-term prior probability of each user in each preset activity area;

[0031] Based on the user activity pattern history, calculate the long-term prior probability of each user in each preset activity area;

[0032] The updated prior probability is obtained by weighted fusion of the short-term prior probability and the long-term prior probability.

[0033] According to some embodiments of this application, the step of calculating the posterior probability of a user in a preset activity area using the updated prior probability in Bayesian inference includes:

[0034] Maintain an activity feature vector for each user and each preset activity area. The activity feature vector includes the user's typical interactive device type, interaction frequency distribution, voice command content category distribution, and micro-motion sensor data pattern within a specific time period in that area.

[0035] When multiple users are active in the same preset activity area, the real-time activity features of each user in the area are extracted. The real-time activity features are obtained by analyzing the individual micro-motion patterns reflected in the user presence sensor data, the specific interaction objects and interaction sequences in the smart device interaction records, as well as the sound source localization and voiceprint recognition results.

[0036] In the likelihood calculation stage of Bayesian inference, the real-time activity features of each user are matched with the user's activity feature vector in the region, and the matching degree is calculated as the likelihood of the user in the current region.

[0037] In Bayesian inference, the updated prior probability is used to calculate the posterior probability of a user in a preset activity area.

[0038] According to some embodiments of this application, the step of matching each user's real-time activity features with the user's activity feature vector in the region during the likelihood calculation stage of Bayesian inference, and calculating the matching degree as the user's likelihood in the current region, includes:

[0039] When the real-time activity feature exhibits a multi-task parallel mode, the real-time activity feature is decomposed into multiple independent sub-activity features;

[0040] For each of the sub-activity features, it is matched with the corresponding part in the activity feature vector to calculate multiple sub-matching degrees;

[0041] The matching degree is obtained by weighting and fusing the multiple sub-matching degrees based on the correlation between the sub-activity features corresponding to each sub-matching degree and the preliminary identification result.

[0042] According to some embodiments of this application, the step of weightedly fusing the multiple sub-matching degrees based on the correlation between the sub-activity feature corresponding to each sub-matching degree and the preliminary identification result to obtain the matching degree includes:

[0043] When the semantic similarity between the sub-activity feature and the preliminary identification result is higher than a preset threshold, the semantic similarity is corrected according to the matching degree between the sub-activity feature and the specific operation object and operation behavior of the preliminary identification result;

[0044] Based on the historical activation frequency of the sub-activity features under different time periods and different device types, and the device type and time preference information contained in the preliminary identification results, the historical activation frequency weight of the sub-activity features is corrected;

[0045] Based on the corrected semantic similarity and the corrected historical activation frequency weight, the weights of the weighted fusion of the multiple sub-matching degrees are adjusted to obtain the matching degree.

[0046] Secondly, embodiments of this application provide a speech recognition noise reduction optimization system, including:

[0047] A signal receiving module for receiving audio signals containing residual noise and slightly distorted speech;

[0048] A noise identification module is used to identify the noise type of the audio signal and obtain the residual noise type and intensity.

[0049] The feature extraction module is used to extract speech features from the audio signal based on the type and intensity of the residual noise;

[0050] The preliminary recognition module generates a preliminary recognition result based on the speech features and calculates the initial confidence level of the preliminary recognition result;

[0051] The decision execution module executes the identification decision based on the preliminary identification results and the initial confidence level.

[0052] The technical solution according to the embodiments of this application has at least the following beneficial effects: The speech recognition noise reduction optimization method disclosed in this application receives an audio signal containing residual noise and identifies the noise type and intensity of the audio signal. Based on this, speech features are extracted from the audio signal according to the identified residual noise type and intensity, thereby enabling targeted speech feature extraction and avoiding feature extraction distortion caused by unclear noise characteristics in complex noise environments, as is the case with traditional methods. Subsequently, a preliminary recognition result is generated based on the extracted speech features, and the initial confidence level of the preliminary recognition result is calculated. Finally, a recognition decision is executed based on the preliminary recognition result and the initial confidence level. This method can effectively solve the problem that in the prior art, traditional noise reduction methods significantly reduce noise reduction performance when facing complex noise that is non-stationary, highly transient, has a wide frequency distribution, and drastic intensity changes, as well as composite noise backgrounds introduced by multi-task parallel activities, resulting in a serious impairment of speech recognition accuracy. By accurately identifying the type and intensity of noise, this application can more intelligently adjust the speech feature extraction strategy and make decisions based on recognition confidence, thereby significantly improving the accuracy and robustness of speech recognition in complex and ever-changing smart home environments.

[0053] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0054] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0055] Figure 1 This is a flowchart illustrating a speech recognition noise reduction optimization method provided in one embodiment of this application;

[0056] Figure 2 This is a schematic diagram of a speech recognition noise reduction optimization system provided in one embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical methods, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] It should be noted that the meaning of "multiple" (or "more than") in the description of the embodiments of this application refers to two or more, and "greater than," "less than," "exceeding," etc. are understood to exclude the number itself, while "above," "below," "within," etc. are understood to include the number itself. If "first," "second," etc. are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0059] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: the existence of a alone, the existence of b alone, the existence of c alone, the simultaneous existence of a and b, the simultaneous existence of a and c, the simultaneous existence of b and c, or the simultaneous existence of a, b, and c, where a, b, and c can be single or multiple.

[0060] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0061] Based on the above, this application proposes a speech recognition noise reduction optimization method and system, aiming to improve the accuracy and robustness of speech recognition.

[0062] The speech recognition noise reduction optimization method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application that implements the speech recognition noise reduction optimization method, etc., but is not limited to the above forms.

[0063] This application can be applied to numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via communication networks. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices. It should be noted that in various specific embodiments of this invention, when processing is required based on data related to the characteristics of an object (e.g., user attributes or sets of attribute information), permission or consent from the corresponding object is obtained first, and the collection, use, and processing of this data comply with relevant laws and standards. Furthermore, when the embodiments of the present invention need to obtain the attribute information of an object, they will obtain the separate permission or separate consent of the corresponding object through pop-up windows or redirection to a confirmation page. After obtaining the separate permission or separate consent of the corresponding object, they will then obtain the relevant data of the object necessary for the embodiments of the present invention to operate normally.

[0064] See Figure 1 , Figure 1 This is a flowchart illustrating a speech recognition noise reduction optimization method according to an embodiment of this application. The speech recognition noise reduction optimization method provided in this embodiment includes, but is not limited to, steps S110 to S150, which will be described in detail below.

[0065] Step S110: Receive an audio signal containing residual noise;

[0066] Step S120: Identify the noise type of the audio signal to obtain the residual noise type and intensity;

[0067] Step S130: Extract speech features from the audio signal based on the type and intensity of residual noise;

[0068] Step S140: Generate preliminary recognition results based on speech features, and calculate the initial confidence level of the preliminary recognition results;

[0069] Step S150: Execute the identification decision based on the preliminary identification results and initial confidence level.

[0070] It should be noted that "audio signal" refers to the raw sound data collected through a microphone or other acoustic sensors, which may contain the target user's voice commands, background noise, and other interfering sounds. "Residual noise" refers to noise components that remain in the audio signal after initial noise reduction processing and have not been completely eliminated. This residual noise may be due to limitations of the noise reduction algorithm, the complexity of noise characteristics, or instantaneous changes in noise intensity. "Speech features" refer to key information extracted from the audio signal that characterizes the speech content, such as Mel-frequency cepstral coefficients (MFCC), Linear Predictive Coding (LPC) coefficients, or Perceptual Linear Predictive (PLP) coefficients. These features are used in subsequent speech recognition processes. "Preliminary recognition result" refers to the text or semantic result obtained by the speech recognition engine after preliminary decoding of the user's voice command based on the extracted speech features. "Initial confidence" is a quantitative assessment of the reliability of the preliminary recognition result, usually expressed as a probability value, reflecting the recognition engine's confidence in the correctness of the result. "Recognition decision" refers to the subsequent actions taken by the system based on the preliminary recognition result and initial confidence, such as executing the command, requesting user confirmation, or providing alternative solutions.

[0071] In one embodiment, the system receives an audio signal containing residual noise. When a user issues a voice command to the smart speaker, the speaker's microphone captures ambient sounds, forming a raw audio stream. The audio signal is then analyzed to identify the type and intensity of the residual noise. Noise can be classified by analyzing parameters such as the audio signal's spectral characteristics, temporal envelope, and short-time energy, combined with machine learning models (such as Support Vector Machines (SVM), Neural Networks (NN), or Gaussian Mixture Models (GMM)). Noise types may include, but are not limited to: human voice interference, music interference, ambient white noise, mechanical noise (such as fan noise, air conditioner noise), and transient impact noise (such as door closing sound, object falling sound). Simultaneously, the system quantifies the intensity of each type of noise, for example, by calculating the signal-to-noise ratio (SNR) or noise power spectral density (PSD). Furthermore, speech features are extracted from the audio signal based on the residual noise type and intensity. After identifying the type and intensity of the residual noise, the system can employ an adaptive speech feature extraction strategy. For example, if the main noise type is identified as low-frequency mechanical noise, the parameters of the filter bank can be adjusted to enhance the extraction of mid-to-high frequency speech components. If human voice interference is identified, source separation technology can be used to attempt to separate the target speech from the interfering human voice before feature extraction. This noise-aware feature extraction method can retain the effective information of the speech signal to the greatest extent while suppressing the negative impact of noise. Based on the speech features, a preliminary recognition result is generated, and the initial confidence of the preliminary recognition result is calculated. The extracted speech features are fed into the speech recognition engine to generate a preliminary text or semantic recognition result. For example, if the user says "turn on the living room light," the preliminary recognition result might be "turn on the living room light." At the same time, the speech recognition engine calculates the initial confidence of this preliminary recognition result. The initial confidence reflects the recognition engine's confidence in the correctness of the result, and is usually a probability value between 0 and 1. For example, if the recognition engine is very certain about the recognition result of "turn on the living room light," its initial confidence might be 0.95; if there is ambiguity, the confidence might be lower, such as 0.6. Finally, a recognition decision is executed based on the preliminary recognition result and the initial confidence. After obtaining preliminary identification results and initial confidence levels, the system makes a final identification decision based on this information. For example, if the initial confidence level is higher than a preset threshold (e.g., 0.8), the system can directly execute the instruction corresponding to the preliminary identification result, such as "turn on the living room light." If the initial confidence level is low, the system may not execute the instruction immediately but will adopt other strategies, such as requesting confirmation from the user ("Do you want to turn on the living room light?"), providing a list of alternative instructions, or combining other contextual information for auxiliary judgment. This confidence-based decision-making mechanism avoids blindly executing instructions when identification is uncertain, thereby improving the system's reliability and user experience.

[0072] It should be noted that the initial confidence level refers to the reliability assessment of the preliminary identification results, and its value is usually between 0 and 1, representing the probability that the identification result is correct. When the initial confidence level is lower than the preset threshold, it indicates that the preliminary identification results may have uncertainties or ambiguities, requiring further auxiliary judgment. The preset threshold can be set according to the needs of the actual application scenario and system performance, for example, it can be set to 0.7 or 0.8. Active device status information refers to the relevant data of devices currently in working status or activated by the user in the smart home system, such as the on / off status of smart lights, the content being played on a smart TV, the playlist of a smart speaker, and the operating mode of a smart air conditioner. Time information can be understood as the current date, time, day of the week, season, etc., and its purpose is to provide contextual information in the time dimension. This information can be obtained in real time from the central controller of the smart home system or from various smart devices. In practical applications, scene context auxiliary judgment is performed on the preliminary identification results to obtain the scene matching factor. This refers to using the obtained active device status information and time information, combined with preset scene rules or machine learning models, to evaluate the degree of matching between the preliminary identification results and the current smart home environment and user activity scenario. The scene matching factor is a quantitative indicator used to represent the reasonableness or probability of the preliminary identification result in the current scene. Further, the final confidence score is obtained by weighted fusion based on the initial confidence score and the scene matching factor. This involves combining the initial confidence score of the preliminary identification result with the scene matching factor to obtain a more comprehensive and reliable confidence assessment. Weighted fusion can be implemented through linear weighting, nonlinear weighting, or other fusion algorithms, where the initial confidence score and the scene matching factor can be assigned different weights according to their importance. Therefore, the final confidence score can more accurately reflect the true reliability of the preliminary identification result in the current environment.

[0073] In one embodiment, suppose a user is in the living room with a TV playing a program in the background. The user gives the voice command "Turn on the lights" to a smart speaker. Due to background noise interference from the TV, the smart speaker's initial recognition result is "Turn on, etc.", with an initial confidence level of 0.6, lower than the preset threshold of 0.7. At this point, the system triggers a context-assisted judgment mechanism. First, the system obtains the status information of active devices in the smart home system. For example, it finds that the smart TV in the living room is on, while the smart lights in the living room are off. At the same time, the system obtains the current time information, for example, the current time is 8 pm. Then, based on this active device status information and time information, the system performs a scene context-assisted judgment on the initial recognition result "Turn on, etc." Considering that 8 pm is usually the time when users are active at home, and living room lights are usually turned on at night, and that "Turn on, etc." and "Turn on the lights" are highly similar in pronunciation, and that "Turn on the lights" highly matches the current scene (night, lights off), the system calculates a high scene matching factor, for example, 0.9. The system then weights and fuses the initial confidence level of 0.6 with the scene matching factor of 0.9, for example, by using a simple average weighting (0.6 * 0.5 + 0.9 * 0.5 = 0.75), resulting in a final confidence level of 0.75. Since the final confidence level of 0.75 is higher than the preset decision threshold of 0.7, the system ultimately confirms the recognition result as "turn on the light" and executes the corresponding command, successfully turning on the smart lights in the living room.

[0074] It's important to clarify that user presence sensor data refers to data reflecting a user's physical location or state of being in a smart home environment. This data is collected through devices such as infrared sensors, millimeter-wave radar, ultrasonic sensors, and cameras. This data indicates whether the user is in a specific room or area, or their movement trajectory. Its purpose is to provide direct evidence of the user's physical location. Smart device interaction records refer to historical data of user interactions with various smart devices in the smart home system. Examples include smart light on / off records, smart speaker voice command records, smart TV viewing records, and smart door lock on / off records. These records reflect the user's behavioral habits, preferences, and current activities. Their purpose is to reveal user behavior patterns and potential intentions. Active device status refers to information about devices currently in a working or activated state within the smart home system. Examples include whether lights in a room are on, whether the air conditioner is running, and whether the television is playing a program. This status information provides an objective description of the current environment, helping to determine the user's location. Its purpose is to provide real-time context of the current environment. Preset activity areas can be physical spaces with specific functions or uses that are pre-defined in a smart home environment, such as living rooms, bedrooms, kitchens, and studies.

[0075] It's important to note that the historical accuracy of user presence sensor data can be obtained through statistical analysis comparing past sensor data with the actual user location, while the current signal strength reflects the quality of the sensor data in the current environment. By comprehensively considering these two factors, a reliability weight for the user presence sensor data can be calculated. This weight quantifies the reliability of the data in Bayesian inference. For example, when the historical accuracy is high and the current signal strength is good, the reliability weight will be set to a higher value, and vice versa. Smart device interaction records refer to historical data of user interactions with devices in the smart home system, such as records of turning lights on and off, adjusting air conditioning, and playing music. The shorter the interval between the interaction time and the current time, the stronger the timeliness of the interaction record, and the greater its indicative significance of the current user state. The timeliness decay factor can be designed as a function that decreases with increasing time interval, such as an exponential decay function, to ensure that recent interaction records occupy a more significant position in inference. The interaction type and duration of smart device interaction records can be used to assess the strength of user intent. For example, prolonged television viewing or continuous temperature adjustment may indicate a clear intention to engage in activity within that area, while brief light switching operations may suggest a lower intention intensity. Intention intensity can be evaluated using preset rules or machine learning models; for instance, assigning a higher intention intensity value to the interaction type "watching a movie" and a lower value to "briefly turning on the lights." A reliability weight is applied to the likelihood calculation corresponding to the user's presence sensor data, meaning that the more reliable the sensor data, the greater its contribution to the likelihood. A timeliness decay factor and intention intensity are applied to the likelihood calculation corresponding to smart device interaction records, ensuring that recent and clearly intentional interaction records have a more significant impact on the likelihood. By combining active device status and time information, the posterior probability of a user within a preset activity area can be comprehensively calculated, resulting in a more accurate probability distribution of the user within that area.

[0076] In one embodiment, assume the smart home system includes three preset activity areas: a living room, a bedroom, and a kitchen. When a user is in the living room, the millimeter-wave radar (user presence sensor) detects subtle movements, with a historical accuracy of 95% and a current signal strength of medium. Based on this information, the system calculates a reliability weight of 0.8 for the millimeter-wave radar data. Simultaneously, 5 minutes prior, the user turned on the living room television via voice command (smart device interaction record), with the interaction type being "entertainment" and a duration of 2 hours. The system adjusts the timeliness attenuation factor to 0.95 based on the interval between the interaction time and the current time (5 minutes). Based on the interaction type "entertainment" and duration, the intent strength is assessed as 0.9 (high intent). During the likelihood calculation phase of Bayesian inference, the likelihood corresponding to the living room millimeter-wave radar data is multiplied by the reliability weight of 0.8. The likelihood corresponding to the living room television interaction record is multiplied by both the timeliness attenuation factor of 0.95 and the intent strength of 0.9. If the user briefly opened the refrigerator in the kitchen 1 hour prior (smart device interaction record), with the interaction type being "retrieve item" and a duration of 1 minute... The system adjusts the timeliness decay factor to 0.6 based on the time interval (1 hour). Based on the interaction type "object retrieval" and duration, the intent strength is assessed at 0.4 (low intent). In Bayesian inference, the likelihood corresponding to the kitchen refrigerator interaction record is multiplied by the timeliness decay factor of 0.6 and the intent strength of 0.4. In this way, the system can more finely weigh the contributions of different data sources. For example, recent high-intent interactions in the living room and reliable sensor data will have a significant positive impact on the user's posterior probability in the living room, while outdated low-intent interactions in the kitchen have a smaller impact. Ultimately, the system can calculate the posterior probability distribution of the user in the living room, bedroom, and kitchen; for example, the probability of the user in the living room is 0.85, in the bedroom is 0.1, and in the kitchen is 0.05, thus obtaining a more accurate probability distribution of the user in the preset activity areas.

[0077] It's important to note that activity pattern history is a structured data storage method used to track and record a specific user's activity habits in different areas of the smart home environment over a long period. For example, activity pattern history can include the average time a user spends in preset activity areas such as the living room, bedroom, and kitchen during different time periods (morning, afternoon, evening), as well as the frequency of interaction with devices such as smart lights, smart speakers, and smart TVs in these areas. This data can be accumulated and updated using user presence sensor data and smart device interaction records continuously collected by the smart home system. Changes in the environmental layout can refer to the addition, removal, or repositioning of physical devices in the smart home environment, or the redefinition of area functions. The system can detect these changes using preset sensors (e.g., door and window sensors, motion sensors) or through manual user input. When a change in the environmental layout is detected, the system considers that the original prior probabilities may no longer be applicable, thus triggering a prior probability update mechanism. Furthermore, when the system analyzes user presence sensor data and smart device interaction records and discovers a significant and persistent deviation between a user's actual activity pattern in a specific area (e.g., a user suddenly spending a long time in an area they normally don't frequent, or frequently interacting with new devices in a certain area) and the prior probabilities used in the current Bayesian inference, a prior probability update mechanism should be triggered. This deviation can be assessed using statistical methods, such as the chi-square test or KL divergence, to quantify the difference between the actual activity pattern and the prior probability distribution. Therefore, in the Bayesian inference calculation process, instead of simply using a preset static prior probability, an updated prior probability is employed. The updated prior probability more accurately reflects the user's current activity habits and environmental state, thus making the calculated posterior probability of the user in the preset activity area more precise.

[0078] In one embodiment, suppose Mr. Zhang typically spends time in the kitchen between 7 and 8 AM, frequently using the smart coffee machine and smart oven during this period. The system sets a higher prior probability that Mr. Zhang is in the kitchen between 7 and 8 AM based on his historical activity patterns. One day, Mr. Zhang changes his habits and begins exercising in the living room between 7 and 8 AM, frequently interacting with the smart speaker in the living room via voice. Initially, the system may still tend to assume Mr. Zhang is in the kitchen based on the old prior probability. However, as the system continuously collects data from Mr. Zhang's user presence sensors (e.g., micro-motion sensor data in the living room) and smart device interactions (e.g., interactions with the smart speaker in the living room), the system will detect a persistent deviation between Mr. Zhang's actual activity pattern in the living room and the current prior probability of him being in the kitchen. At this point, the system will trigger a prior probability update mechanism, increasing the prior probability of Mr. Zhang being in the living room between 7 and 8 AM and decreasing the prior probability of him being in the kitchen based on the new historical activity pattern. When Mr. Zhang issues the voice command again, even if the confidence level of the initial recognition result is low, the system will use the updated prior probability when performing context-assisted judgment to more accurately infer that Mr. Zhang is in the living room. This, combined with the status of active devices in the living room (e.g., the smart speaker is playing) and time information, will more accurately assist in judging the intent of the voice command, such as "play music" instead of "make coffee", ultimately improving the accuracy of voice recognition.

[0079] It's important to note that calculating short-term prior probabilities aims to reflect user behavioral trends in the recent or current time period. For example, based on historical user activity patterns over the past few hours or days, combined with information from the current time period, methods such as sliding time window averaging and exponentially weighted moving averages can be used to dynamically assess the likelihood of short-term stays in various preset activity areas. The goal is to capture immediate changes and temporary activities in user behavior. Calculating long-term prior probabilities aims to reflect stable, periodic user behavior habits. For example, based on data from longer time spans (such as weeks, months, or even longer) in historical user activity patterns, the average dwell time and interaction frequency in a specific area (such as the kitchen) during a specific time period (e.g., 8 AM to 9 AM daily) can be analyzed to obtain the user's long-term activity tendencies in that area. The goal is to reflect the user's inherent life patterns and habits. Weighted fusion of short-term and long-term prior probabilities refers to linearly or non-linearly combining short-term and long-term prior probabilities according to preset weighting factors to obtain the final updated prior probability. For example, the weights of short-term and long-term prior probabilities can be dynamically adjusted based on the volatility of short-term behavior and the stability of long-term habits. When user behavior patterns change rapidly, the weight of short-term prior probabilities can be appropriately increased; when user behavior patterns are relatively stable, the weight of long-term prior probabilities can be increased. Thus, updating prior probabilities can comprehensively reflect both immediate user behavior and long-term habits, providing a more comprehensive and accurate basis for user location inference.

[0080] In one embodiment, suppose a user in a smart home environment has a historical activity pattern showing that on weekdays, the user is typically active in the kitchen between 7:00 and 8:00 AM (high long-term prior probability), while on weekends, the user may be active in the living room between 9:00 and 10:00 AM. Specifically, when the system needs to update the user's prior probability of being in the kitchen at 7:30 AM on a weekday: First, based on the user's historical activity pattern and the current time period (weekday, 7:30 AM), the system calculates the short-term prior probability of the user being in the kitchen. If the user has been active in the kitchen on two days within this time period in the past few days (e.g., the past three weekdays), the short-term prior probability is likely to be high, for example, 0.8. Second, the system calculates the long-term prior probability of the user being in the kitchen based on the user's historical activity pattern. If the user has spent 90% of their time in the kitchen between 7:00 and 8:00 AM on weekdays over the past few months, the long-term prior probability is likely to be 0.9. Finally, the system fuses these two probabilities using a weighted average. For example, we can set the weight of the short-term prior probability to 0.4 and the weight of the long-term prior probability to 0.6. Then, the updated prior probability = (0.8 * 0.4) + (0.9 * 0.6) = 0.32 + 0.54 = 0.86.

[0081] It's important to note that activity feature vectors can be understood as a dataset formed through long-term statistical analysis and modeling of a specific user's behavioral patterns within a specific preset activity area. The vectors detail the types of devices a user typically interacts with within that area, the frequency distribution of these interactions, the types of commonly used voice commands, and typical activity patterns over a specific time period reflected by micro-motion sensor data. For example, a user's activity feature vector in the living room might include their frequent use of the smart TV in the evening, issuing "play a movie" commands, and exhibiting micro-motion patterns of prolonged sitting on the sofa. These feature vectors provide a benchmark for subsequent real-time activity matching. Real-time activity features refer to the system's real-time capture and analysis of each user's current behavioral data when multiple users are simultaneously active within the same preset activity area. The acquisition of these real-time features is the result of multimodal fusion, specifically including: analyzing individual micro-motion patterns reflected in user presence sensor data, such as the subtle movement trajectories and frequencies detected by millimeter-wave radar or infrared sensors; analyzing specific interaction objects and sequences in smart device interaction records, such as which smart light or air conditioner the user recently operated, and the order of operation; and combining sound source localization technology to determine the location of voice commands and using voiceprint recognition technology to confirm the identity of the voice command issuer. Through this real-time data, a detailed activity profile of each user at the current moment can be constructed.

[0082] In one embodiment, assume there are two users, A and B, in a smart living room. The system pre-maintains activity feature vectors for users A and B in the living room area. For example, user A's activity feature vector might show that they frequently watch TV in the sofa area at night and issue the voice command "turn on the TV," while user B's activity feature vector might show that they frequently use a smart speaker to play music in the dining area during the same time period and issue the command "play music." When the system detects that users A and B are active in the living room simultaneously and receives an audio signal containing residual noise, initially identifying it as "turn on the TV," the system extracts the real-time activity features of users A and B. For example, it detects micro-movements in the sofa area and in the dining area by analyzing individual micro-motion patterns reflected in the user presence sensor data; it discovers that user A has not recently interacted with the smart speaker, while user B has recently interacted with it, based on the specific interaction objects and sequences in the smart device interaction records; it discovers that the voice command originated from the sofa area through sound source localization and voiceprint recognition results, and confirms that the speaker is user A through voiceprint recognition. Subsequently, in the likelihood calculation stage of Bayesian inference, the system matches user A's real-time activity features with user A's activity feature vector in the living room to calculate the matching degree of user A executing the "turn on the TV" command in the living room. Simultaneously, the system also matches user B's real-time activity features with user B's activity feature vector in the living room to calculate the matching degree of user B executing the "turn on the TV" command in the living room. Since user A's real-time activity features have a higher matching degree with the activity feature vector related to "turn on the TV," and the sound source localization and voiceprint recognition results also point to user A, user A's likelihood is significantly higher than user B's. Finally, by combining the updated prior probabilities, the system can more accurately infer that user A is the user who issued the "turn on the TV" command, thus providing strong contextual support for subsequent recognition decisions and avoiding incorrectly attributing the command to user B or causing recognition ambiguity.

[0083] Real-time activity features exhibiting a multi-task parallel mode refer to the activity patterns reflected by a user within the same time period, based on their presence sensor data, smart device interaction records, and sound source localization and voiceprint recognition results. These patterns include two or more distinguishable, simultaneous, or alternating independent tasks or intentions. For example, a user might be cooking in the kitchen while simultaneously controlling music playback in the living room via voice commands. When the system detects the simultaneous presence of these independent activity features, it considers a multi-task parallel mode to be present.

[0084] It's important to note that decomposing real-time activity features into multiple independent sub-activity features means that the system uses pattern recognition, semantic analysis, or pre-defined rules to break down detected complex real-time activity features into several smaller, more atomic activity units. For example, a real-time activity feature containing "cooking" and "listening to music" can be decomposed into two sub-activity features: "kitchen cooking activity" and "living room music playback activity." Each sub-activity feature represents a relatively independent user intent or behavior. For each sub-activity feature, matching it with the corresponding part in the activity feature vector to calculate multiple sub-matching degrees means that for each decomposed sub-activity feature, the system compares it with the part of the user's activity feature vector in the pre-defined activity area that corresponds to the type or content of that sub-activity feature. For example, the "kitchen cooking activity" sub-activity feature will be matched with parts of the activity feature vector related to kitchen equipment interaction and cooking-related voice commands, thereby calculating a sub-matching degree reflecting the degree to which the sub-activity feature matches typical activity patterns. This process generates an independent sub-matching degree for each sub-activity feature.

[0085] It's important to clarify that the multi-task parallel mode of real-time activity features refers to a user's activity pattern within the same time period, as reflected by their presence sensor data, smart device interaction records, and sound source localization and voiceprint recognition results. This pattern includes two or more distinguishable, simultaneous, or alternating independent tasks or intentions. For example, a user might be cooking in the kitchen while simultaneously controlling music playback in the living room via voice commands. When the system detects the simultaneous existence of these independent activity features, it considers a multi-task parallel mode to be present. Furthermore, decomposing real-time activity features into multiple independent sub-activity features means that the system uses pattern recognition, semantic analysis, or preset rules to break down the detected complex real-time activity features into several smaller, more atomic activity units. For example, a real-time activity feature containing "cooking" and "listening to music" can be decomposed into two sub-activity features: "kitchen cooking activity" and "living room music playback activity." Each sub-activity feature represents a relatively independent user intention or behavior. Specifically, for each sub-activity feature, it is matched with the corresponding part of the activity feature vector to calculate multiple sub-matching degrees. This means that for each decomposed sub-activity feature, the system compares it with the part of the user's activity feature vector in the preset activity area that corresponds to the type or content of that sub-activity feature. For example, the sub-activity feature of "kitchen cooking activity" will be matched with the parts of the activity feature vector related to kitchen equipment interaction and cooking-related voice commands, thereby calculating a sub-matching degree that reflects the degree to which the sub-activity feature matches typical activity patterns. This process generates an independent sub-matching degree for each sub-activity feature.

[0086] See Figure 2 , Figure 2 This is a schematic diagram of a speech recognition noise reduction optimization system provided in one embodiment of this application. The speech recognition noise reduction optimization system 200 includes:

[0087] The signal receiving module 210 is used to receive audio signals containing residual noise and slightly distorted speech;

[0088] The noise identification module 220 is used to identify the noise type of the audio signal and obtain the residual noise type and intensity.

[0089] The feature extraction module 230 is used to extract speech features from the audio signal based on the type and intensity of residual noise;

[0090] The preliminary recognition module 240 generates preliminary recognition results based on speech features and calculates the initial confidence level of the preliminary recognition results;

[0091] The decision execution module 250 executes the identification decision based on the preliminary identification results and the initial confidence level.

[0092] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0093] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0094] The foregoing has provided a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined in this application.

Claims

1. A speech recognition noise reduction optimization method, characterized in that, include: Receive audio signals containing residual noise; The audio signal is subjected to noise type identification to obtain the residual noise type and intensity; Based on the type and intensity of the residual noise, speech features are extracted from the audio signal; Based on the speech features, a preliminary recognition result is generated, and the initial confidence level of the preliminary recognition result is calculated; Based on the preliminary identification results and initial confidence level, an identification decision is made; The step of making an identification decision based on the preliminary identification results and initial confidence level includes: When the initial confidence level is lower than a preset threshold, obtain the status information and time information of active devices in the smart home system; Based on the active device status information and the time information, the preliminary identification result is subjected to scene context-assisted judgment to obtain the scene matching factor; The final confidence level is obtained by weighted fusion based on the initial confidence level and the scene matching factor. The final confidence level is compared with a preset decision threshold to confirm the final identification result; Execute an identification decision based on the final identification result; The step of performing scene context-assisted judgment on the preliminary identification result based on the active device status information and the time information to obtain the scene matching factor includes: Collect user presence sensor data, smart device interaction records, and active device status; By analyzing the user presence sensor data, the smart device interaction records, the active device status, and the time information through Bayesian inference, the probability distribution of the user in the preset activity area can be inferred. Based on the probability distribution and the content of the preliminary identification results, the degree of correlation between user intent and activity area is calculated to obtain the scene matching factor.

2. The speech recognition noise reduction optimization method according to claim 1, characterized in that, The step of analyzing the user presence sensor data, smart device interaction records, active device status, and time information using a Bayesian inference mechanism to infer the probability distribution of the user in a preset activity area includes: Calculate the reliability weight of the user presence sensor data based on the historical accuracy and current signal strength. Adjust the timeliness decay factor of the smart device interaction record based on the interval between the interaction time recorded by the smart device and the current time. Based on the interaction type and duration of the smart device interaction records, assess the intent intensity of the smart device interaction records; In Bayesian inference, the reliability weight is applied to the likelihood calculation corresponding to the user presence perceptron data, the timeliness decay factor and the intent intensity are applied to the likelihood calculation corresponding to the smart device interaction record, and the posterior probability of the user in the preset activity area is calculated by combining the active device status and the time information, so as to obtain the probability distribution of the user in the preset activity area.

3. The speech recognition noise reduction optimization method according to claim 2, characterized in that, The step of calculating the posterior probability of a user in a preset activity area includes: Maintain an activity pattern history record for each user, which records the duration of the user's stay and interaction frequency in various preset activity areas at different time periods; When the smart home system detects changes in the environmental layout, or when the system discovers, through user presence sensor data belonging to the user and smart device interaction records belonging to the user, that a user's actual activity pattern in a specific area has a persistent deviation from the current prior probability, it triggers the prior probability update mechanism to obtain an updated prior probability. In Bayesian inference, the updated prior probability is used to calculate the posterior probability of a user in a preset activity area.

4. The speech recognition noise reduction optimization method according to claim 3, characterized in that, The steps for obtaining the updated prior probabilities include: Retrieve user activity pattern history and current time period information; Based on the user activity pattern history and the current time period information, calculate the short-term prior probability of each user in each preset activity area; Based on the user activity pattern history, calculate the long-term prior probability of each user in each preset activity area; The updated prior probability is obtained by weighted fusion of the short-term prior probability and the long-term prior probability.

5. The speech recognition noise reduction optimization method according to claim 3, characterized in that, In Bayesian inference, the step of calculating the posterior probability of a user in a preset activity area using the updated prior probability includes: Maintain an activity feature vector for each user and each preset activity area. The activity feature vector includes the user's typical interactive device type, interaction frequency distribution, voice command content category distribution, and micro-motion sensor data pattern within a specific time period in that area. When multiple users are active in the same preset activity area, the real-time activity features of each user in the area are extracted. The real-time activity features are obtained by analyzing the individual micro-motion patterns reflected in the user presence sensor data, the specific interaction objects and interaction sequences in the smart device interaction records, as well as the sound source localization and voiceprint recognition results. In the likelihood calculation stage of Bayesian inference, the real-time activity features of each user are matched with the user's activity feature vector in the region, and the matching degree is calculated as the likelihood of the user in the current region. In Bayesian inference, the updated prior probability is used to calculate the posterior probability of a user in a preset activity area.

6. The speech recognition noise reduction optimization method according to claim 5, characterized in that, The step of matching each user's real-time activity features with the user's activity feature vector in the region during the likelihood calculation stage of Bayesian inference, and calculating the matching degree as the user's likelihood in the current region, includes: When the real-time activity feature exhibits a multi-task parallel mode, the real-time activity feature is decomposed into multiple independent sub-activity features; For each of the sub-activity features, it is matched with the corresponding part in the activity feature vector to calculate multiple sub-matching degrees; The matching degree is obtained by weighting and fusing the multiple sub-matching degrees based on the correlation between the sub-activity features corresponding to each sub-matching degree and the preliminary identification result.

7. The speech recognition noise reduction optimization method according to claim 6, characterized in that, The step of weightedly fusing the multiple sub-matching degrees based on the correlation between the sub-activity features corresponding to each sub-matching degree and the preliminary identification result to obtain the matching degree includes: When the semantic similarity between the sub-activity feature and the preliminary identification result is higher than a preset threshold, the semantic similarity is corrected according to the matching degree between the sub-activity feature and the specific operation object and operation behavior of the preliminary identification result; Based on the historical activation frequency of the sub-activity features under different time periods and different device types, and the device type and time preference information contained in the preliminary identification results, the historical activation frequency weight of the sub-activity features is corrected; Based on the corrected semantic similarity and the corrected historical activation frequency weight, the weights of the weighted fusion of the multiple sub-matching degrees are adjusted to obtain the matching degree.

8. A speech recognition noise reduction optimization system, characterized in that, include: A signal receiving module for receiving audio signals containing residual noise and slightly distorted speech; A noise identification module is used to identify the noise type of the audio signal and obtain the residual noise type and intensity. The feature extraction module is used to extract speech features from the audio signal based on the type and intensity of the residual noise; The preliminary recognition module generates a preliminary recognition result based on the speech features and calculates the initial confidence level of the preliminary recognition result; The decision execution module executes an identification decision based on the preliminary identification results and the initial confidence level. It is also used to obtain the status information and time information of active devices in the smart home system when the initial confidence level is lower than a preset threshold; Based on the active device status information and the time information, the preliminary identification result is subjected to scene context-assisted judgment to obtain the scene matching factor; The final confidence level is obtained by weighted fusion based on the initial confidence level and the scene matching factor. The final confidence level is compared with a preset decision threshold to confirm the final identification result; Execute an identification decision based on the final identification result; It is also used to collect user presence sensor data, smart device interaction records, and active device status; By analyzing the user presence sensor data, the smart device interaction records, the active device status, and the time information through Bayesian inference, the probability distribution of the user in the preset activity area can be inferred. Based on the probability distribution and the content of the preliminary identification results, the degree of correlation between user intent and activity area is calculated to obtain the scene matching factor.

Images