Intelligent loudspeaker box permission design management method and system

Abstract

The invention provides an intelligent loudspeaker box permission design management method and system. The method comprises the following steps: collecting user audio data and preprocessing the audio data; a user-situation joint feature vector is constructed to carry out recognition precision adjustment; if a complex voice environment with multiple languages and multiple accents appears, designing a fuzzy matching and dynamic adjustment model to dynamically manage the authority of the intelligent sound box in real time; a behavior monitoring and anomaly detection model is constructed, user permissions are analyzed in real time through feature engineering and time sequence modeling, and permission levels are dynamically adjusted according to anomaly detection and behavior monitoring results; proposing a mapping function of an emotional state and permission response to dynamically adjust the permission level, and immediately applying new permission configuration by the intelligent sound box; and constructing a self-adaptive permission optimization model based on reinforcement learning, and performing self-adaptive learning on the permission strategy of the intelligent sound box through reinforcement learning to perform long-term strategy optimization. The wide application requirements of the intelligent loudspeaker box in diversified scenes such as families and offices are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of permission design management, and in particular, relates to a smart speaker permission design management method and system. Background Art

[0002] With the rapid development of Internet of Things (IoT) technology, smart speakers have gradually become indispensable smart devices in home and office environments. They provide users with a variety of convenient services such as music playback, information query, smart home control, etc. through voice interaction. However, the widespread use of smart speakers has also caused a series of problems related to privacy protection, permission management and multi-user collaboration. Existing smart speakers mainly control the use rights of devices through simple static permission settings, which has many limitations and shortcomings.

[0003] First, the existing static permission management model lacks dynamism and intelligence, and cannot adapt to the complex needs of multiple users and multiple scenarios. In a family environment, different members may have different usage needs and permission requirements for the functions of the speaker. For example, children and adults should have different permissions to access content, and the existing static permission settings cannot flexibly respond to these changes, often resulting in children accessing inappropriate content or adults being unable to obtain the required functions in a timely manner. Secondly, traditional static rule-based permission management cannot provide a personalized experience, lacks in-depth understanding and learning of individual user behaviors and usage habits, and leads to poor user experience. As a device that is used very frequently in daily life, smart speakers should have higher user experience optimization capabilities.

[0004] In addition, with the popularity of smart speakers in homes, offices and other environments, their permission management faces more severe privacy and security challenges. The traditional permission setting method usually requires users to manually adjust permissions through cumbersome operations, which is not only inconvenient but also prone to errors. What is more serious is that speakers are usually exposed in public environments, and unauthorized users may perform unauthorized operations through voice commands, which poses a huge security risk. For example, unauthorized users can easily access confidential information, control smart home devices, and even make improper voice purchases, resulting in property and privacy losses. In the existing technology, the solutions to these security issues are mostly post-event remedial measures, lacking forward-looking protection strategies.

[0005] Finally, the permission management of smart speakers lacks an effective management mechanism in multi-user scenarios, especially in scenarios where multiple people are using the speaker at the same time. How to distinguish user identities and provide personalized services is a major challenge. The current permission management system can only distinguish simple user roles or rely on manual login to switch users, which is neither convenient nor reliable in actual use and cannot meet diverse usage needs. Faced with the alternating use of users of different age groups and roles, the existing technology cannot provide an intelligent and dynamic permission management solution, which limits the function of smart speakers and the optimization of user experience.

[0006] Therefore, the current smart speaker rights management technology has significant deficiencies, mainly manifested in the lack of dynamic intelligence, personalized experience, security assurance, and insufficient support for multi-user scenarios. These problems have seriously limited the potential and market application of smart speakers. Summary of the invention

[0007] The purpose of the present invention is to design a smart speaker permission design management method and system, introduce advanced artificial intelligence technology and data analysis methods, overcome the defects of the prior art, and provide a smart speaker permission management system that is more intelligent, highly secure, and has excellent user experience.

[0008] In order to achieve the above object, a first aspect of the present invention provides a smart speaker permission design management method, the method comprising:

[0009] S1, collect user audio data and preprocess the audio data, build a Gaussian mixture model based on the preprocessed data to output the audio signature;

[0010] S2. The audio signature is used as input to represent the identity characteristics of the user. The real-time environment data is collected and preprocessed. A user-context joint feature vector is constructed based on the user's identity characteristics and the preprocessed real-time environment data. A regularization term is added to the user-context joint feature vector to adjust the recognition accuracy. Then, context awareness and dynamic permission configuration initialization are performed to obtain the optimized permission level and permission score.

[0011] S3. Based on the optimized permission level and permission score, if a complex voice environment with multiple languages ​​and multiple accents appears, a fuzzy matching and dynamic adjustment model is designed to dynamically manage the permissions of the smart speaker in real time; wherein the fuzzy matching and dynamic adjustment model uses a feature weighted aggregation method to calculate the voice feature vector C new The credibility score γ is expressed as follows:

[0012]

[0013] Among them, γ represents the credibility score of the speech input, which indicates the credibility of the current speech feature matching; M represents the total number of speech features; w j represents the weight of the jth feature, indicating the importance of this feature in calculating the credibility of speech; α j represents the fuzzy adjustment parameter of the jth feature, controlling the sensitivity of feature matching; C new,j represents the component of the jth speech feature vector; μ j represents the expected value of the jth feature, which represents the typical value of the feature under normal circumstances;

[0014] Using the credibility score γ of voice input and the optimized authority score Adjusting functions based on permissions Dynamically adjust user permission levels It is expressed as follows:

[0015]

[0016] in, Indicates the dynamically adjusted permission level, indicating the permission setting adjusted according to the voice credibility and the original permission score; represents the initial authority level output in the second step; η represents the authority adjustment gain coefficient, which controls the authority adjustment range; γ represents the credibility score of the speech input; δ represents the intermediate threshold of the credibility score; represents the optimized permission score; τ represents the baseline threshold of the permission score; based on the calculated new permission level The smart speaker system instantly adjusts the user's permission configuration and dynamically changes the accessibility of functions;

[0017] S4. New permission level optimized based on complex voice environment Combined with real-time user behavior data, a behavior monitoring and anomaly detection model is built to analyze user permissions in real time through feature engineering and time series modeling, perform anomaly detection and behavior monitoring on user behavior, and dynamically adjust permission levels based on the results of anomaly detection and behavior monitoring.

[0018] S5. According to the permission level Combined with the user's emotional state, the emotional state is classified through the emotion classifier model to generate emotional labels. According to the emotional state labels and abnormal behavior response strategies, a mapping function between emotional state and permission response is proposed to dynamically adjust the permission level. The permission level calculated by the mapping function between emotional state and permission response is Smart speakers instantly apply new permission configurations;

[0019] S6. According to the authority level Build an adaptive permission optimization model based on reinforcement learning, combine users' long-term behavior and usage patterns, and use reinforcement learning to adaptively learn the permission strategy of smart speakers for long-term strategy optimization.

[0020] Preferably, the preprocessing is to process the collected audio data x(t) to remove noise and perform standardization processing;

[0021] Wherein, the probability density function of the Gaussian mixture model is expressed as:

[0022]

[0023] Wherein, C represents the extracted Mel frequency cepstral coefficient feature vector, which is used to represent the voiceprint feature of the audio; Represents the parameter set of the Gaussian mixture model; π k represents the mixing weight of the kth Gaussian component, satisfying And π k ≥0;μ k represents the mean vector of the kth Gaussian component; Σ k represents the covariance matrix of the kth Gaussian component; K represents the number of Gaussian components, which means the number of Gaussian distributions in the model; represents the multivariate normal distribution, expressed as:

[0024]

[0025] Where d represents the dimension of the feature vector C.

[0026] Preferably, for each new audio input, the user identity is identified by calculating the log-likelihood estimate of the feature vector of the new audio under each user model, which is expressed as follows:

[0027]

[0028] Among them, logL(C new |S i ) represents the log-likelihood value of the new feature vector under the i-th user model, which is used to measure the matching degree between the audio input and the user model; C new : The feature vector representing the new audio input; S i represents the audio signature of the i-th user, that is, the GMM parameter set Θ i ; T represents the total number of frames of the new audio input, which represents the number of frames after the audio signal is segmented; P(C new (t)|Θ i ) represents the feature vector C of the tth frame new (t) In the user model Θ i The probability of the following.

[0029] Preferably, the real-time environmental data is collected through a variety of sensors, including microphones, cameras, light sensors, temperature and humidity sensors, and accelerometers, and the real-time environmental feature vector is represented by E;

[0030] The user-context joint feature vector is fused through a weighted multilayer perceptron model, which is expressed as follows:

[0031] C e =σ(W2·σ(W1·[E norm ,S i ]+b1)+b2)

[0032] Among them, C e represents the generated context feature vector, which represents the user's behavior characteristics in the current environment; [E norm ,S i ] represents the input feature after combining the environment feature vector and the user audio signature; W1, W2 represent weight matrices with dimensions of m×(n+1) and m×m, respectively, which are learned through model training; b1, b2 represent bias vectors with dimensions of m and m, respectively, which are learned through model training; σ represents a nonlinear activation function, which is used to capture nonlinear relationships;

[0033] Combined with the context feature vector C e and user audio signature S i , generate the user-context joint feature vector F = [S i ,C e ];

[0034] In the user-context joint feature vector F = [S i ,C e ]Adding regularization terms can achieve higher recognition accuracy by weighted combination of user and environment features, as shown below:

[0035] F opt =F+λ·(F⊙W reg )

[0036] Among them, F opt represents the optimized joint feature vector; λ represents the regularization parameter, which is used to balance the influence of the original features and the regularization term; ⊙ represents the element-wise product operation, which is used for element-by-element weighting; W reg Represents the weight matrix, which represents the relationship between features.

[0037] Preferably, the configuration initialization includes:

[0038] Design a permission scoring function based on the adjusted user-context joint feature vector to calculate the user's permission score in the current context It is expressed as follows:

[0039]

[0040] in, Indicates the permission score, which is used to reflect the user's permission level in the current context; represents the weight vector, indicating the importance of each feature; β represents the weight adjustment coefficient, controlling the overall weight of the joint features; The bi-norm of the joint features is used as a measure of feature importance, which adds constraints on the balance between features. represents the bias term;

[0041] Based on the calculated authority score and preset permission level threshold, and dynamically adjust permissions. It is expressed as follows:

[0042]

[0043] in, Indicates the dynamically adjusted permission level; Indicates the permission threshold, which is set according to user needs and system configuration.

[0044] Preferably, in S3, an error correction feedback mechanism is designed for the fuzzy matching and dynamic adjustment model to dynamically adjust the weights and parameters of the fuzzy matching model. The error correction mechanism is expressed as follows:

[0045] Δw=∈·(γ target -γ)·C new

[0046] Among them, Δw represents the weight adjustment vector, which is used to correct the weight of speech features; ∈ represents the learning rate, which is used to control the step size of weight adjustment; γ target represents the target credibility score, which is usually set to the credibility level expected by the system; γ represents the credibility score of the current speech input; C new The feature vector representing the current speech input.

[0047] Preferably, according to the weighted behavior feature vector B w A monitoring and anomaly detection model based on long short-term memory network is constructed to predict the behavior characteristics of the next moment and identify abnormal behavior patterns. The update formula of the behavior monitoring and anomaly detection model is expressed as follows:

[0048] h t =f LSTM (B w,t ,h t-1 )

[0049] Among them, ht represents the hidden state at the current time t, capturing the time dependency of user behavior; B w,t represents the weighted behavior feature vector at the current time t; h t-1 represents the hidden state at the previous moment t-1; f LSTM Represents the state update function of the long short-term memory network;

[0050] By calculating the residual between the actual behavior characteristics and the predicted behavior characteristics, it is determined whether there is abnormal behavior; wherein the abnormal score ξ is expressed as follows:

[0051]

[0052] Among them, ξ represents the anomaly score, which indicates the difference between the current behavior and the predicted behavior; B w,t Represents the actual weighted behavior feature vector at the current time t; Represents the weighted behavior feature vector of the next moment predicted by the long short-term memory network; σ 2 represents the variance of the prediction residuals, which is used to normalize the anomaly scores;

[0053] According to the abnormal score ξ and the preset abnormal threshold θ, it is determined whether to trigger the permission adjustment or warning mechanism; the adjustment strategy is as follows:

[0054] If ξ≤θ, no adjustment is made and the current permission level is maintained

[0055] If ξ>θ, it is identified as abnormal behavior, and permissions are tightened or the user is prompted to authenticate based on the severity of the abnormality.

[0056] Preferably, the collection of the user's emotional state includes the collection of audio and video features, and then a fusion model based on a self-attention mechanism is used to capture the mutual influence between multimodal features. The specific formula is as follows:

[0057]

[0058] Where E represents the fused emotional feature vector, which is used to describe the user's current comprehensive emotional state; A represents the audio feature vector, with a dimension of m; V represents the video feature vector, with a dimension of n; W A and W V Represents the projection matrix, which is used to map audio and video features to the same space, with a dimension of m×d k and n×d k ;d k Represents the dimension of the key vector in the attention mechanism, which is used to scale the dot product operation; softmax represents the normalization function used to calculate the attention weight;

[0059] Using the fused emotional feature vector E, the emotional state is classified through the emotional classifier model C(E) to generate the emotional label c, which is expressed as follows:

[0060]

[0061] Among them, c represents the emotional state label of the current user, expressed as a discrete category; w i Represents the weight vector of the classifier model, the weight corresponding to the i-th class, with a dimension of d k ; b i Represents the bias term of the classifier model, the bias corresponding to the i-th class;

[0062] According to the emotional state label c and the abnormal behavior response strategy, a mapping function between emotional state and authority response is proposed Used to dynamically adjust permission levels It is expressed as follows:

[0063]

[0064] in, It represents the permission level dynamically adjusted according to the emotional state, and takes the user's emotions into consideration when setting the permission. ρ(c) represents the permission adjustment increment function, which depends on the user's emotional state c and is expressed as a positive or negative integer.

[0065] Preferably, the adaptive permission optimization model based on reinforcement learning combines the user's current permission level and emotional state, as well as the user's historical behavior pattern and system feedback, that is, the state vector Among them, H represents the user's historical behavior pattern feature vector; F represents the system feedback feature vector; the system's executable actions are defined as the permission adjustment operation set Each action a i represents the adjustment of the permission level; it means optimizing the objective function J(θ), by adjusting the strategy π θ (S) is optimized to maximize the expected cumulative reward;

[0066] The formula for optimizing the objective function J(θ) is as follows:

[0067]

[0068] Where J(θ) represents the optimization objective function of the policy parameter θ, which represents the expected cumulative reward; S represents the current state vector, which consists of the authority level, emotional state, historical behavior pattern, and system feedback; represents the state distribution set, which represents the possible state space; γ represents the discount factor, which ranges from 0≤γ≤1 and is used to measure the importance of future rewards; r(S t ,a t) indicates that in state S t Execute action a t The immediate reward obtained when the authority is adjusted reflects the effectiveness and rationality of the authority adjustment; t represents the action performed at time t, given by the strategy π θ (S) confirm;

[0069] In the process of policy optimization, the policy regularization term Ω(θ) is introduced to prevent the policy from overfitting the user's short-term behavior pattern, and the policy parameter θ is updated to maximize the regularized optimization objective, which is expressed as follows:

[0070]

[0071] Among them, θ t+1 represents the updated policy parameters; θ t represents the policy parameter at the current time t; α represents the learning rate, which controls the step size of parameter update; represents the policy gradient, which represents the gradient of the optimization objective function to the policy parameters; λ represents the regularization strength parameter, which controls the influence of the regularization term on the policy update; Ω(θ t ) represents the policy regularization term, which is a sparsity regularization that encourages simplicity and generalization of policy parameters, and is expressed as follows:

[0072]

[0073] Where N represents the total number of policy parameters; θ i Represents the i-th parameter in the strategy parameters.

[0074] In a second aspect of the present invention, a smart speaker authority design management system is provided, the system comprising:

[0075] A user data collection module is used to collect user audio data and preprocess the audio data, and construct a Gaussian mixture model based on the preprocessed data to output an audio signature;

[0076] Configuration initialization module, used to take audio signature as input to represent the identity characteristics of the user, collect real-time environment data and pre-process it, build a user-context joint feature vector based on the user's identity characteristics combined with the pre-processed real-time environment data, add regularization terms to the user-context joint feature vector to adjust the recognition accuracy, and then perform context awareness and dynamic permission configuration initialization to obtain the optimized permission level and permission score;

[0077] The permission initialization module is used to design a fuzzy matching and dynamic adjustment model to dynamically manage the permissions of the smart speaker in real time based on the optimized permission level and permission score if a complex voice environment with multiple languages ​​and multiple accents appears; wherein the fuzzy matching and dynamic adjustment model uses a feature weighted aggregation method to calculate the voice feature vector C new The credibility score γ is expressed as follows:

[0078]

[0079] Among them, γ represents the credibility score of the speech input, which indicates the credibility of the current speech feature matching; M represents the total number of speech features; w j represents the weight of the jth feature, indicating the importance of this feature in calculating the credibility of speech; α j represents the fuzzy adjustment parameter of the jth feature, controlling the sensitivity of feature matching; C new,j represents the component of the jth speech feature vector; μ j represents the expected value of the jth feature, which represents the typical value of the feature under normal circumstances;

[0080] Using the credibility score γ of voice input and the optimized authority score Adjusting functions based on permissions Dynamically adjust user permission levels It is expressed as follows:

[0081]

[0082] in, Indicates the dynamically adjusted permission level, indicating the permission setting adjusted according to the voice credibility and the original permission score; represents the initial authority level output in the second step; η represents the authority adjustment gain coefficient, which controls the authority adjustment range; γ represents the credibility score of the speech input; δ represents the intermediate threshold of the credibility score; represents the optimized permission score; τ represents the baseline threshold of the permission score; based on the calculated new permission level The smart speaker system instantly adjusts the user's permission configuration and dynamically changes the accessibility of functions;

[0083] The permission management module is used to optimize the new permission level based on the complex voice environment. Combined with real-time user behavior data, a behavior monitoring and anomaly detection model is built to analyze user permissions in real time through feature engineering and time series modeling, perform anomaly detection and behavior monitoring on user behavior, and dynamically adjust permission levels based on the results of anomaly detection and behavior monitoring. Based on permission level Combined with the user's emotional state, the emotional state is classified through the emotion classifier model to generate emotional labels. According to the emotional state labels and abnormal behavior response strategies, a mapping function between emotional state and permission response is proposed to dynamically adjust the permission level. The permission level calculated by the mapping function between emotional state and permission response is Smart speakers instantly apply new permission configurations;

[0084] System optimization module for Build an adaptive permission optimization model based on reinforcement learning, combine users' long-term behavior and usage patterns, and use reinforcement learning to adaptively learn the permission strategy of smart speakers for long-term strategy optimization.

[0085] The beneficial technical effects of the present invention are at least as follows:

[0086] First, the present invention adopts user audio signature recognition technology, which can identify the user's identity in real time through audio features during the use of the speaker. This method can not only effectively distinguish different users in a home or office environment, but also automatically adjust the permission settings according to each user's usage habits and permission requirements, realizing truly personalized services and overcoming the problem of the lack of flexibility of traditional static permission settings.

[0087] Secondly, the present invention introduces context-aware permission management technology. Through the fusion of multi-sensor data, such as microphones, cameras, light sensors, etc., combined with user behavior data and environmental information (such as time, location, schedule, etc.), the system can intelligently perceive the current situation and dynamically adjust permission settings. For example, in the evening family mode, the system will automatically enable the child protection mode to restrict access to inappropriate functions, thereby greatly improving the intelligence and user experience of the system and overcoming the problem of lack of dynamic adjustment capabilities in the prior art.

[0088] The present invention designs an adaptive speech fuzzy matching algorithm, which can adaptively adjust the speech recognition accuracy under various accents and speech speeds, ensuring that the user's intention is accurately understood while flexibly adjusting the permission settings to ensure security. This adaptive mechanism solves the problem of low recognition accuracy in the existing technology in multi-language and multi-accent usage scenarios, while reducing the risk of misoperation of permissions.

[0089] In addition, the present invention also integrates an abnormal permission behavior detection module and an emotional reasoning permission management module. The abnormal permission behavior detection module uses a machine learning algorithm to monitor and analyze the user's permission usage behavior in real time, and can quickly identify abnormal operations and take corresponding security measures, thereby enhancing the security and protection capabilities of the system. The emotional reasoning permission management module analyzes the user's emotional state through voice, and automatically adjusts permissions when the user is emotionally excited, to prevent inappropriate behavior caused by emotional fluctuations. These innovations effectively solve the defects of the existing technology that lack response to user emotions and abnormal behaviors, and significantly improve the security and humanized experience of the system.

[0090] Through these innovations, the present invention provides a comprehensive solution that can effectively overcome the various defects of the prior art and realize dynamic management of smart speaker permissions, personalized services, security protection, and multi-user support. This multi-level, all-round intelligent permission management system not only improves the user experience, but also significantly enhances the security and adaptability of the device, meeting the wide application needs of smart speakers in diverse scenarios such as home and office. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0092] Figure 1 The present invention provides a flowchart of a smart speaker permission design and management method according to an embodiment of the present invention.

[0093] Figure 2 The present invention provides a framework diagram of a smart speaker permission design and management system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0094] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0095] In one or more embodiments, Figure 1 As shown, a smart speaker permission design management method of the present invention is disclosed, and the method includes steps 1 to 6, including:

[0096] S1. Collect user audio data and preprocess the audio data, and construct a Gaussian mixture model based on the preprocessed data to output the audio signature.

[0097] Specifically, during the preprocessing process, the collected audio data x(t) is processed to remove noise and perform normalization.

[0098]

[0099] Where y(t) represents the denoised audio signal and x(t) represents the original audio signal, expressed as a time function. represents the estimated noise signal, extracted from the environment using an adaptive filter.

[0100] Signal standardization process, the standardized signal is expressed as:

[0101]

[0102] Among them, y norm (t) represents the normalized audio signal. μ(y) represents the mean of the denoised signal y(t), which is calculated as Where N is the number of signal samples. σ(y) represents the standard deviation of the denoised signal y(t), calculated as

[0103] Furthermore, a Gaussian mixture model is used to model the user's audio features to form an audio signature. The probability density function of GMM is expressed as:

[0104]

[0105] Wherein, C represents the extracted Mel-Frequency Cepstral Coefficient (MFCC) feature vector, which is used to represent the voiceprint features of the audio. Represents the parameter set of GMM. k represents the mixing weight of the kth Gaussian component, satisfying And π k ≥0.μ k Represents the mean vector of the kth Gaussian component. Σ k Represents the covariance matrix of the kth Gaussian component. K represents the number of Gaussian components, which represents the number of Gaussian distributions in the model. represents the multivariate normal distribution, expressed as:

[0106]

[0107] Where d is the dimension of the feature vector C.

[0108] Furthermore, for each new audio input, the user identity is identified by calculating the log-likelihood estimate of its feature vector under each user model:

[0109]

[0110] Among them, logL(C new |S i ) represents the log-likelihood value of the new feature vector under the i-th user model, which is used to measure the matching degree between the audio input and the user model. new The feature vector representing the new audio input. S i represents the audio signature of the i-th user, that is, the GMM parameter set Θ i T represents the total number of frames of the new audio input, and represents the number of frames after the audio signal is segmented. new (t)|Θ i ) represents the feature vector C of the tth frame new (t) In the user model Θ i The probability of the following.

[0111] S2. The audio signature is used as input to represent the user's identity characteristics. Real-time environmental data is collected and preprocessed. A user-context joint feature vector is constructed based on the user's identity characteristics and the preprocessed real-time environmental data. A regularization term is added to the user-context joint feature vector to adjust the recognition accuracy. Then, context awareness and dynamic permission configuration initialization are performed to obtain the optimized permission level and permission score.

[0112] Among them, the smart speaker collects environmental data in real time through a variety of sensors (such as microphones, cameras, light sensors, temperature and humidity sensors, accelerometers, etc.) to form an environmental feature vector E = [e1, e2, ..., e n ]. These environmental characteristics include:

[0113] e1 represents the ambient noise level, ranging from 0 to 100 decibels.

[0114] e2 represents light intensity, ranging from 0 to 10,000 lux.

[0115] e3 indicates the distance between the user and the speaker, ranging from 0 to 5 meters.

[0116] e4 represents the current time, in 24-hour format, ranging from 0 to 23.99.

[0117] e5 represents the number of people detected, and the integer range is 0 to 10.

[0118] e6 represents the user activity status, a categorical variable, and its value range is {stationary, walking, jumping, etc.}.

[0119] Other sensor data such as temperature e7, humidity e8, etc.

[0120] Furthermore, due to the different ranges and units of different sensor data, the environmental feature vector E needs to be standardized to unify the data scale:

[0121]

[0122] in, represents the standardized environmental characteristics. min(e i ) and max(e i ) represents environmental characteristics e i The minimum and maximum values ​​of are used for normalization.

[0123] Further, using the user audio signature S obtained in step 1 i Fusion of user identity and environmental features. This paper introduces a new fusion method to fuse user identity features and environmental data to construct a context feature vector C e The specific method is through a weighted multi-layer perceptron (MLP) model, which can dynamically adjust the weights of user features and environmental features during the learning process to reflect the user's behavior patterns in different situations.

[0124] C e =σ(W2·σ(W1·[E norm ,S i ]+b1)+b2)

[0125] Among them, C e Represents the generated context feature vector, which represents the user’s behavior characteristics in the current environment. norm ,S i ] represents the input feature after merging the environment feature vector and the user audio signature. W1, W2 represent weight matrices with dimensions of m×(n+1) and m×m, respectively, which are learned through model training. b1, b2 represent bias vectors with dimensions of m and m, respectively, which are learned through model training. σ represents a nonlinear activation function (such as ReLU or Sigmoid) used to capture nonlinear relationships.

[0126] Furthermore, combined with the context feature vector C e and user audio signature S i , generate the user-context joint feature vector F = [S i ,C e ]. This is a fused feature vector that represents the comprehensive state of a specific user in the current environment.

[0127] In order to better capture the complex relationship between user features and environmental features, a joint feature modeling method with an innovative regularization term is proposed. This regularization term achieves higher recognition accuracy by weighted combination of user and environmental features. The model formula is as follows:

[0128] F opt =F+λ·(F⊙W reg )

[0129] Among them, F opt W represents the optimized joint feature vector, which takes into account the weighted relationship between user and environment features. λ represents the regularization parameter, which is used to balance the influence of the original features and the regularization term, and is usually determined by cross-validation. ⊙ represents the element-wise product operation (Hadamard product), which is used for element-by-element weighting. reg Represents the weight matrix, which represents the relationship between features and is obtained by optimizing dynamic learning.

[0130] Furthermore, based on the optimized joint feature vector F opt , proposes a permission scoring function with an innovative additional term to calculate the user's permission score in the current context

[0131]

[0132] in, represents the permission score, which is used to reflect the user's permission level in the current context. w represents the weight vector, which is obtained through model training and represents the importance of each feature. β represents the weight adjustment coefficient, which controls the overall weight of the joint features. ∥F opt ∥ 2 It represents the bi-norm of the joint features, which is used as a measure of feature importance and adds constraints on the balance between features. b represents the bias term, which is learned through training data.

[0133] Based on the calculated authority score And the preset permission level threshold, dynamically adjust the permissions. Permission adjustment function is defined as:

[0134]

[0135] in, It indicates the permission level after dynamic adjustment, and the specific permission configuration will be set according to this level. T1 and T2 indicate the permission threshold, which will be set according to user needs and system configuration.

[0136] The final permission level will be determined Applied to smart speaker systems to dynamically control the functions and data permissions accessible to users.

[0137] S3. Based on the optimized permission level and permission score, if a complex voice environment with multiple languages ​​and accents appears, a fuzzy matching and dynamic adjustment model is designed to dynamically manage the permissions of the smart speaker in real time.

[0138] Specifically, when the user issues a voice command, the speaker collects the current voice input signal x(t), pre-processes the voice signal (such as denoising and standardization) using the method in the first step, and generates a processed voice feature vector C new .

[0139] At the same time, in order to adapt to the multi-language and multi-accent environment, an adaptive speech fuzzy matching model is proposed. This model uses a feature weighted aggregation method to calculate the speech feature vector C new The credibility score γ is as follows:

[0140]

[0141] Where γ represents the credibility score of the speech input, which represents the credibility of the current speech feature matching. M represents the total number of speech features. j Represents the weight of the jth feature, indicating the importance of this feature in calculating the speech credibility. j Represents the fuzzy adjustment parameter of the jth feature, which controls the sensitivity of feature matching. new,j Represents the component of the jth speech feature vector. j It represents the expected value of the jth feature, which indicates the typical value of the feature under normal circumstances.

[0142] Furthermore, a credibility-based permission adjustment strategy is defined using the credibility score γ of the speech input and the permission score in the second step Propose a "permission adjustment function" To dynamically adjust the user's permission level

[0143]

[0144] in, Indicates the dynamically adjusted permission level, which indicates the permission setting adjusted according to the voice credibility and the original permission score. represents the initial permission level output in the second step. η represents the permission adjustment gain coefficient, which controls the permission adjustment range. γ represents the credibility score of the speech input. δ represents the middle threshold of the credibility score, which is usually set to 0.5. represents the permission score calculated in the second step. τ represents the baseline threshold of the permission score, which is usually set based on actual usage.

[0145] Further, based on the calculated new permission level The smart speaker system instantly adjusts the user's permission configuration and dynamically changes the accessibility of functions.

[0146] In order to further improve the robustness and accuracy of the system, an "error correction feedback mechanism" is introduced. This mechanism dynamically adjusts the weights and parameters of the fuzzy matching model by detecting and analyzing user feedback or usage behavior after permission adjustment. The core formula of error correction is as follows:

[0147] Δw=∈·(γ target -γ)·C new

[0148] Where Δw represents the weight adjustment vector, which is used to correct the weight of speech features. ∈ represents the learning rate, which is used to control the step size of weight adjustment. γ target represents the target credibility score, which is usually set to the credibility level expected by the system. γ represents the credibility score of the current speech input. new The feature vector representing the current speech input.

[0149] The parameters of the speech fuzzy matching model are updated according to the error correction formula, so that the system can more accurately adjust the permission configuration when facing voice input from different users and contexts.

[0150] Furthermore, by combining the user's immediate feedback and long-term usage behavior data, the permission adjustment function Continuously optimize the fuzzy matching model. By real-time monitoring of user operation habits and permission usage, dynamically adjust system parameters to improve the adaptability and accuracy of permission adjustment.

[0151] Collect user usage data (such as user behavior and satisfaction feedback under adjusted permission levels), and regularly retrain the fuzzy matching model and permission adjustment function to ensure that model parameters are consistent with dynamic changes in user needs.

[0152] S4. New permission level optimized based on complex voice environment Combined with real-time user behavior data, a behavior monitoring and anomaly detection model is built to analyze user permissions in real time through feature engineering and time series modeling, perform anomaly detection and behavior monitoring on user behavior, and dynamically adjust permission levels based on the results of anomaly detection and behavior monitoring.

[0153] Specifically, the user behavior features are extracted, including:

[0154] User behavior data collection: The smart speaker continuously collects the user's operation behavior data and generates a behavior feature vector B = [b1, b2, ..., b k ], these features include:

[0155] b1 represents the type of voice command issued by the user (such as playing music, setting an alarm, etc.), which is represented as a categorical variable.

[0156] b2 represents the time interval between instruction occurrences (unit: seconds), expressed as a continuous variable.

[0157] b3 represents the frequency of instructions issued by the user in a specific time period (unit: times / minute), expressed as a continuous variable.

[0158] b4 represents the range of change in the permission level corresponding to each instruction, expressed as an integer.

[0159] b5 represents the recognized emotional state of the user, expressed as a categorical variable (such as calm, excited, etc.).

[0160] Behavior feature standardization and weighting: The behavior feature vector B is standardized to eliminate the scale differences between different features. Then, a weighted combination is performed based on the importance of the features to generate a weighted behavior feature vector B. w :

[0161] B w =W B ·B norm

[0162] Among them, B w W represents the weighted behavior feature vector, which is the behavior feature after weighting with different weights. B Represents the weight matrix, which is used to weight different features. norm represents the standardized behavioral feature vector, and is represented as the standardized eigenvalue.

[0163] Furthermore, using the weighted behavioral feature vector B w , a behavior monitoring and anomaly detection model based on the long short-term memory network (LSTM) is constructed to capture the time dependency of user behavior. The goal of the model is to predict the behavior characteristics of the next moment and thus identify abnormal patterns of behavior. The update formula of the LSTM model is:

[0164] h t =f LSTM (B w,t ,h t-1 )

[0165] Among them, h t represents the hidden state at the current time t, capturing the temporal dependency of user behavior. w,t Represents the weighted behavior feature vector at the current time t. t-1 represents the hidden state at the previous moment t-1. LSTMRepresents the state update function of LSTM, which is defined as the regular LSTM unit calculation formula.

[0166] By calculating the residual between the actual behavior characteristics and the predicted behavior characteristics, it is determined whether there is abnormal behavior. The abnormal score ξ is calculated by the following formula:

[0167]

[0168] Among them, ξ represents the anomaly score, which indicates the difference between the current behavior and the predicted behavior. w,t Represents the actual weighted behavior feature vector at the current time t. Represents the weighted behavior feature vector of the next moment predicted by the LSTM model. 2 Represents the variance of the prediction residuals, used to normalize the anomaly scores.

[0169] Furthermore, based on the anomaly score ξ and the preset anomaly threshold θ, it is decided whether to trigger the permission adjustment or warning mechanism. The adjustment strategy is as follows:

[0170] If ξ≤θ, no adjustment is made and the current permission level is maintained

[0171] If ξ>θ, it is identified as abnormal behavior, and permissions are tightened or the user is prompted to authenticate based on the severity of the abnormality.

[0172] Furthermore, define the dynamic adjustment function of permissions under abnormal behavior conditions Dynamically adjust permission levels based on anomaly scores

[0173]

[0174] in, Indicates the permission level after dynamic adjustment, which takes abnormal behavior into consideration. represents the permission level outputted in the third step. ζ represents the adjustment step size parameter, which controls the amplitude of permission adjustment. ξ represents the current anomaly score. θ represents the threshold of anomaly detection, which is usually set based on actual user behavior data.

[0175] Furthermore, we continuously optimize the time series behavior model and anomaly detection mechanism by combining user feedback and actual behavior data. Through online learning and regular model updates, we ensure the model’s adaptability to changes in user behavior. We collect user behavior data (such as the number of abnormal behaviors, the frequency of permission adjustments, etc.), regularly update the parameters of the LSTM model, and adjust the anomaly detection threshold θ and the weight matrix W. B , ensuring that the system can operate stably in a changing user behavior environment.

[0176] S5. According to the permission level Combined with the user's emotional state, the emotional state is classified through the emotion classifier model to generate emotional labels. According to the emotional state labels and abnormal behavior response strategies, a mapping function between emotional state and permission response is proposed to dynamically adjust the permission level. The permission level calculated by the mapping function between emotional state and permission response is The smart speaker applies the new permission configuration instantly.

[0177] Furthermore, the user’s emotional state feature extraction uses a smart speaker combined with a microphone and a camera to collect the user’s multimodal emotional data in real time, including audio and video features. Audio feature vector A = [a1, a2, …, a m ] may include:

[0178] a1 represents the change in speech pitch frequency and is expressed as a continuous variable.

[0179] a2 represents the change in speaking rate and is expressed as a continuous variable.

[0180] a3 represents the speech volume intensity, expressed as a continuous variable.

[0181] Other audio features such as speech discontinuity, timbre, etc.

[0182] Video feature vector V = [v1, v2, ..., v n ] may include:

[0183] v1 represents facial expression features (such as upturned eye corners, downturned mouth corners, etc.), expressed as a categorical variable.

[0184] v2 represents eye movement characteristics (such as eye movement speed and direction), expressed as a continuous variable.

[0185] v3 represents the head posture (such as rotation angle, pitch angle, etc.), which is expressed as a continuous variable.

[0186] Furthermore, the audio feature vector A and the video feature vector V are fused to generate a fused emotion feature vector E. A fusion model based on the self-attention mechanism is used to capture the mutual influence between multimodal features. The specific formula is as follows:

[0187]

[0188] Where E represents the fused emotional feature vector, which is used to describe the user's current comprehensive emotional state. A represents the audio feature vector with a dimension of m. V represents the video feature vector with a dimension of n. W A and W V Represents the projection matrix, which is used to map audio and video features to the same space, with a dimension of m×d k and n×d k.d k Represents the dimension of the key vector in the attention mechanism, used to scale the dot product operation. Softmax represents the normalization function used to calculate the attention weight.

[0189] Furthermore, using the fused emotional feature vector E, a sentiment classifier model Classify the emotional state and generate the emotional label c:

[0190]

[0191] Where c represents the emotional state label of the current user, expressed as discrete categories (such as calm, angry, happy, etc.). i Represents the weight vector of the classifier model, the weight corresponding to the i-th class, with a dimension of d k . b i Represents the bias term of the classifier model, the bias corresponding to the i-th class, scalar.

[0192] Furthermore, according to the emotional state label c and the abnormal behavior response strategy, a mapping function between emotional state and authority response is proposed: Used to dynamically adjust permission levels

[0193]

[0194] in, Indicates the permission level that is dynamically adjusted according to the emotional state, and the permission setting that takes the user's emotions into consideration. It represents the permission level outputted in the fourth step. ρ(c) represents the permission adjustment increment function, which depends on the user's emotional state c and is expressed as a positive or negative integer.

[0195] Furthermore, according to the emotional state and authority response mapping function Calculated permission level The smart speaker instantly applies the new permission configuration to ensure that the user's operations are safe and reasonable under the current emotional state.

[0196] Combine users' immediate feedback and long-term behavior pattern data to continuously optimize sentiment classification models and permission response strategies. By real-time monitoring of user behavior and sentiment changes, regularly update sentiment classifier parameters, and adjust permission response functions. The model is fused with emotional features to ensure that the model can adapt to the dynamic changes of the user's emotional state.

[0197] S6. According to the authority level Build an adaptive permission optimization model based on reinforcement learning, combine users' long-term behavior and usage patterns, and use reinforcement learning to adaptively learn the permission strategy of smart speakers for long-term strategy optimization.

[0198] Specifically, the state of the adaptive permission optimization model based on reinforcement learning is defined as the user's current permission level and emotional state, as well as the user's historical behavior pattern and system feedback, that is, the state vector in, represents the dynamically adjusted permission level outputted from the fifth step, and represents the current permission configuration optimized according to the user's emotional state. c represents the user's current emotional state label, expressed as discrete categories (such as calm, angry, happy, etc.). H represents the user's historical behavior pattern feature vector, which represents the statistical characteristics of the user's behavior over a period of time in the past, such as operation frequency, preferred operation type, etc. F represents the system feedback feature vector, which represents the user feedback data of the system under different permission configurations, such as user satisfaction score, frequency of erroneous operations, etc.

[0199] Define the actions that the system can perform as a set of permission adjustment operations Each action a i Indicates the adjustment of permission level (such as increase, decrease, maintain, etc.). The design of these actions needs to take into account the security of permissions, user experience and functional limitations of smart speakers.

[0200] Furthermore, in order to enable the system to maintain the optimal permission configuration under the changing user behavior and emotional state, we adopt the reinforcement learning method of policy optimization. Define the optimization objective function J(θ), and θ The parameter θ of (S) is optimized to maximize the expected cumulative reward. The formula for optimizing the objective function J(θ) is as follows:

[0201]

[0202] Among them, J(θ) represents the optimization objective function of the policy parameter θ, which represents the expected cumulative reward. S represents the current state vector, which consists of the authority level, emotional state, historical behavior pattern, and system feedback. Represents the state distribution set, which represents the possible state space. γ represents the discount factor, which ranges from 0≤γ≤1 and is used to measure the importance of future rewards. r(S t ,a t ) indicates that in state S t Execute action a t The immediate reward obtained when the authority is adjusted reflects the effectiveness and rationality of the authority adjustment. t represents the action performed at time t, given by the strategy π θ (S) OK.

[0203] In the policy optimization process, in order to avoid the policy from overfitting the user's short-term behavior pattern, we introduce a novel policy regularization term Ω(θ) and update the policy parameter θ to maximize the regularized optimization objective:

[0204]

[0205] Among them, θ t+1 represents the updated policy parameters. t Represents the policy parameters at the current time t. α represents the learning rate, which controls the step size of parameter update. represents the policy gradient, which represents the gradient of the optimization objective function to the policy parameters. λ represents the regularization strength parameter, which controls the influence of the regularization term on the policy update. Ω(θ t ) represents the policy regularization term, which is designed as a sparse regularization to encourage simplicity and generalization of policy parameters. The formula is:

[0206]

[0207] Where N is the total number of policy parameters. i Represents the i-th parameter in the strategy parameters.

[0208] Furthermore, the immediate reward function r(S t ,a t ) is designed by combining user behavior feedback, emotional state changes, and the rationality of permission adjustment, aiming to balance the security of the system and user experience. The formula of the reward function is as follows:

[0209]

[0210] Among them, r(S t ,a t ) represents the immediate reward, which measures the t Execute action a t The effect of t represents the change in user satisfaction score at the current time t, and the function f(U t ) indicates positive rewards, which are positively correlated with reasonable authority adjustments. t Indicates the change in the user's emotional state fluctuation, function g(E t ) indicates negative rewards, which are negatively correlated with negative emotional reactions caused by authority adjustments. Indicates the extent of permission adjustment, function Indicates negative rewards, punishing overly frequent permission adjustment operations. ω1, ω2, ω3 represent weight coefficients, controlling the impact of various factors on immediate rewards.

[0211] To further improve the adaptability and generalization ability of the model, we introduced an adaptive learning mechanism based on user behavior patterns. This mechanism continuously monitors the user's behavioral characteristics and emotional state changes, and dynamically adjusts the learning rate and regularization parameters of the strategy model to adapt to the user's personalized needs. The formula for dynamic adjustment of the learning rate and regularization parameters is:

[0212] α t+1 =α t ·(1-η·|ΔH t |)

[0213] λ t+1 =λ t ·(1+ζ·|ΔF t |)

[0214] Among them, α t+1 Represents the learning rate at the next moment. t Represents the current learning rate. η represents the learning rate adjustment coefficient, which controls the adjustment range of the learning rate. |ΔH t | represents the change amplitude of the user behavior pattern feature vector, indicating the degree of change of user behavior. t+1 Represents the regularization strength parameter at the next moment. t represents the regularization strength parameter at the current moment. ζ represents the regularization parameter adjustment coefficient, which controls the adjustment range of the regularization strength. |ΔF t | represents the change amplitude of the system feedback feature vector, which represents the degree of change of user feedback.

[0215] Furthermore, we combine the user's long-term behavior data and immediate feedback to establish a strategy optimization and feedback loop. By continuously monitoring the system's performance under different permission configurations and regularly updating the parameters of the strategy model, we can ensure the system's optimal performance in the long run.

[0216] Collect users’ long-term behavior data (such as permission adjustment frequency, changes in user satisfaction, fluctuations in emotional state, etc.), and regularly update the parameters of the reinforcement learning model, including the strategy parameter θ, regularization parameter λ, and learning rate α, to ensure that the model can adapt to long-term changes in user behavior and emotional state.

[0217] The present application also provides a smart speaker permission design management system, such as Figure 2 As shown, the system comprises:

[0218] The user data collection module 101 is used to collect user audio data and pre-process the audio data, and construct a Gaussian mixture model based on the pre-processed data to output an audio signature;

[0219] Configuration initialization module 102, used to use the audio signature as input to represent the identity characteristics of the user, collect real-time environment data and pre-process it, build a user-context joint feature vector based on the user's identity characteristics combined with the pre-processed real-time environment data, add a regularization term to the user-context joint feature vector to adjust the recognition accuracy, and then perform context awareness and dynamic permission configuration initialization to obtain an optimized permission level and permission score;

[0220] The permission initialization module 103 is used to design a fuzzy matching and dynamic adjustment model to dynamically manage the permissions of the smart speaker in real time based on the optimized permission level and permission score if a complex voice environment with multiple languages ​​and multiple accents appears; wherein the fuzzy matching and dynamic adjustment model uses a feature weighted aggregation method to calculate the voice feature vector C new The credibility score γ is expressed as follows:

[0221]

[0222] Among them, γ represents the credibility score of the speech input, which indicates the credibility of the current speech feature matching; M represents the total number of speech features; w j represents the weight of the jth feature, indicating the importance of this feature in calculating the credibility of speech; α j represents the fuzzy adjustment parameter of the jth feature, controlling the sensitivity of feature matching; C new,j represents the component of the jth speech feature vector; μ j represents the expected value of the jth feature, which represents the typical value of the feature under normal circumstances;

[0223] Using the credibility score γ of voice input and the optimized authority score Adjusting functions based on permissions Dynamically adjust user permission levels It is expressed as follows:

[0224]

[0225] in, Indicates the dynamically adjusted permission level, indicating the permission setting adjusted according to the voice credibility and the original permission score; represents the initial authority level output in the second step; η represents the authority adjustment gain coefficient, which controls the authority adjustment range; γ represents the credibility score of the speech input; δ represents the intermediate threshold of the credibility score; represents the optimized permission score; τ represents the baseline threshold of the permission score; based on the calculated new permission level The smart speaker system instantly adjusts the user's permission configuration and dynamically changes the accessibility of functions;

[0226] The authority management module 104 is used to optimize the new authority level based on the complex voice environment. Combined with real-time user behavior data, a behavior monitoring and anomaly detection model is built to analyze user permissions in real time through feature engineering and time series modeling, perform anomaly detection and behavior monitoring on user behavior, and dynamically adjust permission levels based on the results of anomaly detection and behavior monitoring. Based on permission level Combined with the user's emotional state, the emotional state is classified through the emotion classifier model to generate emotional labels. According to the emotional state labels and abnormal behavior response strategies, a mapping function between emotional state and permission response is proposed to dynamically adjust the permission level. The permission level calculated by the mapping function between emotional state and permission response is Smart speakers instantly apply new permission configurations;

[0227] System optimization module 105, used to optimize the system according to the permission level Build an adaptive permission optimization model based on reinforcement learning, combine users' long-term behavior and usage patterns, and use reinforcement learning to adaptively learn the permission strategy of smart speakers for long-term strategy optimization.

[0228] The above disclosures are only some preferred embodiments of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention are still within the scope of the invention.

Claims

1. A smart speaker permission design and management method, characterized in that: The method comprises: S1, collect user audio data and preprocess the audio data, build a Gaussian mixture model based on the preprocessed data to output the audio signature; S2. The audio signature is used as input to represent the identity characteristics of the user. The real-time environment data is collected and preprocessed. A user-context joint feature vector is constructed based on the user's identity characteristics and the preprocessed real-time environment data. A regularization term is added to the user-context joint feature vector to adjust the recognition accuracy. Then, context awareness and dynamic permission configuration initialization are performed to obtain the optimized permission level and permission score. S3. Based on the optimized permission level and permission score, if a complex voice environment with multiple languages ​​and multiple accents appears, a fuzzy matching and dynamic adjustment model is designed to dynamically manage the permissions of the smart speaker in real time; wherein the fuzzy matching and dynamic adjustment model uses a feature weighted aggregation method to calculate the voice feature vector C new The credibility score γ is expressed as follows: Among them, γ represents the credibility score of the speech input, which indicates the credibility of the current speech feature matching; M represents the total number of speech features; w j represents the weight of the jth feature, indicating the importance of this feature in calculating the credibility of speech; α j represents the fuzzy adjustment parameter of the jth feature, controlling the sensitivity of feature matching; C new,j represents the component of the jth speech feature vector; μ j represents the expected value of the jth feature, which represents the typical value of the feature under normal circumstances; Using the credibility score γ of voice input and the optimized authority score Adjusting functions based on permissions Dynamically adjust user permission levels It is expressed as follows: in, Indicates the dynamically adjusted permission level, indicating the permission setting adjusted according to the voice credibility and the original permission score; represents the initial authority level output in the second step; η represents the authority adjustment gain coefficient, which controls the authority adjustment range; γ represents the credibility score of the speech input; δ represents the intermediate threshold of the credibility score; represents the optimized permission score; τ represents the baseline threshold of the permission score; based on the calculated new permission level The smart speaker system instantly adjusts the user's permission configuration and dynamically changes the accessibility of functions; S4. New permission level optimized based on complex voice environment Combined with real-time user behavior data, a behavior monitoring and anomaly detection model is built to analyze user permissions in real time through feature engineering and time series modeling, perform anomaly detection and behavior monitoring on user behavior, and dynamically adjust permission levels based on the results of anomaly detection and behavior monitoring. S5. According to the permission level Combined with the user's emotional state, the emotional state is classified through the emotion classifier model to generate emotional labels. According to the emotional state labels and abnormal behavior response strategies, a mapping function between emotional state and permission response is proposed to dynamically adjust the permission level. The permission level calculated by the mapping function between emotional state and permission response is Smart speakers instantly apply new permission configurations; S6. According to the authority level Build an adaptive permission optimization model based on reinforcement learning, combine users' long-term behavior and usage patterns, and use reinforcement learning to adaptively learn the permission strategy of smart speakers for long-term strategy optimization.

2. A smart speaker permission design and management method according to claim 1, characterized in that: The preprocessing is to process the collected audio data x(t) to remove noise and perform standardization; Wherein, the probability density function of the Gaussian mixture model is expressed as: Wherein, C represents the extracted Mel frequency cepstral coefficient feature vector, which is used to represent the voiceprint feature of the audio; Represents the parameter set of the Gaussian mixture model; π k represents the mixing weight of the kth Gaussian component, satisfying And π k ≥0;μ k represents the mean vector of the kth Gaussian component; Σ k represents the covariance matrix of the kth Gaussian component; K represents the number of Gaussian components, which means the number of Gaussian distributions in the model; represents the multivariate normal distribution, expressed as: Where d represents the dimension of the feature vector C.

3. A smart speaker permission design management method according to claim 2, characterized in that: For each new audio input, the user identity is identified by calculating the log-likelihood estimate of the feature vector of the new audio under each user model, which is expressed as follows: Among them, logL(C new |S i ) represents the log-likelihood value of the new feature vector under the i-th user model, which is used to measure the matching degree between the audio input and the user model; C new The feature vector representing the new audio input; S i represents the audio signature of the i-th user, that is, the GMM parameter set Θ i ; T represents the total number of frames of the new audio input, which represents the number of frames after the audio signal is segmented; P(C new (t)|Θ i ) represents the feature vector C of the tth frame new (t) In the user model Θ i The probability of the following.

4. The method for designing and managing permissions for a smart speaker according to claim 1, characterized in that: The real-time environmental data is collected through a variety of sensors, including microphones, cameras, light sensors, temperature and humidity sensors, and accelerometers. The real-time environmental feature vector is represented by E; The user-context joint feature vector is fused through a weighted multilayer perceptron model, which is expressed as follows: C e =σ(W2·σ(W1·[E norm ,S i ]+b1)+b2) Among them, C e represents the generated context feature vector, which represents the user's behavior characteristics in the current environment; [E norm ,S i ] represents the input feature after combining the environment feature vector and the user audio signature; W1, W2 represent weight matrices with dimensions of m×(n+1) and m×m, respectively, which are learned through model training; b1, b2 represent bias vectors with dimensions of m and m, respectively, which are learned through model training; σ represents a nonlinear activation function, which is used to capture nonlinear relationships; Combined with the context feature vector C e and user audio signature S i , generate the user-context joint feature vector F = [S i ,C e ]; In the user-context joint feature vector F = [S i ,C e ]Adding regularization terms can achieve higher recognition accuracy by weighted combination of user and environment features, as shown below: F opt =F+λ·(F⊙W reg ) Among them, F opt represents the optimized joint feature vector; λ represents the regularization parameter, which is used to balance the influence of the original features and the regularization term; ⊙ represents the element-wise product operation, which is used for element-by-element weighting; W reg Represents the weight matrix, which represents the relationship between features.

5. The method for designing and managing permissions for a smart speaker according to claim 1, characterized in that: The configuration initialization includes: Design a permission scoring function based on the adjusted user-context joint feature vector to calculate the user's permission score in the current context It is expressed as follows: in, Indicates the permission score, which is used to reflect the user's permission level in the current context; represents the weight vector, indicating the importance of each feature; β represents the weight adjustment coefficient, controlling the overall weight of the joint features; The bi-norm of the joint features is used as a measure of feature importance, which adds constraints on the balance between features. represents the bias term; Based on the calculated authority score and preset permission level threshold, and dynamically adjust permissions. It is expressed as follows: in, Indicates the dynamically adjusted permission level; Indicates the permission threshold, which is set according to user needs and system configuration.

6. A smart speaker permission design and management method according to claim 1, characterized in that: In S3, an error correction feedback mechanism is designed for the fuzzy matching and dynamic adjustment model to dynamically adjust the weights and parameters of the fuzzy matching model. The error correction mechanism is expressed as follows: Δw=∈·(γ target -c)·C new Among them, Δw represents the weight adjustment vector, which is used to correct the weight of speech features; ∈ represents the learning rate, which is used to control the step size of weight adjustment; γ target represents the target credibility score, which is usually set to the credibility level expected by the system; γ represents the credibility score of the current speech input; C new The feature vector representing the current speech input.

7. The method for designing and managing permissions for a smart speaker according to claim 1, characterized in that: According to the weighted behavior feature vector B w A monitoring and anomaly detection model based on long short-term memory network is constructed to predict the behavior characteristics of the next moment and identify abnormal behavior patterns. The update formula of the behavior monitoring and anomaly detection model is expressed as follows: h t =f LSTM (B w,t ,h t-1 ) Among them, h t represents the hidden state at the current time t, capturing the time dependency of user behavior; B w,t represents the weighted behavior feature vector at the current time t; h t-1 represents the hidden state at the previous moment t-1; f LSTM Represents the state update function of the long short-term memory network; By calculating the residual between the actual behavior characteristics and the predicted behavior characteristics, it is determined whether there is abnormal behavior; wherein the abnormal score ξ is expressed as follows: Among them, ξ represents the anomaly score, which indicates the difference between the current behavior and the predicted behavior; B w,t Represents the actual weighted behavior feature vector at the current time t; Represents the weighted behavior feature vector of the next moment predicted by the long short-term memory network; σ 2 represents the variance of the prediction residuals, which is used to normalize the anomaly scores; According to the abnormal score ξ and the preset abnormal threshold θ, it is determined whether to trigger the permission adjustment or warning mechanism; the adjustment strategy is as follows: If ξ≤θ, no adjustment is made and the current permission level is maintained If ξ>θ, it is identified as abnormal behavior, and permissions are tightened or the user is prompted to authenticate based on the severity of the abnormality.

8. The smart speaker permission design and management method according to claim 1, characterized in that: The collection of the user's emotional state includes the collection of audio and video features, and then a fusion model based on the self-attention mechanism is used to capture the mutual influence between multimodal features. The specific formula is as follows: Where E represents the fused emotional feature vector, which is used to describe the user's current comprehensive emotional state; A represents the audio feature vector, with a dimension of m; V represents the video feature vector, with a dimension of n; W A and W V Represents the projection matrix, which is used to map audio and video features to the same space, with a dimension of m×d k and n×d k ;d k Represents the dimension of the key vector in the attention mechanism, which is used to scale the dot product operation; softmax represents the normalization function used to calculate the attention weight; Using the fused emotional feature vector E, the emotional state is classified through the emotional classifier model C(E) to generate the emotional label c, which is expressed as follows: Among them, c represents the emotional state label of the current user, expressed as a discrete category; w i Represents the weight vector of the classifier model, the weight corresponding to the i-th class, with a dimension of d k ; b i Represents the bias term of the classifier model, the bias corresponding to the i-th class; According to the emotional state label c and the abnormal behavior response strategy, a mapping function between emotional state and authority response is proposed Used to dynamically adjust permission levels It is expressed as follows: in, It represents the permission level dynamically adjusted according to the emotional state, and takes the user's emotions into consideration when setting the permission. ρ(c) represents the permission adjustment increment function, which depends on the user's emotional state c and is expressed as a positive or negative integer.

9. A smart speaker permission design management method according to claim 8, characterized in that: The adaptive permission optimization model based on reinforcement learning is based on the user's current permission level and emotional state, as well as the user's historical behavior pattern and system feedback, that is, the state vector Among them, H represents the user's historical behavior pattern feature vector; F represents the system feedback feature vector; the system's executable actions are defined as the permission adjustment operation set Each action a i represents the adjustment of the permission level; it means optimizing the objective function J(θ), by adjusting the strategy π θ (S) is optimized to maximize the expected cumulative reward; The formula for optimizing the objective function J(θ) is as follows: Where J(θ) represents the optimization objective function of the policy parameter θ, which represents the expected cumulative reward; S represents the current state vector, which consists of the authority level, emotional state, historical behavior pattern, and system feedback; represents the state distribution set, which represents the possible state space; γ represents the discount factor, which ranges from 0≤γ≤1 and is used to measure the importance of future rewards; r(S t ,a t ) indicates that in state S t Execute action a t The immediate reward obtained when the authority is adjusted reflects the effectiveness and rationality of the authority adjustment; t represents the action performed at time t, given by the strategy π θ (S) confirm; In the process of policy optimization, the policy regularization term Ω(θ) is introduced to prevent the policy from overfitting the user's short-term behavior pattern, and the policy parameter θ is updated to maximize the regularized optimization objective, which is expressed as follows: Among them, θ t+1 represents the updated policy parameters; θ t represents the policy parameter at the current time t; α represents the learning rate, which controls the step size of parameter update; represents the policy gradient, which represents the gradient of the optimization objective function to the policy parameters; λ represents the regularization strength parameter, which controls the influence of the regularization term on the policy update; Ω(θ t ) represents the policy regularization term, which is a sparsity regularization that encourages simplicity and generalization of policy parameters, and is expressed as follows: Where N is the total number of policy parameters; θ i Represents the i-th parameter in the strategy parameters.

10. A smart speaker authority design and management system, characterized in that: The system comprises: A user data collection module is used to collect user audio data and preprocess the audio data, and construct a Gaussian mixture model based on the preprocessed data to output an audio signature; Configuration initialization module, used to take audio signature as input to represent the identity characteristics of the user, collect real-time environment data and pre-process it, build a user-context joint feature vector based on the user's identity characteristics combined with the pre-processed real-time environment data, add regularization terms to the user-context joint feature vector to adjust the recognition accuracy, and then perform context awareness and dynamic permission configuration initialization to obtain the optimized permission level and permission score; The permission initialization module is used to design a fuzzy matching and dynamic adjustment model to dynamically manage the permissions of the smart speaker in real time based on the optimized permission level and permission score if a complex voice environment with multiple languages ​​and accents appears; wherein the fuzzy matching and dynamic adjustment model uses a feature weighted aggregation method to calculate the voice feature vector C new The credibility score γ is expressed as follows: Among them, γ represents the credibility score of the speech input, which indicates the credibility of the current speech feature matching; M represents the total number of speech features; w j represents the weight of the jth feature, indicating the importance of this feature in calculating the credibility of speech; α j represents the fuzzy adjustment parameter of the jth feature, controlling the sensitivity of feature matching; C new,j represents the component of the jth speech feature vector; μ j represents the expected value of the jth feature, which represents the typical value of the feature under normal circumstances; Using the credibility score γ of voice input and the optimized authority score Adjusting functions based on permissions Dynamically adjust user permission levels It is expressed as follows: in, Indicates the dynamically adjusted permission level, indicating the permission setting adjusted according to the voice credibility and the original permission score; represents the initial authority level output in the second step; η represents the authority adjustment gain coefficient, which controls the authority adjustment range; γ represents the credibility score of the speech input; δ represents the intermediate threshold of the credibility score; represents the optimized permission score; τ represents the baseline threshold of the permission score; based on the calculated new permission level The smart speaker system instantly adjusts the user's permission configuration and dynamically changes the accessibility of functions; The permission management module is used to optimize the new permission level based on the complex voice environment. Combined with real-time user behavior data, a behavior monitoring and anomaly detection model is built to analyze user permissions in real time through feature engineering and time series modeling, perform anomaly detection and behavior monitoring on user behavior, and dynamically adjust permission levels based on the results of anomaly detection and behavior monitoring. Based on permission level Combined with the user's emotional state, the emotional state is classified through the emotion classifier model to generate emotional labels. According to the emotional state labels and abnormal behavior response strategies, a mapping function between emotional state and permission response is proposed to dynamically adjust the permission level. The permission level calculated by the mapping function between emotional state and permission response is Smart speakers instantly apply new permission configurations; System optimization module for Build an adaptive permission optimization model based on reinforcement learning, combine users' long-term behavior and usage patterns, and use reinforcement learning to adaptively learn the permission strategy of smart speakers for long-term strategy optimization.

Images