A voiceprint template hierarchical update control method, system and electronic device

Abstract

The application belongs to the technical field of voice biometric feature recognition and voiceprint verification, and particularly relates to a voiceprint template hierarchical update control method and system and electronic equipment. The method takes the automatic speech recognition confidence of a voice signal, voiceprint similarity and state confidence as the decision basis of three-level gating, generates a hierarchical decision of rejecting update, only updating an identity template or simultaneously updating the identity template and a state sub-template according to the passing combination of the three-level gating, and updates the identity template and the state sub-template respectively according to the decision result by using a differentiated update rate. The application can realize hierarchical control of voice quality, identity consistency and state recognition reliability, balance the long-term stability of the identity template and the short-term adaptability of the state sub-template, improve the robustness of the voiceprint verification system, and can be used in intelligent terminal unlocking, voice assistant awakening, remote identity authentication, vehicle-mounted interaction and health monitoring scenes.

Description

Technical Field

[0001] This application belongs to the field of voice biometric recognition and voiceprint verification technology, specifically involving a voiceprint template hierarchical update control method, system and electronic device. Background Technology

[0002] Voiceprint recognition technology identifies users by extracting speaker-specific features from speech signals and has been widely applied in various scenarios such as smart terminal unlocking, voice assistant wake-up, remote identity authentication, in-vehicle human-machine interaction, and health monitoring. In practical deployments, voiceprint recognition systems typically rely on pre-registered identity templates for comparison and verification. To maintain system availability, existing technologies mainly employ two template maintenance strategies: a static template strategy, where an initial template is used after user registration and is not updated further (this approach is simple and highly stable); and a dynamic template update strategy, where the original template is iteratively updated using currently collected speech features during the recognition process when the confidence level or matching score reaches a preset standard, in order to adapt to the natural changes in the user's voice over time. Furthermore, some advanced solutions introduce independent state recognition modules to analyze the user's emotions or health status and output the recognition results as auxiliary information to support richer interactive functions or health management services.

[0003] However, in complex real-world applications, existing voiceprint template update mechanisms face the common challenge of balancing long-term stability with short-term adaptability. A user's acoustic characteristics are not only affected by long-term factors such as age and device replacement, but also by short-term physiological or psychological states such as colds, fatigue, and emotional fluctuations. Existing solutions often lack layered isolation and refined control mechanisms for risks related to voice quality, identity consistency, and uncertainty in state recognition. This leads to the system easily and incorrectly embedding abnormal features into the long-term identity template when the user is in an abnormal state, causing template contamination or drift, which can result in the wrongful rejection of legitimate users or a decrease in system security. Furthermore, existing state recognition results are usually independent of the template update process and fail to be effectively transformed into key variables controlling the template lifecycle and update strategy, resulting in significant deficiencies in the system's robustness and adaptability in the face of changing user states. Summary of the Invention

[0004] The purpose of this application is to provide a hierarchical update control method, system and electronic device for voiceprint templates, which can solve the technical problems that the existing voiceprint template update mechanism cannot take into account both long-term stability and short-term state adaptability, and lacks hierarchical control over voice quality, identity consistency and state recognition reliability.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] In a first aspect, this application provides a voiceprint template hierarchical update control method, comprising: S1, acquiring a user-input speech signal and preprocessing the speech signal to obtain a preprocessed speech signal; S2, extracting a voiceprint embedding vector and a state feature vector from the preprocessed speech signal, performing automatic speech recognition on the preprocessed speech signal to obtain an automatic speech recognition confidence level, and comparing the voiceprint embedding vector with a pre-stored user identity template to obtain a voiceprint similarity; S3, comparing the automatic speech recognition confidence level with a first threshold to execute a first gating, and comparing the voiceprint similarity with a second threshold to execute a second gating, and performing quality screening on the current speech sample based on the passing status of the first and second gating; S4, when both the first and second gatings pass, performing state recognition on the preprocessed speech signal to obtain a state label and a state confidence level, and setting the state... S5. A third gating is performed by comparing the confidence level with a third threshold; S6. A hierarchical update decision is generated based on the passing combination of the first, second, and third gatings. The hierarchical update decision satisfies at least the following conditions: when the first or second gating fails, a rejection update decision is generated; when both the first and second gatings pass and the third gating fails, a decision to update only the identity template is generated; when all three gatings pass, a decision to update both the identity template and the state sub-template is generated; S7. Template updates are performed based on the hierarchical update decision. When the hierarchical update decision is to update only the identity template, the identity template is updated using a first update rate. When the hierarchical update decision is to update both the identity template and the state sub-template simultaneously, the identity template is updated using the first update rate, and the state sub-template corresponding to the state label is updated using a second update rate, wherein the second update rate is greater than the first update rate.

[0007] The method described above in this application can be further specified as follows:

[0008] In an optional embodiment, the first threshold and / or the second threshold in step S3 are dynamic thresholds, which are determined based on at least one of the current ambient noise level, the user's recent voiceprint matching history, the stability of automatic speech recognition, and the security level; the first threshold is increased when the noise level increases, the second threshold is decreased when the user's recent matching is stable and the ambient noise is below the noise threshold, and the first threshold and / or the second threshold is increased when the security level increases.

[0009] In an optional embodiment, the status label in step S4 includes at least one of emotion status label and health status label; the emotion status label includes at least one of calm, happy, sad and angry, and the health status label includes at least one of healthy, cold, fatigue and allergy; the status sub-template establishes a three-level index according to user identifier, status type and status label, and initializes the corresponding status sub-template with the current voiceprint embedding vector when the corresponding status sub-template does not exist and the third gating passes.

[0010] In an optional embodiment, the generation of the hierarchical update decision in step S5 is judged sequentially according to the priority order of the first gating, the second gating, and the third gating; when the first gating fails or the second gating fails, a rejection update decision is generated, and the state recognition is skipped or the subsequent gating judgment is terminated.

[0011] In an optional embodiment, in step S6, the first update rate decreases as the cumulative number of updates to the identity template increases and is not lower than a preset minimum update rate; the second update rate adopts a normal second update rate when the cumulative number of updates to the state sub-template is less than a preset switching threshold, and switches to a slow second update rate after reaching the switching threshold; the normal second update rate is greater than the first update rate, and the slow second update rate is less than the normal second update rate but greater than the first update rate.

[0012] Furthermore, the method may also include: determining template contamination risk based on the state switching index and identity template change index within the most recent window; triggering template contamination protection when the template contamination risk meets preset protection conditions; and reducing the first update rate used to update the identity template.

[0013] In one optional embodiment, the state switching indicator includes at least one of the following: the number of state label switching times, the number of different state labels, or the state switching frequency; the identity template change indicator includes at least one of the following: the normalized vector change magnitude before and after the identity template update, the cosine distance, or the template center drift. When the state switching indicator reaches the state switching threshold and the identity template change indicator reaches the template change threshold, a conservative update mode is triggered. In the conservative update mode, the first update rate is reduced to a preset proportion of the base first update rate. When the number of consecutive stable windows or the number of consecutive stable times reaches the recovery threshold, the conservative update mode is exited.

[0014] Furthermore, the method may further include: during the voiceprint verification stage, dynamically fusing identity template similarity and state sub-template similarity based on the current state label, state confidence, and the timeliness of the corresponding state sub-template to obtain the final verification score. Specifically, when the state sub-template corresponding to the current state label exists and the state confidence reaches a preset threshold, the state sub-template similarity is fused with a state weight. The state weight is positively correlated with the state confidence and negatively correlated with the age of the state sub-template. When the corresponding state sub-template does not exist or the state confidence does not reach the preset threshold, only the identity template similarity is used as the final verification score. The age of the state sub-template is the time interval or update cycle number between the current verification time and the most recent update time of the corresponding state sub-template.

[0015] Secondly, this application provides a voiceprint template hierarchical update control system, the system being configured to execute the method described in any of the above claims, the system comprising: a voice acquisition and preprocessing module, used to acquire a user-input voice signal and preprocess the voice signal to obtain a preprocessed voice signal; a shared feature extraction module, used to extract a voiceprint embedding vector and a state feature vector from the preprocessed voice signal; an automatic speech recognition module, used to perform automatic speech recognition on the preprocessed voice signal to obtain an automatic speech recognition confidence level; a voiceprint comparison module, used to compare the voiceprint embedding vector with a user identity template to obtain a voiceprint similarity; and a state recognition module, used to perform state recognition on the preprocessed voice signal when both the first and second gates are passed. The system obtains state labels and state confidence scores; a multi-level gating coordination module executes the first, second, and third gating controls and generates hierarchical update decisions based on the gating controls; a template storage and update module maintains user identity templates and state sub-templates and executes template updates based on hierarchical update decisions; a template contamination prevention module reads the state label sequence and the vector before and after the update of the user identity template within the most recent window, and determines whether to enter a conservative update mode based on state switching indicators and identity template change indicators; and a state-aware verification module dynamically fuses identity template similarity and state sub-template similarity to obtain the final verification score during the voiceprint verification stage, based on the current state label, state confidence score, and the timeliness of the corresponding state sub-template.

[0016] Thirdly, this application also proposes an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor causes the processor to implement the method described in any of the above claims.

[0017] Compared with the prior art, this application has the following beneficial effects:

[0018] This application employs a multi-level gating process based on automatic speech recognition confidence, voiceprint similarity, and state confidence, incorporating speech content clarity, speaker identity consistency, and state recognition reliability into template update judgments. This allows for the layered isolation of low-quality speech, non-target user speech, and unreliable state recognition samples, reducing the probability of erroneous samples entering the template update process and minimizing template contamination at its source.

[0019] This application achieves separate control of identity templates and state sub-templates through hierarchical update decisions and differentiated update rates: identity templates are updated slowly to maintain long-term stability, while state sub-templates are updated quickly to capture short-term changes. The two work together to reduce the risk of short-term state deviations being solidified into identity templates and to provide more accurate verification when the user is in a specific state.

[0020] This application employs a cascaded three-level gating design, which rejects updates if the first or second gating fails, thus avoiding further state identification and subsequent update judgments. This reduces the risk of template contamination while minimizing unnecessary state identification calculations, thereby improving the computational efficiency of the template update control process. Attached Figure Description

[0021] Figure 1 A flowchart of a voiceprint template hierarchical update control method provided in this application embodiment;

[0022] Figure 2 A schematic diagram of the index structure of the identity template and status sub-template provided in the embodiments of this application;

[0023] Figure 3 This is a structural block diagram of a voiceprint template hierarchical update control system provided in an embodiment of this application.

[0024] Figure labeling: 301, Voice acquisition and preprocessing module; 302, Shared feature extraction module; 303, Automatic speech recognition module; 304, Voiceprint comparison module; 305, Status recognition module; 306, Multi-level gating coordination module; 307, Template storage and update module; 308, Template contamination protection module; 309, Status perception verification module. Detailed Implementation

[0025] The technical solutions in the embodiments of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0026] like Figure 1The diagram shown is a flowchart of a voiceprint template hierarchical update control method provided in an embodiment of this application. The method includes:

[0027] Step S101: Acquire the user-input voice signal and preprocess the voice signal to obtain the preprocessed voice signal.

[0028] In this embodiment, the voice signal can refer to the raw acoustic waveform data containing user voice information captured by a collection device such as a microphone array, a single-channel microphone, or a far-field pickup device. The voice content can include wake words, voice commands, authentication phrases, or natural interactive voice. The sampling rate can be from 8 kHz to 48 kHz, preferably 16 kHz; the effective voice duration can be from 0.5 seconds to 10 seconds. If the endpoint detection result shows that the effective voice duration is less than the minimum duration, the system records the sample as unusable and does not proceed to the template update process.

[0029] Preprocessing of the speech signal aims to eliminate environmental interference and channel distortion, thereby improving the robustness of features in subsequent voiceprint recognition or state recognition processes. In this embodiment, preprocessing includes a signal enhancement stage and a feature extraction stage. The signal enhancement stage may include DC removal, pre-emphasis, speech activity detection, and noise reduction to filter out non-speech segments and environmental background noise. The feature extraction stage includes framing and windowing the enhanced speech signal, and calculating acoustic feature sequences (e.g., Mel spectrum, filter bank features, or Mel frequency cepstral coefficients), followed by mean-variance normalization of the extracted acoustic feature sequences. Preferably, the framing process can use a frame length of 20 ms to 30 ms and a frame shift of 5 ms to 15 ms. This range takes into account both the short-term stationarity of the speech signal (usually remaining stationary within 20-30 ms) and the information continuity between frames, while also adapting to the input dimension requirements of the subsequent neural network model.

[0030] Step S102: Extract the voiceprint embedding vector and state feature vector from the preprocessed speech signal, perform automatic speech recognition on the preprocessed speech signal to obtain the automatic speech recognition confidence, and compare the voiceprint embedding vector with the pre-stored user identity template to obtain the voiceprint similarity.

[0031] In one embodiment, the extraction of the voiceprint embedding vector and the state feature vector can employ a joint architecture of a shared feature extraction network, which includes an input layer, a shared acoustic coding layer, a statistical pooling layer, a voiceprint branch, and a state feature branch. The input layer receives the acoustic feature sequence; the shared acoustic coding layer can employ one or more combinations of convolutional neural networks, time-delay neural networks, recurrent neural networks, and Transformer encoders to encode the acoustic feature sequence frame by frame, extracting a deep acoustic representation containing speaker identity information; the statistical pooling layer performs mean pooling, standard deviation pooling, or attention pooling along the time dimension on the aforementioned deep acoustic representation to map the variable-length speech frame sequence into a fixed-length global feature vector. The voiceprint branch receives the global feature vector, processes it through a fully connected layer, a batch normalization layer, and an L2 normalization layer, and outputs a voiceprint embedding vector with an output dimension of 128 to 512 dimensions. In one implementation, the fully connected layer has a two-layer structure. The first layer has an output dimension of 512, and the second layer has an output dimension of 128. Each layer is followed by a batch normalization layer and a ReLU activation function. Finally, an L2 normalization layer is used to output a 128-dimensional voiceprint embedding vector. The state feature branch receives the global feature vector, processes it through the fully connected layer and the batch normalization layer, and then outputs a state feature vector.

[0032] In another embodiment, the extraction of the voiceprint embedding vector and the state feature vector can also be set as two independent models, each using its own independent acoustic coding layer and statistical pooling layer, without sharing parameters.

[0033] In another embodiment, the state feature vector may be extracted directly from the speech signal without using a neural network, and may include one or more of the following: fundamental frequency, energy, fundamental frequency jitter, amplitude perturbation, harmonic-to-noise ratio, speech rate, pause ratio, Mel frequency cepstral coefficient statistics, and filter bank statistics.

[0034] Automatic speech recognition confidence is used to measure the probability that the content of a speech signal is correctly recognized. In one embodiment, the preprocessed speech signal can be input into an end-to-end automatic speech recognition model (such as a model based on the Listen-Attend-Spell architecture or the Wav2Vec2.0 model), and the decoding path probability or posterior probability output by the model is the automatic speech recognition confidence. In another embodiment, a weighted sum of the acoustic model score and the language model score can be calculated using a traditional hybrid model (HMM-DNN), and this score can be mapped to the [0,1] interval using a function such as the Sigmoid function as the confidence index, where the weighting coefficients can be determined by adjusting the development set. Furthermore, those skilled in the art should understand that the above models are merely examples, and other models with similar functions can also be applied.

[0035] Voiceprint similarity characterizes the degree of matching between the current speech and a known user identity. It is obtained by using any one of the following as a similarity index: cosine similarity, negative Euclidean distance, probabilistic linear discriminant analysis score, or calibrated normalized score. In one specific implementation, voiceprint similarity is obtained by calculating the cosine similarity or Euclidean distance between the voiceprint embedding vector and the feature vector of the user identity template. The user identity template is a multi-dimensional vector set generated during user registration to represent the user's baseline voiceprint features.

[0036] The aforementioned automatic speech recognition and voiceprint comparison can be performed in parallel without affecting the dependencies.

[0037] Step S103: Compare the automatic speech recognition confidence with a first threshold to perform a first gating, and compare the voiceprint similarity with a second threshold to perform a second gating. Perform quality screening on the current speech sample based on the passing status of the first and second gating.

[0038] The first and second gating mechanisms are used to initially screen the current speech sample based on two dimensions: speech content quality and identity consistency, respectively. When the automatic speech recognition confidence level is greater than or equal to the first threshold, the first gating is considered passed, indicating that the current speech content is clear and highly identifiable; otherwise, it is considered failed. When the voiceprint similarity is greater than or equal to the second threshold, the second gating is considered passed, indicating that the current speech is highly likely to originate from the target user; otherwise, it is considered failed.

[0039] Based on the pass / fail status of these two gating mechanisms, the quality of the current voice sample can be screened: if both the first and second gating mechanisms pass, the current voice sample is considered a high-quality and reliable candidate sample; if either gating mechanism fails, the current voice sample is considered to have quality defects or identity risks and is not suitable for direct template updating. This dual gating mechanism effectively filters out low-quality voice and non-target user voice, preventing abnormal samples from contaminating the subsequent template library.

[0040] Step S104: When both the first and second gating passes, perform state recognition on the preprocessed speech signal to obtain a state label and state confidence, and compare the state confidence with a third threshold to execute the third gating.

[0041] State recognition is used to further analyze the user's physiological or psychological state at the moment of pronunciation, and the state label is a classification description of the state. Specifically, the state feature vector obtained in step S102 is input into the state recognition classifier, which includes a shared encoding layer (fully connected layer, batch normalization layer, and dropout layer) and two independent output heads: an emotion output head and a health output head. After Softmax normalization, the outputs the probability distributions of the emotion class and the health class, respectively, thereby obtaining the state labels (such as emotion category and health category). On this basis, the state labels can be further temporally smoothed by sliding window mode voting to reduce single-frame recognition jumps.

[0042] State confidence is used to characterize the reliability of state recognition results. Its value can be, for example, the probability corresponding to the maximum probability value in the Softmax probability distribution output by the model.

[0043] The third threshold is a threshold value used to measure the reliability of state recognition. Provided that both the first and second gating mechanisms pass, the system performs state recognition and compares the obtained state confidence score with the third threshold: if the state confidence score is greater than or equal to the third threshold, the third gating is considered passed, indicating that the state recognition result is reliable; if the state confidence score is less than the third threshold, the third gating is considered failed, indicating that the state is ambiguous or the recognition is uncertain. By introducing state recognition and the third gating, the template update process can be finely controlled using user state information, avoiding erroneous template updates when the state is uncertain.

[0044] Step S105: Generate a hierarchical update decision based on the passing combination of the first gate, the second gate, and the third gate. The hierarchical update decision shall at least satisfy the following conditions: when the first gate or the second gate fails, generate a decision to refuse to update; when both the first and second gates pass and the third gate fails, generate a decision to update only the identity template; when all three gates pass, generate a decision to update both the identity template and the state sub-template simultaneously.

[0045] The hierarchical update decision refers to a control command generated based on the logical combination of the judgment results of the aforementioned three gating mechanisms. The trigger condition for the update rejection decision is strictly limited to the failure of either the first or second gating mechanism. This implies that the current voice sample either has substandard content quality or insufficient identity consistency, posing a potential security risk. The "update only identity template" decision refers to an instruction that updates only the user's basic identity template without involving specific state sub-templates. Its trigger condition is that both the first and second gating mechanisms pass, but the third gating mechanism fails. This indicates that although the voice quality is good and the identity is confirmed, the user's state recognition is unreliable, therefore only general identity features are fine-tuned. The "update both identity template and state sub-template simultaneously" decision refers to an instruction that updates both the identity template and the corresponding state sub-templates. Its trigger condition is that the first, second, and third gating mechanisms all pass. This indicates that the current voice sample has high quality, confirmed identity, and a clear state, and can be used for a comprehensive model update. Through this hierarchical decision logic, the update strategy can be dynamically selected based on the sample's confidence level, maximizing the utilization of effective data while ensuring system security.

[0046] In a specific numerical example, if the automatic speech recognition confidence is 0.86, the voiceprint similarity is 0.80, the first threshold is 0.75, and the second threshold is 0.75, then the first gate 202 and the second gate 203 pass. If the state recognition model outputs a health state as a cold, with a state confidence of 0.88 and a third threshold of 0.70, then the third gate 205 passes, generating an update decision that simultaneously updates the identity template and the state sub-template. If the automatic speech recognition confidence and voiceprint similarity of another speech sample both meet the standards, but the highest state confidence is only 0.45, then only the identity template is updated, and the state sub-template is not created or updated. If the automatic speech recognition confidence is 0.50 or the voiceprint similarity is 0.58, which is lower than the corresponding threshold, then the update rejection path is entered.

[0047] Step S106: Perform template update according to the hierarchical update decision. When the hierarchical update decision is to update only the identity template, the identity template is updated using the first update rate. When the hierarchical update decision is to update both the identity template and the status sub-template, the identity template is updated using the first update rate, and the status sub-template corresponding to the status label is updated using the second update rate. The second update rate is greater than the first update rate.

[0048] The template update process integrates current speech features into an existing template library. Its core lies in controlling the rate of feature integration, i.e., the update rate. An identity template is a set of model parameters or feature vectors representing a user's long-term stable voiceprint features. The first update rate controls the update step size of the identity template; its value is usually set relatively small to ensure that the identity template does not drift drastically due to fluctuations in individual samples, thus maintaining long-term stability. A state sub-template is a dedicated feature model built for specific state labels (such as a cold or anger) to adapt to short-term changes in the user's state. The second update rate controls the update step size of the state sub-template; its value is configured to be greater than the first update rate. For example, when the first update rate is 0.05, the second update rate can be set to 0.2, 0.3, or 0.5, etc.

[0049] When making the decision to update only the identity template, the system uses only the first update rate to smoothly update the identity template. When making the decision to update both the identity template and the state sub-template simultaneously, the system uses the first update rate to slowly update the identity template, while simultaneously using a larger second update rate to quickly update the state sub-template corresponding to the current state label. This allows the state sub-template to quickly adapt to the user's current state characteristics. By setting differentiated update rates, a balance is achieved between the stability of the long-term identity template and the sensitivity of the short-term state sub-template, effectively improving the verification performance of the voiceprint recognition system under different states.

[0050] In an optional embodiment, the first threshold and / or the second threshold in step S103 are dynamic thresholds, determined based on at least one of the current ambient noise level, the user's recent voiceprint matching history, the stability of automatic speech recognition, and the security level; specifically, the first threshold is increased when the noise level increases, the second threshold is decreased when the user's recent matching is stable and the ambient noise is lower than a preset noise threshold, and the first threshold and / or the second threshold is increased when the security level increases.

[0051] The parameters are defined as follows: This represents the index of the current speech sample, which is a non-negative integer; , These represent the current first threshold and the current second threshold, respectively. , These represent the initial first threshold and the initial second threshold, respectively. , These represent the current environmental noise level and the noise reference level, respectively, both normalized to [0,1]. This represents the recent stability of the user's voiceprint matching, normalized to [0,1]. This represents the current security level, normalized to [0,1]. , , , These are dynamic threshold adjustment coefficients, all of which are non-negative numbers; , These represent the lower and upper bounds of the first threshold clip, respectively, with values ​​ranging from [0,1]. , These represent the lower and upper bounds of the second threshold clip, respectively, with values ​​ranging from [0,1].

[0052] When the first and second thresholds are dynamic thresholds, they can be calculated using the following formula:

[0053] (1)

[0054] (2)

[0055] in, Indicates will Limited to the range Inside. If dynamic thresholds are not enabled, then let , .

[0056] In an optional embodiment, the status label in step S104 is used to characterize the user's state when the voice is collected. The status label includes at least one of emotion-related status labels and health-related status labels. Specifically, the emotion-related status labels include at least one of calm, happy, sad, and angry, while the health-related status labels include at least one of healthy, cold, fatigued, and allergic. Since the user's voice characteristics may change under different emotional or health states, this embodiment uses status labels to distinguish the voiceprint characteristics of the same user under different states.

[0057] Furthermore, such as Figure 2 The diagram shown illustrates the index structure of the identity template and state sub-template provided in this embodiment. The user identifier distinguishes different users, the state type distinguishes between emotional and health states, and the state label indicates the specific state under the corresponding state type. The identity template is indexed based on the user identifier; the state sub-template is indexed using the user identifier, state type, and state label, with the third-level index pointing to the corresponding state sub-template.

[0058] In a specific embodiment, for example, for user U1's health-related cold state, its corresponding state sub-template can be located through the index path "U1-health-cold"; for user U1's emotion-related happy state, its corresponding state sub-template can be located through the index path "U1-emotion-happy".

[0059] During the template update process, the system determines the target index path based on the user identifier, state type, and state label corresponding to the current speech sample, and checks whether a corresponding state sub-template already exists under that target index path. If the corresponding state sub-template does not exist, and the third gating passes, the state sub-template is initialized with the speaker embedding vector corresponding to the current speech sample. Specifically, the current speaker embedding vector can be used as the initial template vector of the state sub-template, or as the initial feature record of the state sub-template, to form a new state sub-template corresponding to the user identifier, state type, and state label. If the corresponding state sub-template already exists, it can be updated according to the aforementioned hierarchical update rules.

[0060] Through the aforementioned three-level index structure, this embodiment can manage voiceprint sub-templates under different users, different state types, and different state labels in a hierarchical manner, enabling the system to quickly locate the target state sub-template that needs to be queried or updated. Simultaneously, by initializing a new state sub-template only when the corresponding state sub-template does not exist and the third gating passes, the risk of incorrectly creating state sub-templates from low-quality or abnormal samples can be reduced, thereby improving the management accuracy and recognition robustness of voiceprint templates in state-changing scenarios.

[0061] In an optional embodiment, the generation of the hierarchical update decision in step S105 is performed sequentially according to the priority order of the first gating, the second gating, and the third gating. When the first gating fails or the second gating fails, a rejection update decision is generated, and the state recognition is skipped or the subsequent gating judgment is terminated, thereby avoiding redundant state recognition calculations for invalid samples.

[0062] In an optional embodiment, different update rates are applied to the identity template and the state sub-template in step S106.

[0063] For identity templates, the first update rate decreases as the cumulative number of updates to the identity template increases, but is not lower than a preset minimum update rate. For example, the first update rate is determined by the following formula:

[0064] (3)

[0065] Where n represents the cumulative number of times the identity template has been updated; , , Let represent the initial first update rate, the minimum first update rate, and the basic first update rate, respectively. , ; Indicates the attenuation coefficient. .

[0066] For a state sub-template, the second update rate switches based on the cumulative number of updates to the state sub-template. When the cumulative number of updates to the state sub-template is less than a preset switching threshold, the normal second update rate is used; when the cumulative number of updates to the state sub-template reaches the switching threshold, it switches to a slow second update rate. For example, the second update rate is determined by the following formula:

[0067] (4)

[0068] Where m represents the cumulative number of updates to the state sub-template; , , These represent the normal second update rate, the slow second update rate, and the actual second update rate, respectively. , ; This represents the update rate switching threshold, which is a positive integer. It is an indicator function that takes the value 1 if the condition is true, and 0 otherwise.

[0069] The above three update rate settings must meet the following requirements: > > That is, the normal second update rate is greater than the slow second update rate, and the slow second update rate is greater than the first update rate.

[0070] This method achieves coordinated control of the long-term stability and short-term state adaptability of the voiceprint model by distinguishing the update strategies of identity templates and state sub-templates.

[0071] In one optional implementation, the method provided in this application embodiment further includes: determining template contamination risk based on the state switching index and identity template change index within the most recent window; when the template contamination risk meets the preset protection conditions, triggering template contamination protection and reducing the first update rate used to update the identity template.

[0072] Furthermore, state transition metrics include at least one of the following: the number of state label changes, the number of different state labels, or the state transition frequency. Specifically, the number of state label changes refers to the cumulative frequency of state label changes within the most recent time window or a fixed number of recent voice interaction sequences. The number of different state labels refers to the count of distinct state label categories appearing within that window, reflecting the dispersion of the state distribution. The state transition frequency can refer to the number of state label changes per unit time, used to capture the rate of state change.

[0073] Furthermore, the identity template change index includes at least one of the following: the normalized vector change magnitude before and after the identity template update, cosine distance, or template center drift. The normalized vector change magnitude can refer to the geometric distance or the magnitude of the difference vector between the vectors before and after the identity template update, mapped onto a unit sphere. This index directly reflects the degree of template displacement in the feature space. Cosine distance is calculated by converting the cosine of the angle between the vectors before and after the update into a distance metric. It focuses on the deviation in the representation direction rather than the absolute distance and is insensitive to changes in the magnitude of the feature vector, making it suitable for capturing changes in the distribution pattern of voiceprint features. Template center drift can refer to the Euclidean or Mahalanobis distance between the new template center point and the original center point after adding the current sample vector to the template library. This measures the pulling effect of the sample on the overall cluster center.

[0074] When the state switching indicator reaches the state switching threshold and the identity template change indicator reaches the template change threshold, the conservative update mode is triggered, and the first update rate is reduced to a preset ratio of the base first update rate; when the number of consecutive stable windows or the number of consecutive stable times reaches the recovery threshold, the conservative update mode is exited.

[0075] Template contamination protection trigger conditions can be expressed as:

[0076] (5)

[0077] in, This indicates a template contamination protection flag; it is set to 1 when triggered and 0 when not triggered. , These represent the number of different status labels and the threshold for status switching types within the most recent window, respectively. , , Let these represent the initial identity template vector, the identity template vector after the nth update, and the identity template vector after the (n+1)th update, respectively. Indicates the threshold for the proportion of template mutations. ; This represents the L2 norm of a vector.

[0078] For example, when template contamination protection is triggered (i.e. The actual first update rate is reduced to the preset percentage of the base first update rate:

[0079] (6)

[0080] in, To reduce pollution factors, ∈ (0,1); , These represent the basic first update rate and the first update rate after pollution protection adjustments, respectively.

[0081] For example, the identity template and the state sub-template can be updated and normalized respectively using the following formula:

[0082] (7)

[0083] (8)

[0084] in, This represents the speaker embedding vector of the current speech sample; Represents the state sub-template vector after the m-th update of state k; ε is the numerical stability term, taking a positive value (e.g., 10). -12 Up to 10 -6 ), used to avoid the normalization denominator being zero.

[0085] In a specific embodiment, if the state labels of the most recent 6 high-confidence samples are calm, cold, fatigued, angry, cold and calm in sequence, then the number of different state labels is 4. If the state switching type threshold is 3, and the vector change of the identity template relative to the identity template before the update reaches 0.12 times the norm of the initial identity template, and the template mutation ratio threshold is 0.10, then equation (5) holds, triggering the conservative update mode. If the basic first update rate is 0.12 and the pollution protection reduction factor is 1 / 3, then the actual first update rate becomes 0.04. If the subsequent 4 consecutive high-confidence samples do not experience frequent state switching or identity template mutation, then the conservative update mode is exited.

[0086] In one optional implementation, the method provided in this application further includes: during the voiceprint verification stage, dynamically fusing identity template similarity and state sub-template similarity based on the current state label, state confidence, and the timeliness of the corresponding state sub-template to obtain the final verification score. Specifically, when the state sub-template corresponding to the current state label exists and the state confidence reaches a preset threshold, the state sub-template similarity is fused with a state weight. The state weight is positively correlated with the state confidence and negatively correlated with the age of the state sub-template. When the corresponding state sub-template does not exist or the state confidence does not reach the preset threshold, only the identity template similarity is used as the final verification score. The age of the state sub-template is the time interval or update cycle number between the current verification time and the most recent update time of the corresponding state sub-template.

[0087] For example, the state template weights and the final validation score can be calculated using the following formula:

[0088] (9)

[0089] (10)

[0090] in, , These represent the minimum weight of the state template and the weight of the current state template, respectively, with values ​​ranging from [0,1]. The flag indicates whether the state sub-template is available; it is 1 if available and 0 if unavailable. Indicates the age decay coefficient; Indicates the age of the current state sub-template; This represents the confidence level of the current sample's state. ; Indicates the third threshold; This indicates the current final verification score; This represents the speaker embedding vector of the current speech sample; Let $k$ represent the state sub-template vector after the $m$-th update of state $k$.

[0091] When the corresponding state sub-template exists and hour ,otherwise .when hour, Equation (10) degenerates into voiceprint verification based solely on identity templates.

[0092] In a verification numerical example, the third threshold is 0.70, the minimum weight of the state template is 0.35, and the age decay coefficient of the state template is 0.03. If the confidence of the current state is 0.86, the age of the cold state sub-template is 2 time units, the similarity between the current voiceprint embedding vector and the identity template is 0.68, and the similarity with the cold state sub-template is 0.84. Substituting into equation (10), the final verification score is 0.7882, which is higher than the similarity of 0.68 when using the identity template alone. This indicates that in the cold state, the weighted fusion of the state sub-templates improves the final verification score by about 15.9%, thereby effectively reducing the false rejection rate.

[0093] Further reference Figure 3This document illustrates a voiceprint template hierarchical update control system provided in an embodiment of this application. The system includes: a voice acquisition and preprocessing module 301, used to acquire user-input voice signals and preprocess the voice signals to obtain preprocessed voice signals; a shared feature extraction module 302, used to extract voiceprint embedding vectors and state feature vectors from the preprocessed voice signals; an automatic speech recognition module 303, used to perform automatic speech recognition on the preprocessed voice signals to obtain automatic speech recognition confidence; a voiceprint comparison module 304, used to compare the voiceprint embedding vectors with user identity templates to obtain voiceprint similarity; and a state recognition module 305, used to perform state recognition on the preprocessed voice signals to obtain state when both the first and second gates are passed. The system includes: a state label and state confidence level; a multi-level gating coordination module 306, which executes the first, second, and third gating controls and generates hierarchical update decisions based on the gating controls; a template storage and update module 307, which maintains user identity templates and state sub-templates and executes template updates based on hierarchical update decisions; a template contamination prevention module 308, which reads the state label sequence and the vector before and after the update of the user identity template within the most recent window, and determines whether to enter the conservative update mode based on the state switching index and the identity template change index; and a state awareness verification module 309, which, during the voiceprint verification stage, dynamically fuses the identity template similarity and the state sub-template similarity based on the current state label, state confidence level, and the timeliness of the corresponding state sub-template to obtain the final verification score.

[0094] Each module in the above system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0095] In an optional embodiment, this application also provides an electronic device, including a processor and a memory. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the methods of any of the above embodiments. Specifically, when the processor executes the computer program, it implements at least some steps of speech signal acquisition, voiceprint embedding vector extraction, automatic speech recognition, state recognition, multi-level gating judgment, hierarchical update decision generation, and voiceprint template updating, thereby enabling the electronic device to perform hierarchical update control of the voiceprint template based on the recognition results and gating judgment results of the speech samples. The electronic device can be a terminal device, an edge computing device, or a server.

[0096] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for hierarchical update control of voiceprint templates, characterized in that, include: S1, acquire the user-input voice signal, and preprocess the voice signal to obtain a preprocessed voice signal; S2, extract the voiceprint embedding vector and state feature vector from the preprocessed speech signal, perform automatic speech recognition on the preprocessed speech signal to obtain automatic speech recognition confidence, and compare the voiceprint embedding vector with the pre-stored user identity template to obtain voiceprint similarity; S3, compare the automatic speech recognition confidence with a first threshold to perform a first gating, and compare the voiceprint similarity with a second threshold to perform a second gating, and perform quality screening on the current speech sample based on the passing status of the first gating and the second gating; S4, when both the first gate and the second gate pass, perform state recognition on the preprocessed voice signal to obtain a state label and a state confidence level, and compare the state confidence level with a third threshold to execute the third gate; S5, generate a hierarchical update decision based on the passing combination of the first gating, the second gating, and the third gating. The hierarchical update decision at least satisfies the following conditions: when the first gating fails or the second gating fails, generate a rejection update decision; when both the first and second gatings pass and the third gating fails, generate a decision to update only the identity template; when all three gatings pass, generate a decision to update both the identity template and the state sub-template simultaneously. S6, perform template update according to the hierarchical update decision. When the hierarchical update decision is to update only the identity template, update the identity template with a first update rate. When the hierarchical update decision is to update both the identity template and the status sub-template, update the identity template with the first update rate and update the status sub-template corresponding to the status label with a second update rate, wherein the second update rate is greater than the first update rate.

2. The method according to claim 1, characterized in that, The first threshold and / or the second threshold in step S3 are dynamic thresholds, which are determined based on at least one of the current ambient noise level, the user's recent voiceprint matching history, the stability of automatic speech recognition, and the security level. The first threshold is increased when the noise level increases, the second threshold is decreased when the user's recent matching is stable and the ambient noise is lower than the noise threshold, and the first threshold and / or the second threshold is increased when the security level increases.

3. The method according to claim 1, characterized in that, The status labels in step S4 include at least one of emotion status labels and health status labels; the emotion status labels include at least one of calm, happy, sad and angry, and the health status labels include at least one of healthy, cold, fatigue and allergies; the status sub-template establishes a three-level index according to user identifier, status type and status label, and initializes the corresponding status sub-template with the current voiceprint embedding vector when the corresponding status sub-template does not exist and the third gating passes.

4. The method according to claim 1, characterized in that, In step S5, the generation of the hierarchical update decision is judged sequentially according to the priority order of the first gating, the second gating, and the third gating; when the first gating fails or the second gating fails, the update rejection decision is generated, and the state recognition is skipped or the subsequent gating judgment is terminated.

5. The method according to claim 1, characterized in that, In step S6, the first update rate decreases as the cumulative number of updates to the identity template increases and is not lower than a preset minimum update rate; the second update rate adopts a normal second update rate when the cumulative number of updates to the state sub-template is less than a preset switching threshold, and switches to a slow second update rate after reaching the switching threshold; the normal second update rate is greater than the first update rate, and the slow second update rate is less than the normal second update rate but greater than the first update rate.

6. The method according to claim 1, characterized in that, Also includes: Template contamination risk is determined based on the state switching index and identity template change index within the most recent window. When the template contamination risk meets the preset protection conditions, template contamination protection is triggered, and the first update rate used to update the identity template is reduced.

7. The method according to claim 6, characterized in that, The state switching indicator includes at least one of the following: the number of state label switching times, the number of different state labels, or the state switching frequency. The identity template change indicator includes at least one of the following: the normalized vector change magnitude before and after the identity template update, the cosine distance, or the template center drift. When the state switching indicator reaches the state switching threshold and the identity template change indicator reaches the template change threshold, a conservative update mode is triggered. In the conservative update mode, the first update rate is reduced to a preset proportion of the base first update rate. When the number of consecutive stable windows or the number of consecutive stable times reaches the recovery threshold, the conservative update mode is exited.

8. The method according to claim 1, characterized in that, Also includes: During the voiceprint verification stage, the identity template similarity and the state sub-template similarity are dynamically fused based on the current state label, state confidence, and the timeliness of the corresponding state sub-template to obtain the final verification score. Specifically, when the state sub-template corresponding to the current state label exists and the state confidence reaches a preset threshold, the state sub-template similarity is fused with a state weight. The state weight is positively correlated with the state confidence and negatively correlated with the age of the state sub-template. When the corresponding state sub-template does not exist or the state confidence does not reach the preset threshold, only the identity template similarity is used as the final verification score; the state sub-template age is the time interval or number of update cycles between the current verification time and the most recent update time of the corresponding state sub-template.

9. A voiceprint template hierarchical update control system, characterized in that, The system is configured to perform the method of any one of claims 1 to 8, the system comprising: The voice acquisition and preprocessing module is used to acquire the voice signal input by the user and preprocess the voice signal to obtain the preprocessed voice signal; A shared feature extraction module is used to extract speaker embedding vectors and state feature vectors from the preprocessed speech signal; An automatic speech recognition module is used to perform automatic speech recognition on the preprocessed speech signal to obtain automatic speech recognition confidence level; The voiceprint comparison module is used to compare the voiceprint embedding vector with the user identity template to obtain voiceprint similarity. The state recognition module is used to perform state recognition on the preprocessed voice signal to obtain state labels and state confidence when both the first gate and the second gate are passed. The multi-level gating coordination module is used to execute the first gating, the second gating, and the third gating, and to generate hierarchical update decisions by combining the gatings. The template storage and update module is used to maintain the user identity template and status sub-template, and to perform template updates according to the hierarchical update decision; The template contamination prevention module is used to read the status label sequence in the most recent window and the vector before and after the update of the user identity template, and determine whether to enter the conservative update mode based on the status switching index and the identity template change index. The state-aware verification module is used in the voiceprint verification stage to dynamically fuse identity template similarity and state sub-template similarity based on the current state label, state confidence, and the timeliness of the corresponding state sub-template to obtain the final verification score.

10. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 8.

Images