Voiceprint authentication method and device based on multi-node parameter aggregation, equipment and medium

Abstract

The application relates to the technical field of voiceprint recognition and discloses a voiceprint authentication method and device based on multi-node parameter aggregation, equipment and a medium, which comprises the following steps: performing array noise reduction on a multi-channel audio signal, extracting and fusing voiceprint features, physiological acoustic features and content features; performing living body analysis based on random interactive reply audio; performing fraud early warning based on content features and emotional outliers; performing identity recognition by using a voiceprint recognition model obtained through joint training, and determining a voiceprint authentication result in combination with a living body determination result and a fraud early warning determination result. The application can be applied to business scenes such as financial technology and medical health, comprehensive authentication bases are formed through array noise reduction, multi-dimensional feature fusion, living body analysis and fraud early warning, and identity recognition is performed by using a voiceprint recognition model obtained through joint training, so that the stability and reliability of voiceprint authentication in a complex voice interaction scene are improved.

Description

Technical Field

[0001] This invention relates to the field of voiceprint recognition technology, and in particular to a voiceprint authentication method, apparatus, device, and medium based on multi-node parameter aggregation. Background Technology

[0002] Existing voiceprint authentication technologies typically determine identity based on feature matching between collected speech and pre-stored voiceprint data. In complex voice interaction scenarios, environmental noise, changes in collection conditions, variations in individual vocalization states, and non-real-time voice input can all affect the stability of voiceprint features and matching results. Current technologies often rely primarily on voiceprint comparisons from single speech samples, and their comprehensive analytical capabilities remain limited when the voice environment and user state are constantly changing.

[0003] In the fintech sector, services such as insurance claims, claims processing, remote identity verification, account transactions, and credit reviews may involve collecting user voice data via mobile devices. Users' environments and vocal states vary significantly, and issues such as recording playback, identity theft, and unusual speech may arise during these processes. Current technologies primarily rely on voiceprint database comparisons for identity verification, lacking a unified analysis of environmental changes, real-time vocal states, and anomalies in the voice content, making it difficult to meet the voice authentication requirements of complex financial business scenarios.

[0004] In the healthcare sector, services such as remote consultations, online follow-up visits, health consultations, and medical service identity verification also rely on voice interaction. Patients' acoustic features may shift due to changes in age, colds, vocal cord condition, or respiratory status. Background noise in home, hospital ward, and public environments can also affect voice feature extraction. Existing processing methods based on comparing fixed voiceprint features have limited compatibility with different acquisition environments and individual changes, and there is still room for improvement in continuously evolving voice interaction scenarios. Summary of the Invention

[0005] The main objective of this invention is to provide a voiceprint authentication method, apparatus, device, and storage medium based on multi-node parameter aggregation, aiming to solve the technical problem that existing technologies mainly rely on fixed voiceprint features for single comparison, making it difficult to form stable and reliable voiceprint authentication results in complex voice interaction scenarios.

[0006] To achieve the above objectives, the present invention provides a voiceprint authentication method based on multi-node parameter aggregation, comprising: Acquire multi-channel audio signals containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and process the multi-channel audio signals using array noise reduction to obtain noise-reduced audio signals; The voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal are extracted, and the voiceprint features, physiological acoustic features, and content features are fused to obtain the fused features; A random interaction command is generated, the response audio signal of the object to be verified in response to the random interaction command is obtained, and a liveness analysis is performed based on the response audio signal and the reference dynamic characteristic data to obtain a liveness determination result; Text risk features are generated based on the content features, and emotional anomaly values ​​are generated based on the noise-reduced frequency signal. Fraud warning judgment results are determined based on the text risk features and the emotional anomaly values. The fused features are input into a voiceprint recognition model jointly trained using a multi-node parameter aggregation method to obtain the identity recognition result. The voiceprint authentication result is determined based on the identity recognition result, the liveness detection result, and the fraud warning result.

[0007] Furthermore, to achieve the above objectives, the present invention provides a voiceprint authentication device based on multi-node parameter aggregation, comprising: An array noise reduction module is used to acquire a multi-channel audio signal containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and to process the multi-channel audio signal using an array noise reduction method to obtain a noise-reduced audio signal. A multi-dimensional feature fusion module is used to extract the voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal, and to fuse the voiceprint features, physiological acoustic features, and content features to obtain fused features; The liveness analysis module is used to generate random interaction commands, obtain the response audio signal of the object to be verified in response to the random interaction commands, and perform liveness analysis based on the response audio signal and the benchmark dynamic characteristic data to obtain a liveness determination result. The fraud warning module is used to generate text risk features based on the content features, generate emotional anomaly values ​​based on the noise-reduced frequency signal, and determine the fraud warning judgment result based on the text risk features and the emotional anomaly values. The voiceprint recognition module is used to input the fused features into the voiceprint recognition model jointly trained by a multi-node parameter aggregation method to obtain the identity recognition result; The authentication decision module is used to determine the voiceprint authentication result based on the identity recognition result, the liveness determination result, and the fraud warning determination result.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a voiceprint authentication program based on multi-node parameter aggregation stored in the memory and executable on the processor, wherein when the voiceprint authentication program based on multi-node parameter aggregation is executed by the processor, it implements the steps of the voiceprint authentication method based on multi-node parameter aggregation as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a voiceprint authentication program based on multi-node parameter aggregation, wherein when the voiceprint authentication program based on multi-node parameter aggregation is executed by a processor, it implements the steps of the voiceprint authentication method based on multi-node parameter aggregation as described above.

[0010] Beneficial Effects: This invention relates to the field of voiceprint recognition technology, and discloses a voiceprint authentication method, apparatus, device, and medium based on multi-node parameter aggregation. The method includes: array denoising of a multi-channel audio signal containing the speech of the person to be verified; extracting and fusing voiceprint features, physiological acoustic features, and content features; performing liveness analysis based on random interactive response audio and baseline dynamic feature data; providing fraud warning based on content features and emotional anomalies; inputting the fused features into a voiceprint recognition model jointly trained via multi-node parameter aggregation to obtain an identity recognition result; and combining the liveness determination result and the fraud warning result to determine the voiceprint authentication result. This invention can be applied to business scenarios such as fintech and healthcare. By forming a multi-dimensional authentication basis through array denoising, multi-dimensional feature fusion, random interactive liveness analysis, and fraud warning, and combining it with a jointly trained voiceprint recognition model for identity recognition, this invention reduces the impact of environmental changes, changes in vocal state, and non-real-time voice input on the authentication result, improving the stability and reliability of voiceprint authentication in complex voice interaction scenarios. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for a voiceprint authentication method based on multi-node parameter aggregation in one embodiment of the present invention. Figure 2 This is a flowchart illustrating an embodiment of the voiceprint authentication method based on multi-node parameter aggregation according to the present invention. Figure 3 for Figure 2 A detailed flowchart of step S10; Figure 4 for Figure 2 A detailed flowchart of step S20; Figure 5 for Figure 2A detailed flowchart of step S30; Figure 6 for Figure 2 A detailed flowchart of step S40; Figure 7 for Figure 2 A detailed flowchart of step S50; Figure 8 for Figure 2 A detailed flowchart of step S60; Figure 9 This is a schematic diagram of the functional modules of a preferred embodiment of the voiceprint authentication device based on multi-node parameter aggregation of the present invention. Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 11 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The voiceprint authentication method based on multi-node parameter aggregation provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can use the client to perform array denoising on multi-channel audio signals containing the voice of the person to be verified, extract and fuse voiceprint features, physiological acoustic features, and content features; perform liveness analysis based on random interactive response audio and baseline dynamic feature data; and perform fraud warnings based on content features and emotional anomalies. The fused features are input into a voiceprint recognition model jointly trained through multi-node parameter aggregation to obtain the identity recognition result, and the voiceprint authentication result is determined by combining the liveness determination result and the fraud warning result. This invention can be applied to business scenarios such as fintech and healthcare. By forming multi-dimensional authentication criteria through array denoising, multi-dimensional feature fusion, random interactive liveness analysis, and fraud warnings, and combining them with a jointly trained voiceprint recognition model for identity recognition, this invention reduces the impact of environmental changes, changes in vocal state, and non-real-time voice input on the authentication result, improving the stability and reliability of voiceprint authentication in complex voice interaction scenarios. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The present invention will now be described in detail through specific embodiments.

[0014] Please see Figure 2 , Figure 2This is a flowchart illustrating an embodiment of the voiceprint authentication method based on multi-node parameter aggregation provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2 As shown, the voiceprint authentication method based on multi-node parameter aggregation proposed in this invention includes the following steps: S10: Acquire a multi-channel audio signal containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and process the multi-channel audio signal using array noise reduction to obtain a noise-reduced audio signal. In this embodiment, the multi-channel audio signal is obtained by synchronously acquiring the speech of the target object from multiple acquisition channels located at different spatial positions. Each acquisition channel retains corresponding time information, so that the differences in amplitude, phase, and arrival time of the same speech component in different channels can be used to distinguish the target sound source from the ambient sound source. The multi-channel audio signal can be acquired by a linear array, a circular array, or a planar array. When there is channel sampling offset, time alignment between channels can be performed based on the time information.

[0016] Baseline dynamic feature data is used to characterize the dynamic distribution of vocalizations in the existing speech of the subject to be verified. It can originate from verified registered speech, historical interactive speech, or multiple speech acquisition results. Baseline dynamic feature data may include fundamental frequency, respiratory rhythm, and their corresponding distribution range, or it may utilize other acoustic data capable of characterizing changes in vocalization cycles and respiration. Data generated over time can be used to generate corresponding baseline dynamic feature data through statistical summarization, time-segmented updates, or sliding updates.

[0017] Array-based noise reduction utilizes the spatial differences between different acquisition channels to suppress sound components from non-target directions. The multi-channel audio signal can be converted to the time-frequency domain. Spatial noise information is generated based on the distribution of environmental noise across the acquisition channels, and the direction of the target sound source is determined based on the arrival differences of the speech to be verified in different channels. Channel combination weights are generated by combining the spatial noise information and the target sound source direction. The multi-channel audio signal is then weighted, synthesized, and reconstructed in the time domain to obtain a noise-reduced frequency signal, thereby reducing interference from background noise, side sources, and spatial reverberation from the speech of the subject to be verified.

[0018] In far-field acquisition scenarios with continuously changing background noise, frequency-domain beamforming is employed to generate noise-reduced audio signals. After time alignment and frequency-domain transformation, the audio data from each acquisition channel are updated based on the spatial distribution of noise in non-speech segments, and the target sound source direction is updated based on the channel arrival differences in speech segments, dynamically adjusting the channel combination weights. For environments with rapidly changing wind noise, vehicle noise, or surrounding human speech, the update cycle for noise statistics and target sound source direction can be shortened.

[0019] When the terminal structure is fixed and the location of the object to be verified changes little, a noise-reduced frequency signal is generated by combining time delay compensation and channel weighting. The time delay of each acquisition channel is corrected according to the microphone array element position and the direction of the target sound source, and then the combination weights are assigned according to the signal-to-noise ratio of each channel. When indoor reverberation is significant, frequency domain noise suppression can also be combined to reduce continuous background noise.

[0020] Baseline dynamic characteristic data can be summarized according to a fixed time range or updated based on recently confirmed voice recordings. When the vocal state of the subject to be verified is constantly changing, the proportion of recent voice data can be increased; when historical data is scarce, it can be supplemented and updated after new confirmed voice recordings are added.

[0021] In the fintech business, users can file insurance claims or remotely verify their identity via mobile devices, collecting ambient sounds such as vehicle noise, wind noise, or the voices of people in the surrounding environment. Multiple microphones simultaneously capture the user's voice, determining the direction of the user's voice based on the spatial differences between the channels and suppressing sound components from other directions to generate a noise-reduced frequency signal. At the same time, the baseline dynamic characteristic data of existing confirmation voices are read.

[0022] In the healthcare sector, users conduct remote health consultations or online follow-up visits via home smart terminals, where there may be television noise, equipment operation noise, or indoor reverberation. After recording user voice through multiple acquisition channels, array noise reduction is performed based on the direction of the sound source and the distribution of ambient noise to obtain a noise-reduced frequency signal; the baseline dynamic characteristic data can be formed from previously confirmed remote voice interaction data and updated according to long-term changes in vocalization status.

[0023] This embodiment acquires spatial difference information of the speech of the object to be verified through multi-channel acquisition, and uses array noise reduction to generate a noise-reduced frequency signal by combining the direction of the target sound source and the distribution of environmental noise. This can reduce the interference of complex acoustic environment on speech acquisition results. At the same time, it acquires the corresponding reference dynamic feature data to keep the current speech corresponding to the existing vocal dynamic data, thereby improving the data stability in complex speech interaction environment.

[0024] S20, extract the voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal, and fuse the voiceprint features, physiological acoustic features, and content features to obtain fused features; In this embodiment, the noise-reduced audio signal retains the main speech components of the object to be verified. Voiceprint features are used to characterize individual vocal differences and can be extracted from spectral envelope, cepstral distribution, or acoustic coding results. Physiological acoustic features are used to characterize vocal cord vibration and respiratory rhythm, and can include fundamental frequency, fundamental period, and breathing intervals, obtained through the periodicity of the speech waveform, harmonic structure, and changes in energy envelope. Content features are used to characterize semantic information; these can be achieved by transing the noise-reduced audio signal into text and then performing contextual encoding, or by directly extracting semantic representations from the speech coding results.

[0025] Voiceprint features, physiological acoustic features, and content features can be formed into a unified representation through dimensional mapping or feature projection, and then fused using dynamic weights, attention weights, gating coefficients, or feature splicing to obtain fused features that simultaneously contain individual acoustic information, vocal physiological information, and semantic information.

[0026] In situations where speech is short and processing resources are limited, cepstral representation is used to extract voiceprint features, physiological acoustic features are formed based on periodicity and energy changes, and semantic encoding is performed on the transcribed text. After dimensional mapping of the three types of features, they are weighted and fused.

[0027] In cases where the speech is long and the content is rich, an acoustic coding network is used to generate voiceprint features, physiological acoustic features are formed based on time-frequency changes, content features are generated through a contextual language coding network, and dynamic fusion weights are generated through interactive attention.

[0028] In the fintech business, when users submit insurance claims or conduct remote identity verification, three types of features can be extracted from the noise-reduced audio signal and fused to form a multi-dimensional voice representation.

[0029] In the field of healthcare, when users conduct remote health consultations or online follow-up visits, for some acoustic parameter shifts caused by colds, vocal cord conditions, or age changes, information on individual vocalization, physiological acoustics, and consultation content can be preserved through the fusion of three types of features.

[0030] This embodiment integrates voiceprint features, physiological acoustic features, and content features, so that the integrated features simultaneously include individual acoustic differences, physiological information of vocalization, and semantic information. This reduces the degree to which a single voiceprint representation is affected by changes in vocalization state and improves the integrity and stability of speech feature representation.

[0031] S30, generate a random interaction command, obtain the response audio signal of the object to be verified in response to the random interaction command, and perform a liveness analysis based on the response audio signal and the reference dynamic characteristic data to obtain a liveness determination result; In this embodiment, random interaction instructions are used to temporarily generate the content that the object to be verified needs to respond to during the current interaction. These instructions can be randomly selected from a preset interaction content library, or generated by combining time information, random characters, or dynamic semantic content, thus changing the response requirements for different interaction processes. Random interaction instructions can require the object to be verified to read specified content, answer current dynamic questions, or complete predetermined vocal tasks, and can be output through voice, text, or interface information.

[0032] The response audio signal is collected in real time by the subject to be verified in response to random interactive commands. The response audio signal may contain information such as response semantics, pitch changes, vocal rhythm, and breathing intervals. During the acquisition process, framing, endpoint detection, and effective speech segment extraction can be performed to reduce the impact of silent segments and irrelevant sounds on liveness analysis.

[0033] The baseline dynamic characteristic data describes the dynamic distribution of the vocalization process of the object to be verified, and may include fundamental frequency distribution, fundamental tone variation range, respiratory interval distribution, or vocal rhythm data. Liveness analysis extracts corresponding real-time dynamic information from the response audio signal and performs a difference analysis with the baseline dynamic characteristic data, while simultaneously determining the response relationship between the response content and random interactive commands. Based on the degree of response matching and dynamic differences, a liveness determination result is generated to distinguish between real-time vocalization that meets the current interaction requirements and pre-recorded or abnormally generated speech.

[0034] In situations requiring increased randomness, dynamic content generation is employed to create random interactive commands. Response requests are generated based on the current time, random characters, or temporary semantic content. Corresponding audio signals are then acquired, and liveness analysis is performed from both the response content and vocal dynamics dimensions. For interactions with short response times, the analysis proportion of pitch variation and vocal rhythm can be increased.

[0035] In situations where interaction duration needs to be controlled, instruction content is randomly selected from a pre-defined interaction content library. Instruction content is grouped according to different lengths and pronunciation structures, and the subject to be verified is required to respond immediately after selection. The response audio signal, after effective speech extraction, is compared with baseline dynamic characteristic data for difference analysis, and a liveness determination result is generated based on whether the response content meets the requirements of the current instruction.

[0036] In the fintech business, when users conduct remote identity verification or report insurance claims, the interactive interface temporarily generates random interactive commands corresponding to the current date, dynamic characters, or specified phrases, collects the user's real-time response to form a response audio signal, and then combines it with existing benchmark dynamic feature data to determine whether the current voice conforms to real-time speech characteristics.

[0037] In the healthcare sector, when users conduct remote health consultations or online follow-up visits, the terminal generates temporary voice response requests and collects the user's response to form a reply audio signal. To address the possibility of changes in the user's vocalization, in vivo analysis can be performed by combining the range of pitch variation and respiratory rhythm distribution, allowing for certain physiological fluctuations while determining whether the current response originates from real-time interaction.

[0038] This embodiment generates a response requirement specific to the current interaction through random interactive commands, and performs liveness analysis by combining the response audio signal with the reference dynamic feature data. This allows the liveness determination to utilize both the instantaneous response relationship and the dynamic difference in vocalization, thereby reducing the possibility that pre-recorded or abnormally generated voice will meet the current interaction requirement and improving the reliability of voice liveness determination.

[0039] S40, generate text risk features based on the content features, and generate emotional anomaly values ​​based on the noise-reduced frequency signal, and determine the fraud warning judgment result based on the text risk features and the emotional anomaly values; In this embodiment, content features include terms, phrases, semantic relationships, and contextual information within the speech content. Text risk features can be generated based on the intensity of the occurrence of risk semantic units in the content features, the degree of contextual relevance, and their distribution in the corpus data. These text risk features reflect the proximity between the current speech content and the preset risk semantics. Risk semantic units can be represented by terms, phrases, or semantic categories, and text risk features can be represented by a single risk value or a multi-dimensional risk vector.

[0040] Emotional anomalies are obtained from prosody and acoustic variations in noise-reduced audio signals. They can be used to analyze pitch variations, energy fluctuations, speech rate, pause distribution, or vocal stability, and compared with preset emotion distributions or the range of variation within the current speech. Multiple acoustic indicators can be normalized and weighted to form emotional anomalies, reflecting the degree to which the current vocal state deviates from the reference state.

[0041] Textual risk features reflect the semantic content dimension, while sentiment outliers reflect the acoustic performance dimension. These two types of data can be fused according to preset weights, or combined based on their respective judgment intervals, and then the fraud warning result can be determined accordingly.

[0042] In cases where the audio content is long and the keywords are scattered, a joint analysis of term frequency and corpus distribution is used to generate text risk features, and the participation weights are adjusted according to the discriminative power of different risk semantic units. Emotional anomalies can be statistically analyzed based on the pitch, energy, and speech rate changes of multiple audio segments, thus reducing the impact of short-term acoustic fluctuations on the results.

[0043] In cases where the speech content is short and the emotional changes are significant, contextual semantic matching is used to generate text risk features, and the participation of emotional outliers in joint judgment is increased. For speech with sudden changes in speech rate, abnormal pauses, or significant energy fluctuations, emotional outliers can be adjusted according to the degree of deviation of the corresponding acoustic indicators.

[0044] In the field of fintech business, when users report insurance claims, remotely verify their identity, or handle account business, text risk features can be generated based on abnormal semantic information in the voice content, and emotional abnormality values ​​can be generated based on changes in pitch, energy, and speech rate. The two types of data are used together to form fraud warning judgment results.

[0045] In the field of healthcare, when users conduct remote health consultations or online follow-up visits, text risk features can be generated based on abnormal statements in the voice content. Combined with acoustic changes corresponding to tension, urgency, or abnormal pauses, emotional abnormality values ​​can be generated to make a comprehensive judgment on the abnormal risks in the current voice.

[0046] This embodiment generates text risk features from content features and emotional anomalies from noise-reduced audio signals, enabling fraud warnings to utilize both semantic content and vocal state information simultaneously. This reduces bias caused by relying solely on a single keyword or acoustic change for judgment and improves the completeness of abnormal voice content recognition.

[0047] S50, the fused features are input into the voiceprint recognition model jointly trained by multi-node parameter aggregation to obtain the identity recognition result; In this embodiment, the fused features are used as input to the voiceprint recognition model. They are first processed through an input mapping layer to unify dimensions, then the feature encoding layer extracts identity differentiation information, and finally, the identity output layer generates the identity recognition result. The voiceprint recognition model can employ a convolutional coding structure, a temporal coding structure, or an attention coding structure. The identity recognition result can be represented as an identity category, an identity matching probability, or an identity consistency score.

[0048] The multi-node parameter aggregation method involves multiple training nodes updating the same model structure using their local training data and exchanging model parameters to complete joint training. The training endpoint sends the same initial model parameters to each training node and receives node model parameters generated by each node based on local fusion features and identity tags. Then, it assigns aggregation weights based on the number of training samples, data quality, or node contribution, and performs weighted aggregation of the node model parameters. The aggregated model parameters are then sent back to each training node for iterative updates. Once the training termination condition is met, a voiceprint recognition model is formed. During training, it is not necessary to centrally aggregate the complete speech data stored by each training node.

[0049] When the data scale varies significantly across training nodes, the node aggregation weight is determined by the number of training samples, ensuring that nodes with a larger number of samples have a corresponding participation ratio in model parameter updates. The voiceprint recognition model employs an input mapping layer, an attention encoding layer, and an identity output layer. The fused features are encoded to generate an identity representation, and the identity output layer then forms the identity recognition result.

[0050] When dealing with a large number of training nodes and significant network transmission pressure, a hierarchical parameter aggregation approach is adopted. Multiple training nodes are divided into several training groups, each group performs local parameter aggregation, and then the aggregated parameters from each group are aggregated again. After training, a unified voiceprint recognition model with fused feature inputs is used for feature encoding and identity recognition.

[0051] In the fintech business, different business nodes store verified user voice data. Each node participates in updating model parameters based on local data. The trained voiceprint recognition model receives fusion features generated from insurance claims, remote identity verification, or account business scenarios and outputs identity recognition results.

[0052] In the healthcare sector, different remote health service nodes each store verified voice interaction data, and joint training is completed through multi-node parameter aggregation. When a user conducts a remote health consultation or online follow-up visit, the voiceprint recognition model, trained by integrating feature inputs, generates the corresponding identity recognition result.

[0053] This embodiment uses a multi-node parameter aggregation method to jointly train the voiceprint recognition model, so that training information scattered across different training nodes can participate in model training through model parameter updates, and use fused features for identity recognition, thereby expanding the coverage of model training data and improving the adaptability of identity recognition results to different speech feature distributions.

[0054] S60, determine the voiceprint authentication result based on the identity recognition result, the liveness determination result, and the fraud warning determination result.

[0055] In this embodiment, the identity recognition result reflects the matching status between the current voice and the identity of the object to be verified, and can be represented by identity category, matching score, or trust status. The liveness detection result reflects whether the current voice originates from real-time speech, and can be represented by liveness status or liveness trust level. The fraud warning detection result reflects the abnormal risk corresponding to the current voice content and speech state, and can be represented by risk level or risk score. When the data formats of the three types of results are inconsistent, they can be converted into a unified detection state or a unified numerical range, and then jointly analyzed according to a preset combination relationship. The voiceprint authentication result can be generated based on the combined state of the three types of results, or it can be generated after weighted fusion of the normalized results, and is used to represent the comprehensive authentication state of the current voice interaction.

[0056] In situations where it is necessary to clearly distinguish between different authentication states, a state combination approach is used for joint analysis. The identity recognition result, liveness detection result, and fraud warning result are each converted into a corresponding state, and then the voiceprint authentication result is determined based on a preset state mapping relationship. A rejection state can be formed when the identity does not match or the liveness detection is abnormal; a review state can be formed when the identity matches and the liveness detection is normal, but a fraud warning exists.

[0057] In situations where it is necessary to preserve the degree of difference among various results, a weighted fusion approach is used for joint analysis. The three types of results are converted to a unified numerical range, and participation weights are set according to the credibility of different results to generate a comprehensive authentication value. Then, a voiceprint authentication result is formed based on a preset judgment range. For interactions with high requirements for liveness detection, the participation ratio of liveness detection results can be increased.

[0058] In the fintech business, when users file insurance claims, remotely verify their identity, or handle account transactions, voiceprint authentication results can be generated by combining identity recognition results, liveness detection results, and fraud warning results. Interactions with matching identities but with abnormal voice risks can be transferred for further review.

[0059] In the field of healthcare, when users conduct remote health consultations or online follow-up visits, voiceprint authentication results can be generated by combining the current voice identity recognition status, real-time voice output status, and abnormal risk status, and abnormal interactions can be handled in a graded manner.

[0060] This embodiment combines identity recognition results, liveness detection results, and fraud warning results to enable voiceprint authentication to simultaneously consider identity matching, real-time voice authenticity, and abnormal risk information. This reduces authentication bias caused by local anomalies affecting a single judgment result and improves the completeness and credibility of voiceprint authentication results.

[0061] In one embodiment, such as Figure 3 As shown, step S10 above includes: S101, read the historical fundamental frequency sequence and historical respiratory interval sequence corresponding to the object to be verified from the benchmark feature library, generate fundamental frequency benchmark center data and fundamental frequency benchmark discrete data based on the historical fundamental frequency sequence, generate respiratory interval benchmark center data and respiratory interval benchmark discrete data based on the historical respiratory interval sequence, and generate benchmark dynamic feature data based on the fundamental frequency benchmark center data, the fundamental frequency benchmark discrete data, the respiratory interval benchmark center data and the respiratory interval benchmark discrete data; S102, the voice of the object to be verified is collected through a microphone array to obtain channel audio data corresponding to multiple collection channels, and the sampling time markers corresponding to the channel audio data are recorded. The channel audio data is time-aligned according to the sampling time markers to generate a multi-channel audio signal. S103, based on the multi-channel audio signal, divide the speech activity segment and non-speech segment, extract the noise frequency domain data of each acquisition channel from the non-speech segment, and generate a noise covariance matrix based on the noise frequency domain data; S104, extract the time difference of arrival between each acquisition channel based on the voice activity segment, generate the target sound source direction based on the time difference of arrival and the array element position data of the microphone array, and generate the target steering vector based on the target sound source direction and the array element position data; S105, Based on the noise covariance matrix and the target steering vector, a spatial filter weight vector is generated under the distortion-free constraint of the target sound source direction; S106, the multi-channel audio signal is frequency domain transformed to generate multi-channel frequency domain data, the multi-channel frequency domain data is weighted and synthesized using the spatial filtering weight vector, and the weighted synthesis result is reconstructed in the time domain to obtain a noise-reduced frequency signal.

[0062] In this embodiment, the benchmark feature library stores historical vocalization data corresponding to the object to be verified. The historical fundamental frequency sequence reflects the changes in vocal cord vibration frequency in different historical speech segments, and the historical respiratory interval sequence reflects the temporal distribution between adjacent respiratory events during continuous vocalization. The benchmark center data of the fundamental frequency can be generated from the mean, median, or robust center value of the historical fundamental frequency sequence, and is used to represent the central location of the fundamental frequency; the benchmark discrete data of the fundamental frequency can be generated from the standard deviation, interquartile range, or absolute deviation, and is used to represent the fluctuation range of the fundamental frequency. The benchmark center data and the benchmark discrete data of the respiratory interval are generated in a corresponding manner. The combination of the four types of data forms the benchmark dynamic feature data, so that both the vocal frequency and respiratory rhythm retain the central distribution and variation amplitude.

[0063] Multiple acquisition channels in the microphone array record the speech of the object to be verified. Each channel's audio data is accompanied by a sampling time stamp. Time alignment is achieved through timestamp correction, sampling point offset compensation, or resampling to ensure that sound components at the same moment maintain temporal correspondence in the multi-channel audio signal. The time-aligned multi-channel audio signal preserves the amplitude, phase, and arrival time differences of the target speech and ambient sound at different array element positions.

[0064] Speech activity segments and non-speech segments can be distinguished based on short-time energy, zero-crossing changes, spectral changes, or speech activity detection results. Non-speech segments mainly include ambient sound and background noise. Noise frequency domain data is obtained by performing a short-time frequency domain transformation on the non-speech segments in each acquisition channel. By statistically analyzing the correlation between different acquisition channels at the same frequency location, a noise covariance matrix can be generated to describe the energy distribution of ambient noise in the array space and channel correlation. When ambient sound changes, the noise covariance matrix can be updated according to the new non-speech segments.

[0065] The same speech component within a speech activity segment arrives at different array elements with a time difference. This arrival time difference can be obtained through inter-channel cross-correlation, generalized cross-correlation, or phase difference analysis, and then combined with the array element position data to determine the direction of the target sound source. The array element position data describes the spatial position of each microphone in the array coordinate system. Based on the target sound source direction and the array element position data, the time delay and phase relationship of the target speech propagating to different array elements can be determined, forming a target steering vector.

[0066] The noise covariance matrix reflects the spatial noise distribution that needs to be suppressed, while the target steering vector reflects the target speech spatial response that needs to be preserved. When generating the spatial filter weight vector based on these two types of data, the amplitude and phase weights of each acquisition channel can be adjusted to suppress sound components from non-target directions while maintaining the target sound source direction response. Distortion-free constraints are used to control the combined response from the target sound source direction, preventing the target speech from being synchronously attenuated during spatial filtering.

[0067] Multi-channel audio signals undergo a short-time frequency domain transformation to form multi-channel frequency domain data. Spatial filtering weight vectors are applied to multiple channels at the same time and frequency position, and a single-channel frequency domain result is obtained through complex weighting and channel synthesis. The weighted synthesis result is then subjected to inverse frequency domain transformation and inter-frame overlap synthesis to recover the continuous time-domain waveform, forming a noise-reduced audio signal. This preserves the main spectral and temporal information of the speech to be verified while reducing the proportion of environmental noise and sounds from non-target directions.

[0068] For example, the formula for generating the spatial filter weight vector based on the noise covariance matrix and the target steering vector is:

[0069] in, Represents the spatial filtering weight vector; This represents the noise covariance matrix generated based on the noise frequency domain data in the non-speech segment; The matrix representing the inverse of the noise covariance matrix; Represents the target orientation vector; Indicates the direction of the target sound source; This represents the conjugate transpose of the target orientation vector.

[0070] This embodiment models the central distribution and dispersion of historical fundamental frequencies and breathing intervals to form benchmark dynamic characteristic data that reflects the dynamic range of vocal changes of the object to be verified. Through multi-channel time alignment, non-speech segment noise statistics, target sound source direction estimation, and distortion-free spatial filtering, environmental noise and non-target direction sounds can be suppressed by utilizing the spatial differences between different array elements. Then, after frequency domain weighted synthesis and time domain reconstruction, a noise-reduced frequency signal is obtained, thereby improving the purity and stability of the speech data of the object to be verified.

[0071] In one embodiment, such as Figure 4 As shown, step S20 above includes: S201, perform frame division and frequency domain transformation on the noise-reduced frequency signal to generate spectral energy data, and perform filter bank mapping, logarithmic transformation and discrete cosine transformation on the spectral energy data to generate cepstral feature sequence, and generate voiceprint features based on the cepstral feature sequence; S202, perform autocorrelation analysis on the noise-reduced frequency signal to generate fundamental frequency data, generate fundamental frequency data based on the fundamental frequency data, generate breathing interval data based on the energy envelope change of the noise-reduced frequency signal, and generate physiological acoustic features based on the fundamental frequency data and the breathing interval data. S203, the noise-reduced frequency signal is transcribed into speech to generate transcribed text, and the transcribed text is semantically encoded to generate content features; S204, the voiceprint features, the physiological acoustic features, and the content features are mapped to a unified feature space to generate mapped voiceprint features, mapped physiological acoustic features, and mapped content features. Based on the feature interactions between the mapped voiceprint features, the mapped physiological acoustic features, and the mapped content features, fusion weights are generated. Based on the fusion weights, the mapped voiceprint features, the mapped physiological acoustic features, and the mapped content features are weighted and fused to obtain fused features.

[0072] In this embodiment, the noise-reduced audio signal is divided into continuous audio frames according to a preset frame length and frame shift. Each audio frame is windowed and then subjected to frequency domain transformation, converting the time-domain waveform into amplitude information at different frequency positions, and forming spectral energy data based on the amplitude. Filter bank mapping is used to aggregate the spectral energy into bands according to a preset frequency scale, so that adjacent frequency components form multiple frequency bands. Logarithmic transformation compresses the energy amplitude differences between different frequency bands, and discrete cosine transformation further reduces the correlation between frequency band energies, forming a cepstral feature sequence arranged according to dimensions. The cepstral features of multiple audio frames can be spliced, statistically summarized, or time-series encoded to generate acoustic signature features that characterize the spectral structure of the object to be verified.

[0073] Autocorrelation analysis of the denoised audio signal determines periodic repetition positions by comparing the similarity between the speech waveform and waveforms with different time delays, and generates pitch period data based on these periodic delay positions. The pitch period data, combined with the sampling frequency, is converted into pitch frequency data to reflect the periodic vibration state of the vocal cords. Respiratory interval data can be obtained from the energy envelope changes of the denoised audio signal. By identifying time segments with respiratory acoustic characteristics or sustained low-energy characteristics between consecutive vocalization segments, the time interval between adjacent respiratory events is determined. The pitch frequency data and respiratory interval data are combined or statistically summarized according to their time positions to form a physiological acoustic feature that simultaneously reflects vocal cord vibration and respiratory rhythm.

[0074] Noise-reduced audio signals are transcribed into text through speech-to-text transcription. The transcription process identifies corresponding characters, terms, or sub-words based on the acoustic frame sequence and reconstructs the text content in chronological order. After word segmentation or text unit division, the transcribed text undergoes contextual semantic encoding, updating the representation of each text unit in conjunction with the relationships between the preceding and following text. Contextual semantic encoding can employ temporal coding or attention coding structures to convert local term information and overall semantic relationships into vector representations, generating content features.

[0075] Voiceprint features, physiological acoustic features, and content features originate from different information dimensions, and their feature lengths and numerical distributions may differ. These three types of features are transformed into a unified feature space through linear mapping, nonlinear mapping, or normalized projection, generating mapped voiceprint features, mapped physiological acoustic features, and mapped content features, respectively. These three types of mapped features can interact through splicing, attention, or gating, and fusion weights are generated based on the response relationships between each mapped feature and other mapped features. The fusion weights are used to determine the degree of participation of the three types of mapped features in the current speech, and then weighted and combined according to their corresponding weights to obtain the fused features, ensuring that the same feature representation simultaneously preserves spectral individual differences, vocal physiological information, and semantic content information.

[0076] For example, the formula for generating cepstral feature sequences based on spectral energy data is:

[0077] in, Represents the cepstral feature in the m-th dimension; The spectrum energy data is represented by the k-th frequency band energy obtained after being mapped by the filter bank; k represents the filter bank frequency band index; K represents the number of frequency bands participating in the discrete cosine transform; m represents the cepstral feature dimension index; M represents the number of generated cepstral feature dimensions, which can be 39 dimensions in the embodiment; ε represents a positive number to avoid the inability to perform logarithmic transformation when the frequency band energy is zero.

[0078] The formula for generating pitch period data based on autocorrelation analysis is as follows:

[0079]

[0080] in, This represents the autocorrelation value when the delay is τ. This represents the nth speech sample value; N represents the number of sampling points involved in the autocorrelation analysis. Indicates the number of delayed sampling points; This indicates the non-zero delay position corresponding to the pitch period; Indicates the sampling frequency of the speech signal; This represents the fundamental frequency data.

[0081] The formula for generating fusion features based on fusion weights is:

[0082] in, Indicates fusion characteristics; This indicates the mapping of voiceprint features; Indicates the characteristics of the mapped content; This indicates the mapping of physiological acoustic features; This represents the fusion weights corresponding to the mapped voiceprint features; This represents the fusion weight corresponding to the mapped content features; This represents the fusion weights corresponding to the mapped physiological acoustic features.

[0083] This embodiment extracts cepstral structure, pitch and breathing rhythm, and contextual semantic information from the noise-reduced frequency signal, which can characterize the vocal state and speech content of the object to be verified from different dimensions. Then, the three types of features are mapped to a unified feature space and fusion weights are generated according to feature interaction, so that different information can participate in fusion according to the actual response level in the current speech, thereby reducing the influence of local changes on a single acoustic feature and improving the information integrity and stability of the fused features.

[0084] In one embodiment, such as Figure 5 As shown, step S30 above includes: S301, extract candidate interactive content from a preset interactive content library, randomly select target interactive content from the candidate interactive content, and generate random interactive instructions and interactive response conditions based on the target interactive content; S302, output the random interaction command, collect the audio data of the object to be verified in response to the random interaction command, and generate a reply audio signal; S303, the response audio signal is transcribed into speech to generate response text, and interactive response data is generated based on the matching relationship between the response text and the interactive response conditions; S304, perform autocorrelation analysis on the response audio signal to generate real-time fundamental frequency data, and generate real-time respiratory interval data based on the energy envelope change of the response audio signal; S305, extract the fundamental frequency reference center data, fundamental frequency reference discrete data, respiratory interval reference center data and respiratory interval reference discrete data from the reference dynamic characteristic data, and perform standardized difference processing on the real-time fundamental frequency data based on the fundamental frequency reference center data and the fundamental frequency reference discrete data to generate fundamental frequency standardized difference data; S306, Based on the respiratory interval reference center data and the respiratory interval reference discrete data, perform standardized difference processing on the real-time respiratory interval data to generate respiratory interval standardized difference data, and generate dynamic feature distance based on the fundamental frequency standardized difference data and the respiratory interval standardized difference data; S307, a joint determination is made based on the interactive response data and the dynamic feature distance to obtain the liveness determination result.

[0085] In this embodiment, a preset interactive content library stores voice response content available for real-time interaction. Each candidate interactive content may include reading aloud, dynamic question-and-answer content, or vocal content with definite response requirements. Target interactive content is selected from the candidate interactive content using random number generation, ensuring variability in the response content used in different interaction processes. The target interactive content is converted into a random interactive command to be output to the object to be verified, and interactive response conditions are generated. These interactive response conditions describe the expected response content matching the target interactive content, the allowed range of semantic changes, or the correspondence between text units, thus providing data for subsequent determination of whether a response is generated in response to the current random interactive command.

[0086] Random interaction commands can be output via voice broadcast, text display, or a combination of voice and text. After the subject responds to the random interaction command, the voice recording process is captured, generating a response audio signal. This response audio signal retains information such as the response content, fundamental frequency changes, and breathing rhythm. The response audio signal is then transcribed, converting continuous audio into characters, words, or text units arranged in the order of pronunciation, generating response text. The response text is then compared with the interaction response conditions through text unit matching, semantic similarity analysis, or keyword correspondence to form interaction response data reflecting whether the current response meets the requirements of random interaction.

[0087] The periodic vocal components in the response audio signal can be obtained through autocorrelation analysis. The speech waveform is compared with waveforms under different delays, and the pitch period is determined based on the delay position corresponding to the periodic correlation peak. This, combined with the sampling frequency, generates real-time pitch frequency data. Real-time pitch frequency data can be represented as a single statistical value from the current response, a continuous pitch frequency sequence, or a summary of data from multiple speech segments, reflecting the vocal cord vibration state during the current vocalization process.

[0088] Real-time respiratory interval data can be generated based on changes in the energy envelope of the response audio signal. After smoothing the audio amplitude or performing frame-by-frame energy statistics, time segments with sustained low energy, inspiratory acoustic components, or interrupted vocalization characteristics between consecutive vocal segments are identified, and real-time respiratory interval data is generated based on the time distance between adjacent respiratory events. For response audio signals containing multiple respiratory events, the respiratory interval sequence can be preserved, or statistical data representing the current respiratory rhythm can be generated.

[0089] In the baseline dynamic characteristic data, the fundamental frequency baseline center data represents the central location of the existing fundamental frequency of the object to be verified, while the fundamental frequency baseline discrete data represents the fluctuation range of the existing fundamental frequency. The offset of the real-time fundamental frequency data relative to the fundamental frequency baseline center data is scaled according to the fundamental frequency baseline discrete data to generate standardized fundamental frequency difference data. Through scaling, the influence of different fundamental frequency variation ranges of different objects to be verified on the difference results can be reduced.

[0090] The baseline center data and baseline discrete data of respiratory intervals represent the central location and fluctuation range of existing respiratory intervals, respectively. The offset of real-time respiratory interval data relative to the baseline center data of respiratory intervals is scaled according to the baseline discrete data of respiratory intervals to generate standardized difference data of respiratory intervals. The standardized difference data of fundamental frequency and the standardized difference data of respiratory intervals can be combined according to a distance metric to form a dynamic characteristic distance, which is used to reflect the overall deviation between the current vocal dynamics of the response and the existing vocal dynamics of the object to be verified.

[0091] Interactive response data reflects the degree of correspondence between the current response and the random interactive requirement, while the dynamic feature distance reflects the degree of difference between the current vocal dynamics and the baseline dynamic distribution. Joint judgment can be performed by setting interactive response conditions and dynamic difference conditions separately, or by converting the two types of data to a unified judgment interval and then combining them. When the response content meets the current random interactive requirement and the dynamic feature distance is within the allowable range, a corresponding liveness determination result is generated; when the response content does not match the current random interactive requirement, or when the vocal dynamics deviate from the baseline dynamic distribution beyond the allowable range, a corresponding anomaly determination result is generated.

[0092] For example, the formula for generating the dynamic feature distance based on standardized difference data is:

[0093] in, N represents the distance between dynamic features; N represents the number of valid feature data points participating in the dynamic difference analysis; i represents the feature data index. This represents the i-th real-time fundamental frequency data; This represents the fundamental frequency reference center data; This represents discrete data based on the fundamental frequency. This represents the i-th real-time respiratory interval data; This represents the baseline center data for respiratory intervals; This represents the baseline discrete data for respiratory intervals.

[0094] This embodiment randomly selects interactive content and synchronously generates interactive response conditions, giving each interaction dynamically changing response requirements. By matching the response text with the interactive response conditions, it can be determined whether the current voice is generated in real time for this interaction. By performing standardized difference analysis on the real-time fundamental frequency and real-time breathing interval, and combining the two types of differences into a dynamic feature distance, the current vocal dynamics can be analyzed while considering the vocal fluctuation range of the object to be verified. By combining the interactive response data and the dynamic feature distance for joint judgment, the ability to distinguish between pre-recorded voice, non-current interactive voice, and abnormal vocal dynamics is improved.

[0095] In one embodiment, such as Figure 6 As shown, step S40 above includes: S401, Based on the content features, identify candidate risk terms, generate term frequency data corresponding to the candidate risk terms, generate corpus distribution data corresponding to the candidate risk terms based on preset corpus data, and generate term weights based on the term frequency data and the corpus distribution data; S402, extract the term risk coefficients corresponding to the candidate risk terms from the preset risk feature library, and generate text risk features based on the term weights and term risk coefficients; S403, extract pitch change data, energy change data and speech rate change data from the noise-reduced frequency signal, and generate emotional acoustic features based on the pitch change data, the energy change data and the speech rate change data; S404, Obtain baseline emotion feature data, and generate emotion anomaly values ​​based on the feature differences between the emotion acoustic features and the baseline emotion feature data; S405, generate fraud risk data based on the text risk features and the emotional anomaly values, and determine the fraud warning judgment result based on the fraud risk data.

[0096] In this embodiment, content features include word semantics, contextual relationships, and semantic association information within the speech content. Content features can be matched with the semantic representations corresponding to preset risk words to identify candidate risk words. For each candidate risk word, word frequency data is generated based on its frequency of occurrence in the current speech content, its consecutive occurrence, or its semantic response strength; then, corpus distribution data is generated based on the occurrence range and sparsity of the candidate risk words in the preset corpus data. Word frequency data reflects the local saliency of the candidate risk word in the current speech content, while corpus distribution data reflects the degree to which the candidate risk word distinguishes different speech contents. The combination of these two types of data generates word weights.

[0097] A pre-defined risk feature library stores the term risk coefficients corresponding to different candidate risk terms. These term risk coefficients represent the degree of association between different risk terms and anomalous semantics. Combining term weights with their corresponding term risk coefficients generates the risk contribution of each candidate risk term. These multiple risk contributions are then aggregated to form the text risk features. Text risk features can be represented by a single risk value or formed into multi-dimensional risk vectors according to different risk semantic categories.

[0098] The pitch variation data in the noise-reduced audio signal can be generated based on the pitch difference of consecutive audio frames to reflect the fluctuation of the vocal frequency; the energy variation data can be generated based on the amplitude changes of short-time energy, energy envelope, or adjacent speech segments to reflect changes in vocal intensity; and the speech rate variation data can be generated based on the number of syllables, the number of words, or the effective vocal duration per unit time to reflect changes in vocal rhythm. These three types of data can be combined after normalization and time alignment to form emotional acoustic features, allowing pitch, energy, and speech rate to participate in subsequent analysis within a unified data range.

[0099] The baseline emotional feature data can be derived from existing confirmed speech data of the subject to be verified, or generated from a pre-defined set of reference speech samples. This data describes the distribution range of pitch, energy, and speech rate changes under the reference vocalization state. Component difference analysis is performed between the current emotional acoustic features and the baseline emotional feature data to obtain the degree of deviation in different acoustic dimensions. Then, emotional outliers are generated through weighted combination, distance measurement, or normalization summarization. The larger the emotional outlier, the more significant the deviation of the current vocalization state from the reference distribution.

[0100] Text risk features reflect risk information within semantic content, while emotional outliers reflect the degree of abnormality in vocalization. These two types of data can be fused according to preset weights, or they can be converted separately to a unified risk range and then combined to generate fraud risk data. The fraud risk data is compared with preset judgment criteria to form a fraud warning judgment result, thus simultaneously retaining the judgment criteria for both semantic content and acoustic state dimensions.

[0101] For example, the formula for generating fraud risk data based on text risk features and sentiment outliers is:

[0102] in, W represents fraud risk data; W represents the set of candidate risk terms identified from content features; w represents a candidate risk term. The term weights corresponding to candidate risk terms are generated from term frequency data and corpus distribution data. This represents the term risk coefficient corresponding to the candidate risk term; Overall corresponding text risk characteristics; Indicates abnormal emotional values; This indicates the participation weight corresponding to abnormal emotional values.

[0103] This embodiment generates term weights by combining the frequency of candidate risk terms in the current speech content with their distribution in a preset corpus, and further combines term risk coefficients to form text risk features, which can improve the ability to distinguish risk semantic information. By jointly analyzing changes in pitch, energy, and speech rate and comparing them with baseline emotion feature data, emotion anomaly values ​​reflecting the degree of deviation from the vocal state can be generated. Then, the text risk features and emotion anomaly values ​​are combined to form fraud risk data, thereby reducing the judgment bias caused by a single keyword or a single acoustic change and improving the completeness and stability of fraud warning judgment results.

[0104] In one embodiment, such as Figure 7 As shown, step S50 above includes: S501: Obtain the number of training samples corresponding to multiple training nodes and send the initial model parameters to each training node; S502, receive the node model parameters generated by each training node after local training based on the initial model parameters, identity tags, and local fusion features generated according to their respective saved local training speech samples; S503, generate node aggregation weights based on the number of training samples corresponding to each training node, and aggregate the node model parameters based on the node aggregation weights to generate aggregated model parameters; S504, the aggregated model parameters are sent to each training node, and the updated node model parameters generated by each training node after continuing local training based on the aggregated model parameters, their respective local fusion features and identity tags are received. S505, perform parameter aggregation on the updated node model parameters based on the node aggregation weights to generate updated aggregated model parameters, send the updated aggregated model parameters to each training node, and repeat the steps of receiving updated node model parameters, performing parameter aggregation on the updated node model parameters based on the node aggregation weights to generate updated aggregated model parameters, and sending updated aggregated model parameters until a preset training termination condition is met. Use the updated aggregated model parameters when the preset training termination condition is met as model parameters to obtain the voiceprint recognition model. S506, The fused features are input into the voiceprint recognition model for feature encoding and identity recognition processing to obtain the identity recognition result.

[0105] In this embodiment, multiple training nodes each store local training speech samples and corresponding identity tags. The parameter aggregation terminal is responsible for distributing initial model parameters, receiving node model parameters, and aggregating parameters. Each training node is configured with the same speaker recognition model structure, and the parameter dimensions and connection relationships of each model layer are kept consistent, so that model parameters generated by different training nodes can be aggregated at the same parameter positions.

[0106] The voiceprint recognition model comprises an input mapping layer, a feature encoding layer, an identity representation layer, and an identity output layer, connected in sequence. The input mapping layer receives locally fused features and converts the fused features from different samples into a unified hidden dimension. The feature encoding layer includes one or more sequentially connected encoding units, each of which may include linear transformations, nonlinear activations, and normalization processes to extract high-level features relevant to identity differentiation. The identity representation layer converts the encoding results into a fixed-dimensional identity representation. The identity output layer maps the identity representation to an identity category space, generating output data corresponding to each identity category.

[0107] Each training node generates local training data based on local training speech samples. For a single local training speech sample, voiceprint information, physiological acoustic information, and content information are extracted following a feature generation process consistent with the identity recognition stage. Feature mapping and fusion are then performed to generate local fused features. These local fused features are then combined with the corresponding identity markers to form training data items. Thus, the model input used by each training node has the same data structure and feature dimensions as the fused features received by the subsequent voiceprint recognition model.

[0108] The parameter aggregation terminal initializes the voiceprint recognition model to obtain initial model parameters. These initial model parameters include weight and bias parameters in the input mapping layer, feature encoding layer, identity representation layer, and identity output layer. The parameter aggregation terminal sends the same set of initial model parameters to each training node, while simultaneously recording the number of training samples actually used by each node in the current training.

[0109] After receiving the initial model parameters, each training node loads them into its local voiceprint recognition model and divides the local training data into multiple training batches. The local fusion features in each batch are input into the voiceprint recognition model, sequentially passing through an input mapping layer, a feature encoding layer, and an identity representation layer to generate identity representations. Then, the identity output layer generates identity prediction data. An identity classification loss is generated based on the difference between the predicted identity data and the identity label, and backpropagation is performed based on this loss to update the parameters of each model layer. When further constraints on the identity representation distribution are needed, an identity representation distance loss can be incorporated to ensure that identity representations for the same identity maintain a small feature distance, while identity representations for different identities maintain a distinguishable feature distance.

[0110] Local training parameters can include local training epochs, batch size, learning rate, learning rate update method, and parameter optimization method. Each training node can use the same local training parameters, or the corresponding local training epochs can be pre-configured based on the number of local training samples. In one implementation, each training node iterates through the local training data for several epochs according to a preset batch size. After completing each training batch, the model parameters are updated once based on the current loss. After completing the preset local training epochs, the current model parameters are sent as the node's model parameters to the parameter aggregation end.

[0111] The parameter aggregation layer generates node aggregation weights based on the number of training samples corresponding to each training node. If a training node has a large proportion of training samples compared to the total number of training samples, its node aggregation weight will increase accordingly. During parameter aggregation, data belonging to the same model layer and the same parameter position from different nodes are weighted and combined according to their node aggregation weights to generate aggregated model parameters. Therefore, the influence of each training node on the aggregated model parameters corresponds to the actual scale of data involved in training.

[0112] The parameter aggregation endpoint sends the aggregated model parameters to each training node. Each training node replaces its model parameters from the previous round of local training with the aggregated model parameters and continues batch training based on its saved local fusion features and identity tags to generate updated node model parameters. After receiving the updated node model parameters returned by each training node, the parameter aggregation endpoint continues to aggregate parameters according to the node aggregation weights to obtain the updated aggregated model parameters.

[0113] A joint training round consists of one parameter distribution, one local training, and one parameter aggregation. The first joint training round uses the initial model parameters as the training parameters for each training node. Subsequent joint training rounds use the updated aggregated model parameters generated in the previous round as the new training parameters for each training node. As the joint training rounds increase, parameter changes from different training nodes based on their respective local training speech samples are continuously incorporated into the updated aggregated model parameters.

[0114] The preset training termination conditions can be set based on the maximum number of joint training epochs, the magnitude of model loss change, the magnitude of change between adjacent updated aggregate model parameters, or the identity recognition performance on the validation data. For example, training can stop when the preset maximum number of joint training epochs is reached; or when the model loss change is below a preset range for multiple consecutive joint training epochs; or when the identity recognition performance on the validation data remains stable over multiple consecutive joint training epochs. After the preset training termination conditions are met, the current updated aggregate model parameters are loaded into the voiceprint recognition model, resulting in the trained voiceprint recognition model.

[0115] In fintech business scenarios, each training node can correspond to a different insurance institution, regional claims center, or different business data node. Local training voice samples can come from verified insurance application voices, policy maintenance voices, claims reporting voices, and remote verification voices, with identity tags used to represent the corresponding user's identity. Each training node only uses locally stored voice data to generate local fusion features and update model parameters, sending the node model parameters to the parameter aggregation end, without needing to centrally send complete voice data. After training is complete, the fusion features generated from the current insurance claim voice are input into the voiceprint recognition model, and the model outputs identity category, identity matching score, or identity consistency data, forming the identity recognition result.

[0116] In healthcare scenarios, each training node can correspond to different remote health service nodes or regional health service data nodes. Local training voice samples can come from verified remote health consultations, online follow-up visits, and health service interactions, with identity tags indicating the corresponding user's identity. Each training node generates local fusion features using the same feature generation process and performs local training, then aggregates parameters to form a unified voiceprint recognition model. The fusion features generated from the current remote voice interaction are input into the trained voiceprint recognition model, which outputs the corresponding identity recognition result.

[0117] After the voiceprint recognition model is trained using fused feature inputs, the input mapping layer converts the fused features into hidden dimensions consistent with those used in the model training phase. The feature encoding layer extracts identity-distinguishing information, the identity representation layer generates a fixed-dimensional identity representation, and the identity output layer generates identity prediction data based on the identity representation. From the identity prediction data, identity categories, identity matching scores, or identity consistency data can be generated, and the identity recognition result can be obtained accordingly.

[0118] For example, the formula for generating parameters of an aggregation model based on node aggregation weights is:

[0119] in, This indicates the aggregated model parameters generated after aggregating the current parameters or the updated aggregated model parameters; K represents the number of training nodes participating in the current joint training; k represents the training node index. This represents the number of training samples corresponding to the k-th training node; This represents the total number of training samples from all participating nodes. This represents the node aggregation weight corresponding to the k-th training node; This represents the node model parameters generated or updated by the k-th training node in the t-th joint training round; t represents the joint training round.

[0120] This embodiment unifies the model structure, input feature structure, and parameter dimensions of each training node, clarifying the processes of local fusion feature generation, local batch training, model parameter updating, node parameter uploading, and weighted aggregation. This enables multiple training nodes to jointly train the voiceprint recognition model based on their respective stored training speech data. Using the same fusion features as in the identity recognition stage as the model input during the training phase reduces the data structure differences between the training input and the actual recognition input. Continuously aggregating the model parameters formed by different training nodes through multiple joint training rounds improves the voiceprint recognition model's adaptability to different speech data distributions and the stability of the identity recognition results.

[0121] In one embodiment, such as Figure 8 As shown, step S60 above includes: S601, Match the identity recognition result with the preset identity determination conditions to generate an identity determination status; S602, Match the liveness determination result with the preset liveness determination conditions to generate a liveness determination state; S603, Match the fraud warning judgment result with the preset fraud judgment conditions to generate a fraud warning status; S604, Generate an authentication status combination based on the identity determination status, the liveness determination status and the fraud warning status; S605, the authentication state combination is matched with the preset authentication state mapping relationship to obtain the voiceprint authentication result.

[0122] In this embodiment, the identity recognition result can be represented by an identity matching score, identity consistency data, or identity category data. Preset identity determination conditions are set according to the data format of the identity recognition result and may include score ranges, category correspondences, or consistency determination thresholds. After matching the identity recognition result with the preset identity determination conditions, it is converted into an identity determination state. The identity determination state can distinguish between identity matching, identity mismatch, and intermediate states requiring further confirmation, thus converting identity recognition results in different data formats into unified state information.

[0123] Liveness detection results can be represented by a liveness confidence score, liveness category, or anomaly marker. Preset liveness detection conditions are used to define different liveness detection ranges, and a liveness detection status is generated based on the score range, corresponding category, or anomaly marker of the liveness detection result. The liveness detection status is used to maintain information on whether the current speech meets the real-time voice requirements, and is represented in a composable data format along with the identity detection status.

[0124] Fraud warning results can be represented by risk scores, risk levels, or warning categories. Pre-defined fraud judgment conditions categorize corresponding states based on risk levels, converting fraud warning results into fraud warning states. This allows identity verification results, liveness detection results, and fraud warning results, despite having different data structures, to be converted into state data with clearly defined judgment meanings.

[0125] Identity verification status, liveness verification status, and fraud warning status are combined according to fixed field positions or status code order to generate authentication status combinations. Each authentication status combination retains the status of each of the three verification dimensions without overwriting any of them. A preset authentication status mapping relationship stores the correspondence between different authentication status combinations and voiceprint authentication results; this can be stored using a status combination table, a status code mapping table, or a conditional mapping structure. Matching the current authentication status combination with the preset authentication status mapping relationship yields the corresponding voiceprint authentication result. The voiceprint authentication result can indicate different statuses such as authentication passed, authentication rejected, or further verification required.

[0126] This embodiment converts the identity recognition results, liveness detection results, and fraud warning detection results in different data formats into identity detection status, liveness detection status, and fraud warning status, respectively. These are then combined to form an authentication status combination, and a voiceprint authentication result is generated based on a preset authentication status mapping relationship. This approach can retain the judgment information of three dimensions: identity matching, real-time voice transmission, and abnormal risk, avoiding a single judgment result directly determining the final authentication status, thereby improving the consistency and completeness of the voiceprint authentication result.

[0127] In one embodiment, a user submits a health insurance accidental fall report via the insurance company's mobile app. After entering policy information, the user verbally states their identity, the date and location of the accident, the details of the fall, and their claim request. Given the presence of television playback and indoor reverberation in the home environment, the mobile app uses multiple microphones to simultaneously capture the reported audio, creating a multi-channel audio signal, and aligns the channels according to the sampling time markers. The server retrieves confirmed voice data from the user's previous insurance identity verification, policy maintenance, and historical claims interactions, generating baseline dynamic characteristic data corresponding to the fundamental frequency and breathing interval.

[0128] Non-speech segments in the multi-channel audio signal are used to analyze background noise such as television sound, while speech activity segments are used to determine the user's speaking direction. Spatial filtering weights are then generated based on the microphone array position. The multi-channel audio signal is then weighted, synthesized, and reconstructed in the temporal domain to obtain a noise-reduced frequency signal. The noise-reduced report speech is analyzed in the frequency domain to form voiceprint features, and analyzed using pitch period and breathing rhythm to form physiological acoustic features. Content features are also formed based on the report's content, including the identity statement, accident date, details of the fall, and claim requests. These three types of features are mapped to a unified feature space and then weighted and fused to obtain the fused features.

[0129] To determine whether the current report was made by the user in real time, the mobile device temporarily generates a random interactive command to state the current date. The user immediately replies with the current date, generating a response audio signal. The server matches the response text with the interactive response conditions corresponding to the current date, extracts the real-time fundamental frequency and real-time breathing interval, and performs a standardized difference analysis with the fundamental frequency and breathing interval baseline data corresponding to the user's historical confirmation voice messages. If the response content meets the current random interaction requirements and the vocal dynamics are within the allowable range, a liveness determination result is generated, indicating that the user has passed the liveness assessment.

[0130] For the current claim report, the server identifies candidate risk terms such as accident date, fall injury, and claim application from content features. It generates term weights based on the occurrence of these terms in the current report and their distribution in historical claim corpora, and then combines this with term risk coefficients from a risk feature database to form textual risk features. Simultaneously, it analyzes pitch, energy, and speech rate variations in the report's audio and compares them with baseline emotional feature data from the user's existing confirmation audio to generate emotional anomaly values. Textual risk features and emotional anomalies together form fraud risk data. If the current claim report is complete, has a stable speech rate, and does not exhibit obvious semantic abnormalities, a fraud warning result is generated indicating that no fraud alert has been triggered.

[0131] The voiceprint recognition model is jointly trained by multiple training nodes. Each training node stores its local training speech samples and identity markers, and generates local fusion features based on these samples. The server sends initial model parameters to each training node, receives the node model parameters generated during local training, generates node aggregation weights based on the number of training samples at each node, and aggregates the parameters. Aggregated model parameters are continuously sent out, and each training node continues local training and returns updated node model parameters until the training termination condition is met, resulting in the voiceprint recognition model. The fusion features corresponding to the current report speech are input into the voiceprint recognition model to obtain the identity matching and recognition result.

[0132] The server generates identity verification status, liveness detection status, and fraud warning status based on the identity recognition result, liveness detection result, and fraud warning result, respectively, and combines them to form an authentication status combination. The current result is identity matching, liveness detection passed, and no fraud warning triggered. According to the preset authentication status mapping relationship, the voiceprint authentication result is obtained, and the report record continues to enter the claims material review and subsequent claims processing process.

[0133] In one embodiment, a voiceprint authentication device based on multi-node parameter aggregation is provided, which corresponds one-to-one with the voiceprint authentication method based on multi-node parameter aggregation in the above embodiments. (Refer to...) Figure 9 , Figure 9 This is a schematic diagram of the functional modules of a preferred embodiment of the voiceprint authentication device based on multi-node parameter aggregation of the present invention. The voiceprint authentication device based on multi-node parameter aggregation includes: an array noise reduction module 10, a multi-dimensional feature fusion module 20, a liveness analysis module 30, a fraud warning module 40, a voiceprint recognition module 50, and an authentication decision module 60. Detailed descriptions of each functional module are as follows: The array noise reduction module 10 is used to acquire a multi-channel audio signal containing the speech of the object to be verified and the reference dynamic feature data of the object to be verified, and to process the multi-channel audio signal using an array noise reduction method to obtain a noise-reduced audio signal. The multi-dimensional feature fusion module 20 is used to extract the voiceprint features, physiological acoustic features and content features of the noise-reduced frequency signal, and fuse the voiceprint features, the physiological acoustic features and the content features to obtain fused features; The liveness analysis module 30 is used to generate random interaction commands, acquire the response audio signal of the object to be verified in response to the random interaction commands, and perform liveness analysis based on the response audio signal and the reference dynamic characteristic data to obtain a liveness determination result. Fraud warning module 40 is used to generate text risk features based on the content features, generate emotional anomaly values ​​based on the noise-reduced frequency signal, and determine the fraud warning judgment result based on the text risk features and the emotional anomaly values. The voiceprint recognition module 50 is used to input the fused features into the voiceprint recognition model jointly trained by using a multi-node parameter aggregation method to obtain the identity recognition result; The authentication decision module 60 is used to determine the voiceprint authentication result based on the identity recognition result, the liveness determination result, and the fraud warning determination result.

[0134] In one embodiment, the array noise reduction module 10 is specifically used for: Read the historical fundamental frequency sequence and historical respiratory interval sequence corresponding to the object to be verified from the benchmark feature library. Generate fundamental frequency benchmark center data and fundamental frequency benchmark discrete data based on the historical fundamental frequency sequence. Generate respiratory interval benchmark center data and respiratory interval benchmark discrete data based on the historical respiratory interval sequence. Generate benchmark dynamic feature data based on the fundamental frequency benchmark center data, the fundamental frequency benchmark discrete data, the respiratory interval benchmark center data, and the respiratory interval benchmark discrete data. The voice of the object to be verified is collected by a microphone array to obtain channel audio data corresponding to multiple acquisition channels, and the sampling time mark corresponding to the channel audio data is recorded. The channel audio data is time-aligned according to the sampling time mark to generate a multi-channel audio signal. Based on the multi-channel audio signal, speech activity segments and non-speech segments are divided. Noise frequency domain data of each acquisition channel is extracted from the non-speech segments, and a noise covariance matrix is ​​generated based on the noise frequency domain data. Based on the speech activity segment, the time difference of arrival between each acquisition channel is extracted, and the target sound source direction is generated based on the time difference of arrival and the array element position data of the microphone array. The target steering vector is generated based on the target sound source direction and the array element position data. Based on the noise covariance matrix and the target steering vector, a spatial filter weight vector is generated under the distortion-free constraint of the target sound source direction. The multi-channel audio signal is frequency-domain transformed to generate multi-channel frequency-domain data. The multi-channel frequency-domain data is then weighted and synthesized using the spatial filtering weight vector. Finally, the weighted synthesis result is reconstructed in the time domain to obtain a noise-reduced audio signal.

[0135] In one embodiment, the multidimensional feature fusion module 20 is specifically used for: The noise-reduced frequency signal is framed and frequency domain transformed to generate spectral energy data. The spectral energy data is then subjected to filter bank mapping, logarithmic transformation, and discrete cosine transform to generate a cepstral feature sequence. Based on the cepstral feature sequence, voiceprint features are generated. Autocorrelation analysis is performed on the noise-reduced frequency signal to generate fundamental frequency data, and fundamental frequency data is generated based on the fundamental frequency data. Respiratory interval data is generated based on the energy envelope change of the noise-reduced frequency signal, and physiological acoustic features are generated based on the fundamental frequency data and the respiratory interval data. The noise-reduced frequency signal is transcribed into speech to generate transcribed text, and the transcribed text is then semantically encoded to generate content features. The voiceprint features, physiological acoustic features, and content features are respectively mapped to a unified feature space to generate mapped voiceprint features, mapped physiological acoustic features, and mapped content features. Fusion weights are generated based on the feature interactions between the mapped voiceprint features, mapped physiological acoustic features, and mapped content features. The mapped voiceprint features, mapped physiological acoustic features, and mapped content features are then weighted and fused based on the fusion weights to obtain fused features.

[0136] In one embodiment, the liveness analysis module 30 is specifically used for: Candidate interactive content is extracted from a preset interactive content library, target interactive content is randomly selected from the candidate interactive content, and random interactive instructions and interactive response conditions are generated based on the target interactive content; Output the random interaction command, collect the audio data of the object to be verified in response to the random interaction command, and generate a reply audio signal; The response audio signal is transcribed into speech to generate response text, and interactive response data is generated based on the matching relationship between the response text and the interactive response conditions. Autocorrelation analysis is performed on the response audio signal to generate real-time fundamental frequency data, and real-time respiratory interval data is generated based on the energy envelope change of the response audio signal. The fundamental frequency reference center data, fundamental frequency reference discrete data, respiratory interval reference center data, and respiratory interval reference discrete data are extracted from the reference dynamic characteristic data. Based on the fundamental frequency reference center data and the fundamental frequency reference discrete data, the real-time fundamental frequency data is standardized and differentially processed to generate fundamental frequency standardized differential data. Based on the reference center data and the reference discrete data of the respiratory interval, the real-time respiratory interval data is standardized and differentially processed to generate standardized differential data of the respiratory interval, and a dynamic feature distance is generated based on the standardized differential data of the fundamental frequency and the standardized differential data of the respiratory interval. A liveness determination result is obtained by jointly determining the interaction response data and the dynamic feature distance.

[0137] In one embodiment, the fraud warning module 40 is specifically used for: Based on the content features, candidate risk terms are identified, term frequency data corresponding to the candidate risk terms are generated, and corpus distribution data corresponding to the candidate risk terms is generated based on preset corpus data. Term weights are generated based on the term frequency data and the corpus distribution data. Extract the term risk coefficients corresponding to the candidate risk terms from the preset risk feature library, and generate text risk features based on the term weights and term risk coefficients; Pitch variation data, energy variation data, and speech rate variation data are extracted from the noise-reduced frequency signal, and emotional acoustic features are generated based on the pitch variation data, energy variation data, and speech rate variation data. Obtain baseline emotion feature data, and generate emotion anomaly values ​​based on the feature differences between the emotion acoustic features and the baseline emotion feature data; Fraud risk data is generated based on the text risk features and the emotional anomalies, and the fraud warning judgment result is determined based on the fraud risk data.

[0138] In one embodiment, the voiceprint recognition module 50 is specifically used for: Obtain the number of training samples corresponding to multiple training nodes, and send the initial model parameters to each training node; Receive node model parameters generated by each training node after local training based on the initial model parameters, identity tags, and local fusion features generated from their respective saved local training speech samples; Node aggregation weights are generated based on the number of training samples corresponding to each training node, and the node model parameters are aggregated based on the node aggregation weights to generate aggregated model parameters. The aggregated model parameters are sent to each training node, and the updated node model parameters generated by each training node after continuing local training based on the aggregated model parameters, their respective local fusion features and identity tags are received. Based on the node aggregation weights, the updated node model parameters are aggregated to generate updated aggregated model parameters. The updated aggregated model parameters are sent to each training node. The steps of receiving updated node model parameters, aggregating updated node model parameters based on node aggregation weights, generating updated aggregated model parameters, and sending updated aggregated model parameters are repeated until a preset training termination condition is met. The updated aggregated model parameters that meet the preset training termination condition are used as model parameters to obtain the voiceprint recognition model. The fused features are input into the voiceprint recognition model for feature encoding and identity recognition processing to obtain the identity recognition result.

[0139] In one embodiment, the authentication decision module 60 is specifically used for: The identity recognition result is matched with preset identity determination conditions to generate an identity determination status; The liveness determination result is matched with the preset liveness determination conditions to generate a liveness determination status; The fraud warning determination result is matched with the preset fraud determination conditions to generate a fraud warning status; An authentication status combination is generated based on the identity determination status, the liveness determination status, and the fraud warning status. The authentication state combination is matched with the preset authentication state mapping relationship to obtain the voiceprint authentication result.

[0140] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a voiceprint authentication method based on multi-node parameter aggregation on the server side.

[0141] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a voiceprint authentication method based on multi-node parameter aggregation.

[0142] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Acquire multi-channel audio signals containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and process the multi-channel audio signals using array noise reduction to obtain noise-reduced audio signals; The voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal are extracted, and the voiceprint features, physiological acoustic features, and content features are fused to obtain the fused features; A random interaction command is generated, the response audio signal of the object to be verified in response to the random interaction command is obtained, and a liveness analysis is performed based on the response audio signal and the reference dynamic characteristic data to obtain a liveness determination result; Text risk features are generated based on the content features, and emotional anomaly values ​​are generated based on the noise-reduced frequency signal. Fraud warning judgment results are determined based on the text risk features and the emotional anomaly values. The fused features are input into a voiceprint recognition model jointly trained using a multi-node parameter aggregation method to obtain the identity recognition result. The voiceprint authentication result is determined based on the identity recognition result, the liveness detection result, and the fraud warning result.

[0143] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: Acquire multi-channel audio signals containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and process the multi-channel audio signals using array noise reduction to obtain noise-reduced audio signals; The voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal are extracted, and the voiceprint features, physiological acoustic features, and content features are fused to obtain the fused features; A random interaction command is generated, the response audio signal of the object to be verified in response to the random interaction command is obtained, and a liveness analysis is performed based on the response audio signal and the reference dynamic characteristic data to obtain a liveness determination result; Text risk features are generated based on the content features, and emotional anomaly values ​​are generated based on the noise-reduced frequency signal. Fraud warning judgment results are determined based on the text risk features and the emotional anomaly values. The fused features are input into a voiceprint recognition model jointly trained using a multi-node parameter aggregation method to obtain the identity recognition result. The voiceprint authentication result is determined based on the identity recognition result, the liveness detection result, and the fraud warning result.

[0144] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0145] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0147] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0148] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A voiceprint authentication method based on multi-node parameter aggregation, characterized in that, Includes the following steps: Acquire multi-channel audio signals containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and process the multi-channel audio signals using array noise reduction to obtain noise-reduced audio signals; The voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal are extracted, and the voiceprint features, physiological acoustic features, and content features are fused to obtain the fused features; A random interaction command is generated, the response audio signal of the object to be verified in response to the random interaction command is obtained, and a liveness analysis is performed based on the response audio signal and the reference dynamic characteristic data to obtain a liveness determination result; Text risk features are generated based on the content features, and emotional anomaly values ​​are generated based on the noise-reduced frequency signal. Fraud warning judgment results are determined based on the text risk features and the emotional anomaly values. The fused features are input into a voiceprint recognition model jointly trained using a multi-node parameter aggregation method to obtain the identity recognition result. The voiceprint authentication result is determined based on the identity recognition result, the liveness detection result, and the fraud warning result.

2. The voiceprint authentication method based on multi-node parameter aggregation as described in claim 1, characterized in that, Acquire multi-channel audio signals containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and process the multi-channel audio signals using array denoising to obtain denoised audio signals, including: Read the historical fundamental frequency sequence and historical respiratory interval sequence corresponding to the object to be verified from the benchmark feature library. Generate fundamental frequency benchmark center data and fundamental frequency benchmark discrete data based on the historical fundamental frequency sequence. Generate respiratory interval benchmark center data and respiratory interval benchmark discrete data based on the historical respiratory interval sequence. Generate benchmark dynamic feature data based on the fundamental frequency benchmark center data, the fundamental frequency benchmark discrete data, the respiratory interval benchmark center data, and the respiratory interval benchmark discrete data. The voice of the object to be verified is collected by a microphone array to obtain channel audio data corresponding to multiple acquisition channels, and the sampling time mark corresponding to the channel audio data is recorded. The channel audio data is time-aligned according to the sampling time mark to generate a multi-channel audio signal. Based on the multi-channel audio signal, speech activity segments and non-speech segments are divided. Noise frequency domain data of each acquisition channel is extracted from the non-speech segments, and a noise covariance matrix is ​​generated based on the noise frequency domain data. Based on the speech activity segment, the time difference of arrival between each acquisition channel is extracted, and the target sound source direction is generated based on the time difference of arrival and the array element position data of the microphone array. The target steering vector is generated based on the target sound source direction and the array element position data. Based on the noise covariance matrix and the target steering vector, a spatial filter weight vector is generated under the distortion-free constraint of the target sound source direction. The multi-channel audio signal is frequency-domain transformed to generate multi-channel frequency-domain data. The multi-channel frequency-domain data is then weighted and synthesized using the spatial filtering weight vector. Finally, the weighted synthesis result is reconstructed in the time domain to obtain a noise-reduced audio signal.

3. The voiceprint authentication method based on multi-node parameter aggregation as described in claim 1, characterized in that, Extract the voiceprint features, physiological acoustic features, and content features from the noise-reduced audio signal, and fuse the voiceprint features, physiological acoustic features, and content features to obtain fused features, including: The noise-reduced frequency signal is framed and frequency domain transformed to generate spectral energy data. The spectral energy data is then subjected to filter bank mapping, logarithmic transformation, and discrete cosine transform to generate a cepstral feature sequence. Based on the cepstral feature sequence, voiceprint features are generated. Autocorrelation analysis is performed on the noise-reduced frequency signal to generate fundamental frequency data, and fundamental frequency data is generated based on the fundamental frequency data. Respiratory interval data is generated based on the energy envelope change of the noise-reduced frequency signal, and physiological acoustic features are generated based on the fundamental frequency data and the respiratory interval data. The noise-reduced frequency signal is transcribed into speech to generate transcribed text, and the transcribed text is then semantically encoded to generate content features. The voiceprint features, physiological acoustic features, and content features are respectively mapped to a unified feature space to generate mapped voiceprint features, mapped physiological acoustic features, and mapped content features. Fusion weights are generated based on the feature interactions between the mapped voiceprint features, mapped physiological acoustic features, and mapped content features. The mapped voiceprint features, mapped physiological acoustic features, and mapped content features are then weighted and fused based on the fusion weights to obtain fused features.

4. The voiceprint authentication method based on multi-node parameter aggregation as described in claim 1, characterized in that, A random interaction command is generated, and the response audio signal of the object to be verified in response to the random interaction command is obtained. Based on the response audio signal and the baseline dynamic characteristic data, a liveness analysis is performed to obtain a liveness determination result, including: Candidate interactive content is extracted from a preset interactive content library, target interactive content is randomly selected from the candidate interactive content, and random interactive instructions and interactive response conditions are generated based on the target interactive content; Output the random interaction command, collect the audio data of the object to be verified in response to the random interaction command, and generate a reply audio signal; The response audio signal is transcribed into speech to generate response text, and interactive response data is generated based on the matching relationship between the response text and the interactive response conditions. Autocorrelation analysis is performed on the response audio signal to generate real-time fundamental frequency data, and real-time respiratory interval data is generated based on the energy envelope change of the response audio signal. The fundamental frequency reference center data, fundamental frequency reference discrete data, respiratory interval reference center data, and respiratory interval reference discrete data are extracted from the reference dynamic characteristic data. Based on the fundamental frequency reference center data and the fundamental frequency reference discrete data, the real-time fundamental frequency data is standardized and differentially processed to generate fundamental frequency standardized differential data. Based on the reference center data and the reference discrete data of the respiratory interval, the real-time respiratory interval data is standardized and differentially processed to generate standardized differential data of the respiratory interval, and a dynamic feature distance is generated based on the standardized differential data of the fundamental frequency and the standardized differential data of the respiratory interval. A liveness determination result is obtained by jointly determining the interaction response data and the dynamic feature distance.

5. The voiceprint authentication method based on multi-node parameter aggregation as described in claim 1, characterized in that, Based on the content features, text risk features are generated, and based on the reduced noise frequency signal, sentiment anomaly values ​​are generated. A fraud warning judgment result is determined based on the text risk features and the sentiment anomaly values, including: Based on the content features, candidate risk terms are identified, term frequency data corresponding to the candidate risk terms are generated, and corpus distribution data corresponding to the candidate risk terms is generated based on preset corpus data. Term weights are generated based on the term frequency data and the corpus distribution data. Extract the term risk coefficients corresponding to the candidate risk terms from the preset risk feature library, and generate text risk features based on the term weights and term risk coefficients; Pitch variation data, energy variation data, and speech rate variation data are extracted from the noise-reduced frequency signal, and emotional acoustic features are generated based on the pitch variation data, energy variation data, and speech rate variation data. Obtain baseline emotion feature data, and generate emotion anomaly values ​​based on the feature differences between the emotion acoustic features and the baseline emotion feature data; Fraud risk data is generated based on the text risk features and the emotional anomalies, and the fraud warning judgment result is determined based on the fraud risk data.

6. The voiceprint authentication method based on multi-node parameter aggregation as described in claim 1, characterized in that, The fused features are input into a voiceprint recognition model jointly trained using a multi-node parameter aggregation method to obtain identity recognition results, including: Obtain the number of training samples corresponding to multiple training nodes, and send the initial model parameters to each training node; Receive node model parameters generated by each training node after local training based on the initial model parameters, identity tags, and local fusion features generated from their respective saved local training speech samples; Node aggregation weights are generated based on the number of training samples corresponding to each training node, and the node model parameters are aggregated based on the node aggregation weights to generate aggregated model parameters. The aggregated model parameters are sent to each training node, and the updated node model parameters generated by each training node after continuing local training based on the aggregated model parameters, their respective local fusion features and identity tags are received. Based on the node aggregation weights, the updated node model parameters are aggregated to generate updated aggregated model parameters. The updated aggregated model parameters are sent to each training node. The steps of receiving updated node model parameters, aggregating updated node model parameters based on node aggregation weights, generating updated aggregated model parameters, and sending updated aggregated model parameters are repeated until a preset training termination condition is met. The updated aggregated model parameters that meet the preset training termination condition are used as model parameters to obtain the voiceprint recognition model. The fused features are input into the voiceprint recognition model for feature encoding and identity recognition processing to obtain the identity recognition result.

7. The voiceprint authentication method based on multi-node parameter aggregation as described in claim 1, characterized in that, The voiceprint authentication result is determined based on the identity recognition result, the liveness detection result, and the fraud warning result, including: The identity recognition result is matched with preset identity determination conditions to generate an identity determination status; The liveness determination result is matched with the preset liveness determination conditions to generate a liveness determination status; The fraud warning determination result is matched with the preset fraud determination conditions to generate a fraud warning status; An authentication status combination is generated based on the identity determination status, the liveness determination status, and the fraud warning status. The authentication state combination is matched with the preset authentication state mapping relationship to obtain the voiceprint authentication result.

8. A voiceprint authentication device based on multi-node parameter aggregation, characterized in that, The voiceprint authentication device based on multi-node parameter aggregation includes: An array noise reduction module is used to acquire a multi-channel audio signal containing the speech of the object to be verified and the baseline dynamic feature data of the object to be verified, and to process the multi-channel audio signal using an array noise reduction method to obtain a noise-reduced audio signal. A multi-dimensional feature fusion module is used to extract the voiceprint features, physiological acoustic features, and content features of the noise-reduced frequency signal, and to fuse the voiceprint features, physiological acoustic features, and content features to obtain fused features; The liveness analysis module is used to generate random interaction commands, obtain the response audio signal of the object to be verified in response to the random interaction commands, and perform liveness analysis based on the response audio signal and the benchmark dynamic characteristic data to obtain a liveness determination result. The fraud warning module is used to generate text risk features based on the content features, generate emotional anomaly values ​​based on the noise-reduced frequency signal, and determine the fraud warning judgment result based on the text risk features and the emotional anomaly values. The voiceprint recognition module is used to input the fused features into the voiceprint recognition model jointly trained by a multi-node parameter aggregation method to obtain the identity recognition result; The authentication decision module is used to determine the voiceprint authentication result based on the identity recognition result, the liveness determination result, and the fraud warning determination result.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a voiceprint authentication program based on multi-node parameter aggregation stored in the memory and executable on the processor. When the voiceprint authentication program based on multi-node parameter aggregation is executed by the processor, it implements the steps of the voiceprint authentication method based on multi-node parameter aggregation as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a voiceprint authentication program based on multi-node parameter aggregation. When the voiceprint authentication program based on multi-node parameter aggregation is executed by the processor, it implements the steps of the voiceprint authentication method based on multi-node parameter aggregation as described in any one of claims 1-7.

Images