Far-field voiceprint recognition method and device, electronic equipment and storage medium

Abstract

The invention provides a far-field voiceprint recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting sample near-field audio data, constructing a geometric acoustic propagation model containing a frequency band related reverberation model and a sound source and receiving equipment directionality model based on the data, and generating a room pulse response function; performing convolution on sample near-field voice and the pulse response function to generate simulated far-field audio data; carrying out pre-training based on the sample near-field audio and carrying out fine tuning based on the simulated far-field audio by adopting a speaker representation weight pre-retention training strategy, and constructing a far-field voiceprint recognition model; and inputting the to-be-detected far-field audio after reverberation removal and the reference near-field audio into the model, calculating the similarity and outputting an identification result. The problems of scarcity of far-field voice data, poor model cross-domain adaptability, serious signal reverberation and the like are solved, and the accuracy and robustness of a far-field voiceprint recognition system are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of voiceprint recognition, in particular, to a far-field voiceprint recognition method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, voice-based human-computer interaction has been widely applied in smart home, intelligent security, remote conference, virtual assistant and other scenarios. As a key component of voice interaction system, voiceprint recognition (Speaker Recognition) undertakes important identity recognition and security authentication tasks. Current mainstream voiceprint recognition systems are mostly based on deep learning methods, which extract speaker features from voice signals by training neural networks and realize registration and recognition matching.

[0003] In a near-field environment, voiceprint recognition technology has achieved remarkable results, especially under the condition of fixed microphone and high signal-to-noise ratio, the recognition accuracy is high. However, as the application scenarios expand to open space, long-distance voice collection and other directions, far-field voiceprint recognition technology has attracted widespread attention. However, due to the influence of various factors such as signal attenuation, spatial reverberation, background noise interference, multi-path propagation and device differences during the propagation of far-field voice, its signal-to-noise ratio is much lower than that in the near-field environment, resulting in a significant decline in system recognition performance and insufficient robustness.

[0004] Current far-field voiceprint recognition mainly faces the following technical problems: existing public corpus is mostly near-field voice, lacking of far-field training data with real environment acoustic characteristics, which seriously restricts the training and optimization of the model. Existing models are mostly based on near-field voice, and their embedding space features are unstable in far-field, making it difficult to accurately capture speaker identity information. In actual systems, the registration audio is usually collected from high-quality near-field channels, while the test audio comes from far-field channels, which have significant differences in spectral structure, acoustic distribution and other aspects, resulting in a decline in cross-domain recognition accuracy. SUMMARY

[0005] The embodiments of the present disclosure at least provide a far-field voiceprint recognition method, device, electronic device and storage medium, which effectively solve the problems of insufficient training data, poor model cross-domain adaptability and serious reverberation noise interference in far-field voiceprint recognition by introducing acoustic space parameterization modeling, speaker representation weight pre-reserved training strategy and weighted prediction error dereverberation method. The generated far-field voice data has real spatial characteristics, and the trained model can quickly adapt to complex far-field scenarios, which overall improves the accuracy and robustness of the recognition system in the far-field environment, and meets the high-performance requirements of intelligent security, remote authentication and other practical applications.

[0006] The embodiments of the present disclosure provide a far-field voiceprint recognition method, comprising:

[0007] collecting sample near-field audio data in a room, constructing a geometric acoustic propagation model according to propagation characteristics of the sample near-field audio data, and determining a room impulse response function describing a frequency-domain sound field distribution;

[0008] constructing a reverberation model for independently calculating a reverberation time for each frequency band and reflecting differences in sound absorption characteristics of different materials at different frequencies, and a sound source and receiving device directivity model simulating spatial pickup characteristics of a microphone array;

[0009] performing time-domain convolution on the sample near-field audio data and the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity model, to generate simulated far-field audio data;

[0010] inputting the de-reverberated to-be-tested far-field audio data and reference near-field audio data into a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, a representation position of a far-field speaker class corresponding to the simulated far-field audio data is reserved in pre-training, and a weight vector corresponding to the far-field speaker class is non-randomly initialized according to discriminative feature knowledge learned in the pre-training stage in fine-tuning.

[0011] In an optional implementation, the geometric acoustic propagation model is constructed according to the propagation characteristics of the sample near-field audio data, specifically including:

[0012] for a high-frequency sound wave range, an image source method is adopted, virtual image sources are equivalent to reflected sound sources in the room according to the mirror reflection law based on room sizes and microphone positions corresponding to the sample near-field audio data, sound wave propagation path lengths between each virtual image source and the microphone positions are calculated, and sound wave propagation time delays and sound wave propagation attenuations are determined;

[0013] for a low-frequency sound wave range, a random ray tracing method is adopted, multiple beams of light rays are randomly emitted from sound source positions corresponding to the sample near-field audio data to simulate sound wave propagation and reflection in the room, light ray energies at the microphone positions are recorded, and sound pressure distributions at the microphone positions are determined by counting the light ray energy distributions;

[0014] the sound wave propagation time delays, the sound wave propagation attenuations, and the sound pressure distributions are taken as the propagation characteristics, and the geometric acoustic propagation model is constructed.

[0015] In an optional implementation, the room impulse response function describing the frequency-domain sound field distribution is determined, specifically including:

[0016] The Helmholtz wave equation is used to describe the sound field distribution in the space, and the Helmholtz wave equation is converted into a boundary integral equation;

[0017] Boundary conditions reflecting the room geometry and material acoustic characteristics are established, the Helmholtz equation is solved at each frequency point, and the frequency response between the sound source corresponding to the sample near-field audio data and the microphone receiving position is determined;

[0018] The transfer function generated in the solving process, which contains direct sound, early reflection and late reverberation components, is determined as the room impulse response function.

[0019] In an optional implementation, a reverberation model for independently calculating reverberation time for each frequency band and reflecting the differences in sound absorption characteristics of different materials at different frequencies is constructed, and a sound source and receiving device directivity model simulating the spatial pickup characteristics of a microphone array is constructed, specifically including:

[0020] The Eyring formula is replaced with a sound absorption coefficient matrix related to the frequency band instead of a single sound absorption coefficient in the Sabine formula to construct a multi-band model;

[0021] In the multi-band model, the reverberation time is independently calculated for each frequency band according to the differences in sound absorption characteristics of different materials at different frequencies, the frequency-selective reverberation characteristics in a real environment are simulated, and the reverberation model is generated;

[0022] The directivity pattern of the sound source to the microphone corresponding to the sample near-field audio data is represented as a linear combination of spherical harmonics, the directional sound field distribution in three-dimensional space is characterized by spherical harmonic series expansion, and the complete three-dimensional sound field is described to generate the sound source and receiving device directivity model.

[0023] In an optional implementation, after constructing a reverberation model for independently calculating reverberation time for each frequency band and reflecting the differences in sound absorption characteristics of different materials at different frequencies, and constructing a sound source and receiving device directivity model simulating the spatial pickup characteristics of a microphone array, the method further includes:

[0024] A Lambert cosine law is used to establish a mathematical relationship between the reflected sound intensity and the cosine value of the angle between the incident sound wave and the surface normal, and a statistical diffusion model is constructed;

[0025] For each sound wave reflection point of the sample near-field audio data, the basic reflection intensity is calculated according to the statistical diffusion model, and the distribution characteristics of the reflected energy in each direction are determined according to the scattering technology of the material corresponding to the sound wave reflection point.

[0026] In an optional implementation, the far-field voiceprint recognition model is trained based on the following steps:

[0027] pre-training a preset far-field voiceprint recognition model based on the sample near-field audio data, and constructing a complete embedding space including a near-field speaker category corresponding to the sample near-field audio data and a far-field speaker category corresponding to the simulated far-field audio data;

[0028] learning domain-independent speaker discriminative feature knowledge through pre-training, and reserving a representation position for the far-field speaker category through implicit representation learning in a projection layer;

[0029] fine-tuning the preset far-field voiceprint recognition model based on the simulated far-field audio data, and directly initializing a weight vector corresponding to the far-field speaker category in the projection layer according to the domain-independent speaker discriminative feature knowledge.

[0030] In an optional implementation, the reverberation-removed processing is performed on the to-be-tested far-field audio data based on the following steps:

[0031] constructing a model for decomposing the to-be-tested far-field audio data into a target signal and a reverberation component, describing the reverberation signal using a delay linear prediction, and representing the reverberation component as a weighted sum of past observation signals according to a reverberation-removed filter coefficient and a prediction delay;

[0032] estimating the reverberation-removed filter coefficient by minimizing a weighted prediction error, and using an iterative optimization algorithm to calculate the weighted prediction error using the estimated reverberation-removed filter coefficient in each iteration, updating the reverberation-removed filter coefficient according to the weighted prediction error until the error converges, and determining an optimal reverberation-removed filter coefficient;

[0033] removing the reverberation component in the to-be-tested far-field audio data using a reverberation-removed filter having the optimal reverberation-removed filter coefficient.

[0034] The embodiments of the present disclosure further provide a far-field voiceprint recognition device, and the device comprises:

[0035] a first acoustic space parameterization modeling module configured to collect sample near-field audio data in a room, construct a geometric acoustic propagation model according to a propagation feature of the sample near-field audio data, and determine a room impulse response function describing a frequency domain sound field distribution;

[0036] a second acoustic space parameterization modeling module configured to construct a reverberation model for calculating a reverberation time independently for each frequency band and reflecting differences in sound absorption characteristics of different materials at different frequencies, and a sound source and receiving device directivity model for simulating spatial pickup characteristics of a microphone array;

[0037] The far-field acoustic data simulation generation module is configured to generate simulated far-field audio data by time-domain convolution of the sample near-field audio data and the room impulse response function based on superposition of the geometric acoustic propagation model and the reverberation model and a sound source and receiving device directivity model;

[0038] The identification module is configured to input the de-reverberated to-be-tested far-field audio data and the reference near-field audio data into a trained far-field voiceprint identification model to determine a far-field voiceprint identification result, wherein the far-field voiceprint identification model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, a representation position of a far-field speaker class corresponding to the simulated far-field audio data is reserved in the pre-training, and a weight vector corresponding to the far-field speaker class is non-randomly initialized according to discriminative feature knowledge learned in the pre-training stage in the fine-tuning.

[0039] The embodiments of the present disclosure further provide an electronic device, including a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the above far-field voiceprint identification method or steps in any possible implementation manner of the above far-field voiceprint identification method.

[0040] The embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the above far-field voiceprint identification method or steps in any possible implementation manner of the above far-field voiceprint identification method.

[0041] The embodiments of the present disclosure further provide a computer program product, which includes a computer program / instruction, and the computer program / instruction is executed by a processor to implement the above far-field voiceprint identification method or steps in any possible implementation manner of the above far-field voiceprint identification method.

[0042] The method, device, electronic device, and storage medium provided by the embodiments of the present disclosure collect sample near-field audio data in a room, construct a geometric acoustic propagation model according to the propagation characteristics of the sample near-field audio data, and determine a room impulse response function describing the distribution of a frequency domain sound field; construct a reverberation model for independently calculating a reverberation time for each frequency band and reflecting the differences in sound absorption characteristics of different materials at different frequencies, and a sound source and receiving device directivity model simulating the spatial pickup characteristics of a microphone array; perform time domain convolution on the sample near-field audio data and the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity model, to generate simulated far-field audio data; and input the de-reverberated to-be-tested far-field audio data and reference near-field audio data into a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data, fine-tuned based on the simulated far-field audio data, and in the pre-training, a representation position of a far-field speaker category corresponding to the simulated far-field audio data is reserved, and in the fine-tuning, a non-random initialization is performed on a weight vector corresponding to the far-field speaker category based on discriminative feature knowledge learned in the pre-training stage. By introducing the acoustic space parameterization modeling, the speaker representation weight pre-reservation training strategy, and the weighted prediction error de-reverberation method, the problems of insufficient training data, poor model cross-domain adaptability, and serious reverberation noise interference in far-field voiceprint recognition are effectively solved. The generated far-field speech data has real spatial characteristics, the training model can quickly adapt to complex far-field scenes, the accuracy and robustness of the recognition system in a far-field environment are improved as a whole, and the high-performance requirements of practical applications such as intelligent security and remote authentication are met.

[0043] In order to make the above objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the specification and form a part of the specification, which illustrate the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show certain embodiments of the present disclosure, and therefore should not be regarded as a limitation on the scope, and other related drawings can also be obtained by those of ordinary skill in the art without creative labor.

[0045] Figure 1 A flowchart of a far-field voiceprint recognition method provided by the embodiments of the present disclosure is shown;

[0046] Figure 2 A flowchart of a method for training a far-field voiceprint recognition model is shown.

[0047] Figure 3 A schematic diagram of a far-field voiceprint recognition device is shown.

[0048] Figure 4 A schematic diagram of an electronic device is shown. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings for the embodiments of the present disclosure to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The components of the embodiments of the present disclosure described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0050] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0051] The term “and / or” herein only describes an association relationship, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term “at least one” herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0052] It is found through research that the current far-field voiceprint recognition mainly faces the following technical difficulties: the existing public corpus is mostly near-field speech, and there is a lack of far-field training data with real environment acoustic characteristics, which seriously restricts the training and optimization of the model. The existing model is mostly constructed based on near-field speech, and the embedding space features thereof show instability under far-field, and it is difficult to accurately capture speaker identity information. In actual systems, the registration audio is usually collected from a high-quality near-field channel, while the test audio comes from a far-field channel, and the two are significantly different in spectral structure, acoustic distribution, etc., resulting in a decrease in cross-domain recognition accuracy.

[0053] Based on the above research, the present disclosure provides a far-field voiceprint recognition method, device, electronic equipment and storage medium. Sample near-field audio data is collected, a geometric acoustic propagation model is constructed according to the propagation characteristics of the sample near-field audio data, and a room impulse response function describing the frequency domain sound field distribution is determined; a reverberation model for independently calculating reverberation time for each frequency band and reflecting the sound absorption characteristic differences of different materials at different frequencies is constructed, and a sound source and receiving device directivity model simulating the spatial pickup characteristics of a microphone array is constructed; the sample near-field audio data is time-domain convolved with the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity model, to generate simulated far-field audio data; and the de-reverberated to-be-tested far-field audio data and the reference near-field audio data are input into a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result. The far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, a representation position of a far-field speaker class corresponding to the simulated far-field audio data is reserved in pre-training, and a non-random initialization is performed on a weight vector corresponding to the far-field speaker class in pre-reservation according to discriminative feature knowledge learned in the pre-training stage. By introducing acoustic space parameterization modeling, speaker representation weight pre-reservation training strategy and weighted prediction error de-reverberation method, the problems of insufficient training data, poor model cross-domain adaptability and serious reverberation noise interference in far-field voiceprint recognition are effectively solved. The generated far-field speech data has real spatial characteristics, the training model can quickly adapt to complex far-field scenes, the accuracy and robustness of the recognition system in a far-field environment are overall improved, and the high-performance requirements of actual applications such as intelligent security and remote authentication are met.

[0054] To facilitate the understanding of the present embodiment, first, a far-field voiceprint recognition method disclosed by the present embodiment is introduced in detail. The execution subject of the far-field voiceprint recognition method provided by the present embodiment is generally a computer device with certain computing power, which may, for example, include a terminal device or a server or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the far-field voiceprint recognition method may be realized by a processor invoking computer readable instructions stored in a memory.

[0055] Referring to Figure 1 FIG. 1 shows a flowchart of a far-field voiceprint recognition method provided by the present embodiment, and the method includes steps S101-S104, wherein:

[0056] S101, collect sample near-field audio data in a room, construct a geometric acoustic propagation model according to the propagation characteristics of the sample near-field audio data, and determine a room impulse response function describing the frequency domain sound field distribution.

[0057] In a specific implementation, first, sample near-field audio data in the target room environment is collected, which can be obtained by playing a reference voice at a standard sound source position and collecting at a close distance position, to simulate the original signal characteristics of the sound source in anechoic or weak reverberation conditions. Subsequently, based on the characteristics of the collected near-field audio signal during spatial propagation, a geometric acoustic propagation model is constructed to describe the propagation behavior of the voice signal. The geometric acoustic propagation model applies the mirror reflection law to calculate early reflections in the high frequency region, and combines wave phenomena to accurately capture frequency-dependent acoustic characteristics in the low frequency region.

[0058] Here, the geometric acoustic propagation model adopts a hybrid modeling method combining image source method and random ray tracing method, wherein the image source method is used to simulate the mirror reflection path of sound waves in the high frequency region, and the random ray tracing method is used to describe the complex propagation phenomena such as multi-path propagation and scattering reflection in the low frequency region. Through this geometric acoustic model, the path characteristics and time delay characteristics of the sound source signal propagating and reflecting between different surfaces in the room can be more comprehensively restored.

[0059] Specifically, for the high frequency sound wave range, the image source method is used, and according to the room size and microphone position corresponding to the collected sample near-field audio data, the mirror reflection law is used to equivalent the reflected sound source in the room as a virtual image source, calculate the sound wave propagation path length between each virtual image source and the microphone position, determine the sound wave propagation time delay and sound wave propagation attenuation; for the low frequency sound wave range, the random ray tracing method is used, a plurality of beams of light are randomly emitted from the sound source position corresponding to the sample near-field audio data to simulate the propagation and reflection of sound waves in the room, and the light energy at the microphone position is recorded to determine the sound pressure distribution at the microphone position. The sound wave propagation time delay, sound wave propagation attenuation and sound pressure distribution are used as the propagation characteristics to construct the geometric acoustic propagation model.

[0060] In a specific implementation, to construct a geometric acoustic propagation model that accurately simulates the sound wave propagation behavior in the room, first, according to the sound source position, microphone position and geometric size of the room corresponding to the collected sample near-field audio data, different modeling methods are used to obtain acoustic propagation characteristics for high frequency and low frequency sound wave ranges.

[0061] Here, for the high-frequency sound wave propagation range, an image source method is used for modeling. According to the mirror reflection law, each reflected sound source in the room is equivalent to a virtual image source. Based on the relative position of the sound source and the microphone in the three-dimensional space and the size boundary information of the room, the straight-line propagation path length between each virtual image source and the microphone is calculated, and the corresponding sound wave propagation time delay and the sound wave propagation attenuation caused by the propagation loss on the propagation path are further determined.

[0062] Here, for the low-frequency sound wave propagation range, a random ray tracing method is used for modeling. Starting from the sound source position corresponding to the sample near-field audio data, multiple beams of light are randomly emitted to simulate the sound wave propagation behavior in all directions in the room space. Each beam of light is reflected by the boundary surface such as the wall, ceiling or floor during propagation, and the energy attenuation calculation is performed according to the sound absorption coefficient of different materials. When the light propagates to the microphone position, the arrival time and the carried energy are recorded. By collecting a large number of light paths and statistically analyzing the energy at the microphone position, the sound pressure level distribution characteristics at the microphone position are obtained, thereby reflecting the scattering and attenuation behavior of the low-frequency band in the complex space.

[0063] Specifically, the image source position of the rectangular room is calculated by mirror reflection. For a three-dimensional room, the position of the image source can be represented by the following formula:

[0064]

[0065] Where (x s , y s , z s ) represents the sound source position, (x r , y r , z r ) represents the room size, and (x m , y m , z m ) represents the microphone position. By calculating the straight-line distance from each image source to the receiving point, i.e. the length of the sound wave propagation path, the sound wave propagation time delay and attenuation are obtained.

[0066] In this way, based on the coupling algorithm of the image source method and the random ray tracing method, a complete acoustic energy propagation path and its time distribution characteristics are constructed. In terms of space construction, the model accurately calculates the early reflected sound by the image source method, effectively handles high-frequency sound waves using the mirror reflection law. At the same time, for complex late reflected sound propagation, the random ray tracing method is used, especially in the low-frequency region combined with wave phenomena simulation to accurately capture frequency-dependent acoustic characteristics, thereby realizing high-fidelity near-field to far-field audio conversion.

[0067] Further, a room impulse response function describing the sound field distribution in the room space is calculated. The response function combines the geometric structure of the room, the sound absorption characteristics of the surface materials, and the spatial layout of the sound source and the microphone. Based on the Helmholtz wave equation and the boundary element method, the frequency response information between the given sound source position and the receiving point is obtained by numerical solution. The room impulse response function not only contains the direct sound component, but also includes early reflections and late reverberation components, which can be used for subsequent convolution conversion of near-field speech signals to simulated speech data with far-field acoustic characteristics.

[0068] Specifically, the Helmholtz wave equation is used to describe the sound field distribution in the space, and the Helmholtz wave equation is converted into a boundary integral equation. Boundary conditions reflecting the geometric and material acoustic characteristics of the room are established, and the Helmholtz equation is solved at each frequency point to determine the frequency response between the sound source and the microphone receiving position corresponding to the sample near-field audio data. The transfer function generated during the solving process, which includes direct sound, early reflections and late reverberation components, is determined as the room impulse response function.

[0069] Here, the Helmholtz wave equation is used to describe the propagation of sound waves in the frequency domain, and the room acoustic problem is represented as a solution problem of the Helmholtz equation. Its mathematical form can be expressed as the following formula:

[0070]

[0071] where, is the Laplace operator; p(r) represents the sound pressure at the current time at the spatial position r; k = ω / c is the wave number, ω represents the angular frequency, and c represents the sound speed.

[0072] Here, the boundary element method is a method of discretizing the boundary into boundary elements and using boundary integral equations for numerical analysis. It converts the Helmholtz equation into a boundary integral equation, and discretizes the boundary of the room without the need for meshing the entire three-dimensional space. Further, the solution of the Helmholtz equation can be expressed as the following formula:

[0073]

[0074] where, G(r, r s ) represents the free space Green function, S represents the boundary surface, represents the derivative along the normal direction of the boundary.

[0075] In specific implementation, the Helmholtz equation constructs an accurate description of the sound field distribution in the frequency domain to capture the modal resonance behavior in the low frequency region. The boundary element method first establishes boundary conditions reflecting the geometry and material acoustic characteristics of the room, and then solves the Helmholtz equation at each frequency point to obtain the frequency response between the sound source and the microphone receiving position. The transfer function generated by this process contains direct sound, early reflections and late reverberation components, fully characterizing the acoustic characteristics of the sound source in a specific room, and finally converting the near-field speech into a far-field signal with spatial characteristics through convolution.

[0076] S102, construct a reverberation model for independent calculation of reverberation time for each frequency band and reflect the difference in sound absorption characteristics of different materials at different frequencies, and simulate the directivity model of the sound source and receiving device of the microphone array spatial pickup characteristics.

[0077] In specific implementation, to further improve the physical authenticity and spectral consistency of far-field speech modeling, a frequency band-dependent reverberation model is constructed to reflect the sound absorption difference of different frequency bands, and a sound source and receiving device directivity model is constructed to accurately simulate the spatial pickup effect.

[0078] Specifically, in terms of reverberation modeling, based on the extension of traditional Sabine and Eyring reverberation theory, a multi-band parameterized reverberation model supporting frequency band-independent reverberation time calculation is constructed. This model introduces a frequency-dependent sound absorption coefficient matrix to model the absorption ability of different materials in the room to different frequency sound waves. For each frequency band, its corresponding reverberation time is calculated respectively, and factors such as room volume, surface area, material sound absorption coefficient and sound wave incidence angle are considered in the specific calculation process. In this way, the model can accurately reflect the frequency-selective reverberation characteristics that low frequencies usually have longer reverberation time and high frequencies decay rapidly in actual environments, thereby improving the simulation accuracy of far-field speech in spectral structure.

[0079] At the same time, to truly restore the propagation and reception characteristics of speech in space, the present application further constructs a directivity model of the sound source and the microphone array, which is used to describe the pickup and radiation characteristics of the device in different directions and different frequencies. This model constructs a three-dimensional spatial directivity sound field distribution based on the Spherical Harmonics expansion method, and for each microphone channel or sound source, its directivity pattern in spherical coordinates is represented by a linear combination of spherical harmonics. In the modeling process, independent spherical harmonic coefficient matrices are assigned to different frequency bands to support frequency-dependent directivity changes, thereby accurately describing spatial characteristics such as high-frequency directivity and low-frequency omnidirectional propagation.

[0080] As a possible implementation, the Eyring formula is constructed by replacing the single absorption coefficient in the Sabine formula with a frequency band-related absorption coefficient matrix. In the multi-band model, the reverberation time is calculated independently for each frequency band according to the differences in sound absorption characteristics of different materials at different frequencies, simulating the frequency-selective reverberation characteristics in the real environment to generate a reverberation model. The directivity pattern of the sample near-field audio data from the sound source to the microphone is represented as a linear combination of spherical harmonics, which characterizes the directional sound field distribution in three-dimensional space through spherical harmonic series expansion, and describes the complete three-dimensional sound field to generate a sound source and receiving device directivity model.

[0081] Here, Sabine is a classic reverberation time prediction formula in room acoustics, which represents the time required for sound energy to decay by 60 dB, as shown in the following formula:

[0082]

[0083] where V represents the room volume, S i represents the area of the i-th surface, and ɑ i represents the absorption coefficient of the i-th surface.

[0084] Here, Eyring is an improvement on the Sabine formula, which considers that the decay of sound energy in the room follows an exponential law, as shown in the following formula:

[0085]

[0086] where S represents the total surface area of the room, and ɑ is the average absorption coefficient.

[0087] Specifically, based on the above formula, the single absorption coefficient is replaced by a frequency-dependent absorption coefficient matrix α(f) to construct a multi-band model, where f represents different frequency bands. The reverberation time is calculated independently for each frequency band, taking into account the differences in sound absorption characteristics of different materials at different frequencies, thereby accurately controlling the energy decay characteristics of different frequency bands. This extension enables the model to simulate the frequency-selective reverberation characteristics in the real environment, such as the fact that low-frequency reverberation time is usually longer than high-frequency reverberation time, more accurately reproducing the room acoustics performance, so that the generated far-field speech has a spectral energy distribution consistent with actual far-field recordings, achieving high-fidelity simulation of room acoustics characteristics with unique decay characteristics for different frequency bands.

[0088] Further, high-order sound source and microphone directivity modeling based on spherical harmonics (Spherical Harmonics) functions is used to simulate the spatial pickup characteristics of real microphone arrays. Spherical harmonics are a set of orthogonal basis functions on the sphere, defined as follows:

[0089]

[0090] where n represents the order, m represents the degree, θ represents the polar angle, and φ represents the azimuth angle, denotes the associated Legendre polynomial.

[0091] Here, based on the spherical harmonics, any directivity pattern D(θ, φ) can be expressed as a linear combination of spherical harmonics, and the spherical harmonic series expansion is used to represent the directional sound field distribution in three-dimensional space. The spherical harmonic order determines the complexity and accuracy of the directivity pattern, as shown in the following formula:

[0092]

[0093] wherein, is the expansion coefficient, which determines the characteristics of a specific directivity pattern.

[0094] Here, in the directivity modeling of the microphone array, the spherical harmonics are used as a complete orthogonal basis function system on the sphere. Through accurate spatial frequency domain decomposition, a beamformer with specific directivity is designed, and a complete three-dimensional sound field is described to achieve high-order sound field reconstruction. For sound source directivity representation, spherical harmonics can accurately depict the complex radiation pattern of different sound sources, and by assigning independent spherical harmonic coefficient matrices to different frequency bands, the frequency-dependent directivity change can be accurately simulated. This physical model-based sound field representation enables the far-field modeling to comprehensively consider the spatial filtering response characteristics of the receiving device, accurately reproduce the amplitude-frequency characteristics and phase-frequency characteristics of sound waves at different incident angles, and ensure that the generated audio signal has high consistency with the actual recorded far-field audio signal in terms of spatial and directional characteristics.

[0095] As a possible implementation, after constructing the reverberation model and the directivity model of the sound source and receiving device, in order to accurately simulate the frequency-selective attenuation problem of the far-field audio during propagation, air absorption, material reflection loss, and boundary diffusion effect also need to be considered. A statistical diffusion model is established using Lambert's cosine law to control the proportion of specular reflection and diffuse reflection through the scattering coefficient, and to achieve a mixed reflection characteristic closer to the real far field. Specifically, a mathematical relationship between the reflected sound intensity and the cosine value of the angle between the incident sound wave and the surface normal is established using Lambert's cosine law to construct a statistical diffusion model. For each sound wave reflection point of the sample near-field audio data, the basic reflection intensity is calculated according to the statistical diffusion model, and the distribution characteristics of the reflected energy in each direction are determined according to the scattering technology of the material corresponding to the sound wave reflection point.

[0096] In specific implementation, Lambert's cosine law describes the relationship between the reflected light intensity and the angle between the incident light direction and the surface normal in diffuse reflection. Specifically, the intensity of the reflected light is proportional to the cosine of the angle between the incident light direction and the surface normal, as shown in the following formula:

[0097] I(θ) = I0 x cos(θ)

[0098] where I(θ) is the intensity observed at angle θ, I0 is the intensity normal to the surface, and θ is the angle between the observer's line of sight and the surface normal.

[0099] Here, in the statistical diffusion model of audio propagation, Lambert's cosine law is an important theoretical tool for simulating the reflection and diffusion behavior of sound waves in complex environments. When sound waves encounter various surfaces, Lambert's cosine law establishes a mathematical relationship between the reflected sound intensity and the cosine of the angle between the incident sound wave and the surface normal, accurately predicting the reflected intensity distribution at different angles. In real acoustic environments, sound wave reflection exhibits a mixed characteristic of specular reflection and diffuse reflection. By introducing the scattering coefficient as a key parameter, the proportion of the two reflection modes can be effectively controlled, thereby constructing a more realistic mixed reflection model.

[0100] where the statistical model based on Lambert's cosine law and the scattering coefficient describes the reflection and diffusion behavior of sound waves as a random process with probability characteristics, and its reflected intensity distribution follows a specific probability density function. In simulation implementation, for each sound wave reflection point, the model first calculates the basic reflection intensity according to Lambert's cosine law, and then determines the distribution characteristics of reflected energy in various directions according to the scattering coefficient of the material. This statistical method based on physical principles can more realistically reproduce the complex reflection characteristics in far-field acoustic environments, providing a computational framework for accurately simulating the propagation rules and frequency-selective attenuation of sound waves in various complex environments.

[0101] S103, based on superimposing the reverberation model and the geometric acoustic propagation model of the sound source and the receiving device directivity model, time-domain convolution of the sample near-field audio data and the room impulse response function to generate simulated far-field audio data.

[0102] In specific implementation, to generate simulated far-field speech data with real spatial propagation characteristics, based on the constructed geometric acoustic propagation model, sample near-field audio data is processed with a room impulse response function that describes spatial propagation characteristics. The entire simulation process is realized by time-domain convolution of the original near-field speech signal with the room transfer function calculated through spherical harmonic function expansion, ultimately synthesizing a speech signal with real far-field characteristics at the target microphone position, providing high-fidelity training and evaluation data for far-field voiceprint recognition methods.

[0103] Here, first, the sound wave propagation process in the room is modeled with high fidelity by the aforementioned geometric acoustic propagation model, which integrates a multi-band reverberation model independently modeled for each frequency band and can accurately reflect the sound absorption performance differences of different materials at different frequencies; a directional model of the sound source and receiving device based on spherical harmonics is used to simulate the spatial directivity response of the real microphone array and sound source.

[0104] On this basis, the room impulse response function from the sound source to the microphone position in the target space is solved according to the room structure and the position parameters of the sound source and microphone, which comprehensively contains the direct sound, early reflection sound and late reverberation components, and expresses the frequency domain propagation behavior in time series form.

[0105] Subsequently, the collected sample near-field audio data is time-domain convolved with the above room impulse response function to simulate the actual process of the original speech signal propagating in the room space to the target microphone, thereby generating simulated far-field audio data with real far-field acoustic characteristics.

[0106] It should be noted that the simulated far-field audio not only fully embodies the acoustic propagation law in the real scene in terms of energy attenuation, spectral distortion and reverberation delay, but also accurately reproduces the device directional characteristics in terms of spatial directivity and phase structure, providing high-fidelity, physically consistent training samples for subsequent far-field voiceprint recognition model training.

[0107] For example, the simulation parameters of the far-field audio are set as follows: the room size is set to 6m x 6m x 2.5m, the average value of the frequency-dependent sound absorption coefficient of each interface is 0.4, the ratio of early reflection and late reverberation is controlled by the finite reflection order (N=3) to ensure that the generated room impulse response has moderate reverberation characteristics. The sound source is positioned at the spatial coordinates [3.0, 3.0, 1.5] m. The microphone uses an adaptive positioning strategy to dynamically adjust the spatial position according to different target distances.

[0108] S104, input the de-reverberated far-field audio data to be tested and the reference near-field audio data into the trained far-field voiceprint recognition model to determine the far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, and the representation position of the far-field speaker class corresponding to the simulated far-field audio data is reserved in the pre-training, and the weight vector corresponding to the far-field speaker class is non-randomly initialized according to the discriminative feature knowledge learned in the pre-training stage in the fine-tuning.

[0109] In specific implementation, to achieve accurate recognition of speakers in far-field audio, a two-stage training mechanism based on pre-training and fine-tuning is adopted to construct a far-field voiceprint recognition model, and a reverberation removal process and embedding comparison are performed to complete the final recognition task.

[0110] Specifically, first, the to-be-tested far-field audio data after reverberation removal and the corresponding reference near-field audio data are input into the trained far-field voiceprint recognition model respectively, and the corresponding speaker embedding vectors are extracted; then the similarity (such as cosine similarity) between the two embedding vectors is calculated and compared with a preset recognition threshold, so as to determine whether the two speeches come from the same speaker, and obtain the final far-field voiceprint recognition result.

[0111] Here, referring to Figure 2 The method comprises steps S201-S203, wherein:

[0112] S201, pre-training a preset far-field voiceprint recognition model based on the sample near-field audio data, and constructing a complete embedding space comprising a near-field speaker category corresponding to the sample near-field audio data and a far-field speaker category corresponding to the simulated far-field audio data;

[0113] S202, learning domain-independent speaker discriminative feature knowledge through pre-training, and reserving a representation position for the far-field speaker category through implicit representation learning in a projection layer.

[0114] S203, fine-tuning the preset far-field voiceprint recognition model based on the simulated far-field audio data, and directly initializing a weight vector corresponding to the far-field speaker category in the projection layer according to the domain-independent speaker discriminative feature knowledge.

[0115] In specific implementation, in the pre-training stage of the first stage, large-scale speaker embedding modeling is performed based on the collected sample near-field audio data. In this stage, the model reserves a representation position for the far-field speaker category corresponding to the simulated far-field audio data when constructing the embedding space. Even though no positive sample data of these far-field categories is provided in the pre-training stage, the model forms a discriminative boundary suitable for the far-field categories through implicit feature learning. In the fine-tuning stage of the second stage, the simulated far-field audio data generated through geometric acoustic modeling is used as training sample to perform far-field adaptability optimization while keeping the model structure stable. In this process, the model performs non-random initialization on the previously reserved weight vector of the far-field speaker category based on the domain-independent discriminative feature knowledge learned in the pre-training stage, so that the model can quickly converge to the optimal representation space of the far-field recognition task with a smaller learning rate, thereby significantly improving the recognition performance and robustness of the model under real far-field conditions.

[0116] Here, assuming that the simulated far-field fine-tuning training dataset contains X speaker categories, and the near-field pre-training training dataset contains Y speaker categories, the conventional method usually only learns Y-class representations in the pre-training stage, and randomly initializes the projection layer weights for the newly added x classes in the fine-tuning stage. This discontinuous knowledge transfer makes it difficult for the model parameters to effectively converge to the optimal representation space under the condition of a small learning rate. In the pre-training stage of the present embodiment, a (X+Y)-dimensional speaker embedding space is constructed. Although there is no positive sample training data for far-field speakers (X classes) in this stage, the model can reserve a representation space for far-field speakers through implicit representation learning and indirectly optimize the weight distribution through the decision boundary learned by near-field speakers. In the second stage of fine-tuning, the model can use the domain-independent speaker discriminative feature knowledge obtained in the first stage to perform non-random initialization of the pre-reserved X-class weight vector, thereby achieving efficient convergence under the condition of a low learning rate, effectively solving the cross-domain adaptation problem in far-field voiceprint recognition.

[0117] In the present embodiment, the ResNet101 model is selected as the base network structure of the far-field voiceprint recognition model. For the pre-training data and fine-tuning data, the three-times speed expansion strategy is used to generate twice the amount of data for additional speakers. Each speech segment is adjusted to 0.9 and 1.1 times the speed based on the speed function, and the total number of speakers in the pre-training is Y+X. The following enhancement techniques are used for each speech segment: 1) convolution with simulated RIR data to add reverberation. 2) Randomly select a noise file in MUSAN to add to the original signal with a signal-to-noise ratio range of [0, 15dB]. 3) Randomly select a music file in MUSAN, and perform cutting or repeating operations as needed to match the length of the speech segment, and add it to the original signal with a signal-to-noise ratio range of [5, 15dB]. 4) Add speech mixing in MUSAN to the original signal with a signal-to-noise ratio range of 5-15dB. The training data is enhanced with a probability of 0.6.

[0118] Preferably, the acoustic features in the training stage are 80-dimensional Fbank features with a frame length of 25ms and a frame shift of 10ms, and 200 frames are extracted for each feature. The pooling layer selects MQMHA STP pooling. The SGD optimizer with a momentum of 0.9 and a weight decay of 1e-4 is used. Each batch contains 64 samples. The initial learning rate is set to 0.1, and the minimum learning rate is 5e-5. A total of 120 epochs are trained, and AAM-Softmax is used in combination with InterTopK and Subcenter. The margin gradually increases from 0.0 to 0.2 starting from the 20th epoch. In the second stage, the initial learning rate is set to 2e-5, and the remaining parameters are the same as those in the first stage.

[0119] Further, in the identification stage, first, the registered and test audios are pre-processed and Fbank acoustic features are extracted, then the features are mapped to a high-dimensional speaker embedding space through a trained model, and finally the cosine similarity between the two embedding vectors is calculated to determine whether they are the same speaker through a preset threshold comparison.

[0120] As a possible implementation, the reverberation removal processing is performed on the to-be-tested far-field audio data, a model is constructed to decompose the to-be-tested far-field audio data into a target signal and a reverberation component, the reverberation signal is described using a delay linear prediction, wherein the reverberation component is represented as a weighted sum of past observation signals according to the de-reverberation filter coefficients and a prediction delay; the de-reverberation filter coefficients are estimated by minimizing the weighted prediction error, and an iterative optimization algorithm is used to calculate the weighted prediction error in each iteration using the estimated de-reverberation filter coefficients, and the de-reverberation filter coefficients are updated according to the weighted prediction error until the error converges to determine the optimal de-reverberation filter coefficients; and the de-reverberation filter with the optimal de-reverberation filter coefficients is used to remove the reverberation component in the to-be-tested far-field audio data.

[0121] In a specific implementation, first, a signal model is established, and the far-field signal y(t) is decomposed into a target signal x(t) and a reverberation component r(t). The reverberation signal is described using a delay linear prediction which is represented as a weighted sum of past observation signals: where g i is the filter coefficient, d is the prediction delay, and N is the filter length. The prediction error e(t) is defined as the difference between the actual observation signal y(t) and the predicted signal This model accurately captures the acoustic propagation characteristics in a reverberation environment.

[0122] Here, the optimal filter coefficient g i is estimated by minimizing the weighted prediction error. The objective function J is defined as the sum of the weighted squared errors: J = ∑ t ω(t)|e(t)| 2 where ω(t) is the weight function, which is usually inversely proportional to the prediction error power. This weighting method maximizes the non-Gaussianity of the residual error, effectively distinguishing between speech signals and reverberation components, making the algorithm better preserve the speech characteristics and suppress the reverberation.

[0123] where the filter coefficient estimation uses an iterative optimization method, first assuming g_i = 0, and then estimating the filter coefficient using the iterative optimization method. In each iteration, the weighted prediction error is calculated using the estimated filter coefficient, and the filter coefficient is updated accordingly until the error converges. This process is usually implemented by solving the weighted least squares problem, which can adapt to different reverberation environments and acoustic conditions.

[0124] Finally, the reverberation component in the signal is removed by the de-reverberation filter, restoring the clean target signal: This step subtracts the predicted reverberation component from the original observed signal, effectively preserving the useful information in the direct sound and early reflections while suppressing the negative impact of late reverberation on speech quality and intelligibility.

[0125] In this way, by removing the reverberation component, the intelligibility and intelligibility of the speech signal are significantly improved, enabling the far-field voiceprint recognition system to more accurately extract speech features, thereby improving the overall performance of the system. At the same time, while reducing reverberation interference, the algorithm can maintain the naturalness and integrity of the speech signal, providing more reliable technical support for far-field speech recognition and voiceprint recognition applications.

[0126] The method provided by the embodiments of the present disclosure comprises: collecting sample near-field audio data in a room, constructing a geometric acoustic propagation model according to the propagation characteristics of the sample near-field audio data, and determining a room impulse response function describing the distribution of the frequency domain sound field; constructing a reverberation model for independently calculating the reverberation time of each frequency band and reflecting the differences in sound absorption characteristics of different materials at different frequencies, and a sound source and receiving device directivity model simulating the spatial pickup characteristics of a microphone array; performing time domain convolution on the sample near-field audio data and the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity model, to generate simulated far-field audio data; inputting the de-reverberated to-be-tested far-field audio data and the reference near-field audio data into a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, and a representation position of a far-field speaker class corresponding to the simulated far-field audio data is reserved in the pre-training, and a non-random initialization is performed on a weight vector corresponding to the far-field speaker class according to the discriminative feature knowledge learned in the pre-training stage in the fine-tuning. By introducing acoustic space parameterization modeling, speaker representation weight pre-reservation training strategy and weighted prediction error de-reverberation method, the problems of insufficient training data, poor model cross-domain adaptability and serious reverberation noise interference in far-field voiceprint recognition are effectively solved. The generated far-field speech data has real spatial characteristics, the training model can quickly adapt to complex far-field scenes, and the overall accuracy and robustness of the recognition system in a far-field environment are improved, meeting the high-performance requirements of intelligent security, remote authentication and other practical applications.

[0127] Those skilled in the art can understand that in the above method of the specific embodiments, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0128] Based on the same inventive concept, the embodiments of the present disclosure also provide a far-field voiceprint recognition device corresponding to the far-field voiceprint recognition method. Since the principle of the device in the embodiments of the present disclosure solves the problem similarly to the far-field voiceprint recognition method described above, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0129] Please refer to Figure 3 , Figure 3 A schematic diagram of a far-field voiceprint recognition device provided by the embodiments of the present disclosure is shown in FIG. 3. Figure 3 As shown in FIG. 3, the far-field voiceprint recognition device 300 provided by the embodiments of the present disclosure includes:

[0130] A first acoustic space parameterization modeling module 310 is configured to collect sample near-field audio data in a room, construct a geometric acoustic propagation model according to the propagation characteristics of the sample near-field audio data, and determine a room impulse response function describing the distribution of the sound field in the frequency domain.

[0131] A second acoustic space parameterization modeling module 320 is configured to construct a reverberation model for independently calculating the reverberation time for each frequency band and reflecting the differences in sound absorption characteristics of different materials at different frequencies, and a sound source and receiving device directivity model simulating the spatial pickup characteristics of a microphone array.

[0132] A far-field acoustic data simulation generation module 330 is configured to perform time-domain convolution on the sample near-field audio data and the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity model, to generate simulated far-field audio data.

[0133] An identification module 340 is configured to input the de-reverberated to-be-tested far-field audio data and the reference near-field audio data to a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, a representation position of a far-field speaker class corresponding to the simulated far-field audio data is reserved in the pre-training, and a weight vector corresponding to the far-field speaker class is non-randomly initialized according to the discriminative feature knowledge learned in the pre-training stage in the fine-tuning.

[0134] The processing flow of each module in the device and the interaction flow between the modules can be referred to the related description in the above method embodiments, which will not be described in detail here.

[0135] The device for far-field voiceprint recognition provided by the embodiments of the present disclosure collects sample near-field audio data in a room, constructs a geometric acoustic propagation model according to the propagation characteristics of the sample near-field audio data, and determines a room impulse response function describing the distribution of a sound field in a frequency domain; a reverberation model for calculating a reverberation time independently for each frequency band and reflecting the differences in sound absorption characteristics of different materials at different frequencies is constructed, and a sound source and a receiving device directivity model simulating the spatial pickup characteristics of a microphone array are constructed; the sample near-field audio data is time-domain convolved with the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity model, to generate simulated far-field audio data; the de-reverberated to-be-tested far-field audio data and reference near-field audio data are input into a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data, a representation position of a far-field speaker category corresponding to the simulated far-field audio data is reserved in the pre-training, and a weight vector corresponding to the far-field speaker category is non-randomly initialized according to discriminative feature knowledge learned in the pre-training stage in the fine-tuning. By introducing acoustic space parameterization modeling, speaker representation weight pre-reservation training strategies, and weighted prediction error de-reverberation methods, the problems of insufficient training data, poor model cross-domain adaptability, and serious reverberation noise interference in far-field voiceprint recognition are effectively solved. The generated far-field speech data has real spatial characteristics, the training model can quickly adapt to complex far-field scenes, the accuracy and robustness of the recognition system in a far-field environment are overall improved, and the high-performance requirements of practical applications such as intelligent security and remote authentication are met.

[0136] Corresponding to Figure 1 The embodiments of the present disclosure also provide an electronic device 400, as shown in Figure 4 The structure schematic diagram of the electronic device 400 provided by the embodiments of the present disclosure includes:

[0137] The processor 41, the memory 42, and the bus 43; the memory 42 is used for storing execution instructions, including an internal memory 421 and an external memory 422; the internal memory 421 is also called an internal memory, and is used for temporarily storing operation data in the processor 41 and data exchanged with the external memory 422 such as a hard disk, the processor 41 exchanges data with the external memory 422 through the internal memory 421, and when the electronic device 400 is running, the processor 41 and the memory 42 communicate through the bus 43, so that the processor 41 executes Figure 1 The steps of the far-field voiceprint recognition method in

[0138] The embodiment of the present disclosure further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, steps of the far-field voiceprint recognition method described in the method embodiment are executed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0139] The embodiment of the present disclosure further provides a computer program product, which comprises computer instructions. When the computer instructions are executed by a processor, steps of the far-field voiceprint recognition method described in the method embodiment can be executed. For details, refer to the method embodiment, and details are not described herein.

[0140] The computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium, and in another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the apparatus described above can refer to the corresponding process in the foregoing method embodiments, and details are not described herein. In several embodiments provided by the present disclosure, it should be understood that the disclosed apparatus and method can be implemented by other means. The apparatus embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0142] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.

[0143] In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0144] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0145] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art who is familiar with the technical field of the present disclosure can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A far-field voiceprint recognition method, characterized in that: include: Collecting sample near-field audio data in the room, constructing a geometric acoustic propagation model based on the propagation characteristics of the sample near-field audio data, and determining a room impulse response function that describes the frequency domain sound field distribution; Build a reverberation model that independently calculates reverberation time for each frequency band and reflects the differences in sound absorption characteristics of different materials at different frequencies, as well as a sound source and receiving device directional model that simulates the spatial pickup characteristics of a microphone array; Based on the geometric acoustic propagation model that superimposes the reverberation model and the sound source and receiving device directivity models, performing a time-domain convolution on the sample near-field audio data and the room impulse response function to generate simulated far-field audio data; The dereverberated far-field audio data to be tested and the reference near-field audio data are input into a trained far-field voiceprint recognition model to determine the far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data. During the pre-training, the representation position of the simulated far-field audio data corresponding to the far-field speaker category is reserved. During the fine-tuning, the weight vector corresponding to the pre-reserved far-field speaker category is non-randomly initialized based on the discriminative feature knowledge learned in the pre-training stage.

2. The method according to claim 1, characterized in that Constructing a geometric acoustic propagation model based on the propagation characteristics of the sample near-field audio data, specifically including: For high-frequency sound waves, an image source method is used. Based on the room size and microphone position corresponding to the sample near-field audio data, the reflected sound source in the room is equivalent to a virtual image source through the law of mirror reflection. The sound wave propagation path length between each virtual image source and the microphone position is calculated to determine the sound wave propagation time delay and sound wave propagation attenuation. A random ray tracing method is used for the low-frequency sound wave range. Multiple beams of light are randomly emitted from the sound source position corresponding to the sample near-field audio data to simulate the propagation and reflection of sound waves in the room. The energy of the light beams at the microphone position is recorded, and the distribution of the light beam energy is statistically analyzed to determine the sound pressure distribution at the microphone position. The geometric acoustic propagation model is constructed using the sound wave propagation time delay, the sound wave propagation attenuation, and the sound pressure distribution as the propagation characteristics.

3. The method according to claim 1, characterized in that Determine the room impulse response function that describes the frequency domain sound field distribution, including: Describing the sound field distribution in space based on the Helmholtz wave equation, and converting the Helmholtz wave equation into a boundary integral equation; Establishing boundary conditions reflecting the room geometry and material acoustic properties, solving the Helmholtz wave equation at each frequency point, and determining the frequency response between the sound source and the microphone receiving position corresponding to the sample near-field audio data; A transfer function including direct sound, early reflection and late reverberation components generated during the solution process is determined as the room impulse response function.

4. The method according to claim 1, wherein Build a reverberation model that independently calculates reverberation time for each frequency band and reflects the differences in sound absorption characteristics of different materials at different frequencies. Also, build a sound source and receiving device directional model that simulates the spatial sound pickup characteristics of a microphone array. Specifically, this includes: The single sound absorption coefficient in the Eyring formula and the Sabine formula is replaced by a sound absorption coefficient matrix related to the frequency band to construct a multi-band model; In the multi-band model, the reverberation time is calculated independently for each frequency band based on the differences in sound absorption characteristics of different materials at different frequencies, simulating the frequency-selective reverberation characteristics in a real environment to generate the reverberation model; The directional pattern from the sound source to the microphone corresponding to the sample near-field audio data is represented as a linear combination of spherical harmonics, and the directional sound field distribution in three-dimensional space is characterized by a spherical harmonic series expansion to describe the complete three-dimensional sound field to generate the directional model of the sound source and receiving device.

5. The method according to claim 1, wherein After constructing a reverberation model that independently calculates reverberation time for each frequency band and reflects the differences in sound absorption characteristics of different materials at different frequencies, and a sound source and receiving device directivity model that simulates the spatial sound pickup characteristics of a microphone array, the method further includes: Lambert's cosine law is used to establish the mathematical relationship between the reflected sound intensity being greater than the incident sound wave and the cosine value of the angle between the surface normal and to construct a statistical diffusion model. For each sound wave reflection point of the sample near-field audio data, a basic reflection intensity is calculated according to the statistical diffusion model, and distribution characteristics of the reflected energy in various directions are determined according to the scattering technology of the material corresponding to the sound wave reflection point.

6. The method according to claim 1, characterized in that The far-field voiceprint recognition model is trained based on the following steps: Pre-training a preset far-field voiceprint recognition model based on the sample near-field audio data to construct a complete embedding space including the near-field speaker category corresponding to the sample near-field audio data and the far-field speaker category corresponding to the simulated far-field audio data; Learning domain-independent speaker discriminative feature knowledge through pre-training, and reserving representation positions for the far-field speaker category through implicit representation learning in the projection layer; A preset far-field voiceprint recognition model is fine-tuned based on the simulated far-field audio data, and a weight vector corresponding to the far-field speaker category is directly initialized at the projection layer according to the domain-independent speaker discriminative feature knowledge.

7. The method according to claim 1, characterized in that De-reverberation processing is performed on the far-field audio data to be measured based on the following steps: Constructing a model for decomposing the far-field audio data to be measured into a target signal and a reverberation component, and describing the reverberation signal using delayed linear prediction, wherein the reverberation component is represented as a weighted sum of past observation signals according to dereverberation filter coefficients and prediction delays; Estimating dereverberation filter coefficients by minimizing weighted prediction errors, calculating weighted prediction errors using the estimated dereverberation filter coefficients in each iteration using an iterative optimization algorithm, updating the dereverberation filter coefficients according to the weighted prediction errors until the errors converge, and determining optimal dereverberation filter coefficients; A dereverberation filter having the optimal dereverberation filter coefficient is used to remove the reverberation component in the far-field audio data to be measured.

8. A far-field voiceprint recognition device, characterized in that: include: a first acoustic space parameterized modeling module, configured to collect sample near-field audio data in a room, construct a geometric acoustic propagation model based on the propagation characteristics of the sample near-field audio data, and determine a room impulse response function that describes the frequency domain sound field distribution; The second acoustic space parametric modeling module is used to build a reverberation model that independently calculates the reverberation time for each frequency band and reflects the differences in the sound absorption characteristics of different materials at different frequencies, as well as a sound source and receiving device directional model that simulates the spatial pickup characteristics of the microphone array; a far-field acoustic data simulation generation module, configured to perform time-domain convolution of the sample near-field audio data with the room impulse response function based on the geometric acoustic propagation model superimposed with the reverberation model and the sound source and receiving device directivity models, to generate simulated far-field audio data; A recognition module is configured to input the dereverberated far-field audio data to be tested and the reference near-field audio data into a trained far-field voiceprint recognition model to determine a far-field voiceprint recognition result, wherein the far-field voiceprint recognition model is pre-trained based on the sample near-field audio data and fine-tuned based on the simulated far-field audio data. During the pre-training, representation positions corresponding to the far-field speaker categories in the simulated far-field audio data are reserved. During the fine-tuning, weight vectors corresponding to the pre-reserved far-field speaker categories are non-randomly initialized based on the discriminative feature knowledge learned during the pre-training phase.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the far-field voiceprint recognition method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the far-field voiceprint recognition method according to any one of claims 1 to 7.

Images