A voiceprint wake-up algorithm based on distributed edge computing

Abstract

The present invention relates to the field of Internet technology, and discloses a voiceprint wake-up algorithm based on distributed edge computing. By combining a traditional wake-up algorithm with distributed edge computing, the advantages of computing power sharing and distributed decision-making are effectively brought into play, and high computing power is shared among nodes and distributed decisions are made among devices, which effectively saves computing resources and reduces the cost of a single unit. This removes technical barriers for further promotion of voiceprint wake-up in the mid- and low-end markets. By sharing the high computing power of voiceprint wake-up among different computing chips, and ensuring that only one device generates a response when adjacent nodes are awakened at the same time through a central decision-making method, the problem of high computing power and multi-device linkage of the voiceprint wake-up algorithm is effectively solved. At the same time, the computing power sharing brought by distributed edge computing can further realize a voiceprint wake-up algorithm with a larger resource model running on a terminal, which also brings feasibility for further optimization of the effect.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a voiceprint wake-up algorithm based on distributed edge computing. Background Art

[0002] Voiceprint recognition, a type of biometric technology, is also called speaker recognition. The theoretical basis of voiceprint recognition is that each voice has unique characteristics, through which the voices of different people can be effectively distinguished. The general steps of voiceprint recognition include: voice collection -> noise suppression and effective voice detection (VAD) -> voiceprint feature extraction -> voice modeling of the speaker -> recognition and matching.

[0003] With the rapid development of the global economy and technology, the voiceprint wake-up function is used more and more frequently, from the personalized wake-up function of fast-moving consumer goods such as mobile phones and speakers to the secondary authentication in the fields of banking and finance. The computing power requirements of traditional voiceprint wake-up algorithms are much higher than those of similar front-end acoustic intelligent products. High computing power brings high chip costs, which makes it difficult to further popularize and promote them in the mid- and low-end markets. At the same time, traditional voiceprint wake-up algorithms have no effect advantages in multi-device linkage. The classic processing problem is that a wake-up word wakes up all the smart devices in the home and starts a continuous dialogue between devices. Summary of the invention

[0004] The main purpose of this invention is to propose a voiceprint wake-up algorithm based on distributed edge computing, which aims to distribute the high computing power of voiceprint wake-up to different computing chips through distributed edge computing methods, and ensure that only one device responds when adjacent nodes are awakened at the same time through central decision-making, thereby effectively solving the problems of high computing power and multi-device linkage of the voiceprint wake-up algorithm.

[0005] To achieve the above purpose, the voiceprint wake-up algorithm based on distributed edge computing proposed in the present invention includes the following steps:

[0006] Voiceprint registration: the user collects voice recordings through one of the nodes of the distributed device. The user triggers an event by saying the command "start voiceprint registration". The distributed device extracts multiple features from the user's data and generates the corresponding voiceprint feature model Model-X after feature extraction. Distributed storage: the voiceprint model data is compressed and stored in multiple storage media of the distributed node devices at the same time.

[0007] Voiceprint verification: users verify through any device in the distributed microphones. The user triggers the event by saying the command "start voiceprint verification". After the microphone device records the user's voice data, the host extracts and compresses the user's voice data through a feature model, decompresses it, and matches it with the voiceprint feature model data on the storage medium. If the match is successful, the corresponding user information is selected.

[0008] Furthermore, the voiceprint registration includes: starting a registration program, recording a text recording of a user's wake-up word, wherein the number of characters in the wake-up word text is between 4 and 10 characters.

[0009] Furthermore, the distributed device extracts multiple features from the user's data, including: framing the audio before feature extraction, currently using 512 sampling points as 1 frame data block, corresponding to 32ms one frame data, performing windowing on the framed signal, and then performing corresponding feature extraction to reduce spectrum leakage. The Hamming window function used to extract voiceprint feature parameters is as follows:

[0010]

[0011] Where a0=0.53856.

[0012] Furthermore, the distributed device extracts multiple features from the user's data, including: using a convolutional neural network (CNN) to extract voiceprint features for both voiceprint and keyword wakeup:

[0013] CNN consists of convolutional layers. Each convolutional layer defines a set of convolutional kernels. Each convolutional kernel is repeatedly applied to different positions of the input data, and element-by-element multiplication and addition operations are performed on the local data corresponding to the position. The result is the output of the convolutional kernel at the input data position. The size of the output obtained after the input data passes through a convolutional layer depends on:

[0014] 1. The number of convolution kernels;

[0015] 2. The size of each convolution kernel;

[0016] 3. The spacing between adjacent positions when applying the convolution kernel (the stride of the kernel).

[0017] The one-dimensional convolutional neural network currently used uses a convolution kernel with a kernel size of 3 and a kernelcount of 4. At the same time, it is defined that the output matrix after convolution processing has the same number of columns as the input matrix.

[0018] Furthermore, the host extracts and compresses the user's voice data into a feature model, decompresses it and matches it with the voiceprint feature model data on the storage medium. If the match is successful, the corresponding user information is selected, including: the voiceprint model matching process uses the cosine similarity method to perform relevant score evaluation. The cosine similarity uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0 degrees. When performing multi-model similarity matching, the model with the largest cosine value is taken as the target detection model.

[0019] The technical solution of the present invention has the following beneficial effects: the technical solution of the present invention, by combining the traditional wake-up algorithm with distributed edge computing, effectively gives play to the advantages of computing power sharing and distributed decision-making, distributes high computing power among nodes and distributes decisions among devices, effectively saves computing resources, and reduces the cost of the monomer, thereby removing technical barriers for the further promotion of voiceprint wake-up in the mid- and low-end markets. By distributing the high computing power of voiceprint wake-up to different computing chips, and ensuring that only one device generates a response when adjacent nodes are awakened at the same time through central decision-making, the problem of high computing power and multi-device linkage of the voiceprint wake-up algorithm is effectively solved. At the same time, the computing power sharing brought by distributed edge computing can further realize the voiceprint wake-up algorithm with a larger resource model running on the terminal. By uniformly extracting features from the user's audio data, the feature can be used to construct a voiceprint model and can also be used to identify the current wake-up word. While reducing the complexity of algorithm calculation, the response speed of voiceprint feature identification is also effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0021] Figure 1 This is an overall framework flow chart of a voiceprint wake-up algorithm based on distributed edge computing proposed by the present invention;

[0022] Figure 2 A user voiceprint registration flow chart of a voiceprint wake-up algorithm based on distributed edge computing proposed by the present invention;

[0023] Figure 3 A voiceprint verification flow chart of a voiceprint wake-up algorithm based on distributed edge computing proposed by the present invention;

[0024] Figure 4A schematic diagram of communication establishment of distributed devices based on a voiceprint wake-up algorithm based on distributed edge computing proposed in the present invention.

[0025] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0028] In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0029] The present invention proposes a voiceprint wake-up algorithm based on distributed edge computing.

[0030] like Figures 1 to 4 As shown, in one embodiment of the present invention, the voiceprint wake-up algorithm based on distributed edge computing includes the following steps:

[0031] Voiceprint registration: the user collects voice recordings through one of the nodes of the distributed device. The user triggers an event by saying the command "start voiceprint registration". The distributed device extracts multiple features from the user's data and generates the corresponding voiceprint feature model Model-X after feature extraction. Distributed storage: the voiceprint model data is compressed and stored in multiple storage media of the distributed node devices at the same time.

[0032] Voiceprint verification: users verify through any device in the distributed microphones. The user triggers the event by saying the command "start voiceprint verification". After the microphone device records the user's voice data, the host extracts and compresses the user's voice data through a feature model, decompresses it, and matches it with the voiceprint feature model data on the storage medium. If the match is successful, the corresponding user information is selected.

[0033] Specifically, the voiceprint registration includes: starting a registration program, recording a text recording of a user's wake-up word, wherein the number of characters in the wake-up word text is between 4 and 10 characters.

[0034] The user first uses one of the nodes in the distributed microphone to collect audio in the voiceprint wake-up application (the user says the command "start voiceprint registration" to trigger the event). The algorithm will perform multi-feature extraction (time-frequency feature distribution, etc.) on the user's data and generate the corresponding model Model-X after feature extraction. The model data can be stored in multiple copies in multiple storage media of the distributed microphone device (such as Flash or Sdcard) at the same time.

[0035] The purpose of feature parameter extraction is to extract feature parameters from the speaker's speech that can characterize the speaker's specific acoustic organ structure or habitual behavior. The feature parameters are relatively stable for the same speaker and cannot be inconsistent with changes in time or environment. They should also be consistent for different utterances of the same speaker. Even if different speakers say the same thing, they should be well distinguishable, difficult to imitate, and have strong noise resistance. At present, we use a neural network model, which adopts a distributed parallel processing structure network model with organizational and self-learning capabilities, strong complex classification boundary distinction capabilities, and robustness to incomplete information. During the training process, it can continuously adjust its own parameter weights and structural topology to adapt to the needs of environmental and system performance optimization.

[0036] Specifically, the distributed device extracts multiple features from the user's data, including: framing the audio before feature extraction, currently using 512 sampling points as 1 frame data block, corresponding to 32ms one frame data, performing windowing on the framed signal, and then performing corresponding feature extraction to reduce spectrum leakage. The Hamming window function used to extract voiceprint feature parameters is as follows:

[0037]

[0038] Where a0=0.53856.

[0039] Specifically, the distributed device extracts multiple features from the user's data, including: In the voiceprint recognition system process, the voiceprint modeling process from audio features to speaker models can be regarded as a complex nonlinear function, which defines a mapping from audio feature space to speaker model space. The problem to be solved by voiceprint recognition based on deep learning methods is to represent this complex function with a suitable neural network. Considering the control of the network parameter scale within a reasonable range and the fusion with the feature extraction of the keyword wake-up engine, the voiceprint and keyword wake-up are both subjected to voiceprint feature extraction using convolutional neural networks (CNN):

[0040] CNN consists of convolutional layers. Each convolutional layer defines a set of convolutional kernels. Each convolutional kernel is repeatedly applied to different positions of the input data, and element-by-element multiplication and addition operations are performed on the local data corresponding to the position. The result is the output of the convolutional kernel at the input data position. The size of the output obtained after the input data passes through a convolutional layer depends on:

[0041] 1. The number of convolution kernels;

[0042] 2. The size of each convolution kernel;

[0043] 3. The spacing between adjacent positions when applying the convolution kernel (the stride of the kernel).

[0044] The one-dimensional convolutional neural network currently used uses a convolution kernel with a kernel size of 3 and a kernelcount of 4. At the same time, it is defined that the output matrix after convolution processing has the same number of columns as the input matrix.

[0045] Where N=512, M (M=3) dimensional feature parameters are fixedly extracted for each frame, and the dimension of the parameters and Kernel Count can be configured in real time.

[0046] Kernel is as follows:

[0047] 1 0 0 0 1 0 0 0 1

[0048] Kernel 1

[0049] 0 0 1 0 1 0 1 0 0

[0050] Kernel 2

[0051] 0 1 0 1 0 1 0 0 0

[0052] Kernel 3

[0053] 0 0 0 1 0 1 0 1 0

[0054] Kernel 4

[0055] The convolution is defined as: X(m) is a 3*3 kernel, K(n) is a 3*3 matrix corresponding to a single frame, and Z(m) is the output result of different kernels.

[0056]

[0057] After the above calculation, each frame of input data will eventually obtain m vectors. At the same time, multiple frames of the target wake-up segment audio are processed to obtain results. The multi-frame results obtained above and the speaker ID are saved in the corresponding model file DatI. Through the distributed network channel, the model file DatI is copied to multiple nodes in the distributed network.

[0058] Specifically, the host extracts and compresses the user's voice data into a feature model, decompresses it and matches it with the voiceprint feature model data on the storage medium. If the match is successful, the corresponding user information is selected, including: the voiceprint model matching process uses the cosine similarity method to perform relevant score evaluation. Cosine similarity uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0 degrees. When performing multi-model similarity matching, the model with the largest cosine value is taken as the target detection model.

[0059] When a user performs voiceprint verification, he or she can use any device in the distributed microphone to perform verification (the user says the command "start voiceprint verification" to trigger the event). After the microphone device records the user data, multiple features are extracted from the user data and matched with the model data on the storage medium. If the match is successful, the corresponding user information is picked up. The feature extraction method used for user verification is the same as the method used in the registration stage.

[0060] The model matching process uses the cosine similarity method to evaluate the relevant scores. Cosine similarity uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0 degrees, that is, the more similar the two vectors are, which is called "cosine similarity".

[0061] Specifically, a computing power handshake mechanism is established at the voiceprint awakening communication end, and the host node distributes the computing power of part of the local algorithm to peer-to-peer devices for distributed computing. Figure 4 As shown in the figure, when N devices communicate, the computing power of the kth device is Tk, and the voice of the kth device needs to be broadcast to N-1 devices to complete the voiceprint wake-up process. The feature extraction computing power part that needs to be completed on the kth device is placed on the N-1 devices for distributed computing. The computing power requirement of the feature extraction algorithm is G, and the pre-computing power (the computing power required for the algorithm part that must be executed locally) is Gsk. Then the kth device only needs to have a computing power of Gs+G*(Tk-Gsk) / (T1+T2+…+Tk+…+TN-Gs1-…Gsk-…-Gsn). The average computing power required for the devices participating in the distributed computing must not be lower than the algorithm pre-computing power Gsk. Figure 4 shown.

[0062] Specifically, when the user registers the voiceprint, the distributed node device will perform data sampling, and at the same time, by judging the time when each node triggers the registration wake-up word (selecting the node with the shortest trigger time in the entire distributed system) and the energy of the node recording the audio (selecting the node with the largest energy), a host node for voiceprint registration / verification is selected. In addition, the technical difficulty of distributed voiceprint wake-up lies in the consistency of the MIC Sensor, the acoustic structure, and the acoustic coefficients between multiple distributed microphones. In order to better share the voiceprint model data, the algorithm contains an acoustic automatic calibration algorithm, which controls the surrounding sound field environment of the microphone device during calibration by playing a specific signal source externally. After recording, the device can automatically analyze and determine the difference between the current microphone and the standard acoustic indicators, and calculate the calibration coefficients for the digital gain corresponding to the MIC Sensor, data compensation in a specific frequency band, etc., and save the calibration coefficients in the storage medium.

[0063] Voiceprint recognition, a type of biometric technology, is also called speaker recognition. The theoretical basis of voiceprint recognition is that each voice has unique characteristics, through which the voices of different people can be effectively distinguished. The general steps of voiceprint recognition include: voice collection -> noise suppression and effective voice detection (VAD) -> voiceprint feature extraction -> voice modeling of the speaker -> recognition and matching.

[0064] Specifically, the present invention combines the traditional wake-up algorithm with distributed edge computing, effectively exerts the advantages of computing power sharing and distributed decision-making, distributes high computing power among nodes and distributes decisions among devices, effectively saves computing resources, and reduces the cost of the unit, thereby removing technical barriers for the further promotion of voiceprint wake-up in the mid- and low-end markets. By distributing the high computing power of voiceprint wake-up to different computing chips, and ensuring through central decision-making that when adjacent nodes are awakened at the same time, only one device generates a response, the problem of high computing power and multi-device linkage of the voiceprint wake-up algorithm is effectively solved. At the same time, the computing power sharing brought by distributed edge computing can further realize the voiceprint wake-up algorithm with a larger resource model running on the terminal. By performing unified feature extraction on the user's audio data, the feature can be used to construct a voiceprint model and can also be used to identify the current wake-up word. While reducing the complexity of algorithm calculation, the response speed of voiceprint feature identification is also effectively improved.

[0065] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A voiceprint wake-up algorithm based on distributed edge computing, characterized in that: The following steps are involved: Voiceprint registration: the user collects voice recordings through one of the nodes of the distributed device. The user triggers an event by saying the command "start voiceprint registration". The distributed device extracts multiple features from the user's data and generates the corresponding voiceprint feature model Model-X after feature extraction. Distributed storage: voiceprint model data is compressed and stored in multiple storage media of the distributed node devices at the same time. Voiceprint verification: the user verifies through any device in the distributed microphone. The user triggers the event by saying the command word "start voiceprint verification". After the microphone device records the user's voice data, the host extracts and compresses the user's voice data through a feature model, decompresses it, and matches it with the voiceprint feature model data on the storage medium. If the match is successful, the corresponding user information is selected; The distributed device extracts multiple features from the user's data, including: framing the audio before feature extraction, currently using 512 sampling points as 1 frame data block, corresponding to 32ms 1 frame data, performing windowing on the framed signal, and then performing corresponding feature extraction to reduce spectrum leakage; using the Hamming window function to extract voiceprint feature parameters, as follows: Where a0=0.53856; The host extracts and compresses the user's voice data through a feature model, decompresses it, and matches it with the voiceprint feature model data on the storage medium. If the match is successful, the corresponding user information is selected, including: the voiceprint model matching process uses the cosine similarity method to perform relevant score evaluation. The cosine similarity uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0 degrees. When performing multi-model similarity matching, the model with the largest cosine value is taken as the target detection model; When N devices are communicating, the computing power of the kth device is Tk. The voice of the kth device needs to be broadcast to N-1 devices to complete the voiceprint wake-up process. The feature extraction computing power that needs to be completed on the kth device is placed on the N-1 devices for distributed computing. The computing power requirement of the feature extraction algorithm is G, and the front-end computing power is Gsk. The kth device only needs to have a computing power of Gsk+G*(Tk-Gsk) / (T1+T2+…+Tk+…+TN-Gs1-…Gsk-…-Gsn). The average computing power required for devices participating in distributed computing must not be lower than the algorithm front-end computing power Gsk.

2. The voiceprint wake-up algorithm based on distributed edge computing according to claim 1 is characterized in that: The voiceprint registration includes: starting a registration program, recording a user wake-up word text recording, and the number of words in the wake-up word text is between 4 and 10 words.

3. The voiceprint wake-up algorithm based on distributed edge computing according to claim 1 is characterized in that: The distributed device extracts multiple features from the user's data, including: using a convolutional neural network (CNN) to extract voiceprint features for both voiceprint and keyword wakeup: CNN consists of convolutional layers. Each convolutional layer defines a set of convolutional kernels. Each convolutional kernel is repeatedly applied to different positions of the input data, and element-by-element multiplication and addition operations are performed on the local data corresponding to the position. The result is the output of the convolutional kernel at the input data position. The size of the output obtained after the input data passes through a convolutional layer depends on:

1. The number of convolution kernels; 2. The size of each convolution kernel; 3. The spacing between adjacent positions when applying the convolution kernel (the kernel's stride); The one-dimensional convolutional neural network currently used uses a convolution kernel with a kernel size of 3 and a kernel count of 4. At the same time, it is defined that the output matrix after convolution processing has the same number of columns as the input matrix.

Images