Training method of voice signal processing model for high-end ceiling microphone, electronic equipment and storage medium
By embedding beamforming algorithms and speech activity detection in the data simulation of high-end ceiling microphones, combined with the output of the replacement microphone signal after passing through the fixed beam matrix in the mute area, the problem of poor shielding effect of the mute area of the high-end ceiling microphone is solved, achieving better speech separation and user experience.
Patent Information
- Application Number
- CN202411891445.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-03-28
AI Technical Summary
The multi-sound zone function of high-end ceiling mic has poor shielding effect and cannot flexibly adapt to the needs of different scenarios, which affects the user experience.
By embedding beamforming algorithms in data simulation, regional voice differences can be improved and model training difficulty can be reduced. The speech activity detection of the priority sound pickup area is used to realize a priority but unblocked learning strategy, and the calculation of computing power is reduced by replacing the output of the microphone signal after passing through the fixed beam matrix of the mute area.
It improves regional voice differences, enhances the mute area shielding effect, flexibly adapts to the needs of different scenarios, and improves the user experience.
Smart Images
Figure CN119724210A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of speech recognition technology, and in particular to a training method, electronic device, and storage medium for a speech signal processing model for a high-end ceiling microphone. Background Art
[0002] A ceiling microphone is a microphone mounted on the ceiling, usually used in large spaces such as conference rooms, classrooms, etc., to enable voice transmission without the need for a handheld microphone. This microphone features beamforming technology and intelligent feedback suppression, which can clearly capture voices and suppress excess echoes and noise, providing high-quality audio transmission.
[0003] Ceiling microphones usually have dozens or even hundreds of microphones. Adding an algorithm module will increase computing power and memory, which puts a lot of pressure on the chip. Therefore, the industry usually uses additional equipment to implement some functions. Even if there is algorithm development, it is only a relatively simple traditional algorithm with weak effect.
[0004] For example, the vocal ducking function in the audio processor and the multi-zone function of the ceiling microphone. Among them, the vocal ducking function is usually used in the audio processor to automatically reduce the volume of the background music when the human voice is detected. This is common in broadcasts, podcasts, and live performances. The multi-zone function of the ceiling microphone is usually used in conference rooms or large event venues to capture sounds in different areas. The disadvantage is that it needs to be processed with the help of professional audio processing equipment, which increases the system cost and installation complexity, and has high requirements for technicians. The shielding effect of the quiet zone is poor, which basically depends on the effect of beamforming. Noise in non-target areas may be picked up, affecting the audio quality. There is no compatible solution for priority pickup areas and quiet areas, and it cannot flexibly adapt to the needs of different scenarios, affecting the user experience. Summary of the invention
[0005] The embodiments of the present invention provide a training method for a speech signal processing model for a high-end ceiling microphone, an electronic device, and a storage medium, which are used to solve at least one of the above-mentioned technical problems.
[0006] In a first aspect, an embodiment of the present invention provides a training method for a speech signal processing model for a high-end ceiling microphone, comprising: simulating human voices and noise in multiple areas of the high-end ceiling microphone respectively, wherein the multiple areas include at least a sound pickup area and a silent area; subjecting the simulated sound signal to speech signal processing in the multiple areas respectively to obtain an input signal of the speech signal processing model; inputting the input signal into the speech signal processing model to obtain an output signal of the speech signal processing model; and training the speech signal processing model based on the output signal, a preset training label, and a preset loss function.
[0007] In a second aspect, an embodiment of the present invention provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the above-mentioned training methods for a speech signal processing model for a high-end ceiling microphone of the present invention.
[0008] In a third aspect, an embodiment of the present invention provides a storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute any of the above-mentioned training methods for a speech signal processing model for a high-end ceiling microphone of the present invention.
[0009] In a fourth aspect, an embodiment of the present invention further provides a computer program product, comprising a computer program stored on a storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer executes any one of the above-mentioned training methods for a speech signal processing model for a high-end ceiling microphone.
[0010] The method of the present application embeds a beamforming algorithm in data simulation, thereby improving regional speech differences and reducing the difficulty of model training; then, voice activity detection is performed only on the speech in the priority pickup area, thereby realizing a learning strategy of priority but not shielding; further, the microphone signal is selected to replace the output of the microphone signal after passing through the fixed beam matrix in the silent area and input into the model, thereby reducing computing power. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0012] Figure 1 A flowchart of a method for training a speech signal processing model for a high-end ceiling microphone provided by an embodiment of the present invention;
[0013] Figure 2 A training flow chart of a speech signal processing model for a specific example of a training method for a speech signal processing model for a high-end ceiling microphone provided in one embodiment of the present invention;
[0014] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0015] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0016] Please refer to Figure 1 , which shows a flow chart of a method for training a speech signal processing model for a high-end ceiling microphone provided by an embodiment of the present invention.
[0017] like Figure 1 As shown, in step 101, human voices and noises in multiple areas of the high-end ceiling microphone are simulated respectively, wherein the multiple areas at least include a sound pickup area and a silent area;
[0018] In step 102, the simulated sound signal is processed by the speech signals of the plurality of regions respectively to obtain an input signal of the speech signal processing model;
[0019] In step 103, the input signal is input into the speech signal processing model to obtain an output signal of the speech signal processing model;
[0020] In step 104, the speech signal processing model is trained based on the output signal, a preset training label, and a preset loss function.
[0021] In this embodiment, for step 101, the training device for the speech signal processing model of the high-end ceiling microphone simulates the human voice and noise of multiple areas of the high-end ceiling microphone respectively, and adjusts the signal-to-interference ratio, signal-to-noise ratio and other information in combination with the actual application scenario. Further, it can also simulate the scene without a speaker, the scene without ordinary pickup area, and the scene with only silent area, and finally simulate the pickup signal of multiple microphones. For example, three groups of room impulse response data are randomly obtained from the prepared data, and the distance between the three groups of room impulse response data is obtained using the coordinate information, and the angle centered on the microphone board and the pitch angle difference information with the microphone board are used as information to distinguish multiple areas.
[0022] Then, for step 102, the training device of the speech signal processing model for the high-end ceiling microphone processes the simulated sound signal through the speech signals of multiple areas respectively to obtain the input signal of the speech signal processing model. For example, the multiple areas include a priority pickup area, a normal pickup area and a quiet area. The simulated sound signal is convolved with the beamforming fixed beam matrix of the priority pickup area to obtain the output of the simulated sound signal after passing through the fixed beam matrix of the priority pickup area; the simulated sound signal is convolved with the beamforming fixed beam matrix of the normal pickup area to obtain the output of the simulated sound signal after passing through the fixed beam matrix of the normal pickup area; the simulated sound signal is convolved with the beamforming fixed beam matrix of the quiet area to obtain the output of the simulated sound signal after passing through the fixed beam matrix of the quiet area. Then, the output of the simulated sound signal after passing through the fixed beam matrix of the quiet area is replaced with the speech signal recorded by the microphone, thereby reducing computing power. Furthermore, the output of the simulated sound signal after passing through the fixed beam matrix of the priority pickup area and the output after passing through the fixed beam matrix of the ordinary pickup area can be selected and input through amplitude comparison and other related logic, and then the output after passing through the fixed beam matrix of the quiet area is combined with the output to eliminate voice silence using a neural network, thereby ensuring that the voice is not distorted.
[0023] Then, for step 103, the training device for the speech signal processing model of the high-end ceiling microphone inputs the input signal into the speech signal processing model to obtain the output signal of the speech signal processing model. For example, the training data of the speech signal processing model includes clean speech and room impulse response data, and then superimposed with the recorded real machine noise data, wherein the clean speech is a training label preset by the speech signal processing model.
[0024] Finally, for step 104, the training device of the speech signal processing model for the high-end ceiling microphone trains the speech signal processing model based on the output signal, the pre-set training label and the preset loss function. For example, the speech signal processing model adopts multi-channel input and uses the CNN-RNN structure, wherein CNN-RNN is a neural network fusion algorithm that combines CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Network). For example, LSTM (Long Short-Term Memory) is a special recurrent neural network that can effectively capture and utilize dependencies in long time series, thereby achieving speech separation. The main learning goal of the network is to retain the direct sound of the priority pickup area and the ordinary pickup area, weaken the interference transmitted from the sound source in other areas, and secondly, learn the priority logic of the sound in the two areas. The final speech signal processing model outputs a 2-channel mask. The loss function is a multi-scale MSE (mean-square error), and multi-scale refers to FFT (fast Fourier transform) with different window lengths, wherein MSE is a measure of the degree of difference between the estimator and the estimated amount.
[0025] The method of this embodiment obtains an input signal of a speech signal processing model by subjecting a simulated sound signal to speech signal processing in multiple regions respectively, thereby improving regional speech differences and reducing the difficulty of model training.
[0026] In some optional embodiments, when the area is the sound pickup area, the sound pickup area includes a priority sound pickup area and a common sound pickup area, and the voice signal processing of the simulated sound signal passing through the multiple areas respectively includes:
[0027] The training device of the speech signal processing model for the high-end ceiling microphone performs fixed beam matrix convolution and speech activity detection on the sound signal simulated in the priority pickup area; the training device of the speech signal processing model for the high-end ceiling microphone performs fixed beam matrix convolution on the sound signal simulated in the ordinary pickup area, thereby realizing a priority but not shielding learning strategy.
[0028] In some optional embodiments, all areas except the sound pickup area are set as silent areas. When the area is a silent area, the voice signal processing of the simulated sound signal through the multiple areas respectively includes:
[0029] The training device for the speech signal processing model of the high-end ceiling microphone replaces the signal of the quiet zone after beamforming with a preset microphone signal, thereby reducing computing power calculations; further, all areas outside the priority pickup area and the ordinary pickup area are used as quiet zones without the need for beamforming calculations, thereby expanding the concept of quiet zones.
[0030] In some optional embodiments, the multiple areas include a priority sound pickup area, a normal sound pickup area, and a silent area, and the human voice and noise of the multiple areas of the high-end ceiling microphone are respectively simulated including:
[0031] The training device for the speech signal processing model of the high-end ceiling microphone obtains room impulse response data and noise data, corresponds the room impulse response data to multiple coordinate areas divided within the pickup range, and obtains multiple groups of room impulse response data; then, the training device for the speech signal processing model of the high-end ceiling microphone simulates the human voice and noise in the priority pickup area, the ordinary pickup area and the quiet area based on the multiple groups of room impulse response data and noise data, and obtains simulated multi-microphone pickup signals.
[0032] For example, three groups of pure speech, three groups of room impulse response data and one group of noise data are prepared, three groups of room impulse response data are randomly obtained, and the distance between the three groups of room impulse response data, the angle centered on the microphone plate and the pitch angle difference information with the microphone plate are obtained by using the coordinate information as information to distinguish the priority pickup area, the ordinary pickup area and the silent area. In a specific example, the coordinate distance between each two groups of room impulse response data is more than 1m, the angle difference is more than 20°, and the pitch angle difference is more than 20°. For example, the room impulse response data is prepared in advance, and the coordinate search is performed using beamforming to obtain the specific coordinates xy of each room impulse response data relative to the ceiling microphone. Different room impulse response data can use the actual coordinates to calculate the distance between the two points and the angle and pitch angle relative to the ceiling microphone; the coordinates of the ceiling microphone are fixed (0, 0, 3.0 or other heights). Then, the three groups of pure speech and the three groups of RIR are convolved respectively to adjust the signal-to-interference ratio, signal-to-noise ratio, amplitude and other information. At the same time, the scene without a speaker, the scene without speaking in the ordinary pickup area, and the scene with only speaking in the silent area are considered. Finally, the pickup signal of multiple microphones is simulated. For example, there are three types of areas here, and there are scenes with or without people speaking in this area, that is, the permutation and combination mode. In the data simulation, the data is simulated in proportion. If someone speaks, there is pure speech, and if no one speaks, the pure speech is removed.
[0033] In some optional embodiments, the acquiring room impulse response data and noise data includes:
[0034] Play and record white noise, and obtain the room impulse response data based on the adaptive filtering algorithm; then simulate or play the noise on site for recording to obtain the noise data. For example, the room impulse response data is recorded by a real machine, the microphone board is connected to the power amplifier speaker, and the white noise is played and recorded by itself, which helps to automatically obtain the timing information, including the spatial distance propagation information, and then the room impulse response data is obtained by using the adaptive filtering algorithm, so that the phase information of multiple microphones can be obtained.
[0035] In some optional embodiments, the step of making the room impulse response data correspond to a plurality of coordinate areas divided within the sound pickup range to obtain a plurality of groups of room impulse response data includes:
[0036] The training device for the speech signal processing model of the high-end ceiling microphone divides a plurality of coordinate areas within the pickup range of the microphone board, searches for the coordinates of the room impulse response data based on the beamforming algorithm and renames the room impulse response data.
[0037] For example, thousands of coordinate areas are divided within the pickup range of the microphone board. The acquired room impulse response data needs to correspond to the coordinates of the specific divided areas. The recorded room impulse response data is used to convolve the pure speech, and the coordinates of the room impulse response data are obtained through the beamforming algorithm search and renamed.
[0038] The method of this embodiment embeds a beamforming algorithm, thereby improving regional speech differences and reducing the difficulty of model training.
[0039] In some optional embodiments, based on the multiple sets of room impulse response data and noise data, respectively simulating human voices and noises in the priority pickup area, the common pickup area and the quiet area to obtain simulated pickup signals of multiple microphones includes:
[0040] The training device for the speech signal processing model of the high-end ceiling microphone randomly obtains three groups of room impulse response data, and obtains the distance between the three groups of room impulse response data based on the coordinate information of the three groups of room impulse response data; then, based on the distance between the three groups of room impulse response data, obtains the angle information centered on the microphone board and the pitch angle difference information with the microphone board; finally, distinguishes the priority pickup area, the common pickup area and the silent area based on the angle information and the pitch angle difference information. For example, coordinate search is performed using beamforming to obtain the specific coordinates xy of each room impulse response data relative to the ceiling microphone. Different room impulse response data can use the actual coordinates to calculate the distance between the two points and the angle and pitch angle relative to the ceiling microphone; the coordinates of the ceiling microphone are fixed (0, 0, 3.0 or other heights).
[0041] In some optional embodiments, based on the multiple sets of room impulse response data and noise data, respectively simulating human voices and noises in the priority sound pickup area, the common sound pickup area and the quiet area to obtain simulated multi-microphone pickup signals further includes:
[0042] The training device for the speech signal processing model of the high-end ceiling microphone obtains three sets of clean speech and three sets of room impulse response data, convolves the three sets of clean speech and three sets of room impulse response data respectively, and adjusts the signal-to-interference ratio, signal-to-noise ratio and amplitude. For example, the clean speech is convolved with the room impulse response data, and then the recorded real machine noise data is superimposed. This is the training data of the model, and the clean speech is the label of the model.
[0043] Please refer to Figure 2 , which shows a training flow chart of a speech signal processing model for a specific example of a training method for a speech signal processing model for a high-end ceiling microphone provided by an embodiment of the present invention.
[0044] like Figure 2 As shown, step 1: data simulation, simulates the human voice and noise in three areas respectively, adjusts the signal-to-interference ratio, signal-to-noise ratio and other information in combination with the actual application scenario, and considers the scene without a speaker, the scene without speaking in the ordinary pickup area, and the scene with only speaking in the silent area, etc., and finally simulates the pickup signal of multiple microphones.
[0045] Step 2: Convolve the Mic signal with the fixed beam matrix of Beamforming (beamforming algorithm) in different areas to obtain the input signal of the model, namely the priority area VIP_bf (the output of the Mic signal after passing through the fixed beam matrix of the priority pickup area), Normal_bf (the output of the Mic signal after passing through the fixed beam matrix of the normal pickup area), and Mic (the voice signal recorded by the microphone).
[0046] Step 3: Set the model training label, perform beamforming on the sound source in each area during data simulation, and perform Vad (voice activation detection) on the VIP voice. The final label = VIP_target + (1-Vad) * Normal_target. Among them, VIP_target is the output of the pure VIP signal after passing through the fixed beam matrix of the priority pickup area, and Normal_target is the output of the pure Normal signal after passing through the fixed beam matrix of the normal pickup area.
[0047] Step 4: Model training, the final model output is a 2-channel mask, and the total output of the algorithm is output = VIP_bf*mask1+Normal_target*mask2. Among them, mask1 / mask2 is the mask output of the neural network. Among them, the loss function is multi-scale MSE, and multi-scale refers to FFT with different window lengths.
[0048] The training method of the speech signal processing model for high-end ceiling microphones consists of data preparation, beamforming labeling, data simulation, model training, etc.:
[0049] Data preparation: The main data are room impulse response (RIR) and noise data. RIR is recorded with a real machine. The microphone board is connected to the amplifier speaker, and white noise is played and recorded by itself. Self-playing and self-recording helps to automatically obtain timing information, including spatial distance propagation information, and then use the adaptive filtering algorithm to obtain RIR; noise data is recorded by live simulation or playing various noises. Data recording takes into account room scenes of different sizes and reverberation times. The purpose of real machine recording is to obtain phase information of multiple microphones.
[0050] Beamforming marking: Thousands of coordinate areas are divided within the pickup range of the microphone board. The RIR data obtained needs to correspond to the coordinates of the specific divided areas. The recorded RIR convolution pure speech is used to search and obtain the coordinates of the RIR through the beamforming algorithm and rename the RIR.
[0051] Data simulation: prepare three groups of pure speech, three groups of RIR and one group of noise data, randomly obtain three groups of RIR, use the coordinate information to obtain the distance between the three groups of RIR, the angle centered on the microphone board and the pitch angle difference information with the microphone board, as the information to distinguish the priority pickup area, the normal pickup area and the silent area. Combined with the actual application scenario, it is recommended that the coordinate distance between each two groups of RIRs is more than 1m, the angle difference is more than 20°, and the pitch angle difference is more than 20°. Among them, there are RIRs prepared in advance, and beamforming is used to perform coordinate search to obtain the specific coordinates xy of each RIR relative to the ceiling microphone. Different RIRs can use the actual coordinates to calculate the distance between the two points and the angle and pitch angle relative to the ceiling microphone; the coordinates of the ceiling microphone are fixed at (0, 0, 3.0), where 3.0 is the height of the ceiling microphone, and it can also be other heights. This application is not limited here. The three groups of pure speech and three groups of RIR are convolved to adjust the signal-to-interference ratio, signal-to-noise ratio, amplitude and other information, and consider the scene without a speaker, the scene without ordinary pickup area speaking, and the scene with only silent area speaking. Specifically, there are three types of areas here (VIP pickup area, ordinary pickup area and silent area), and there are scenes with or without people speaking in this area, that is, the pattern of permutation and combination. In the data simulation, the data is simulated in proportion. If someone speaks, there is pure speech, and if no one speaks, the pure speech is removed.
[0052] Model training: The model uses multi-channel input and a CNN-RNN structure, in which LSTM can effectively capture and utilize dependencies in long time series to achieve speech separation. The main learning goal of the network is to retain the direct sound of the priority pickup area and the ordinary pickup area, weaken the interference transmitted from the sound sources in other areas, and secondly learn the priority logic of the sound in the two areas. Among them, the training data of the model is the clean speech convolution RIR, and then the recorded real machine noise data is superimposed. The clean speech is the label of the model. The loss function can use multi-scale MSE, and multi-scale refers to FFT with different window lengths.
[0053] When preparing data, the concept of quiet zone is expanded. Instead of independently dividing an area as a quiet zone, all areas outside the priority pickup area and the ordinary pickup area are regarded as quiet zones. At the same time, considering that other areas may exceed the range of pickup division, a microphone signal is used to replace the beamformed signal of the quiet zone as the model input of the third channel. In addition, beamforming is a type of speech enhancement algorithm, and the output of the speech in the corresponding area will have an enhancement effect, so the use of microphone signals in the third channel can play a certain role in identification.
[0054] In some other embodiments, the present invention also provides a non-volatile computer storage medium, the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the training method for the speech signal processing model for the high-end ceiling microphone in any of the above method embodiments;
[0055] As an implementation mode, the non-volatile computer storage medium of the present invention stores computer executable instructions, and the computer executable instructions are configured as follows:
[0056] Simulating human voices and noises in multiple areas of the high-end ceiling microphone respectively, wherein the multiple areas at least include a sound pickup area and a silent area;
[0057] Processing the simulated sound signal through the speech signals of the plurality of regions respectively to obtain an input signal of the speech signal processing model;
[0058] Inputting the input signal into the speech signal processing model to obtain an output signal of the speech signal processing model;
[0059] The speech signal processing model is trained based on the output signal, a preset training label and a preset loss function.
[0060] The non-volatile computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the training device for the speech signal processing model of the high-end ceiling microphone, etc. In addition, the non-volatile computer-readable storage medium may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the training device for the speech signal processing model of the high-end ceiling microphone via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0061] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any of the above-mentioned training methods for a speech signal processing model for a high-end ceiling microphone.
[0062] Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, the device includes: one or more processors 310 and a memory 320, Figure 3 A processor 310 is taken as an example. The device for training the speech signal processing model of the high-end ceiling microphone may also include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330 and the output device 340 may be connected via a bus or other means. Figure 3 The example of connection via bus is taken. The memory 320 is the above-mentioned non-volatile computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, implements the training method of the speech signal processing model for high-end ceiling microphones in the above-mentioned method embodiment. The input device 330 can receive input digital or character information, and generate key signal input related to user settings and function controls of the training device for the speech signal processing model of high-end ceiling microphones. The output device 340 may include display devices such as display screens.
[0063] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.
[0064] As an implementation mode, the electronic device is applied to a training device for a speech signal processing model for a high-end ceiling microphone, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can:
[0065] Simulating human voices and noises in multiple areas of the high-end ceiling microphone respectively, wherein the multiple areas at least include a sound pickup area and a silent area;
[0066] Processing the simulated sound signal through the speech signals of the plurality of regions respectively to obtain an input signal of the speech signal processing model;
[0067] Inputting the input signal into the speech signal processing model to obtain an output signal of the speech signal processing model;
[0068] The speech signal processing model is trained based on the output signal, a preset training label and a preset loss function.
[0069] The electronic device of the embodiment of the present application exists in various forms, including but not limited to:
[0070] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.
[0071] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.
[0072] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.
[0073] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.
[0074] (5) Other electronic devices with data interaction functions.
[0075] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for training a speech signal processing model for a high-end ceiling microphone, comprising: Simulating human voices and noises in multiple areas of the high-end ceiling microphone respectively, wherein the multiple areas at least include a sound pickup area and a silent area; Processing the simulated sound signal through the speech signals of the plurality of regions respectively to obtain an input signal of the speech signal processing model; Inputting the input signal into the speech signal processing model to obtain an output signal of the speech signal processing model; The speech signal processing model is trained based on the output signal, a preset training label and a preset loss function.
2. The method according to claim 1, wherein: When the area is the sound pickup area, the sound pickup area includes a priority sound pickup area and a common sound pickup area, and the voice signal processing of the simulated sound signal passing through the multiple areas respectively includes: Performing fixed beam matrix convolution and voice activity detection on the sound signal simulated in the priority pickup area; The sound signal simulated in the common sound pickup area is subjected to fixed beam matrix convolution.
3. The method according to claim 1, wherein: All areas except the sound pickup area are set as silent areas. When the area is the silent area, the voice signal processing of the simulated sound signal through the multiple areas includes: The beamformed signal of the quiet zone is replaced by a preset microphone signal.
4. The method according to claim 1, wherein: The multiple areas include a priority sound pickup area, a normal sound pickup area and a silent area, and the human voices and noises in the multiple areas of the high-end ceiling microphone are respectively simulated as follows: Acquire room impulse response data and noise data, and correspond the room impulse response data to a plurality of coordinate areas divided within the sound pickup range to obtain a plurality of groups of room impulse response data; Based on the multiple groups of room impulse response data and noise data, human voices and noises in the priority sound pickup area, the common sound pickup area and the silent area are simulated respectively to obtain simulated pickup signals of multiple microphones.
5. The method according to claim 4, wherein: The obtaining of room impulse response data and noise data comprises: Play and record white noise, and obtain the room impulse response data based on an adaptive filtering algorithm; The noise is simulated on site or played and recorded to obtain the noise data.
6. The method according to claim 4, wherein: The step of making the room impulse response data correspond to a plurality of coordinate areas divided within the sound pickup range to obtain a plurality of groups of room impulse response data includes: A plurality of coordinate areas are divided within the sound pickup range of the microphone board, and the coordinates of the room impulse response data are searched based on a beamforming algorithm and the room impulse response data are renamed.
7. The method according to claim 4, wherein: The method of simulating the human voice and noise in the priority sound pickup area, the common sound pickup area and the quiet area based on the multiple groups of room impulse response data and noise data to obtain simulated multi-microphone pickup signals includes: Randomly acquiring three groups of room impulse response data, and acquiring distances between the three groups of room impulse response data based on coordinate information of the three groups of room impulse response data; Based on the distances between the three groups of room impulse response data, obtaining angle information centered on the microphone panel and pitch angle difference information with respect to the microphone panel; A priority sound pickup area, a common sound pickup area and a silent area are distinguished based on the included angle information and the pitch angle difference information.
8. The method according to claim 7, wherein: The method of simulating the human voice and noise in the priority sound pickup area, the common sound pickup area and the quiet area based on the multiple groups of room impulse response data and noise data to obtain simulated multi-microphone pickup signals also includes: Three groups of clean speech and three groups of room impulse response data are obtained, and the three groups of clean speech and the three groups of room impulse response data are convolved respectively, and the signal-to-interference ratio, signal-to-noise ratio and amplitude are adjusted.
9. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 1 to 8.
10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.