Model training method and device, electronic equipment, vehicle and storage medium
Patent Information
- Application Number
- CN202210675255.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-15
- Publication Date
- 2026-10-09
- Estimated Expiration
- 2042-06-15
AI Technical Summary
然而，回波消除模型与语音唤醒模型的训练过程通常是相互独立的，是对回波消除模型与语音唤醒模型各自对应的局部优化，对语音唤醒率的提高仍有一定的局限性
[0059] Another embodiment of this disclosure provides a vehicle that includes the aforementioned electronic equipment.
Smart Images
Figure CN117275501B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of acoustic technology, and more particularly to a model training method, apparatus, electronic device, vehicle, and storage medium. Background Technology
[0002] With the widespread adoption of smart electronic devices, voice wake-up functionality is increasingly being used in these devices, such as in smart in-vehicle terminals. Smart electronic devices typically incorporate microphones or speakers to implement voice wake-up. They use microphones to capture user voice and employ speech recognition technology to understand the user's intentions or commands. However, when sound is played from the speaker of the smart electronic device or other devices, the echo signal from the speaker travels through the air or other media to the microphone. At this point, the audio signal captured by the microphone may contain a superimposed combination of the user's voice signal and the echo signal from the speaker. This echo signal can interfere with speech recognition and affect voice wake-up, resulting in a high false wake-up rate.
[0003] In related technologies, audio signals collected by a microphone can be used to train an echo cancellation model, and the echo-cancelled audio signals can be used to train a voice wake-up model. However, the training processes of the echo cancellation model and the voice wake-up model are usually independent of each other, representing local optimizations of their respective models, and thus have certain limitations in improving the voice wake-up rate. Summary of the Invention
[0004] This disclosure proposes a model training method, apparatus, electronic device, vehicle, and storage medium, aiming to at least partially solve one of the technical problems in the related art.
[0005] One embodiment of this disclosure proposes a model training method, which includes:
[0006] Acquire training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal;
[0007] The reference signal and the microphone signal are input into the acoustic echo cancellation model to obtain the echo-cancelled audio signal;
[0008] The echo-cancelled audio signal is input into the voice wake-up model to obtain the inferred wake-up result;
[0009] Based on the original audio signal and the echo-cancelled audio signal, determine the first loss value of the acoustic echo cancellation model;
[0010] Based on the actual wake-up result and the inferred wake-up result, a second loss value for the voice wake-up model is determined;
[0011] The acoustic echo cancellation model is trained based on the first loss value and the second loss value;
[0012] The voice wake-up model is trained based on the second loss value.
[0013] In one embodiment of this disclosure, training the acoustic echo cancellation model based on the first loss value and the second loss value includes:
[0014] The first loss value and the second loss value are weighted and summed to obtain the total loss value;
[0015] The acoustic echo cancellation model is trained based on the total loss value.
[0016] In one embodiment of this disclosure, training the acoustic echo cancellation model based on the total loss value includes:
[0017] Based on the total loss value, determine the parameter gradient of the acoustic echo cancellation model;
[0018] Based on the parameter gradient, the model parameters in the acoustic echo model are adjusted in the direction of decreasing the total loss value until the total loss value meets the preset condition, so as to obtain the trained acoustic echo model.
[0019] In one embodiment of this disclosure, determining the first loss value of the acoustic echo cancellation model based on the original audio signal and the echo-cancelled audio signal includes:
[0020] Obtain the first loss function of the acoustic echo cancellation model;
[0021] Using the first loss function, loss calculations are performed on the original audio signal and the echo-cancelled audio signal to obtain the first loss value of the acoustic echo cancellation model.
[0022] In one embodiment of this disclosure, determining the second loss value of the voice wake-up model based on the actual wake-up result and the inferred wake-up result includes:
[0023] Obtain the second loss function of the voice wake-up model;
[0024] The second loss function is used to calculate the loss between the actual wake-up result and the inferred wake-up result to obtain the second loss value of the voice wake-up model.
[0025] Another embodiment of this disclosure provides a voice wake-up method, the method comprising:
[0026] Acquire the first audio signal picked up by the microphone and the second audio signal output by the speaker;
[0027] The second audio signal is determined as the reference signal for echo cancellation;
[0028] The first audio signal and the reference signal are input into the acoustic echo cancellation model to obtain the third audio signal after echo cancellation;
[0029] The third audio signal is input into the voice wake-up model to obtain a wake-up result, wherein the acoustic echo cancellation model and the voice wake-up model are trained according to the training method of the model.
[0030] The voice wake-up method disclosed herein can eliminate the reference signal in the audio signal collected by the microphone, making the audio signal input into the voice wake-up model closer to the original audio signal, and the wake-up result of the voice wake-up model closer to the real wake-up result.
[0031] Another embodiment of this disclosure provides a model training apparatus, which includes:
[0032] The first acquisition module is used to acquire training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal.
[0033] The first input module is used to input the reference signal and the microphone signal into the acoustic echo cancellation model to obtain the echo-cancelled audio signal;
[0034] The second input module is used to input the echo-cancelled audio signal into the voice wake-up model to obtain the inferred wake-up result;
[0035] The first determining module is used to determine the first loss value of the acoustic echo cancellation model based on the original audio signal and the audio signal after echo cancellation.
[0036] The second determining module is used to determine the second loss value of the voice wake-up model based on the actual wake-up result and the inferred wake-up result;
[0037] The first training module is used to train the acoustic echo cancellation model based on the first loss value and the second loss value.
[0038] The second training module is used to train the voice wake-up model based on the second loss value.
[0039] In one embodiment of this disclosure, the first training module includes:
[0040] The weighted summation unit is used to perform a weighted summation of the first loss value and the second loss value to obtain the total loss value;
[0041] The training unit is used to train the acoustic echo cancellation model based on the total loss value.
[0042] In one embodiment of this disclosure, the training unit is specifically used for:
[0043] Based on the total loss value, determine the parameter gradient of the acoustic echo cancellation model;
[0044] Based on the parameter gradient, the model parameters in the acoustic echo model are adjusted in the direction of decreasing the total loss value until the total loss value meets the preset condition, so as to obtain the trained acoustic echo model.
[0045] In one embodiment of this disclosure, the first determining module includes:
[0046] The first acquisition unit is used to acquire the first loss function of the acoustic echo cancellation model;
[0047] The first calculation unit is used to perform loss calculation on the original audio signal and the echo-cancelled audio signal using the first loss function to obtain the first loss value of the acoustic echo cancellation model.
[0048] In one embodiment of this disclosure, the second determining module includes:
[0049] The second acquisition unit is used to acquire the second loss function of the voice wake-up model;
[0050] The second calculation unit is used to perform loss calculation on the actual wake-up result and the inferred wake-up result using the second loss function to obtain the second loss value of the voice wake-up model.
[0051] Another embodiment of this disclosure provides a voice wake-up device, the device comprising:
[0052] The second acquisition module is used to acquire the first audio signal collected by the microphone and the second audio signal output by the speaker;
[0053] The third determining module is used to determine the second audio signal as a reference signal for echo cancellation;
[0054] The third input module is used to input the first audio signal and the reference signal into the acoustic echo cancellation model to obtain the echo-cancelled third audio signal;
[0055] The fourth input module is used to input the third audio signal into the voice wake-up model to obtain a wake-up result, wherein the acoustic echo cancellation model and the voice wake-up model are trained according to the training method of the model.
[0056] The voice wake-up device disclosed herein can eliminate the reference signal in the audio signal collected by the microphone, so that the audio signal input into the voice wake-up model is closer to the original audio signal, and the wake-up result of the voice wake-up model is closer to the real wake-up result.
[0057] Another embodiment of this disclosure proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described model training method or the above-described voice wake-up method.
[0058] Another embodiment of this disclosure proposes a non-transitory computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the above-described model training method or the above-described voice wake-up method.
[0059] Another embodiment of this disclosure provides a vehicle that includes the aforementioned electronic equipment.
[0060] Another embodiment of this disclosure proposes a computer program product having a computer program stored thereon, which, when executed by a processor, implements the above-described model training method or the above-described voice wake-up method.
[0061] The model training method, apparatus, electronic device, and storage medium disclosed herein first acquire training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal. The reference signal and microphone signal are input into an acoustic echo cancellation model to obtain an echo-cancelled audio signal. The echo-cancelled audio signal is then input into a speech wake-up model to obtain an inferred wake-up result. Next, based on the original audio signal and the echo-cancelled audio signal, a first loss value is determined for the acoustic echo cancellation model. Based on the actual wake-up result and the inferred wake-up result, a second loss value is determined for the speech wake-up model. The acoustic echo cancellation model is trained based on the first and second loss values, and the speech wake-up model is trained based on the second loss value. Thus, by jointly training the acoustic echo cancellation model and the speech wake-up model, joint optimization of the acoustic echo cancellation model and the speech wake-up model is achieved, resulting in a speech wake-up model with high accuracy.
[0062] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description
[0063] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:
[0064] Figure 1 A schematic flowchart of a model training method provided in an embodiment of this disclosure;
[0065] Figure 2 A flowchart illustrating another model training method provided in this embodiment of the disclosure;
[0066] Figure 3 A flowchart illustrating another model training method provided in this embodiment of the disclosure;
[0067] Figure 4 A flowchart illustrating another model training method provided in this embodiment of the disclosure;
[0068] Figure 5 A flowchart illustrating a voice wake-up method provided in an embodiment of this disclosure;
[0069] Figure 6 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;
[0070] Figure 7 This is a schematic diagram of another model training device provided in an embodiment of the present disclosure;
[0071] Figure 8 This is a schematic diagram of the structure of a voice wake-up device provided in an embodiment of the present disclosure;
[0072] Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation
[0073] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0074] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0075] The model training method and apparatus of this disclosure are described below with reference to the accompanying drawings.
[0076] Figure 1 This is a schematic flowchart of a model training method provided in an embodiment of the present disclosure.
[0077] It should be noted that the model training method provided in this embodiment is applied in a model training device, which can be implemented by software and / or hardware. The model training device can be an electronic device or can be configured in an electronic device. In this embodiment, the electronic device can be a PC (Personal Computer), mobile device, tablet computer, terminal device or server, etc., without specific limitations.
[0078] The electronic device is equipped with a microphone for collecting audio signals and a speaker for outputting audio signals.
[0079] like Figure 1 As shown, the model training method includes:
[0080] Step 101: Obtain training data.
[0081] The training data includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal.
[0082] In one embodiment of this disclosure, the microphone signal is an audio signal collected by a microphone, wherein the microphone signal includes the original audio signal, a reference signal corresponding to the microphone signal, and various environmental noises, etc.
[0083] In this context, the original audio signal can be understood as the clean audio signal input into the microphone. The wake-up result produced by inputting this original audio signal into the voice wake-up model is the actual wake-up result.
[0084] For example, the original audio signal could be the sound produced by a person's vocal organs, in which case the original audio signal is considered to be free of any other sounds.
[0085] In some exemplary embodiments, the audio signal output by the speaker can be used as a reference signal.
[0086] In one embodiment of this disclosure, the audio signal output by the speaker during playback can reach the microphone and interfere with the microphone's original audio signal. Therefore, the audio signal output by the speaker can be used as a reference signal, and this reference signal can be eliminated in the acoustic echo cancellation model.
[0087] In some other exemplary embodiments, in a full-duplex communication system, the original audio signal can be a near-end speech signal, and the reference signal can be the speech signal of a far-end speaker in the full-duplex communication system.
[0088] In one embodiment of this disclosure, in a full-duplex communication system, there are, for example, users A and B communicating. User A is defined as the near-end user of the full-duplex communication system, and user B is defined as the far-end user. When user A and user B communicate, user B's voice signal can interfere with user A's voice signal at user A's location. In this case, user B's voice signal can be used as a reference signal, and this reference signal can be eliminated using an acoustic echo cancellation model.
[0089] Step 102: Input the reference signal and microphone signal into the acoustic echo cancellation model to obtain the echo-cancelled audio signal.
[0090] In one embodiment of this disclosure, a reference signal and a microphone signal can be simultaneously input into an acoustic echo cancellation model. As described above, the microphone signal also contains a reference signal. Therefore, the reference signal contained in the microphone signal can be eliminated in the acoustic echo cancellation (AEC) model based on the reference signal input simultaneously with the microphone signal to obtain an echo-cancelled audio signal.
[0091] Step 103: Input the echo-cancelled audio signal into the voice wake-up model to obtain the inferred wake-up result.
[0092] In one embodiment of this disclosure, the voice wake-up model can detect the wake-up word for the input echo-cancelled audio signal and generate a wake-up result, which is referred to as the inferred wake-up result.
[0093] The wake word can be a pre-set word or phrase used to wake up a smart device.
[0094] The inferred wake-up result may be the same as or different from the actual wake-up result.
[0095] Step 104: Determine the first loss value of the acoustic echo cancellation model based on the original audio signal and the audio signal after echo cancellation.
[0096] It is understandable that some loss may occur during the propagation and transmission of speech. The first loss value of the acoustic echo cancellation model can be determined based on the original audio signal and the audio signal after echo cancellation.
[0097] As an exemplary implementation, the first loss function of the acoustic echo cancellation model can be used to calculate the loss of the original audio signal and the echo-cancelled audio signal to obtain the first loss value of the acoustic echo cancellation model.
[0098] As another exemplary implementation, the difference information between the original audio signal and the echo-cancelled audio signal can be determined, and a first loss value of the acoustic echo cancellation model can be determined based on the difference information.
[0099] Step 105: Determine the second loss value of the voice wake-up model based on the actual wake-up result and the inferred wake-up result.
[0100] As an exemplary implementation, the second loss function of the acoustic echo cancellation model can be used to calculate the loss between the actual wake-up result and the inferred wake-up result, so as to obtain the second loss value of the acoustic echo cancellation model.
[0101] As another exemplary implementation, the difference information between the actual wake-up result and the inferred wake-up result can be determined, and a second loss value of the acoustic echo cancellation model can be determined based on the difference information.
[0102] Step 106: Train the acoustic echo cancellation model based on the first loss value and the second loss value.
[0103] For a detailed description of training the acoustic echo cancellation model, please refer to steps 201-203, which will not be repeated here.
[0104] Step 107: Train the voice wake-up model based on the second loss value.
[0105] The model training method disclosed herein first acquires training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal. The reference signal and microphone signal are input into an acoustic echo cancellation model to obtain an echo-cancelled audio signal. This echo-cancelled audio signal is then input into a speech wake-up model to obtain an inferred wake-up result. Next, based on the original audio signal and the echo-cancelled audio signal, a first loss value is determined for the acoustic echo cancellation model. Based on the actual wake-up result and the inferred wake-up result, a second loss value is determined for the speech wake-up model. The acoustic echo cancellation model is trained based on the first and second loss values, and the speech wake-up model is trained based on the second loss value. Thus, by jointly training the acoustic echo cancellation model and the speech wake-up model, joint optimization of the acoustic echo cancellation model and the speech wake-up model is achieved, resulting in a speech wake-up model with high accuracy.
[0106] Figure 2 This is a flowchart illustrating another model training method provided in an embodiment of the present disclosure.
[0107] like Figure 2 As shown, one possible implementation of training the acoustic echo cancellation model based on the first and second loss values in step 106 is as follows:
[0108] Step 201: Perform a weighted summation of the first loss value and the second loss value to obtain the total loss value.
[0109] In one embodiment of this disclosure, it is assumed that the first loss value and the second loss value are respectively represented by Loss AEC and Loss KWS This indicates that the first loss value (Loss) is... AEC Second loss value Loss KWS During the weighted summation process, a first loss value (Loss) is assigned to each. AEC Second loss value Loss KWS The corresponding parameters.
[0110] Specifically, a first loss value, Loss, can be assigned. AEC A weight α, then the second loss value Loss KWS The weight is (1-α). At this point, the first loss value is Loss. AEC Second loss value Loss KWS The total loss obtained by weighted summation is the Loss. Total =α*Loss AEC +(1-α)*Loss KWS.
[0111] Step 202: Determine the parameter gradient of the acoustic echo cancellation model based on the total loss value.
[0112] In one embodiment of this disclosure, one way to determine the parameter gradient of the acoustic echo cancellation model is to take a partial derivative with respect to the total loss value.
[0113] Specifically, assuming the acoustic echo cancellation model is represented as Model AEC The partial derivative with respect to the total loss value can then be expressed as:
[0114] The range of values for α is: 0 < α < 1.
[0115] Step 203: Based on the parameter gradient, adjust the model parameters in the acoustic echo model in the direction that reduces the total loss value until the total loss value meets the preset condition, thereby obtaining the trained acoustic echo model.
[0116] As can be seen from step 201 above, the first loss value is Loss. AEC Second loss value Loss KWS The corresponding weights α and 1-α sum to 1. This is the first loss value. AEC Second loss value Loss KWS The corresponding weights α and 1-α are not fixed values; their sum is sufficient to be 1.
[0117] Based on this characteristic, the first loss value Loss can be... AEC Second loss value Loss KWS The corresponding parameters α and 1-α are adjusted in the direction of decreasing the total loss value until the total loss value meets the preset conditions to obtain the trained acoustic echo model.
[0118] The preset conditions are the conditions that must be met when the acoustic echo model training ends. These preset conditions can be configured according to actual needs. For example, the preset condition for the total loss value to meet the requirement can be that the total loss value is less than a preset value, or that the change in the total loss value approaches stability, meaning that the difference between the total loss values corresponding to two or more adjacent training iterations is less than the set value, and the total loss value basically no longer changes.
[0119] Figure 3 This is a flowchart illustrating another model training method provided in an embodiment of the present disclosure.
[0120] like Figure 3 As shown, one possible implementation of determining the first loss value of the acoustic echo cancellation model in step 104 based on the original audio signal and the audio signal after echo cancellation is as follows:
[0121] Step 301: Obtain the first loss function of the acoustic echo cancellation model.
[0122] In one embodiment of this disclosure, assuming the input microphone signal is M, the reference signal is R, the echo-cancelled audio signal is E, and the original audio signal is C, the acoustic echo cancellation model is represented as Model AEC Therefore, according to the above embodiments, E = Model AEC (M,R).
[0123] The first loss function is denoted as X. Based on the audio signal after echo cancellation being E and the original audio signal being C, we can derive: X = Loss AEC (C,E).
[0124] Step 302: Using the first loss function, calculate the loss of the original audio signal and the audio signal after echo cancellation to obtain the first loss value of the acoustic echo cancellation model.
[0125] Specifically, the echo-cancelled audio signal E and the original audio signal C are used as inputs, and the first loss value is calculated using the first loss function X of the acoustic echo cancellation model.
[0126] Figure 4 This is a flowchart illustrating another model training method provided in an embodiment of the present disclosure.
[0127] like Figure 4 As shown, one possible implementation of determining the second loss value of the voice wake-up model in step 105 based on the actual wake-up result and the inferred wake-up result is as follows:
[0128] Step 401: Obtain the second loss function of the voice wake-up model.
[0129] In one embodiment of this disclosure, the voice wake-up model can be represented as Model KWS The wake-up result inferred by the voice wake-up model is Hyp. KWS The actual wake-up result is Ref KWS .
[0130] It can be seen that the voice wake-up model is... KWS The input is the acoustic echo cancellation model, represented as Model. AEC The output is the audio signal E after echo cancellation. Therefore, Hyp... KWS =Model KWS (E).
[0131] The second loss function is denoted as Y, and the inferred wake-up result is Hyp. KWS The actual wake-up result is RefKWS We can conclude that: Y = Loss KWS (Hyp KWS Ref KWS ).
[0132] Step 402: Using the second loss function, the loss is calculated on the actual wake-up result and the inferred wake-up result to obtain the second loss value of the voice wake-up model.
[0133] Specifically, the inferred arousal result is Hyp KWS The actual wake-up result is Ref KWS As input, the second loss value is calculated using the second loss function Y of the voice wake-up model.
[0134] The model training method disclosed herein jointly trains the acoustic echo cancellation model and the voice wake-up model, thereby achieving joint optimization of the acoustic echo cancellation model and the voice wake-up model, so that the accuracy of the trained voice wake-up model can reach the optimal level.
[0135] This disclosure also proposes a voice wake-up method. Figure 5 This is a flowchart illustrating a voice wake-up method provided in an embodiment of the present disclosure.
[0136] This embodiment describes the application of the voice wake-up method in a smart electronic device as an example. The electronic device is equipped with a microphone and a speaker.
[0137] like Figure 5 As shown, the voice wake-up method includes:
[0138] Step 501: Acquire the first audio signal collected by the microphone and acquire the second audio signal output by the speaker.
[0139] It is understood that, in this embodiment, the audio signal output by the speaker can also be used as the input audio signal for the microphone when propagated through a medium. That is to say, the first audio signal collected by the microphone includes the second audio signal output by the speaker.
[0140] Step 502: The second audio signal is determined as the reference signal for echo cancellation.
[0141] In one embodiment of this disclosure, the audio signal that needs to be eliminated from the first audio signal collected by the microphone, i.e. the second audio signal output by the speaker, is used as a reference signal.
[0142] Step 503: Input the first audio signal and the reference signal into the acoustic echo cancellation model to obtain the third audio signal after echo cancellation.
[0143] Step 504: Input the third audio signal into the voice wake-up model to obtain the wake-up result.
[0144] In one embodiment of this disclosure, after a third audio signal is input into a voice wake-up model, the voice wake-up model can detect the wake-up word for the input third audio signal.
[0145] If a wake word is detected, the smart electronic device can be woken up; if no wake word is detected, the smart electronic device cannot be woken up.
[0146] Among them, the acoustic echo cancellation model and the voice wake-up model are trained according to the above model training method.
[0147] The voice wake-up method disclosed herein can eliminate the reference signal in the audio signal collected by the microphone, making the audio signal input into the voice wake-up model closer to the original audio signal, and the wake-up result of the voice wake-up model closer to the real wake-up result.
[0148] To implement the above embodiments, this disclosure also proposes a model training apparatus.
[0149] Figure 6 This is a schematic diagram of the structure of a model training device provided in an embodiment of this disclosure.
[0150] like Figure 6 As shown, the model training device 60 includes: a first acquisition module 61, a first input module 62, a second input module 63, a first determination module 64, a second determination module 65, a first training module 66, and a second training module 67. Wherein:
[0151] The first acquisition module 61 is used to acquire training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal.
[0152] The first input module 62 is used to input the reference signal and the microphone signal into the acoustic echo cancellation model to obtain the echo-cancelled audio signal.
[0153] The second input module 63 is used to input the echo-cancelled audio signal into the voice wake-up model to obtain the inferred wake-up result.
[0154] The first determining module 64 is used to determine the first loss value of the acoustic echo cancellation model based on the original audio signal and the audio signal after echo cancellation.
[0155] The second determining module 65 is used to determine the second loss value of the voice wake-up model based on the actual wake-up result and the inferred wake-up result.
[0156] The first training module 66 is used to train the acoustic echo cancellation model based on the first loss value and the second loss value.
[0157] The second training module 67 is used to train the voice wake-up model based on the second loss value.
[0158] Figure 7 This is a schematic diagram of another model training device provided in an embodiment of the present disclosure.
[0159] like Figure 7 As shown, in the model training device 70, the first determining module 74 may further include: a first acquiring unit 741 and a first calculation unit 742; the second determining module 75 may further include: a second acquiring unit 751 and a second calculation unit 752; and the first training module 76 may further include: a weighted summation unit 761 and a training unit 762. Wherein:
[0160] The first acquisition unit 741 is used to acquire the first loss function of the acoustic echo cancellation model.
[0161] The first calculation unit 742 is used to perform loss calculation on the original audio signal and the audio signal after echo cancellation using a first loss function, so as to obtain the first loss value of the acoustic echo cancellation model.
[0162] The second acquisition unit 751 is used to acquire the second loss function of the voice wake-up model.
[0163] The second calculation unit 752 is used to perform loss calculation on the actual wake-up result and the inferred wake-up result using the second loss function to obtain the second loss value of the voice wake-up model.
[0164] The weighted summation unit 761 is used to perform a weighted summation of the first loss value and the second loss value to obtain the total loss value.
[0165] Training unit 762 is used to train the acoustic echo cancellation model based on the total loss value.
[0166] The training unit 762 is specifically used to determine the parameter gradient of the acoustic echo cancellation model based on the total loss value; and to adjust the model parameters in the acoustic echo model in the direction of decreasing the total loss value according to the parameter gradient, until the total loss value meets the preset conditions, so as to obtain the trained acoustic echo model.
[0167] The model training apparatus disclosed herein first acquires training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal. The reference signal and microphone signal are input into an acoustic echo cancellation model to obtain an echo-cancelled audio signal. The echo-cancelled audio signal is then input into a speech wake-up model to obtain an inferred wake-up result. Furthermore, based on the original audio signal and the echo-cancelled audio signal, a first loss value is determined for the acoustic echo cancellation model. Based on the actual wake-up result and the inferred wake-up result, a second loss value is determined for the speech wake-up model. The acoustic echo cancellation model is trained based on the first and second loss values, and the speech wake-up model is trained based on the second loss value. Thus, by jointly training the acoustic echo cancellation model and the speech wake-up model, joint optimization of the acoustic echo cancellation model and the speech wake-up model is achieved, resulting in a speech wake-up model with high accuracy.
[0168] Another aspect of this disclosure is a voice wake-up device. Figure 8 This is a schematic diagram of the structure of a voice wake-up device provided in an embodiment of this disclosure.
[0169] like Figure 8 As shown, the voice wake-up device 80 includes: a second acquisition module 81, a third determination module 82, a third input module 83, and a fourth input module 84. Wherein:
[0170] The second acquisition module 81 is used to acquire the first audio signal collected by the microphone and the second audio signal output by the speaker.
[0171] The third determining module 82 is used to determine the second audio signal as a reference signal for echo cancellation.
[0172] The third input module 83 is used to input the first audio signal and the reference signal into the acoustic echo cancellation model to obtain the third audio signal after echo cancellation.
[0173] The fourth input module 84 is used to input the third audio signal into the voice wake-up model to obtain the wake-up result. The acoustic echo cancellation model and the voice wake-up model are trained according to the model training method.
[0174] The voice wake-up device disclosed herein can eliminate the reference signal in the audio signal collected by the microphone, so that the audio signal input into the voice wake-up model is closer to the original audio signal, and the wake-up result of the voice wake-up model is closer to the real wake-up result.
[0175] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, a vehicle, and a computer program product.
[0176] To implement the above embodiments, this disclosure also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method described in the foregoing method embodiments.
[0177] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing method embodiments.
[0178] To implement the above embodiments, this disclosure also proposes a vehicle that includes the aforementioned electronic equipment.
[0179] To implement the above embodiments, this disclosure also proposes a computer program product having a computer program stored thereon, which, when executed by a processor, implements the method described in the foregoing method embodiments.
[0180] Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0181] like Figure 9 As shown, the electronic device 90 includes a processor 91, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 92 or a program loaded from a memory 96 into a random access memory (RAM) 93. The RAM 93 also stores various programs and data required for the operation of the electronic device 90. The processor 91, ROM 92, and RAM 93 are interconnected via a bus 94. An input / output (I / O) interface 95 is also connected to the bus 94.
[0182] The following components are connected to I / O interface 95: memory 96 including hard disk; and communication section 97 including network interface card such as LAN (Local Area Network) card, modem, etc., which performs communication processing via a network such as the Internet; and driver 98 is also connected to I / O interface 95 as needed.
[0183] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 97. When the computer program is executed by processor 91, it performs the functions defined in the methods of this disclosure.
[0184] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 96 including instructions, which can be executed by a processor 91 of an electronic device 90 to perform the above-described method. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.
[0185] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0186] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0187] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.
[0188] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0189] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0190] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0191] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.
Claims
1. A model training method, characterized in that, The method includes: Acquire training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal; The reference signal and the microphone signal are input into the acoustic echo cancellation model to obtain the echo-cancelled audio signal; The echo-cancelled audio signal is input into the voice wake-up model to obtain the inferred wake-up result; Based on the original audio signal and the audio signal after echo cancellation, determine the first loss value of the acoustic echo cancellation model; Based on the actual wake-up result and the inferred wake-up result, a second loss value for the voice wake-up model is determined; The acoustic echo cancellation model is trained based on the first loss value and the second loss value; The voice wake-up model is trained based on the second loss value.
2. The method as described in claim 1, characterized in that, The step of training the acoustic echo cancellation model based on the first loss value and the second loss value includes: The first loss value and the second loss value are weighted and summed to obtain the total loss value; The acoustic echo cancellation model is trained based on the total loss value.
3. The method as described in claim 2, characterized in that, The step of training the acoustic echo cancellation model based on the total loss value includes: Based on the total loss value, determine the parameter gradient of the acoustic echo cancellation model; Based on the parameter gradient, the model parameters in the acoustic echo cancellation model are adjusted in the direction of decreasing the total loss value until the total loss value meets the preset condition, so as to obtain the trained acoustic echo cancellation model.
4. The method as described in claim 1, characterized in that, The step of determining the first loss value of the acoustic echo cancellation model based on the original audio signal and the echo-cancelled audio signal includes: Obtain the first loss function of the acoustic echo cancellation model; Using the first loss function, loss calculations are performed on the original audio signal and the echo-cancelled audio signal to obtain the first loss value of the acoustic echo cancellation model.
5. The method as described in claim 1, characterized in that, The step of determining the second loss value of the voice wake-up model based on the actual wake-up result and the inferred wake-up result includes: Obtain the second loss function of the voice wake-up model; The second loss function is used to calculate the loss between the actual wake-up result and the inferred wake-up result to obtain the second loss value of the voice wake-up model.
6. A voice wake-up method, characterized in that, The method includes: Acquire the first audio signal picked up by the microphone and the second audio signal output by the speaker; The second audio signal is determined as the reference signal for echo cancellation; The first audio signal and the reference signal are input into the acoustic echo cancellation model to obtain the third audio signal after echo cancellation. The third audio signal is input into the voice wake-up model to obtain a wake-up result, wherein the acoustic echo cancellation model and the voice wake-up model are trained by the method according to any one of claims 1-5.
7. A model training device, characterized in that, The device includes: The first acquisition module is used to acquire training data, which includes the original audio signal, the microphone signal corresponding to the original audio signal, the reference signal corresponding to the microphone signal, and the actual wake-up result of the original audio signal. The first input module is used to input the reference signal and the microphone signal into the acoustic echo cancellation model to obtain the echo-cancelled audio signal; The second input module is used to input the echo-cancelled audio signal into the voice wake-up model to obtain the inferred wake-up result; The first determining module is used to determine the first loss value of the acoustic echo cancellation model based on the original audio signal and the audio signal after echo cancellation. The second determining module is used to determine the second loss value of the voice wake-up model based on the actual wake-up result and the inferred wake-up result; The first training module is used to train the acoustic echo cancellation model based on the first loss value and the second loss value. The second training module is used to train the voice wake-up model based on the second loss value.
8. A voice wake-up device, characterized in that, The device includes: The second acquisition module is used to acquire the first audio signal collected by the microphone and the second audio signal output by the speaker; The third determining module is used to determine the second audio signal as a reference signal for echo cancellation; The third input module is used to input the first audio signal and the reference signal into the acoustic echo cancellation model to obtain the echo-cancelled third audio signal; The fourth input module is used to input the third audio signal into the voice wake-up model to obtain a wake-up result, wherein the acoustic echo cancellation model and the voice wake-up model are trained by the method according to any one of claims 1-5.
9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the model training method as described in any one of claims 1-5, or the voice wake-up method as described in claim 6.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the model training method as described in any one of claims 1-5, or the voice wake-up method as described in claim 6.
11. A vehicle, characterized in that, Including the electronic device as described in claim 9.