Children sound detection model training method, children mode starting method and related devices

Abstract

The invention discloses a child sound detection model training method, a child mode starting method and related devices, and the child sound detection model training method comprises the steps: inputting a training voice frame into a child sound detection model, and obtaining a prediction category and a prediction probability outputted by the child sound detection model, the prediction category comprising an adult male, an adult female and a child; determining classification loss by using the prediction category and an actual category corresponding to the training voice frame; and when the prediction category is a child and the actual category is not a child, determining a misinformation loss by using a prediction probability corresponding to the prediction category, and performing parameter adjustment on the child sound detection model by using the classification loss and the false alarm loss. The child sound detection model obtained through training according to the scheme has higher recognition accuracy, and mistaken detection is reduced.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a child voice detection model training method, a child mode activation method and related devices. Background Art

[0002] With the advancement of technology, speech processing technology has been widely used, and the child voice detection model used to detect whether a voice contains a child has also been used to a certain extent. The child voice detection model determines whether there is a child in the collected sound to determine whether to perform subsequent operations related to children. The child voice detection model used in related technologies has the disadvantage of low recognition accuracy, which affects the user experience. Summary of the invention

[0003] The present application provides a child voice detection model training method, a child mode activation method and related devices.

[0004] A technical solution adopted in this application is to provide a training method for a child voice detection model, the method comprising:

[0005] Input the training speech frames into the child voice detection model to obtain the predicted categories and predicted probabilities output by the child voice detection model. The predicted categories include adult males, adult females, and children.

[0006] Determine the classification loss using the predicted category and the actual category corresponding to the training speech frame; and determine the target false alarm loss using the prediction probability corresponding to the predicted category when the predicted category is child and the actual category is not child;

[0007] The classification loss and target false alarm loss are used to adjust the parameters of the child voice detection model.

[0008] Optionally, when the predicted category is child and the actual category is not child, the target false alarm loss is determined using the predicted probability corresponding to the predicted category, including:

[0009] When the predicted category is child and the actual category is non-child, obtain a first predicted probability corresponding to the category of adult male, a second predicted probability corresponding to the category of adult female, and a third predicted probability corresponding to the category of child;

[0010] The target false alarm loss is determined using the first predicted probability, the second predicted probability, the third predicted probability and the actual category.

[0011] Optionally, determining a target false alarm loss using the first predicted probability, the second predicted probability, the third predicted probability and the actual category includes:

[0012] The first predicted probability, the second predicted probability, the third predicted probability, and the actual category are calculated using the cross entropy function to obtain the target false alarm loss.

[0013] Optionally, the classification loss and target false alarm loss are used to adjust the parameters of the child voice detection model, including:

[0014] The classification loss and target false alarm loss are weighted to obtain the total loss.

[0015] The total loss is used to adjust the parameters of the child voice detection model.

[0016] Another technical solution adopted by the present application is to provide a method for starting a child mode, which is applied to a vehicle and includes:

[0017] Acquire the voice frames in the vehicle;

[0018] Inputting the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame, wherein the child voice detection model is trained using the above-mentioned child voice detection model training method;

[0019] In response to the child voice detection result indicating that the speech frame contains a child voice, the operating mode of the vehicle is switched to a child mode.

[0020] Optionally, obtaining a voice frame in a vehicle includes:

[0021] Performing a preprocessing operation on the collected original voice signal in the vehicle to obtain a preprocessed voice signal;

[0022] Perform feature extraction on the preprocessed speech signal to obtain audio features;

[0023] Using several audio features, speech frames are constructed.

[0024] Optionally, in response to the child voice detection result indicating that the speech frame contains a child voice, switching the working mode of the vehicle to a child mode includes:

[0025] Obtaining a target number of target speech frames in the first number of continuous speech frames, wherein the target speech frames represent speech frames containing children's voices;

[0026] In response to the target number being greater than a second number, the operating mode of the vehicle is switched to a child mode, wherein the first number is greater than or equal to the second number.

[0027] Optionally, obtaining a voice frame in a vehicle includes:

[0028] In response to the time interval between the current moment and the last door closing moment being less than or equal to the preset interval, acquiring voice frames in the vehicle at a first preset frequency;

[0029] Or, in response to the time interval between the current moment and the last door closing moment being greater than a preset interval, voice frames in the vehicle are acquired at a second preset frequency, and the first preset frequency is greater than the second preset frequency.

[0030] Another technical solution adopted by the present application is to provide a terminal device, the terminal device comprising a memory and a processor connected to the memory;

[0031] The memory is used to store program data, and the processor is used to execute the program data to implement the training method of the child voice detection model and / or the method of starting the child mode as described above.

[0032] Another technical solution adopted in the present application is to provide a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the training method of the child voice detection model and / or the method of starting the child mode as mentioned above.

[0033] The beneficial effects of the present application are as follows: when training a child voice detection model, on the basis of classification loss, a false alarm loss corresponding to a false alarm situation where the predicted category is child and the actual category is non-child is added, and the false alarm loss is used to constrain the classification loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains a child's voice, thereby reducing manual operations and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 It is a flowchart of an embodiment of a training method for a child voice detection model provided by the present application;

[0036] Figure 2 This is a flow chart of an embodiment of a method for starting a child mode provided by the present application;

[0037] Figure 3is a flowchart of another embodiment of a method for activating a child mode provided by the present application;

[0038] Figure 4 yes Figure 3 A schematic diagram of an embodiment of S101-S103;

[0039] Figure 5 It is a structural diagram of an embodiment of a terminal device provided by the present application;

[0040] Figure 6 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] See also Figure 1 , Figure 1 It is a flow chart of an embodiment of a training method for a child voice detection model provided in the present application.

[0043] like Figure 1 As shown, the training method of the child voice detection model in the embodiment of the present application may specifically include the following steps:

[0044] S1, input the training speech frame into the child voice detection model to obtain the predicted category and predicted probability output by the child voice detection model.

[0045] Among them, the prediction categories include adult males, adult females, and children.

[0046] In some possible embodiments, the predicted category may also include other categories.

[0047] The training method of the child voice detection model provided in the present application and / or the startup method of the child mode are mainly performed by a model training device. In some embodiments, the model training device can be the microphone itself. In some embodiments, the distance measuring device can be a device connected to the microphone communication. For example, the device can be a device for monitoring images, a user equipment (UserEquipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital processing (PersonalDigital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and an autonomous driving car, a robot, a security system, and any one or more products in glasses and helmets for augmented reality or virtual reality. In some possible implementations, the training method of the child voice detection model and / or the startup method of the child mode can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0048] Specifically, the model training device inputs the training speech frame into the child voice detection model to obtain the child voice detection result corresponding to the training speech frame. The child voice detection result includes a predicted category and a predicted probability.

[0049] Generally speaking, the label categories of training speech frames can be set as children; children, adults; children, adult men, and adult women. After a period of work and research, the inventors of this application found that the voices of adult women and children are similar to each other to a certain extent, which has a greater impact on the detection accuracy of the child voice detection model, while the voices of adult men have less impact on the detection accuracy of the child voice detection model. Based on this, this application sets the training speech frames corresponding to adult women as a separate category to better guide the model to distinguish between children's voices and women's voices, thereby improving the accuracy of children's voice detection.

[0050] In some possible embodiments, the label categories of the training speech frames provided in the present application may be children, adult females, and others. Then the predicted categories may include children and adult females.

[0051] It should be noted that this application takes into account that boys' voices have not yet changed and the difference between boys' voices and girls' voices is small, so the children's categories are not further subdivided.

[0052] S2, determines the classification loss using the predicted category and the actual category corresponding to the training speech frame.

[0053] It can be understood that the actual category corresponding to the training speech frame is the label category corresponding to the training speech frame.

[0054] Specifically, the model training device determines the classification loss of the child voice detection model by using the difference between the predicted category output by the child voice detection model and the actual category corresponding to the training speech frame.

[0055] In some possible embodiments, the classification loss is determined based on the cross entropy between the predicted category and the actual category.

[0056] S3, when the predicted category is child and the actual category is not child, the target false alarm loss is determined using the predicted probability corresponding to the predicted category.

[0057] It should be noted that for the child voice detection model, its performance can be represented by two key indicators, the first indicator is the False Reject Rate (FRR), and the second indicator is the False Alarm Rate (FAR). The inventors of the present application have found through research that in the scenario of judging whether to enter the child mode by performing child voice detection, more attention should be paid to the false alarm rate. It is not desirable to frequently cause the child mode to be mistakenly entered due to adults talking in the scenario without children. On the contrary, the child voice detection model can be more tolerant of the false alarm rate. When there are children in the scene, it does not have a big impact if it is not detected once or twice.

[0058] To achieve this goal, this embodiment adds a penalty term for false alarms on the basis of the traditional loss function, so that the child voice detection model pays more attention to false alarms.

[0059] Specifically, in a false alarm scenario where the predicted category is child but the actual category is not child, the model training device calculates the target false alarm loss using the predicted probability.

[0060] S4, uses classification loss and target false alarm loss to adjust the parameters of the child voice detection model.

[0061] Specifically, the model training device uses the classification loss and the target false alarm loss to determine the total loss, and uses the total loss to adjust the network parameters in the child voice detection model.

[0062] In some embodiments, the child voice detection model is a neural network model.

[0063] In the above scheme, when training the child voice detection model, on the basis of classification loss, false alarm loss corresponding to false alarm situations where the predicted category is child and the actual category is non-child is added, and the classification loss is constrained by the false alarm loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0064] Another embodiment of the training method of the child voice detection model provided in the present application may specifically include the following steps:

[0065] S11, input the training speech frame into the child voice detection model to obtain the predicted category and predicted probability output by the child voice detection model.

[0066] Among them, the prediction categories include adult males, adult females, and children.

[0067] S12, determining the classification loss using the predicted category and the actual category corresponding to the training speech frame.

[0068] S13, when the predicted category is child and the actual category is non-child, obtain a first predicted probability corresponding to the category of adult male, a second predicted probability corresponding to the category of adult female, and a third predicted probability corresponding to the category of child.

[0069] When the predicted category is child and the actual category is non-child, the model training device obtains the predicted probability corresponding to each category output by the child voice detection model. More specifically, the model training device obtains the first predicted probability corresponding to the adult male category, the second predicted probability corresponding to the adult female category, and the third predicted probability corresponding to the child category output by the child voice detection model.

[0070] It can be understood that when the predicted category of the child voice detection output is child, the third predicted probability is greater than the first predicted probability, and the third predicted probability is greater than the second predicted probability.

[0071] In some embodiments, the case where the predicted category is child and the actual category is non-child can be represented as the following set:

[0072] S={(softmax_output,target)|target≠kid,predict=kid}.

[0073] Wherein, predict=argmax(softmax_output). In this embodiment, target represents the number corresponding to the actual category, and predict represents the number of the category predicted by the model.

[0074] Among them, the argmax function is used to find the position index corresponding to the item with the largest value.

[0075] For example, for the first predicted probability of 0.3, the second predicted probability of 0.3, and the third predicted probability of 0.4, the corresponding function results satisfy:

[0076] Argmax(0.3, 0.3, 0.4)=3.

[0077] Refers to item 3, which is the category corresponding to the third predicted probability (the category is children).

[0078] S14, using the first predicted probability, the second predicted probability, the third predicted probability and the actual category, determine the target false alarm loss.

[0079] In some embodiments, the model training device inputs the first predicted probability, the second predicted probability, the third predicted probability, and the actual category into the loss function to obtain the target false alarm loss.

[0080] S15, using classification loss and target false alarm loss, adjust the parameters of the child voice detection model.

[0081] In the above scheme, when training the child voice detection model, on the basis of classification loss, false alarm loss corresponding to false alarm situations where the predicted category is child and the actual category is non-child is added, and the classification loss is constrained by the false alarm loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0082] Another embodiment of the training method of the child voice detection model provided in the present application may specifically include the following steps:

[0083] S21, input the training speech frame into the child voice detection model to obtain the predicted category and predicted probability output by the child voice detection model.

[0084] Among them, the prediction categories include adult males, adult females, and children.

[0085] S22, determining the classification loss using the predicted category and the actual category corresponding to the training speech frame.

[0086] S23, when the predicted category is child and the actual category is non-child, obtain a first predicted probability corresponding to the category of adult male, a second predicted probability corresponding to the category of adult female, and a third predicted probability corresponding to the category of child.

[0087] S24, using a cross entropy function, calculating the first predicted probability, the second predicted probability, the third predicted probability, and the actual category to obtain the target false alarm loss.

[0088] For the three-class child voice detection model, when the sample S_i is input, the model output will output three probability values, which satisfy the following relationship:

[0089] softmax_output_i=(P1_i, P2_i, P3_i).

[0090] Among them, P1_i is the first prediction probability, P2_i is the second prediction probability, and P3_i is the third prediction probability.

[0091] The first prediction probability corresponds to the probability of an adult male, the second prediction probability corresponds to the probability of an adult female, and the third prediction probability corresponds to the probability of a child. Due to the characteristics of the softmax function, the sum of the first prediction probability, the second prediction probability and the third prediction probability is 1.

[0092] Exemplarily, the model training device sets the true category label of a certain training speech frame S_i as L_i, and the loss can be calculated on this training speech frame:

[0093] loss_i=CE(softmax_output_i,L_i)

[0094] =CE((P1_i,P2_i,P3_i),L_i)

[0095] Among them, loss_i is the target false alarm loss corresponding to the training speech frame, and CE is the cross entropy function.

[0096] The training speech frame corresponds to a speech frame set, which can be represented as S. Some of the training speech frames meet the false alarm condition, and the set composed of these training speech frames can be represented as S_panaty. It can be understood that S_panaty is a subset of S.

[0097] Specifically, in S_panaty it can be expressed as:

[0098] S_panaty={S_1, S_2,..., S_N}.

[0099] The target false positive loss can be calculated on the set S_panaty:

[0100] loss_penaty=(loss_1+loss_2+…+loss_N) / N

[0101] =average(CE(softmax_output,L))

[0102] Among them, softmax_output = {softmax_output_1, softmax_output_2, ..., softmax_output_N} is the model output corresponding to each training speech frame in S_panaty, and L = {L_1, L_2, ..., L_N} is the actual category corresponding to each training speech frame.

[0103] S28, uses classification loss and target false alarm loss to adjust the parameters of the child voice detection model.

[0104] In the above scheme, when training the child voice detection model, on the basis of classification loss, false alarm loss corresponding to false alarm situations where the predicted category is child and the actual category is non-child is added, and the classification loss is constrained by the false alarm loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0105] Another embodiment of the training method of the child voice detection model provided in the present application may specifically include the following steps:

[0106] S31, input the training speech frame into the child voice detection model to obtain the predicted category and predicted probability output by the child voice detection model.

[0107] Among them, the prediction categories include adult males, adult females, and children.

[0108] S32, determining the classification loss using the predicted category and the actual category corresponding to the training speech frame.

[0109] S33, when the predicted category is child and the actual category is not child, the target false alarm loss is determined using the predicted probability corresponding to the predicted category.

[0110] S34, use the classification loss and target false alarm loss for weighted processing to obtain the total loss.

[0111] Specifically, the total loss satisfies the following relationship:

[0112] loss=Aloss_org+(1-A)loss_penaty.

[0113] Among them, loss is the total loss, loss_org is the classification loss, loss_penaty is the target false positive loss, and A is the weight. Furthermore, A∈[0,1].

[0114] Understandably, the larger A is, the more important the classification loss is. Conversely, the false positive loss is more important.

[0115] S36, using the total loss to adjust the parameters of the child voice detection model.

[0116] Specifically, the model training device uses the total loss to adjust the network parameters of the child voice detection model so that the total loss of the adjusted child voice detection model is iterated in a direction of minimization.

[0117] In the above scheme, when training the child voice detection model, on the basis of classification loss, false alarm loss corresponding to false alarm situations where the predicted category is child and the actual category is non-child is added, and the classification loss is constrained by the false alarm loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0118] See also Figure 2 , Figure 2 It is a flowchart of an embodiment of a method for starting the child mode provided in the present application.

[0119] In related technologies, the vehicle's child mode is mainly activated manually by the driver or other passengers. In an emergency or hurry, the driver and passengers may forget to turn on the child mode, causing the children to be exposed to an inappropriate environment in the car. Some technologies also use sensors to help determine whether a child is in the car. For example, seat sensors or in-car cameras are used to determine whether a child is in the car. Seat sensors can only detect children sitting in a specific position and have limited coverage. In-car cameras have privacy issues. Due to the narrow space in the car, multiple cameras need to be installed in the car to cover the entire car, which greatly increases costs. At the same time, there is a certain delay in using cameras to detect whether a child exists, and children may be exposed to inappropriate content or environment during this time.

[0120] like Figure 2 As shown, an embodiment of a method for starting the child mode provided by the present application may specifically include the following steps:

[0121] S41, obtaining a voice frame in the vehicle.

[0122] Specifically, the model training device uses a vehicle-mounted microphone to collect speech frames inside the vehicle.

[0123] In some embodiments, the speech frame is determined based on a speech signal.

[0124] S42, inputting the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame.

[0125] The child voice detection model is trained using the above-mentioned child voice detection model training method.

[0126] It can be understood that the child voice detection result indicates that the current speech frame contains a child voice, indicating that there is a child in the current car.

[0127] Specifically, the model training device inputs the speech frame into the child voice detection model to determine whether the speaking object in the speech frame includes a child.

[0128] S43, in response to the child voice detection result indicating that the speech frame contains a child voice, switching the operating mode of the vehicle to a child mode.

[0129] Specifically, in response to the child voice detection result indicating that the speech frame contains a child voice, the model training device switches the operating mode of the vehicle to the child mode.

[0130] In some embodiments, the child mode may include adjusting the interactive interface of the vehicle's internal display screen to an interface suitable for children to operate, such as increasing the brightness of the display screen, switching the display screen to an eye protection mode, etc.

[0131] In some embodiments, Kids Mode may include lowering the volume of audio inside the vehicle.

[0132] In some embodiments, the kids mode may include hiding songs, programs, or games that are not suitable for children to listen to or watch.

[0133] In some embodiments, the kids mode may include actively playing children's songs and / or programs.

[0134] In some embodiments, the child mode may include increasing the temperature of the air conditioner in the vehicle and / or reducing the air volume of the air conditioner.

[0135] In some embodiments, the kids mode may include turning up the brightness of the interior lights of the vehicle.

[0136] It should be noted that in the prior art, the method of activating the child mode is often that the occupants actively activate it through the relevant switch buttons in the car, the touch screen in the infotainment system, or voice, or there are relevant sensors installed on the seats in the car to automatically enter the child mode by detecting whether the child seat is connected, or using image detection to determine whether there are children in the car. Compared with the prior art, the method for activating the child mode provided by this application does not require the installation of additional sensors, and realizes the automatic activation of the child mode while ensuring the privacy of the occupants.

[0137] In the above scheme, when training the child voice detection model, on the basis of classification loss, false alarm loss corresponding to false alarm situations where the predicted category is child and the actual category is non-child is added, and the classification loss is constrained by the false alarm loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0138] Another embodiment of the method for activating the child mode provided in the present application may specifically include the following steps:

[0139] S51, performing a preprocessing operation on the collected original voice signal in the vehicle to obtain a preprocessed voice signal.

[0140] Specifically, the model training device collects original voice signals using a vehicle-mounted microphone, and performs preprocessing operations on the original voice signals to obtain preprocessed voice signals.

[0141] In some embodiments, the vehicle-mounted microphone collects the original voice signal at a sampling rate of 16k, which greatly reduces the amount of calculation compared to using a sampling rate of 44.1k or 48k while ensuring the effect.

[0142] In some embodiments, the model training device uses a human voice enhancement algorithm to filter the original speech signal, and uses a speech enhancement algorithm to perform noise reduction to remove wind noise, road noise, engine noise, range extender noise, etc. generated during the driving of the car, and obtain a preprocessed speech signal.

[0143] Furthermore, the model training device performs voice activity detection (VAD) on the preprocessed speech signal to determine whether there is a human voice in the preprocessed speech signal. If yes, jump to S52. If no, jump to S51.

[0144] S52, extracting features from the preprocessed speech signal to obtain audio features.

[0145] Specifically, the model training device sets the Window Size to 25ms and the Hop Size to 10ms. According to the above settings, the model training device performs Mel-frequency cepstral coefficient (MFCC) feature extraction on the preprocessed speech signal to extract an 80-dimensional MFCC feature vector.

[0146] S53, constructing a speech frame using a number of audio features.

[0147] Specifically, the model training device uses every 100 consecutive MFCC feature vectors (approximately 1 second) as an interval and every 200 (approximately 2 seconds) consecutive MFCC feature vectors as a speech frame.

[0148] Exemplarily, the model training device uses the 1st to 200th MFCC feature vectors as the 1st frame of speech, and then uses the 101st to 300th MFCC feature vectors as the 2nd frame of speech, and then uses the 201st to 400th MFCC feature vectors as the 3rd frame of speech, and so on.

[0149] S54, inputting the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame.

[0150] The child voice detection model is trained using the above-mentioned child voice detection model training method.

[0151] S55, in response to the child voice detection result indicating that the speech frame contains a child voice, switching the operating mode of the vehicle to a child mode.

[0152] In the above scheme, when training the child voice detection model, on the basis of classification loss, false alarm loss corresponding to false alarm situations where the predicted category is child and the actual category is non-child is added, and the classification loss is constrained by the false alarm loss, so that the child voice detection model can detect children's voices more accurately, reduce the occurrence of false alarms, and improve the detection accuracy of the child mode. The trained child voice detection model can directly and accurately recognize children's voices through voice without the assistance of other sensing equipment, thereby reducing the detection process of voice recognition and improving recognition efficiency. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0153] Another embodiment of the method for activating the child mode provided in the present application may specifically include the following steps:

[0154] S61, acquiring a voice frame in the vehicle.

[0155] S62, inputting the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame.

[0156] The child voice detection model is trained by the above-mentioned child voice detection model training method, and the child voice detection result includes whether the child voice is contained or not.

[0157] S63: Obtain a target number of target speech frames in the first number of continuous speech frames.

[0158] The target speech frame refers to a speech frame containing a child's voice.

[0159] In order to further reduce the false alarm rate and alleviate the problem of phantom triggering, the model training device needs to post-process the output results of the child voice detection model. Specifically, the model training device obtains the number of target speech frames (ie, the target number) in the first number of continuous speech frames.

[0160] Exemplarily, the target number can be 5, 10, 15, 20, 25, 30, 35, or 40, which is not limited here.

[0161] S64: In response to the target number being greater than the second number, switching the operating mode of the vehicle to the child mode.

[0162] The first number is greater than or equal to the second number.

[0163] In some embodiments, in response to the target number being greater than the second number, it is confirmed that there are children in the vehicle, and the model training device switches the working mode of the vehicle to the child mode.

[0164] It can be understood that in response to the target number being less than or equal to the second number, the model training device does not switch the operating mode of the vehicle to the child mode.

[0165] The above scheme, by setting the prediction categories as adult males, adult females and children, enables the child voice detection model to better distinguish the voices of children, adult males and adult females during the training process; on the basis of the classification loss, the false alarm loss corresponding to the false alarm situation where the predicted category is child and the actual category is not child is added, so that the trained child voice detection model reduces the above false alarm situations. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0166] Another embodiment of the method for activating the child mode provided in the present application may specifically include the following steps:

[0167] S71, in response to the time interval between the current moment and the last door closing moment being less than or equal to the preset interval, acquiring voice frames in the vehicle at a first preset frequency.

[0168] It should be noted that the change of passengers in the car usually only occurs when the car doors are opened or closed, so the detection frequency can be further controlled by combining the opening and closing conditions of the car doors.

[0169] Specifically, in response to the time interval between the current moment and the last door closing moment being less than or equal to a preset interval, the model training device acquires speech frames in the vehicle at a higher sampling frequency, and further, the model training device performs child detection operations at a higher frequency.

[0170] Exemplarily, the preset interval may be 1 minute, 2 minutes, 3 minutes, 4 minutes or 5 minutes, which is not limited here.

[0171] Exemplarily, the first preset frequency may be any one of 2 Hz, 1 Hz, 0.5 Hz, 0.33 Hz, 0.25 Hz, 0.2 Hz, and 0.1 Hz, which is not limited here.

[0172] S72, input the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame.

[0173] The child voice detection model is trained by the above-mentioned child voice detection model training method, and the child voice detection result includes whether the child voice is contained or not.

[0174] S73, in response to the child voice detection result indicating that the speech frame contains a child voice, switching the operating mode of the vehicle to a child mode.

[0175] The above scheme, by setting the prediction categories as adult males, adult females and children, enables the child voice detection model to better distinguish the voices of children, adult males and adult females during the training process; on the basis of the classification loss, the false alarm loss corresponding to the false alarm situation where the predicted category is child and the actual category is not child is added, so that the trained child voice detection model reduces the above false alarm situations. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0176] Another embodiment of the method for activating the child mode provided in the present application may specifically include the following steps:

[0177] S81, in response to the time interval between the current moment and the last door closing moment being greater than a preset interval, acquiring voice frames in the vehicle at a second preset frequency, the first preset frequency being greater than the second preset frequency.

[0178] Specifically, in response to the time interval between the current moment and the last door closing moment being greater than a preset interval, the model training device acquires speech frames in the vehicle at a lower sampling frequency. Further, the model training device performs child detection operations at a lower frequency to save vehicle computing power.

[0179] In some possible embodiments, in consideration of protecting personal privacy and reducing energy consumption, in response to the time interval between the current moment and the last door closing moment being greater than a preset interval, the model training device stops the child detection operation.

[0180] Exemplarily, the second preset frequency may be any one of 0.02 Hz, 0.01 Hz, 0.005 Hz, 0.0025 Hz, 0.0002 Hz, and 0.0001 Hz, which is not limited here.

[0181] S82, input the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame.

[0182] The child voice detection model is trained using the above-mentioned child voice detection model training method.

[0183] S83, in response to the child voice detection result indicating that the speech frame contains a child voice, switching the operating mode of the vehicle to a child mode.

[0184] The above scheme, by setting the prediction categories as adult males, adult females and children, enables the child voice detection model to better distinguish the voices of children, adult males and adult females during the training process; on the basis of the classification loss, the false alarm loss corresponding to the false alarm situation where the predicted category is child and the actual category is not child is added, so that the trained child voice detection model reduces the above false alarm situations. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0185] See also Figure 3 , Figure 3 It is a flow chart of another embodiment of the method for starting the child mode provided in the present application.

[0186] like Figure 3 As shown, another embodiment of the method for starting the child mode provided by the present application may specifically include the following steps:

[0187] S101, collecting voice signals.

[0188] Specifically, the model training device uses a vehicle-mounted microphone to collect voice signals inside the vehicle.

[0189] In some embodiments, the vehicle-mounted microphone collects voice signals at a sampling rate of 16k, which greatly reduces the amount of calculation compared to 44.1k and 48k while ensuring the effect.

[0190] S102, preprocessing and feature extraction are performed on the speech signal in sequence to obtain a speech feature frame.

[0191] In some embodiments, the model training device performs VAD detection, filtering, noise reduction and other processing on the speech signal to obtain a preprocessed speech signal.

[0192] For example, the model training device uses a human voice enhancement algorithm to filter the voice signal. For another example, the model training device uses a voice enhancement algorithm to reduce the noise of the voice signal to remove wind noise, road noise, engine noise, range extender noise, etc. generated during the driving of the car. Finally, the voice signal performs a VAD detection on the voice signal after the above filtering and noise reduction to determine whether there is a human voice in the audio. If so, continue with the subsequent processing.

[0193] In some embodiments, the model training device extracts MFCC features from the processed speech signal and then performs frame processing.

[0194] Exemplarily, the model training device sets the Window Size to 25ms, the Hop Size to 10ms, and extracts 80-dimensional MFCC feature vectors. Every 100 consecutive MFCC feature vectors (approximately 1s) are separated, and every 200 (approximately 2s) consecutive MFCC feature vectors are taken as a speech frame.

[0195] S103, inputting the speech feature frame into a child voice detection model to obtain a prediction result.

[0196] In some possible embodiments, the child voice detection model is constructed based on the CAM++ model.

[0197] The output of the model includes two parts: one is the label (category), and the other is the probability value corresponding to the child label. For the child voice detection task, the label can be set in the following three ways:

[0198] (1) Children;

[0199] (2) Children and adults (including adult men and adult women);

[0200] (3) Children, adult males, and adult females.

[0201] Experiments have shown that better results can be achieved when performing three-category tasks. The reason may be that for the child voice detection task, the voice of an adult female (i.e., an adult female voice) actually has a greater impact on the detection accuracy, while the impact of an adult male voice is relatively small. Based on this, this application treats adult females as a separate category, which is conducive to guiding the model to distinguish between children's voices and adult female voices, thereby improving the detection effect.

[0202] In addition, for detection tasks, there are usually two key indicators, one is the False Reject Rate (FRR) and the other is the False Alarm Rate (FAR). For the child voice detection task of this application, more attention should be paid to the false alarm rate. It is not desirable to frequently generate false triggers due to adults talking when there are no children in the car.

[0203] On the contrary, the truth rate can be more tolerant. When there are children talking in the car, it does not matter if they are not detected once or twice. To achieve this goal, a penalty term for false alarms (false detections) is added to the traditional cross entropy loss function of classification tasks, making it more concerned with false alarms.

[0204] Specifically, the model training device sets the set S_panaty to:

[0205] S_panaty={(softmax_output,target)|target≠kid,predict=kid}.

[0206] Among them, predict = argmax(softmax_output).

[0207] The false positive penalty loss calculated by the model training device on the set S_panaty satisfies the following relationship:

[0208] loss_penaty=averge(CE(softmax_output,target)),(softmax_output,target)∈S.

[0209] Among them, target is the actual label of the speech frame.

[0210] Please refer to the above for the specific calculation of the penalty loss for false alarms, which will not be repeated here.

[0211] Assuming the original loss is loss_org, the loss after adding the penalty term is:

[0212] loss=Aloss_org+(1-A)loss_pendty.

[0213] Where A∈[0,1], when A is larger, it means that the original loss plays a more important role; conversely, the penalty term plays a major role. The original loss can correspond to the classification loss.

[0214] This application trains the model by modifying the loss function, which greatly reduces the probability of false detection of children and makes it possible to detect children's voices using only the audio stream.

[0215] like Figure 4 As shown, Figure 4 yes Figure 3 FIG. 1 is a flow chart of an embodiment of S101-S103. Figure 4 The prediction probabilities in are p1, p2, ..., and the prediction categories are l1, l2, ..., which are used to represent the prediction categories corresponding to different speech frames and the probabilities corresponding to different prediction categories.

[0216] S104: Determine whether to enter the child mode based on the prediction result of each speech feature frame.

[0217] In order to further reduce the false alarm rate and to alleviate the problem of phantom triggering, the model training device needs to post-process the output results of the model, including but not limited to the following three methods: When at least one of the following conditions is met, it is determined to enter the child mode.

[0218] (1) In M consecutive frames, the number of times a child is detected is greater than or equal to N, where M ≥ N;

[0219] (2) Detecting children in N consecutive frames;

[0220] (3) The probability value p corresponding to the child category output by the child voice detection model is greater than a preset threshold.

[0221] In addition, changes in passengers in the car usually only occur when the car door is opened or closed, so the detection frequency of the child voice detection algorithm can be further controlled by combining the opening and closing conditions of the car door. When the car door is opened or closed once, the microphone in the car begins to continuously collect the sound in the car and then detect the child voice. This process can last for 1 minute. In other time periods, the child voice detection algorithm can run at a lower frequency, such as once every 5s or 10s, which can further save computing power.

[0222] Among them, the child voice detection algorithm corresponds to S101-S103.

[0223] Exemplarily, entering the children's mode may include adjusting part of the interface of the vehicle system, such as adjusting the display screen output color; adjusting the indicator sound in the car, and limiting the playback of content that is not suitable for children to listen to or watch.

[0224] The above scheme, by setting the prediction categories as adult males, adult females and children, enables the child voice detection model to better distinguish the voices of children, adult males and adult females during the training process; on the basis of the classification loss, the false alarm loss corresponding to the false alarm situation where the predicted category is child and the actual category is not child is added, so that the trained child voice detection model reduces the above false alarm situations. Furthermore, the method for starting the child mode provided in the present application inputs the collected voice frames into the child voice detection model trained by the training method of the child voice detection model provided in the present application, and automatically enters the child mode when the output result of the child voice detection model contains child voices, thereby reducing manual operations and improving the user experience.

[0225] In addition, the method for starting the child mode provided in the present application automatically identifies whether there are children in the car through child voice detection technology, eliminates manual operation steps, and ensures that the child mode can be started in time under any circumstances; uses audio signals to cover the entire vehicle compartment space, and by identifying the presence of children's voices, it is not limited by seat position or camera field of view, and provides a more comprehensive detection range; compared with video streams, audio streams have lower data volume, require less calculation, and process faster, so real-time monitoring and response mechanisms can be used to ensure that the vehicle's child mode is immediately started the moment a child's voice is detected, reducing delays and improving the system's response speed; based on the results of child voice detection, the playback content and volume are dynamically adjusted to provide entertainment and educational content that meets children's needs, ensuring a safe and comfortable in-car environment.

[0226] Please continue to see Figure 5 , Figure 5 The terminal device 500 of the embodiment of the present application includes a processor 51 and a memory 52 .

[0227] The processor 51 and the memory 52 are connected to the bus. The memory 52 stores program data. The processor 51 is used to execute the program data to implement the training method of the child voice detection model and / or the method of starting the child mode described in the above embodiments.

[0228] In the embodiment of the present application, the processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having the ability to process signals. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 51 may also be any conventional processor, etc.

[0229] This application also provides a computer storage medium, please continue to refer to Figure 6 , Figure 6 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 600 stores program data 61. When the program data 61 is executed by the processor, it is used to implement the training method of the child voice detection model of the above-mentioned embodiment and / or the method of starting the child mode.

[0230] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0231] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Equivalent structures or equivalent process changes made by utilizing the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for training a child voice detection model, characterized in that: The method comprises: Inputting the training speech frame into the child voice detection model to obtain the predicted category and predicted probability output by the child voice detection model, wherein the predicted category includes adult male, adult female and child; Determining a classification loss using the predicted category and the actual category corresponding to the training speech frame; and determining a target false alarm loss using the predicted probability corresponding to the predicted category when the predicted category is a child and the actual category is not a child; The classification loss and the target false alarm loss are used to adjust parameters of the child voice detection model.

2. The method according to claim 1, characterized in that When the predicted category is a child and the actual category is not a child, determining the target false alarm loss by using the predicted probability corresponding to the predicted category includes: When the predicted category is child and the actual category is non-child, obtaining a first predicted probability corresponding to the category of adult male, a second predicted probability corresponding to the category of adult female, and a third predicted probability corresponding to the category of child; The target false alarm loss is determined by using the first predicted probability, the second predicted probability, the third predicted probability and the actual category.

3. The method according to claim 2, characterized in that The determining the target false alarm loss by using the first predicted probability, the second predicted probability, the third predicted probability and the actual category includes: The first predicted probability, the second predicted probability, the third predicted probability, and the actual category are calculated using a cross entropy function to obtain the target false alarm loss.

4. The method according to claim 1, characterized in that: The step of adjusting parameters of the child voice detection model by using the classification loss and the target false alarm loss includes: The classification loss and the target false alarm loss are weighted to obtain a total loss; The total loss is used to adjust parameters of the child voice detection model.

5. A method for starting a child mode, the method being applied to a vehicle, characterized in that: The method comprises: Acquire the voice frames in the vehicle; Inputting the speech frame into a child voice detection model to obtain a child voice detection result corresponding to the speech frame, wherein the child voice detection model is trained according to the method of any one of claims 1 to 4; In response to the child voice detection result indicating that the speech frame contains a child voice, the operating mode of the vehicle is switched to a child mode.

6. The method according to claim 5, characterized in that The acquiring of the voice frame in the vehicle comprises: Performing a preprocessing operation on the collected original voice signal in the vehicle to obtain a preprocessed voice signal; Extracting features from the preprocessed speech signal to obtain audio features; The speech frame is constructed using a number of the audio features.

7. The method according to claim 5, characterized in that In response to the child voice detection result indicating that the speech frame contains a child voice, switching the working mode of the vehicle to a child mode includes: Obtaining a target number of target speech frames in the first number of continuous speech frames, wherein the target speech frames represent speech frames containing children's voices; In response to the target number being greater than a second number, switching the operating mode of the vehicle to a child mode, wherein the first number is greater than or equal to the second number.

8. The method according to claim 5, characterized in that The acquiring of the voice frame in the vehicle comprises: In response to a time interval between a current moment and a last door closing moment being less than or equal to a preset interval, acquiring a voice frame in the vehicle at a first preset frequency; Or, in response to the time interval between the current moment and the last door closing moment being greater than a preset interval, the voice frames in the vehicle are acquired at a second preset frequency, and the first preset frequency is greater than the second preset frequency.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory connected to the processor, wherein: The memory stores program instructions; The processor is configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 8 is implemented.

Images