Function multiplexing method of vehicle-mounted microphone

Abstract

The present invention discloses a method for multiplexing functions of vehicle-mounted microphones, comprising the following steps: outputting the sound signal collected by the road noise active control system to a trigger module; if the trigger module detects a wake-up command signal in the sound signal, outputting the sound signal to the vehicle-mounted intelligent voice system, and executing the next step, if no voice signal is detected for a period of time, stopping outputting the sound signal to the vehicle-mounted intelligent voice system; and downsampling the sound signal to adapt to the voice recognition module of the vehicle-mounted intelligent voice system. The method for multiplexing functions of vehicle-mounted microphones provided by the present invention multiplexes the sound pickup functions of the microphone of the vehicle-mounted road noise active control system and the microphone of the vehicle-mounted intelligent voice system, thereby reducing the number of vehicle-mounted microphones and reducing the cost of the automobile.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a function multiplexing method of a vehicle-mounted microphone. Background Art

[0002] At present, the microphone of the active road noise control system and the microphone of the vehicle-mounted intelligent voice system belong to two independent systems. The microphone of the active road noise control system is used to pick up the error signal. In order to reduce the wiring harness design, the digital microphone of the A2B harness is usually used to collect the error signal and feed it back to the algorithm of the active control system for iterative control; the microphone of the intelligent voice system is used to pick up the voice signal of the passengers. It usually adopts an analog microphone and is designed in an array form. The array algorithm based on signal processing is used to pick up the voice for speaker positioning and voice picking up. This method cannot handle the noise interference in the same direction as the voice, and the positioning and recognition effects are poor. At the same time, the two sets of microphones also increase the cost of the car. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides a method for multiplexing the functions of a vehicle-mounted microphone, and the technical solution is as follows:

[0004] The present invention provides a method for multiplexing functions of a vehicle-mounted microphone, comprising the following steps:

[0005] S1, outputting the sound signal collected by the microphone in the road noise active control system to the trigger module;

[0006] S2. If the trigger module detects a wake-up command signal in the sound signal, the sound signal is output to the vehicle-mounted intelligent voice system, and step S3 is performed. If no voice signal is detected for a period of time, the sound signal is stopped from being output to the vehicle-mounted intelligent voice system.

[0007] S3. Downsampling the sound signal to adapt it to the speech recognition module of the in-vehicle intelligent speech system.

[0008] Furthermore, the road noise active control system and the in-vehicle intelligent voice system share a set of microphones, the microphones include multiple distributed microphones, the distributed microphones are respectively arranged near each seat in the car, the sound signal is collected by the distributed microphones, and the sound signal includes sound information monitored at multiple positions in the car; before the sound signal is downsampled, the sound signal is first used for voice positioning.

[0009] Furthermore, the voice localization comprises the following steps:

[0010] A1. extracting multiple groups of sound data from the data set of the sound signal to form multiple sample sets as different training data sets;

[0011] A2. Performing various feature engineering processes on the training data sets to obtain corresponding feature vector combinations;

[0012] A3, the partitioning module uses different data selection modes to locate and classify the feature vector combination according to the seat distribution in the car, so as to obtain the probability value of each seat in the car making a sound;

[0013] A4. The seat corresponding to the maximum probability value is determined as the speaker's seat, and the sound signal collected by the distributed microphone corresponding to the speaker's seat is output to the speech recognition module.

[0014] Furthermore, the voice recognition module assigns the same or different vehicle control authorities to the sound signals from different seat numbers.

[0015] Furthermore, the signal feature types collected by the microphone include one or more of a mean feature, a minimum feature, a feature combination of an absolute value and a mean, and a feature combination of an absolute value and a minimum.

[0016] Further, the partition module includes a first partition module, a second partition module and a classification layer module, the first partition module includes an A discriminator and a B discriminator, the second partition module includes multiple discrimination layers, each discrimination layer includes a C discriminator, a D discriminator, an E discriminator, and an F discriminator; multiple second partition modules are associated and sorted to form a second partition module with a multi-level structure, the number of the second partition modules is equal to the number of signal feature types used, and the output of the second partition module of the previous level is used as the input of the second partition module of the next level; the classification layer module includes multiple discriminators;

[0017] The feature vector combination of a training data set forms a feature space correspondingly, and the A discriminator, C discriminator and D discriminator will randomly select signal features without replacement in the complete feature space as partition judgment conditions, and the B discriminator, E discriminator and F discriminator will select features according to indicators without replacement in the incomplete feature space as partition judgment conditions;

[0018] Different feature vector combinations are processed successively by the first partition module, the second partition module of the multi-level structure and the classification layer module to output the probability value of the sound emitted by each seat in the car.

[0019] Furthermore, each feature vector combination will correspond to a model training feature vector after being processed by the first partitioning module;

[0020] Output the first model training feature vector to the first-level second partitioning module for processing. In the first-level second partitioning module processing, the output result of each differentiation layer is concatenated with the first model training feature vector to serve as the input of the next differentiation layer.

[0021] The output result of the first-level second partition module is spliced ​​with the first model training feature vector to be input into the second-level second partition module. In the processing of the second-level second partition module, the output result of each differentiation layer is spliced ​​with the second model training feature vector to serve as the input of the next differentiation layer, and so on to obtain the output result of the last-level second partition module, which is spliced ​​with the last model training feature vector and input into the classification layer module to obtain the probability value of sound emitted by each seat in the car.

[0022] Furthermore, the A classifier, the C classifier and the D classifier are recorded as first-class classifiers, and the design of the first-class classifiers includes the following steps:

[0023] B1. The first type of classifier includes X sub-classifiers, each of which randomly selects signal features without replacement in the complete feature space as partition judgment conditions;

[0024] B2. Design an indicator to measure the purity of the training data set, denoted as Purity.

[0025]

[0026]

[0027] Among them, S represents the size of the feature space corresponding to the training data set, C i Represents the number of samples belonging to the i-th category in the feature space;

[0028] B3. Calculate the sample purity of each sub-interval after selecting feature partitioning according to the Purity indicator. When the sample purity or the depth of the sub-classifier meets the corresponding requirements, stop randomly selecting features on the sub-interval and continue partitioning;

[0029] B4. Repeat steps B1-B3 until X sub-classifiers are constructed.

[0030] Furthermore, the B classifier, the E classifier and the F classifier are recorded as the second type of classifiers, and the design of the second type of classifiers includes the following steps:

[0031] C1. The second type of classifier includes Y sub-classifiers, each of which selects features as partition judgment conditions according to indicators in a random feature subspace of the feature space without replacement;

[0032]

[0033]

[0034] Among them, Index represents the index, S represents the feature space size of the training data set, and C i It represents the number of samples belonging to the i-th category in the feature space. The corresponding index is calculated for all selected features, and the feature with the smallest index value is selected as the judgment condition for partitioning each time;

[0035] C2: When the number of samples in a sub-interval or the depth of the sub-classifier meets the corresponding requirements, stop randomly selecting features in the sub-interval and continue partitioning;

[0036] C3. Repeat steps C1-C2 until Y sub-classifiers are constructed.

[0037] Furthermore, the sound signal is output to the road noise active control system in real time, and the trigger module detects the sound signal in real time. If a preset wake-up command signal is detected, the sound signal within a preset time after the detected time node is automatically output to the in-vehicle intelligent voice system, and then automatically stops being output to the in-vehicle intelligent voice system after the preset time ends, waiting for the next trigger.

[0038] The technical solution provided by the present invention brings the following beneficial effects: the microphone of the vehicle-mounted road noise active control system and the microphone of the vehicle-mounted intelligent voice system are reused for sound pickup functions, the number of vehicle-mounted microphones is reduced, and the cost of the car is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 It is a schematic diagram of an active vehicle road noise control system in a method for multiplexing functions of a vehicle microphone provided by an embodiment of the present invention;

[0041] Figure 2 It is a schematic diagram of a signal transmission design for multiplexing a microphone pickup function in a function multiplexing method of a vehicle-mounted microphone provided by an embodiment of the present invention;

[0042] Figure 3 It is a schematic diagram of the voice signal processing flow of the microphone in the function multiplexing method of the vehicle-mounted microphone provided by an embodiment of the present invention;

[0043] Figure 4 This is a diagram showing the effect of speech signal discrimination after mean processing in the function multiplexing method of the vehicle-mounted microphone provided by an embodiment of the present invention;

[0044] Figure 5 This is a diagram showing the effect of speech signal discrimination after minimum value processing in the function multiplexing method of the vehicle-mounted microphone provided by an embodiment of the present invention;

[0045] Figure 6 This is a diagram showing the effect of speech signal discrimination after the ratio of the absolute value to the mean value is processed in the function multiplexing method of the vehicle-mounted microphone provided by an embodiment of the present invention;

[0046] Figure 7 It is a speech signal discrimination effect diagram after the ratio of the absolute value and the minimum value is processed in the function multiplexing method of the vehicle-mounted microphone provided by the embodiment of the present invention;

[0047] Figure 8 It is a schematic diagram of a positioning algorithm framework in a method for multiplexing functions of a vehicle-mounted microphone provided in an embodiment of the present invention;

[0048] Fig. 9 It is a schematic diagram of the structure of the first partition module in the function multiplexing method of the vehicle-mounted microphone provided by an embodiment of the present invention;

[0049] Fig.10 It is a schematic diagram of the structure of the second partition module in the function multiplexing method of the vehicle-mounted microphone provided by an embodiment of the present invention.

[0050] Among them, the reference numerals are as follows: 100 - first A2B transmission line, 101 - second A2B transmission line, 102 - accelerometer, 103 - distributed microphone, 104 - vehicle system, 105 - controller of road noise active control system. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, and to more clearly understand the purpose, technical scheme and advantages of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be noted that the implementation methods not shown or described in the accompanying drawings are forms known to those skilled in the art. In addition, although demonstrations of parameters containing specific values ​​may be provided herein, it should be understood that the parameters do not need to be exactly equal to the corresponding values, but may be approximated to the corresponding values ​​within an acceptable error tolerance or design constraint. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work should belong to the scope of protection of the present invention. In addition, the terms "including" and "having" in the specification and claims of the present invention and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment containing a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0052] In one embodiment of the present invention, a method for multiplexing functions of a vehicle-mounted microphone is provided, characterized in that it includes the following steps:

[0053] S1, outputting the sound signal collected by the microphone in the road noise active control system to the trigger module;

[0054] S2. If the trigger module detects a wake-up command signal in the sound signal, the sound signal is output to the vehicle-mounted intelligent voice system, and step S3 is performed. If no voice signal is detected for a period of time, the sound signal is stopped from being output to the vehicle-mounted intelligent voice system.

[0055] S3. Downsampling the sound signal to adapt it to the speech recognition module of the in-vehicle intelligent speech system.

[0056] Among them, the voice signal refers to the sound signal of human speech. If no human speech is detected in the sound signal, the subsequent sound signal will naturally stop being output to the in-vehicle intelligent voice system. For example, the trigger module is triggered to output the sound signal to the in-vehicle intelligent voice system, and then no voice signal is found in the sound signal within 5 seconds, it is determined that the voice recognition is canceled by default, and the trigger module stops outputting subsequent sound signals to the in-vehicle intelligent voice system. It should be noted that the trigger module can detect a sound signal that is the same or similar to the wake-up command signal. For example, if the similarity of the two signals is higher than 80%, the trigger module is triggered.

[0057] In the previous embodiment, if two people in the car trigger the in-vehicle intelligent voice system and have a continuous conversation after issuing relevant control instructions, it is easy to cause the sound signals of the subsequent conversation to continue to enter the in-vehicle intelligent voice system for calculation and recognition, which greatly increases its calculation burden. In order to reduce the calculation pressure of the in-vehicle intelligent voice system, this embodiment has been improved to limit the single time of the sound signal entering the in-vehicle intelligent voice system. The sound signal is always output to the road noise active control system in real time, and the trigger module detects the sound signal in real time. If a preset wake-up command signal is detected, the sound signal within a preset time after the detected time node is automatically output to the in-vehicle intelligent voice system, and then automatically stops outputting to the in-vehicle intelligent voice system after the preset time ends, waiting for the next trigger. For example, the wake-up command signal is a sound signal of "Zhang San, Zhang San", and the sound signal needs to appear twice in 2 seconds to trigger the trigger module, so that the sound signal of 10s after this time node enters the in-vehicle intelligent voice system. After 10s, the subsequent sound signals will stop entering the in-vehicle intelligent voice system, and the qualified wake-up command signal needs to be detected again to trigger the voice recognition operation accordingly. This embodiment greatly reduces the operation pressure of the in-vehicle intelligent voice system.

[0058] In one embodiment of the present invention, the microphone pickup function of the vehicle-mounted intelligent voice system is reused according to the design specifications and layout positions of the microphone harness of the road noise active control system. Because the road noise active control system has higher requirements on the microphone location than the vehicle-mounted intelligent voice system, the design specifications and layout positions of the microphone harness of the vehicle-mounted road noise active control system are preferred, so that the workload of production modification is smaller. Figure 1 As shown, four distributed microphones 103 are correspondingly arranged near four vehicle seats, and the distributed microphones 103 are all connected to the vehicle system 104 through the first A2B transmission line 100. Four accelerometers 102 are respectively arranged near four tires, and the accelerometers 102 are all connected to the controller 105 of the road noise active control system through the second A2B transmission line 101.

[0059] In order to save the audio signal harness design on the car, the microphone of the road noise active control system adopts the A2B harness structure to connect. Figure 2 As shown, Figure 2Sensor in the figure is the acceleration sensor required by the road noise active control system, and mic is the microphone. The signal collected by the microphone is divided into two signals through the vehicle computer system. One signal is used as the error signal of the road noise active control system to be input into the algorithm for calculation, and the other signal is used as the input path of the voice signal to be transmitted to the vehicle intelligent voice system to complete the voice system task. This transmission path is set with a trigger module, which is equivalent to a trigger switch. When the trigger switch detects the correct wake-up command signal, this path is opened and the signal is input to the vehicle intelligent voice system for recognition. When the trigger switch cannot detect the voice signal or the detected wake-up command signal is incorrect, this path is set to be closed. It is considered that the signal at this time is a noise signal, not a useful voice signal, and is not input into the vehicle intelligent voice system for voice tasks.

[0060] The microphone of the traditional in-vehicle intelligent voice system is an analog signal device, and a 16KHz sampling rate is set to sample for voice tasks. The microphone in the road noise active control system of this embodiment is a digital signal device, and its sampling rate is 48KHz. The collected signal has higher accuracy and restoration. Figure 3 , perform feature engineering design on the signal, use the designed feature combination as the input of the speaker positioning algorithm, and after completing the positioning judgment, downsample the signal to 16KHz and input it into the in-vehicle intelligent voice system in the vehicle system through the transmission path to perform the intelligent voice task. The output is processed by downsampling rate to adapt to the in-vehicle intelligent voice system. A high sampling rate requires too much computing power and storage space, so it is generally reduced to 16KHz to reduce unnecessary burden on voice recognition.

[0061] In one embodiment of the present invention, the road noise active control system and the in-vehicle intelligent voice system share a set of microphones, saving the cost of the microphones. The microphones include multiple distributed microphones, and the distributed microphones are respectively arranged near each seat in the car, such as under, on top or inside the seat, etc. The sound signal is collected by the distributed microphones, and the sound signal includes sound information monitored at multiple positions in the car; for example, there are four seats in the car, and a distributed microphone is arranged under each seat in the car. The voice signal of a person speaking will be collected by the four distributed microphones at the same time, and the sound signal received at the trigger module includes the signals collected by the four distributed microphones respectively.

[0062] Before downsampling the sound signal, multiple groups of sound signals are used for voice localization. Since the microphones of the road noise active control system are distributed and arranged in a single channel, it is impossible to use the traditional microphone array-based algorithm to locate the speaker. Therefore, the localization method of this embodiment is redesigned to distinguish the speaker positions of the main seats in the car, and the high sampling rate signal of the A2B microphone is used to design the feature engineering of the voice signal, and the idea of ​​classification task in machine learning is used to solve the speaker localization problem.

[0063] Wherein, the voice positioning comprises the following steps:

[0064] A1. extracting multiple groups of sound data from the data set of the sound signal to form multiple sample sets as different training data sets;

[0065] A2. Performing various feature engineering processes on the training data sets to obtain corresponding feature vector combinations;

[0066] A3, the partitioning module uses different data selection modes to locate and classify the feature vector combination according to the seat distribution in the car, so as to obtain the probability value of each seat in the car making a sound;

[0067] A4. The seat corresponding to the maximum probability value is determined as the speaker's seat, and the sound signal collected by the distributed microphone corresponding to the speaker's seat is output to the speech recognition module.

[0068] Under the same conditions, the closer to the speaker, the clearer the collected signal will be and the less interference. This embodiment uses voice positioning to find the most suitable set of signals, downsamples them and outputs them to the voice recognition module for voice recognition to improve recognition accuracy.

[0069] In step A2, in the feature engineering process, the signal feature types collected by the microphone include one or more of a mean feature, a minimum feature, a feature combination of an absolute value and a mean, and a feature combination of an absolute value and a minimum.

[0070] Specifically, the steps for calculating the signal characteristics of each type are as follows:

[0071] (1) Mean feature processing

[0072] a. Process the sound signal data with a sampling rate of 48KHz by framing, set the frame length M to 1200 sampling points, and the frame shift K to 480. The relevant expressions are as follows:

[0073]

[0074] Where t is the frame index, i is the index of the speaker position, j is the index of the microphone position, and Z is the frame number. Represents the speech vector of speaker at position i collected by microphone j at frame t.

[0075] b. Perform mean operation on the framed sound vector,

[0076]

[0077] c. Each frame of sound data after averaging Added to an array of shape [1, Z], the speech signal discrimination effect collected by the microphones of the four seats after mean processing is as follows Figure 4 As shown in the figure, A, B, C, and D respectively represent four seats in the car, AA represents the signal picked up by the microphone of seat A when the speaker at seat A speaks, AB represents the signal picked up by the microphone of seat B when the speaker at seat A speaks, and so on.

[0078] (2) Minimum feature processing

[0079] a. Perform the minimum operation on the sound vector of each frame after framing

[0080]

[0081] b. Each frame of speech data after minimum value processing Added to the array of shape [1, Z], the discrimination effect of the speech signals collected by the microphones of the four seats after the minimum value calculation is as follows Figure 5 shown.

[0082] (3) Feature combination of absolute value and mean

[0083] a. Perform an absolute value operation on each frame of sound vector and then perform a mean operation

[0084]

[0085] b. Calculate the absolute value and mean of each frame of speech data Added to an array of shape [1, Z], the speech signal discrimination effect collected by the microphones of the four seats after the feature combination calculation of the absolute value and the mean is as follows Figure 6 shown.

[0086] (4) Feature combination of absolute value and minimum value

[0087] a. Take the absolute value of each frame of speech vector and find the minimum value

[0088]

[0089] b. Calculate the absolute value and minimum value of each frame of speech data Added to an array of shape [1, Z], the speech signal discrimination effect collected by the microphones of the four seats after the absolute value and minimum value feature combination calculation is as follows Figure 7 shown.

[0090] In step A3, see Figure 8 The partitioning module includes a first partitioning module, a second partitioning module and a classification layer module, see Fig. 9 , the first partitioning module includes an A partitioner and a B partitioner, see Fig.10 , the second partitioning module includes multiple partitioning layers, each of which includes C partitioner, D partitioner, E partitioner, and F partitioner; multiple second partitioning modules are associated and sorted to form a second partitioning module with a multi-level structure, the number of the second partitioning modules is equal to the number of signal feature types used, the number of signal feature types is preferably equal to the number of parking seats, and the output of the second partitioning module of the previous level is used as the input of the second partitioning module of the next level; the classification layer module includes multiple partitioners;

[0091] A feature vector combination of a training data set corresponds to forming a feature space. The A discriminator, C discriminator and D discriminator will randomly select signal features without replacement in the complete feature space as partition judgment conditions. The B discriminator, E discriminator and F discriminator will select features according to indicators without replacement in the incomplete feature space as partition judgment conditions. Different feature vector combinations will be processed successively by the first partition module, the second partition module of the multi-level structure and the classification layer module to output the probability value of sound emitted by each seat in the car.

[0092] Among them, the design steps of each classifier can be roughly divided into the following two categories:

[0093] (1) The A classifier, C classifier and D classifier are referred to as first-class classifiers. The design of the first-class classifiers includes the following steps:

[0094] B1. The first type of classifier includes X sub-classifiers, each of which randomly selects signal features without replacement in the complete feature space as partition judgment conditions;

[0095] B2. Design an indicator to measure the purity of the training data set, denoted as Purity.

[0096]

[0097]

[0098] Among them, S represents the size of the feature space corresponding to the training data set, C i Represents the number of samples belonging to the i-th category in the feature space;

[0099] B3. Calculate the sample purity of each sub-interval after selecting feature partitioning according to the Purity indicator. When the sample purity or the depth of the sub-classifier meets the corresponding requirements, stop randomly selecting features on the sub-interval and continue partitioning;

[0100] B4. Repeat steps B1-B3 until X sub-classifiers are constructed.

[0101] (2) The B classifier, the E classifier, and the F classifier are referred to as the second class classifier. The design of the second class classifier includes the following steps:

[0102] C1. The second type of classifier includes Y sub-classifiers, each of which selects features as partition judgment conditions according to indicators in a random feature subspace of the feature space without replacement;

[0103]

[0104]

[0105] Among them, Index represents the index, S represents the feature space size of the training data set, and C i It represents the number of samples belonging to the i-th category in the feature space. The corresponding index is calculated for all selected features, and the feature with the smallest index value is selected as the judgment condition for partitioning each time;

[0106] C2: When the number of samples in a sub-interval or the depth of the sub-classifier meets the corresponding requirements, stop randomly selecting features in the sub-interval and continue partitioning;

[0107] C3. Repeat steps C1-C2 until Y sub-classifiers are constructed.

[0108] Each feature vector combination will obtain a corresponding model training feature vector after being processed by the first partitioning module; the first model training feature vector is output to the first-level second partitioning module for processing. In the processing of the first-level second partitioning module, the output result of each differentiation layer will be spliced ​​with the first model training feature vector to serve as the input of the next differentiation layer;

[0109] The output result of the first-level second partition module is spliced ​​with the first model training feature vector to be input into the second-level second partition module. In the processing of the second-level second partition module, the output result of each differentiation layer is spliced ​​with the second model training feature vector to serve as the input of the next differentiation layer, and so on to obtain the output result of the last-level second partition module, which is spliced ​​with the last model training feature vector and input into the classification layer module to obtain the probability value of sound emitted by each seat in the car.

[0110] In step A4, each seat has a corresponding seat number, and the voice recognition module assigns the same or different vehicle control permissions to sound signals from different seat numbers. The voice recognition module will determine whether to execute the associated voice control function in the car based on the located seat permissions for the recognized voice command. For example, seat A is the driving seat, seat B is the co-pilot seat, C and D are the back seats, and seat A has all voice control permissions, such as voice control of playing music, making calls, turning on lights, automatic driving, etc., while seats C and D only have voice control permissions for playing music. Control permissions are differentiated by seat to avoid misoperation of other people's voices and improve the safety of voice control in the car.

[0111] The following is a specific example to illustrate the positioning operation steps:

[0112] Step 1: Data preprocessing

[0113] Step 1.1 Randomly extract 70% of the samples from four groups of original data sets with replacement from the sound data set with sampling rate of 48KHz as the training data set of the algorithm, which are denoted as D1, D2, D3, and D4 respectively.

[0114] Step 1.2: Perform the above four feature engineering processes on each of the four selected training data sets

[0115] x nm =Featurization m (D n ),n=1,...,4,m=1,...,4

[0116] in, Featurization(*) indicates feature engineering processing of the data set, n indicates the label of the training data set, and m indicates the labels of the above four feature engineering processing methods.

[0117] Step 1.3 When n is constant, combine the feature vectors obtained through the four feature engineering processes

[0118]

[0119] in,

[0120] Step 2: Build the first partition module

[0121] Step 2.1 The function of the first partitioning module is to perform positioning analysis on the voice signal in the sound signal according to the seats in the car. Module 1 includes A classifier and B classifier, which are respectively denoted as ClassifierA and ClassifierB.

[0122] Step 2.2 Build A discriminator

[0123] Step 2.2.1A The classifier contains N1 (N1=30) sub-classifiers, each of which randomly selects signal features without replacement in the complete feature space as the judgment condition for partitioning;

[0124] Step 2.2.2 Design an index to measure the purity of speech sample set

[0125]

[0126] Among them, S represents the size of the feature space corresponding to the training data set, C i Represents the number of samples belonging to the i-th category in the feature space;

[0127] Step 2.2.3 calculates the sample purity of each sub-interval after selecting feature partitioning according to the Purity indicator described in step 2.2.2. When Purity < 0.1 or the depth of the sub-classifier deep > 10, stop randomly selecting features on the sub-interval and continue partitioning;

[0128] Step 2.2.4 Repeat steps 2.2.1-2.2.3 until N1 sub-distinguisers are constructed.

[0129] Step 2.3 Build the B discriminator

[0130] Step 2.3.1B The discriminator contains N2 (N2 = 30) sub-discriminators, each of which is in a random feature subspace of the original feature space (the size of the feature subspace is ) without replacement, select features according to the index as the judgment condition for partitioning,

[0131]

[0132] Among them, S represents the size of the feature space corresponding to the training data set, C i Indicates the number of samples belonging to the i-th category in the feature space. The Index is calculated for all selected features, and the feature with the smallest Index value is selected as the judgment condition for partitioning each time;

[0133] Step 2.3.2 When the number of samples in the subinterval is less than When the depth of the sub-classifier is greater than 10, the feature is randomly selected in the sub-interval to continue partitioning.

[0134] Step 2.3.3 Repeat steps 2.3.1-2.3.2 until N2 sub-distinguisers are constructed.

[0135] Step 3: Feature extraction

[0136] Step 3.1: The feature vector processed by step 1.3 Scanning with a 3×3 square window yields four sets of feature vectors with the shape of [2(Z-2),3,3] To obtain more feature samples, thus preserving the temporal relationship between features.

[0137] Step 3.2 inputs the four sets of feature vectors into the first partition module for four classification tasks.

[0138]

[0139] in, PartitionModel1(*) represents the first partition module, and k is the index value of the classifier in the first partition module (when k=1, it represents the A classifier described in step 2.2, and when k=2, it represents the B classifier described in step 2.3).

[0140] Step 3.3 When n is constant, out nk Transform the shape and combine them to obtain the feature vector required for subsequent model training.

[0141]

[0142]

[0143] in,

[0144] Step 4: Build the second partition module

[0145] Step 4.1 The second partitioning module includes M partitioning layers, for example, M=8, the partitioning layer is denoted as level, each level includes C partitioner, D partitioner, E partitioner, and F partitioner, wherein the structures of C partitioner and D partitioner are the same, and the structures of E partitioner and F partitioner are the same).

[0146] Step 4.2 Build the C Differentiator

[0147] Step 4.2.1C The discriminator contains N3 (N3=51) sub-discriminators, each of which randomly selects features without replacement in the complete feature space S as the judgment condition for partitioning.

[0148] Step 4.2.2 Design the Purity indicator to measure the purity of the sample set.

[0149]

[0150] Step 4.2.3 calculates the sample purity of each sub-interval after feature partitioning based on the Purity indicator described in step 4.2.2. When Purity < 0.1 or the depth of the sub-classifier deep > 15, stop randomly selecting features on the sub-interval and continue partitioning.

[0151] Step 4.2.4 Repeat steps 4.2.1-4.2.3 until N3 sub-distinguisers are constructed.

[0152] Step 4.3 constructs the D classifier, and the steps are the same as those for constructing the C classifier, until N4 (N4=51) sub-classifiers are constructed.

[0153] Step 4.4 Build the E discriminator

[0154] Step 4.4.1E The discriminator contains N5 (N5 = 51) sub-discriminators, each of which is in a random feature subspace of the original feature space (the size of the feature subspace is ) without replacement, select features according to the index as the judgment condition for partitioning

[0155]

[0156] Among them, the index is calculated for all selected features, and the feature with the smallest index value is selected as the judgment condition for partitioning each time;

[0157] Step 4.4.2 When the number of samples in the subinterval is less than When the depth of the sub-classifier is greater than 15, the feature is randomly selected in the sub-interval to continue partitioning.

[0158] Step 4.4.3 Repeat steps 4.4.1-4.4.2 until N5 sub-differentiators are constructed.

[0159] Step 4.5 Step 4.3 constructs the F distinguisher, and the steps are the same as those for constructing the E distinguisher until N6 (N6=51) sub-distinguisers are constructed.

[0160] Step 5: Model training

[0161] Step 5.1: Convert the output obtained in step 3.3 to n They are respectively input into the second partitioning modules with the same four-level structure, where the four-level second partitioning modules correspond to the four feature vectors in step 3.3 respectively, and each level of the second partitioning module has an N-layer structure.

[0162]

[0163] in, PartitionModel2(*) represents the second partition module, s is the index value of the level in each level of the second partition module described in step 4.1, n represents both the label of the feature vector and the label of the four-level second partition module, l is the index value of the classifier in each level in the second partition module, for example, when l=1, it represents the C distinguisher described in step 4.2, when l=2, it represents the D distinguisher described in step 4.3, when l=3, it represents the E distinguisher described in step 4.4, and when l=4, it represents the F distinguisher described in step 4.5.

[0164] Step 5.2 When n and s are constant, combine the outputs of the four PartitionModel2

[0165]

[0166] in,

[0167] Step 5.3 combines the output of each level in step 5.2 with the corresponding feature vector obtained in step 3.3

[0168]

[0169] in,

[0170] Step 5.4 When the first feature vector in step 3.3 completes steps 5.1-5.3 in the first-level PartitionModel2, the output is obtained Then the output is input to the second-level PartitionModel2 to perform steps 5.1 to 5.3, and so on, until all four feature vectors of step 3.3 complete steps 5.1 to 5.3 in the corresponding fourth-level PartitionModel2, and obtain a feature vector with a shape of [(16+16(Z-2)), 1]

[0171] Step 5.5: Input the feature vector output from step 5.4 into the classification layer module composed of the four distinguishers C, D, E, and F described in step 4 to perform the final classification task.

[0172]

[0173] in, ClassifierLayer(*) represents the classification layer described in this step, h represents the index value of the classifier in ClassifieLrayer, where h=1 represents the C distinguisher described in step 4.2, h=2 represents the D distinguisher described in step 4.3, h=3 represents the E distinguisher described in step 4.4, h=4 represents the F distinguisher described in step 4.5, and q is the label of the four categories.

[0174] Step 5.6: Calculate the average of the probability values ​​of the four groups and four categories obtained in step 5.5.

[0175]

[0176] in,

[0177] Step 5.7 takes the maximum value of the probability values ​​of the four categories output in step 5.6

[0178] result=max(result q )

[0179] in, is the final output classification result, i.e., the speaker position, which corresponds to the seat number.

[0180] In one embodiment of the present invention, in order to reduce the amount of computation for voice localization, the trigger module is provided with a human voice discrimination unit, and the human voice discrimination unit is used to determine whether there is a human voice feature in the sound signal. The trigger module detects in the sound signal that before entering the speech recognition module, it first enters the human voice discrimination module to make a judgment based on the features of the human voice. If it is determined to be a human voice, the sound signal is then subjected to a positioning operation to avoid performing a positioning operation on a non-speech signal, that is, to avoid locating an invalid signal.

[0181] The function reuse method of the vehicle microphone provided by the present invention reuses the microphone of the vehicle road noise active control system and the microphone of the vehicle intelligent voice system for sound pickup, and reduces the number of vehicle microphones, reduces the wiring harness of the in-vehicle system, and reduces the cost of the car while ensuring the functions of the two systems; and realizes the positioning function of the speaker in the car based on the installation and layout of the distributed microphone, improves the positioning accuracy, and improves the anti-interference ability and robustness of the speaker positioning function. The present invention effectively solves the problem that when the noise source is in the same direction as the speaker, the existing system has low positioning accuracy and unsatisfactory robustness for the speaker.

[0182] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for multiplexing functions of a vehicle-mounted microphone, characterized in that: The following steps are involved: S1, outputting the sound signal collected by the microphone in the road noise active control system to the trigger module; S2. If the trigger module detects a wake-up command signal in the sound signal, the sound signal is output to the vehicle-mounted intelligent voice system, and step S3 is performed. If no voice signal is detected for a period of time, the sound signal is stopped from being output to the vehicle-mounted intelligent voice system; the road noise active control system and the vehicle-mounted intelligent voice system share a set of microphones; S3, downsampling the sound signal to adapt to the speech recognition module of the vehicle-mounted intelligent speech system; Before downsampling the sound signal, the sound signal is first used for voice localization, including the following steps: A1, extracting multiple groups of sound data from the data set of the sound signal to form multiple sample sets as different training data sets; A2, performing multiple feature engineering processes on the training data sets respectively to obtain corresponding feature vector combinations; A3, a partitioning module uses different data selection modes to locate and classify the feature vector combinations according to the seat distribution in the car to obtain the probability value of each seat in the car emitting sound; A4, judging the seat corresponding to the largest probability value as the speaker's seat, and outputting the sound signal collected by the distributed microphone corresponding to the speaker's seat to the speech recognition module; The partition module includes a first partition module, a second partition module and a classification layer module, the first partition module includes an A discriminator and a B discriminator, the second partition module includes a plurality of discrimination layers, each discrimination layer includes a C discriminator, a D discriminator, an E discriminator, and an F discriminator; the plurality of second partition modules are associated and sorted to form a second partition module with a multi-level structure, the number of the second partition modules is equal to the number of signal feature types collected by the microphone, and the output of the second partition module of the previous level is used as the input of the second partition module of the next level; the classification layer module includes a plurality of discriminators; a feature vector combination of a training data set forms a feature space correspondingly, the A discriminator, the C discriminator and the D discriminator all randomly select signal features in the complete feature space without replacement as partition judgment conditions, and the B discriminator, the E discriminator and the F discriminator all select features according to indicators in the incomplete feature space without replacement as partition judgment conditions; different feature vector combinations are successively processed by the first partition module, the second partition module of the multi-level structure and the classification layer module to output the probability value of sound emitted by each seat in the car.

2. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: The microphone includes a plurality of distributed microphones, and the distributed microphones are respectively arranged near each seat in the vehicle. The sound signal is collected by the distributed microphones, and the sound signal includes sound information monitored at multiple positions in the vehicle.

3. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: The voice recognition module assigns the same or different vehicle control authorities to the sound signals from different seat numbers.

4. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: The signal feature types collected by the microphone include one or more of a mean feature, a minimum feature, a feature combination of an absolute value and a mean, and a feature combination of an absolute value and a minimum.

5. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: Each feature vector combination will correspond to a model training feature vector after being processed by the first partitioning module; Output the first model training feature vector to the first-level second partitioning module for processing. In the first-level second partitioning module processing, the output result of each differentiation layer is concatenated with the first model training feature vector to serve as the input of the next differentiation layer. The output result of the first-level second partition module is spliced ​​with the first model training feature vector to be input into the second-level second partition module. In the processing of the second-level second partition module, the output result of each differentiation layer is spliced ​​with the second model training feature vector to serve as the input of the next differentiation layer, and so on to obtain the output result of the last-level second partition module, which is spliced ​​with the last model training feature vector and input into the classification layer module to obtain the probability value of sound emitted by each seat in the car.

6. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: The A classifier, the C classifier and the D classifier are recorded as the first class classifier, and the design of the first class classifier includes the following steps: B1. The first type of classifier includes X sub-classifiers, each of which randomly selects signal features without replacement in the complete feature space as partition judgment conditions; B2. Design an index to measure the purity of the training data set, denoted as Purity , ； in, S Indicates the size of the feature space corresponding to the training data set, C i Indicates that it belongs to the first i The number of samples in each category; B3. According to the indicators Purity To calculate the sample purity of each sub-interval after selecting feature partitioning, when the sample purity or the depth of the sub-classifier meets the corresponding requirements, stop randomly selecting features on the sub-interval and continue partitioning; B4. Repeat steps B1-B3 until X sub-classifiers are constructed.

7. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: The B classifier, the E classifier and the F classifier are recorded as the second class classifier, and the design of the second class classifier includes the following steps: C1. The second type of classifier includes Y sub-classifiers, each of which selects features as partition judgment conditions according to indicators in a random feature subspace of the feature space without replacement; ； in, Index Indicates the indicator, S represents the feature space size of the training dataset, C i Indicates that it belongs to the first i The number of samples in each category is calculated, and the corresponding index is calculated for all selected features. Each time, the feature with the smallest index value is selected as the judgment condition for partitioning; C2: When the number of samples in a sub-interval or the depth of the sub-classifier meets the corresponding requirements, stop randomly selecting features in the sub-interval and continue partitioning; C3. Repeat steps C1-C2 until Y sub-classifiers are constructed.

8. The function multiplexing method of the vehicle-mounted microphone according to claim 1, characterized in that: The sound signal is continuously output to the road noise active control system in real time, and the trigger module detects the sound signal in real time. If a preset wake-up command signal is detected, the sound signal within a preset time after the detected time node is automatically output to the in-vehicle intelligent voice system, and the sound signal automatically stops being output to the in-vehicle intelligent voice system after the preset time ends, waiting for the next trigger.

Images