Sound mixing and noise reduction method and system

Abstract

The invention discloses a sound mixing and noise reduction method and system, and the method comprises the steps: firstly, carrying out the recognition of the emotional state of a person in a vehicle through an emotion recognition method; and then, whether the person in the vehicle has a bad emotion which can influence the behavior state or not can be judged according to the emotion recognition result. And if the person in the vehicle does not have the impatient emotion, the current noise in the vehicle does not need to be processed. On the contrary, if the person in the vehicle has a restless emotion, the sound signal in the vehicle is immediately collected through the sound pick-up. And finally, a sound wave identification method can be adopted to extract a noise feature vector of a noise sound segment which affects the emotion of people in the vehicle from the collected sound signals, and a corresponding sound mixing sound wave band is generated based on the extracted noise feature vector and is fused with the noise sound segment which affects the people in the vehicle, so that the sound mixing sound wave band becomes a sound segment which can be accepted by people. Therefore, the influence of noise on the behavior state of people is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of methods for regenerating inverted original sound waves by electroacoustic methods, and in particular to a mixing noise reduction method and system. Background Art

[0002] Nowadays, smart cars have been integrated into all aspects of the automotive industry, and the gradual improvement of the interactivity of cars is an inevitable trend. The interaction of cars has gradually become the main way of interaction through sound. In this process, the consideration of internal noise cannot be ignored. Due to the multi-interaction of cars, the probability of internal noise in cars has gradually increased with the increase in the number of voice interaction function points. During long-term driving, if there is a long-term noise in the car in the driving environment, the driver will feel uncomfortable and lose driving ability or increase driving fatigue, thereby affecting the driver's safety. Therefore, how to prevent the impact of internal noise on the driver's driving state needs to be put on the agenda.

[0003] At present, the industry's considerations for noise reduction are mainly active noise reduction technology and passive noise reduction technology. Passive noise reduction technology mainly reduces the noise entering the car by physically isolating or blocking external noise, such as using sound insulation materials and soundproof windows. However, this technology cannot effectively deal with the noise in the internal environment. Active noise reduction technology detects noise using sensors and uses acoustic equipment to emit sound waves opposite to the noise to offset the noise, thereby achieving a noise reduction effect.

[0004] However, the distance between the driver and the sound source in the cockpit is relatively close, and due to the limited speed of sound propagation, it may cause latency problems and reduce the noise reduction effect. At the same time, the driver will frequently change the posture and position of the head while driving, which means that the mixing noise reduction system needs to be able to adjust the sound waves emitted in real time to match the driver's position and direction, which increases the complexity of the system. In addition, various surfaces and structures in the cockpit will cause reflections and interference of sound waves, thus affecting the effect of active noise reduction.

[0005] In 2022, Hiroki Deku, Kenta Sato and others from Meiji University in Japan discovered that the impact of noise on people can be reduced by using mixed sound. By quoting the mixed sound band of the corresponding frequency band and fusing it with the noise, it becomes a sound band that people can accept, thus solving the above problems of active noise reduction.

[0006] Specifically, in the sound band processing of mixing, the delay problem can be solved by adjusting the time delay of each sound element. In digital audio, the sound is recorded at a certain sampling rate. A higher sampling rate can record the start time of the sound more accurately. When mixing, the sound delay can be adjusted more accurately by using the sound band information with high sampling accuracy.

[0007] And the mixing sound band allows complex audio systems to be broken down into multiple levels. In this way, the originally complex audio system is simplified into several relatively independent parts, reducing the difficulty of processing. This modular processing method avoids the complexity of mixing all sound elements together, allowing audio engineers to optimize each sound band in a targeted manner.

[0008] Reflections and interference of sound waves can cause phase problems. In the mixed sound band, this effect can be mitigated by operations such as inverting the phase. Spatial effect plug-ins such as reverb and delay can simulate different acoustic spaces, thereby masking or improving the problems caused by sound wave reflections and interference. Summary of the invention

[0009] In view of the shortcomings of the prior art, the present invention proposes a mixing noise reduction method and system, which can efficiently and accurately solve the impact of noise. The specific technical solution is as follows:

[0010] In a first aspect, a mixing noise reduction method is provided. In a first implementable manner of the first aspect, the method includes:

[0011] Using emotion recognition methods to identify the emotions of organisms, and judging whether the organisms are in a bad mood based on the recognition results;

[0012] In response to the organism being in a bad mood, obtaining a sound signal of the organism's surrounding environment;

[0013] A sound wave recognition method is used to recognize noise feature vectors of noise sound segments in the sound signal, and corresponding mixed sound wave segments are generated based on the noise feature vectors of each noise sound segment.

[0014] In combination with the first implementable manner of the first aspect, in a second implementable manner of the first aspect, using an emotion recognition method to perform emotion recognition on an object includes: using an image emotion recognition method to recognize the emotion of the organism.

[0015] In combination with the first implementable manner of the first aspect, in a third implementable manner of the first aspect, using a sound wave recognition method to identify a noise feature vector of a noise segment in the sound signal includes:

[0016] Using an adaptive filter to filter the sound signal to obtain a noise segment in the sound signal;

[0017] Feature extraction is performed on each of the noise sound segments mentioned to obtain a noise feature vector of the noise sound segment.

[0018] In combination with the third implementable manner of the first aspect, in a fourth implementable manner of the first aspect, an adaptive filter is used to filter the sound signal, including: performing noise reduction processing on the sound signal.

[0019] In combination with the third implementable manner of the first aspect, in a fifth implementable manner of the first aspect, filtering the sound signal by using an adaptive filter includes:

[0020] According to the changes of the sound signal, the least mean square method is used to dynamically adjust the parameters of the adaptive filter.

[0021] In combination with the first implementable manner of the first aspect, in a sixth implementable manner of the first aspect, using a sound wave recognition method to identify a noise feature vector of a noise segment in the sound signal includes:

[0022] The noise feature vector structure of each noise segment extracted is compared with the pre-built noise rule library conditions, and the number of sound segments that do not meet the noise rule library conditions is counted;

[0023] In response to the number of sound segments exceeding a threshold, it is determined that no noise exists in the sound signal.

[0024] In combination with the first implementable manner of the first aspect, in a seventh implementable manner of the first aspect, generating a corresponding mixed sound wave band based on the noise feature vector of each noise sound segment includes:

[0025] The noise characteristic vectors of each noise segment are converted into normal distribution random variables respectively;

[0026] Based on the corresponding normally distributed random variables, respectively determine the mixing feature data and weight coefficients corresponding to each noise sound segment;

[0027] The mixed sound signal data corresponding to each noise sound segment is respectively generated in reverse according to the corresponding mixed sound feature data, and the mixed sound wave band is generated in combination with the corresponding weight coefficient.

[0028] In combination with the seventh implementable manner of the first aspect, in an eighth implementable manner of the first aspect, a Box-Muller algorithm is used to convert the noise feature vector of the noise segment into a normally distributed random variable that satisfies normal distribution conditions.

[0029] In combination with the first implementable manner of the first aspect, in a ninth implementable manner of the first aspect, generating a corresponding mixed sound wave band based on the noise feature vector of each noise sound segment includes:

[0030] Re-adopt the emotion recognition method to recognize the emotion of the organism, and judge whether the organism is still in a bad emotion according to the recognition result;

[0031] In response to the organism still being in a bad mood, the sound signal of the environment surrounding the organism continues to be acquired, and a corresponding mixed sound wave band is generated until the organism returns to normal.

[0032] In a second aspect, a mixing noise reduction system is provided, comprising:

[0033] An emotion recognition module, configured to use an emotion recognition method to perform emotion recognition on a biological subject, and determine whether the biological subject is in a bad emotion according to the recognition result;

[0034] In response to the organism being in a bad mood, obtaining a sound signal of the organism's surrounding environment;

[0035] The noise processing module is configured to identify the noise feature vectors of the noise sound segments in the sound signal by using a sound wave recognition method, and generate corresponding mixed sound wave segments based on the noise feature vectors of each noise sound segment.

[0036] Beneficial effects: The mixed-audio noise reduction method and system of the present invention can accurately identify the emotions of the people in the car by using the emotion recognition method, and the sound signal containing the noise sound segment that affects the behavior state of the people in the car can be accurately collected according to the emotion recognition result. Then, by collecting the sound wave recognition method to extract the features of the noise sound segment in the sound signal, the noise feature vector of the noise sound segment that affects the people in the car can be accurately obtained. Based on the extracted noise feature vector, the corresponding mixed-audio sound wave band can be generated from the perspective of human factors. The generated mixed-audio sound wave band can be merged with the noise sound segment that affects the people in the car, making it a sound segment that can be accepted by people, thereby reducing the impact of noise on people's behavior state. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation of the present invention, the following will briefly introduce the drawings required for use in the specific implementation. In all the drawings, each element or part is not necessarily drawn according to the actual scale.

[0038] Figure 1 A flow chart of a mixing noise reduction method provided by an embodiment of the present invention;

[0039] Figure 2 A system block diagram of a mixing noise reduction system provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0041] like Figure 1 The flowchart of the mixing noise reduction method shown in FIG. 1 includes:

[0042] Step 1: Using emotion recognition method to recognize emotion of organism;

[0043] Step 2: judging whether the organism is in a bad mood according to the recognition result;

[0044] Step 3, in response to the organism being in a bad mood, obtaining a sound signal of the environment surrounding the organism;

[0045] Step 4: using a sound wave recognition method to identify noise feature vectors of noise sound segments in the sound signal, and generating corresponding mixed sound wave segments based on the noise feature vectors of each noise sound segment.

[0046] Specifically, the noise inside the car includes not only the noise segments that affect the behavior of the people in the car, but also other noise segments that can be accepted by the people in the car. If these noise segments that can be accepted by the people are also processed, it will not only increase the resource cost, but also reduce the efficiency of processing the noise inside the car.

[0047] Therefore, in order to improve the efficiency of noise processing and save resource costs. In this embodiment, first, the existing emotion recognition method can be used to identify the emotional state of the people in the car. Then, it can be judged whether the people in the car have bad emotions that will affect the behavioral state, such as anxiety, based on the emotion recognition results. If the people in the car do not have anxiety, there is no need to process the current noise in the car. On the contrary, if the people in the car have anxiety, the sound signal in the car is immediately collected through the microphone. Finally, the existing sound wave recognition method can be used to extract the noise feature vector of the noise sound segment that affects the emotions of the people in the car from the collected sound signal, and generate the corresponding mixed sound wave segment based on the extracted noise feature vector, and merge it with the noise sound segment that affects the people in the car to make it a sound segment acceptable to people, thereby reducing the impact of noise on people's behavioral state.

[0048] In this embodiment, optionally, using an emotion recognition method to perform emotion recognition on an object includes: using an image emotion recognition method to recognize the emotion of the organism.

[0049] Specifically, in step 1, the existing image emotion recognition method can be used to simultaneously recognize the emotions of all people in the car to improve the efficiency of noise processing.

[0050] Specifically, firstly, the camera installed in the car is used to collect the image in the car in real time. Then, the existing face detection method is used to detect the face area in the image in the car. After that, the image of the face area can be feature extracted to obtain facial features related to emotions, such as the shape and position of the eyes, mouth and other parts, the texture and color of the face, etc. Finally, according to all the extracted facial features, the existing emotion classification model, such as support vector machine, can be used to classify the facial features of the occupants in the car, so as to accurately identify the emotions of the occupants in the car, so as to accurately collect the sound signal containing the noise sound segment that affects the behavior state of the occupants in the car.

[0051] In this embodiment, optionally, a sound wave recognition method is used to identify a noise feature vector of a noise segment in the sound signal, including:

[0052] Using an adaptive filter to filter the sound signal to obtain a noise segment in the sound signal;

[0053] Feature extraction is performed on each of the noise sound segments mentioned to obtain a noise feature vector of the noise sound segment.

[0054] Specifically, due to the complex in-car environment, the sound signals in the car will change dynamically all the time. To this end, in step 4, the collected sound signal can be filtered by an adaptive filter to extract pure noise segments from the sound signal so as to subsequently extract accurate noise feature vectors. The adaptive filter can dynamically adjust its own parameters according to the changes in the sound signal, thereby enhancing the robustness of the system, solving the unstable factors caused by sound changes in noise processing, and improving the practicality and reliability of the system.

[0055] In this embodiment, optionally, an adaptive filter is used to filter the sound signal, including: performing noise reduction processing on the sound signal. Specifically, in order to enhance the accuracy of signal feature extraction and improve the quality of the sound segment of noise, and to prevent recognition errors caused by errors in the sound segment transmission process. The sound signal can be subjected to noise reduction processing before the adaptive filter filters the sound signal. The sound signal after noise reduction processing is input into the adaptive filter. The adaptive filter has two inputs, one is the original sound signal x(n) containing noise, and the other is the reference noise signal r(n). In some cases, a reference signal related to the noise in the original sound signal can be obtained. The output y(n) of the filter is an estimate of the noise. The adaptive filter continuously adjusts its filter coefficient w(n) so that the output y(n) is as close as possible to the noise part in the original sound signal.

[0056] In this embodiment, optionally, an adaptive filter is used to filter the sound signal, including: dynamically adjusting the parameters of the adaptive filter by using a least mean square method according to changes in the sound signal.

[0057] Specifically, the least mean square method can be used to dynamically adjust the parameters of the adaptive filter. Specifically, the sample of the sound signal currently input to the adaptive filter can be convolved with the current weight vector to obtain the filtered pure noise sound segment. The specific calculation formula is as follows:

[0058] y(n)=w(n) T *x(n);

[0059] Among them, x(n) is the sample vector of the sound signal, and w(n) is the current weight vector of the adaptive filter.

[0060] The output signal of the adaptive filter can be compared with the expected output signal to determine the error between the two. The specific calculation formula is as follows:

[0061] e(n)=d(n)*y(n);

[0062] Afterwards, the weight vector for the next step can be calculated based on the current weight vector, sample vector, error and step factor μ. The specific calculation formula is as follows:

[0063] w(n+1)=w(n)+μe(n)x(n)

[0064] Repeat the above steps until the maximum number of iterations is reached or the error converges.

[0065] In this embodiment, optionally, a sound wave recognition method is used to identify a noise feature vector of a noise segment in the sound signal, including:

[0066] The noise feature vector structure of each noise segment extracted is compared with the pre-built noise rule library conditions, and the number of sound segments that do not meet the noise rule library conditions is counted;

[0067] In response to the number of sound segments exceeding a threshold, it is determined that no noise exists in the sound signal.

[0068] Specifically, the noise rule library includes a variety of noise features of noise segments that can be processed by mixing. By matching the noise feature vectors corresponding to each noise segment with the noise features stored in the noise rule library, the noise segments in the sound signal that can be processed by mixing can be screened out.

[0069] Specifically, the noise feature vectors corresponding to each noise segment are combined into a mixed sound wave matrix S = [F1, F2, F3, ... F n ], where Fn Represents the noise feature vector corresponding to the nth noise segment. The noise feature vector includes a variety of time domain features and frequency domain features, such as the kurtosis, skewness, and Mel-frequency cepstral coefficients of the noise segment. Traverse each element in the mixed sound wave matrix, and count the number of elements that meet the conditions of the noise rule library. If the proportion of elements that meet the conditions of the noise rule library exceeds the set threshold, it is determined that there is no noise in the sound signal, and there is no need to generate a mixed sound wave band for noise reduction processing. Otherwise, a mixed sound wave band is generated for noise reduction processing. For noise sound segments whose noise feature vectors do not meet the conditions of the noise rule library, they are removed from the mixed sound wave matrix and do not participate in the subsequent mixed sound wave band generation process to reduce the amount of data processing.

[0070] In this embodiment, optionally, in step 4, generating a corresponding mixed sound wave band based on the noise feature vector of each noise sound segment includes:

[0071] The noise characteristic vectors of each noise segment are converted into normal distribution random variables respectively;

[0072] Based on the corresponding normally distributed random variables, respectively determine the mixing feature data and weight coefficients corresponding to each noise sound segment;

[0073] The mixed sound signal data corresponding to each noise sound segment is respectively generated in reverse according to the corresponding mixed sound feature data, and the mixed sound wave band is generated in combination with the corresponding weight coefficient.

[0074] Specifically, when generating mixed sound wave segments, first, the noise feature vectors corresponding to each noise segment in the mixed sound wave matrix after removing the non-noise segment can be transformed into ideal normally distributed random variables, so that the total energy of the noise segment satisfies the normal distribution conditions, so that the sound segment can be easily accepted by humans.

[0075] Then, the mixing feature data and weight coefficient corresponding to the mixing signal for neutralizing each noise segment can be determined according to the corresponding normal distribution random variable. The mixing feature data is the feature data corresponding to the normal distribution random variable. The specific calculation formula of the weight coefficient is as follows:

[0076]

[0077] Among them, μ t , σ t are the target mean and target standard deviation respectively, E i is the energy of the normally distributed random variable corresponding to the i-th noise segment.

[0078] Finally, according to the corresponding mixing feature data, the mixed signal data for neutralizing each noise sound segment can be reversely generated, and the mixed sound wave band can be generated by combining all the mixed signal data and the corresponding weight coefficient. The specific calculation formula is as follows:

[0079]

[0080] Among them, w i , S i are the weight coefficient and mixed signal data corresponding to the i-th noise segment, and m is the number of mixed signal data.

[0081] The generated mixed sound wave band can be merged with the noise wave band that affects the people in the car, making it an acceptable sound wave band for people, thereby reducing the impact of noise on people's behavioral state.

[0082] In this embodiment, optionally, a Box-Muller algorithm is used to convert the noise feature vector of the noise segment into a normal distribution random variable that satisfies normal distribution conditions.

[0083] Specifically, the Box-Muller algorithm can be used to adjust the noise feature vector of the noise segment to achieve energy standardization and normal distribution of the noise segment, that is:

[0084]

[0085] Where N represents the number of characteristic types in the normal distribution random variable, F ij represents the specific element in the normal distribution random variable corresponding to the i-th noise segment, j represents the characteristic type or type in the normal distribution random variable, and Z i Represents a processed normally distributed random variable, that is, the audio frequency in the energy band that can be accepted by humans.

[0086] In this embodiment, optionally, generating a corresponding mixed sound wave segment based on the noise feature vector of each noise sound segment includes:

[0087] Re-adopt the emotion recognition method to recognize the emotion of the organism, and judge whether the organism is still in a bad emotion according to the recognition result;

[0088] In response to the organism still being in a bad mood, the sound signal of the environment surrounding the organism continues to be acquired, and a corresponding mixed sound wave band is generated until the organism returns to normal.

[0089] Specifically, since the scene changes are dynamic, it is impossible to guarantee that the noise situation remains unchanged. Dynamically adjusting the noise processing sound segment according to the feedback from the occupants can better solve the troubles and accuracy of the noise inside the car to the occupants. Therefore, after generating the mixed sound wave segment, the system can re-collect the image inside the car to identify the emotions of the occupants, and judge whether the occupants are still in a bad mood based on the recognition results. If so, the sound signal inside the car can be re-collected, and the noise feature vector of the noise sound segment can be extracted from the sound signal to generate a new mixed sound wave segment until the emotions of the occupants return to normal.

[0090] like Figure 2 The system block diagram of the mixing noise reduction system shown in FIG. 1 includes:

[0091] An emotion recognition module, configured to use an emotion recognition method to perform emotion recognition on a biological subject, and determine whether the biological subject is in a bad emotion according to the recognition result;

[0092] In response to the organism being in a bad mood, obtaining a sound signal of the organism's surrounding environment;

[0093] The noise processing module is configured to identify the noise feature vectors of the noise sound segments in the sound signal by using a sound wave recognition method, and generate corresponding mixed sound wave segments based on the noise feature vectors of each noise sound segment.

[0094] Specifically, the noise reduction system includes an emotion recognition module and a noise processing module. Among them, the emotion recognition module can use existing emotion recognition methods to identify the emotional state of the people in the car. The noise processing module can judge whether the people in the car have bad emotions that will affect their behavior based on the emotion recognition results. If the people in the car are anxious, the noise processing module immediately collects the sound signal in the car through the microphone, and uses the existing sound wave recognition method to extract the noise feature vector of the noise segment that affects the emotions of the people in the car from the collected sound signal, and generates the corresponding mixed sound wave segment based on the extracted noise feature vector, and merges it with the noise segment that affects the people in the car, so that it becomes a sound segment that people can accept, thereby reducing the impact of noise on people's behavior.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A mixing noise reduction method, characterized in that: include: Using emotion recognition methods to identify the emotions of organisms, and judging whether the organisms are in a bad mood based on the recognition results; In response to the organism being in a bad mood, obtaining a sound signal of the organism's surrounding environment; A sound wave recognition method is used to recognize noise feature vectors of noise sound segments in the sound signal, and corresponding mixed sound wave segments are generated based on the noise feature vectors of each noise sound segment.

2. The mixing noise reduction method according to claim 1, characterized in that: Using an emotion recognition method to perform emotion recognition on an object includes: using an image emotion recognition method to recognize the emotion of the organism.

3. The mixing noise reduction method according to claim 1, characterized in that: The method of using a sound wave recognition method to recognize a noise feature vector of a noise segment in the sound signal includes: Using an adaptive filter to filter the sound signal to obtain a noise segment in the sound signal; Feature extraction is performed on each of the noise segments mentioned to obtain the noise feature vector of the noise segment.

4. The mixing noise reduction method according to claim 3, characterized in that: Adopting an adaptive filter to filter the sound signal includes: performing noise reduction processing on the sound signal.

5. The mixing noise reduction method according to claim 3, characterized in that: Adopting an adaptive filter to filter the sound signal includes: According to the changes of the sound signal, the least mean square method is used to dynamically adjust the parameters of the adaptive filter.

6. The mixing noise reduction method according to claim 1, characterized in that: The method of using a sound wave recognition method to recognize a noise feature vector of a noise segment in the sound signal includes: The noise feature vector structure of each noise segment extracted is compared with the pre-built noise rule library conditions, and the number of sound segments that do not meet the noise rule library conditions is counted; In response to the number of sound segments exceeding a threshold, it is determined that no noise exists in the sound signal.

7. The mixing noise reduction method according to claim 1, characterized in that: Generate corresponding mixed sound wave bands based on the noise feature vectors of each noise sound segment, including: The noise feature vectors of each noise segment are converted into normal distribution random variables respectively; Based on the corresponding normally distributed random variables, respectively determine the mixing feature data and weight coefficients corresponding to each noise sound segment; The mixed sound signal data corresponding to each noise sound segment is respectively generated in reverse according to the corresponding mixed sound feature data, and the mixed sound wave band is generated in combination with the corresponding weight coefficient.

8. The mixing noise reduction method according to claim 7, characterized in that: The Box-Muller algorithm is used to convert the noise feature vector of the noise segment into a normally distributed random variable.

9. The mixing noise reduction method according to claim 1, characterized in that: Generate corresponding mixed sound wave bands based on the noise feature vectors of each noise sound segment, including: Re-adopt the emotion recognition method to recognize the emotion of the organism, and judge whether the organism is still in a bad emotion according to the recognition result; In response to the organism still being in a bad mood, the sound signal of the environment surrounding the organism continues to be acquired, and a corresponding mixed sound wave band is generated until the organism returns to normal.

10. A mixing noise reduction system, characterized in that: include: An emotion recognition module, configured to use an emotion recognition method to perform emotion recognition on a biological subject, and determine whether the biological subject is in a bad emotion according to the recognition result; In response to the organism being in a bad mood, obtaining a sound signal of the organism's surrounding environment; The noise processing module is configured to identify the noise feature vectors of the noise sound segments in the sound signal by using a sound wave recognition method, and generate corresponding mixed sound wave segments based on the noise feature vectors of each noise sound segment.

Images