A voice recognition method, device, and computer program product

Abstract

This application relates to a sound recognition method, apparatus, and computer program product, comprising: acquiring a sound signal inside a cockpit; segmenting the sound signal to obtain sound segments; extracting a first sub-segment with long-period characteristics and a second sub-segment with short-period characteristics from the sound segments; extracting a first human voice feature from the first sub-segment; extracting a second human voice feature from the second sub-segment; obtaining a stable human voice feature that is stable in both the long and short periods based on the first and second human voice features; and recognizing the sound signal having the stable human voice feature. This application can improve the accuracy of sound recognition.

Description

Technical Field

[0001] This application relates to the field of in-vehicle monitoring technology, specifically to a voice recognition method, device, and computer program product. Background Technology

[0002] During long drives, drivers may experience fatigue, inattention, or emotional fluctuations, all of which can affect driving skills and reaction time, increasing the risk of traffic accidents. For vehicle safety, it is necessary to monitor the condition of the driver and passengers inside the vehicle. Monitoring allows for the timely detection of problems and the implementation of appropriate measures, such as reminding the driver to rest, adjusting the seat, or modifying the driving environment, thereby reducing the occurrence of traffic accidents. Furthermore, in addition to monitoring the driver's condition, it is also necessary to monitor the condition of other passengers in the vehicle to prevent situations where someone other than the driver is locked inside after the driver leaves the vehicle, or where children are forgotten inside.

[0003] Currently, the common method for monitoring people inside a vehicle is to monitor their breathing. This involves collecting voice recordings to identify the state of the people inside the vehicle. This method is generally used for the driver and requires pre-recording the driver's voiceprint characteristics for subsequent monitoring. If the driver changes or there are many people inside the vehicle, the accuracy of the recognition will be severely affected when multiple types of voices are mixed. Summary of the Invention

[0004] The purpose of this application is to propose a sound recognition method, apparatus, and computer program product to improve the accuracy of sound recognition.

[0005] To achieve the above objectives, according to a first aspect of this application, a voice recognition method is provided, the method comprising:

[0006] Acquire sound signals and segment the sound signals to obtain sound segments;

[0007] Extract a first sub-segment with long-period characteristics and a second sub-segment with short-period characteristics from the sound segment;

[0008] Extract the first human voice feature from the first sub-segment;

[0009] Extract the second human voice features from the second sub-segment;

[0010] Based on the first voice feature and the second voice feature, a stable voice feature that exists stably in both long and short periods is obtained;

[0011] Identify sound signals with the aforementioned stable human voice characteristics.

[0012] The aforementioned method extracts sound segments with long-period and short-period characteristics from the acquired sound signals. Based on these segments, stable human voice features that are stable in both long and short periods can be identified. It should be noted that only sound features that are consistently stable over time are considered valid, thus reducing the possibility of false identification and eliminating transient or accidental sound events, focusing on continuous sound patterns. Finally, by monitoring sound signals with stable human voice features, it ensures that the monitoring is based on reliable data, thereby improving the accuracy of identification. This method can adapt to different in-vehicle sound environments, maintaining high recognition accuracy even in situations with multiple people conversing or high background noise, thus enabling more reliable monitoring of the driver and other passengers, providing strong protection for vehicle safety.

[0013] According to a second aspect of this application, an in-cabin voice recognition device is provided, including a module for performing the method according to the first aspect.

[0014] According to a third aspect of this application, an in-cabin voice recognition device is provided, comprising:

[0015] A communication interface used for communicating with other electronic devices;

[0016] Memory is used to store computer program instructions;

[0017] A processor for executing the computer program instructions to support the in-cabin voice recognition device according to the first aspect in implementing the method according to the first aspect.

[0018] According to a fourth aspect of this application, a computer program product is provided, including computer program instructions that instruct a computer device to perform an operation corresponding to the method described in the first aspect. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a sound recognition method in an embodiment of this application. Detailed Implementation

[0021] The detailed description of the accompanying drawings is intended to illustrate the present embodiments of this application and is not intended to represent only the forms in which this application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included within the spirit and scope of this application.

[0022] See Figure 1 One embodiment of this application provides a voice recognition method that can be used in a cockpit, the method comprising the following steps:

[0023] Step S10: Acquire the sound signal and segment the sound signal to obtain sound segments.

[0024] Specifically, installing one or more microphones inside the cockpit is a basic way to acquire sound signals. These microphones can be omnidirectional to capture sound from all directions, or directional to focus on sound in a specific area, depending on the monitoring range and monitoring requirements.

[0025] It should be noted that when segmenting the acquired sound signals, it is important to determine a reasonable time period for segmentation. If the segmentation time period is too large, the amount of sound data processed in a single operation will be too large, affecting the processing efficiency. If the segmentation time period is too small, the processed sound segments will lack regularity, potentially affecting the accuracy of sound recognition. Specifically, the optimal segmentation period can be determined in advance through experiments. For example, an initial segmentation period can be set first, and then different period lengths can be tested to observe the changes in recognition accuracy and processing efficiency, thus finding the optimal segmentation period.

[0026] Step S20: Extract a first sub-segment with long periodicity and a second sub-segment with short periodicity from the sound segment;

[0027] Specifically, step S20 further processes the segmented sound segments to extract sub-segments. A sub-segment is a part of a sound segment. Long-period features refer to the long-duration fluctuation patterns in the sound signal, while short-period features refer to the short-duration fluctuation patterns in the sound signal. In this embodiment, for ease of description, the sub-segment with long-period features is defined as the first sub-segment, and the sub-segment with short-period features is defined as the second sub-segment.

[0028] Step S30: Extract the first human voice feature from the first sub-segment.

[0029] Specifically, the characteristics of human voices over a long period are easily affected by environmental changes. Therefore, when extracting human voice characteristics from the first sub-segment, it is necessary to analyze and identify the ambient sounds and human voices in the first sub-segment. Ambient sounds include, for example, music played in the car or natural sounds in the cabin environment, while human voices include, for example, human breathing or speaking.

[0030] There are several methods to distinguish between ambient sound and human voice. For example, frequency analysis can be performed on the collected sound. Ambient sound typically has a wide frequency range, while human voice is usually concentrated in a narrower frequency range. By analyzing the audio signal frequency, the frequency characteristics of ambient sound and human voice can be identified. Another method is energy analysis. Ambient sound usually has higher energy, while human voice usually has lower energy. By analyzing the audio signal energy, the energy characteristics of ambient sound and human voice can be identified. However, there are exceptions. When the volume of music playing in the car is high, the energy of ambient sound may be higher than that of human voice. In this case, frequency analysis can be used as an auxiliary analysis. By comparing the energy characteristics between different frames, the energy difference between ambient sound and human voice can be determined, thus achieving the distinction between ambient sound and human voice. It should be noted that if the first sub-segment is found not to contain human voice, the first sub-segment is discarded.

[0031] Step S40: Extract the second human voice feature from the second sub-segment.

[0032] Specifically, human voice features within a short period are more susceptible to noise interference. Therefore, when extracting human voice features from the second sub-segment, it is necessary to analyze and identify the noise and human voice in the second sub-segment. The noise refers to vehicle operating noise, such as engine noise.

[0033] Step S50: Obtain stable human voice features that exist stably in both long and short periods based on the first human voice feature and the second human voice feature;

[0034] Specifically, the first voice feature within a long period contains sound information over a longer time period, which can better reflect a person's voice characteristics and speaking habits, while the second voice feature within a short period is more detailed and can capture more sound details and changes.

[0035] Both the first and second voice features contain various types of features, such as frequency, amplitude, shape of the time-domain signal, and distribution of the frequency-domain signal. The first and second voice features are compared to compare their similarities and differences in different types of features. By analyzing the comparison results, features that are stable in both long and short periods are identified. These features have been verified over long and short time scales and can reflect the stable characteristics of human voices. Based on the identified stable features, the stable voice features are obtained, which can maintain consistency and stability at different time scales, which helps to more accurately monitor and identify human voices inside the vehicle.

[0036] Step S60: Identify the sound signal with the stable human voice characteristics.

[0037] Specifically, only when the sound characteristics remain stable over time are they considered valid, thereby reducing the possibility of misidentification, eliminating brief or accidental sound events, and focusing on continuous sound patterns; therefore, by monitoring sound signals with the stable human voice characteristics in the cockpit, it can be ensured that the monitoring is based on reliable data, thereby improving the accuracy of identification.

[0038] The method described in this embodiment can adapt to different in-vehicle sound environments. Even when multiple people are talking or there is a lot of background noise, it can maintain high recognition accuracy, thereby enabling more reliable monitoring of the status of the driver and other passengers and providing strong protection for vehicle safety.

[0039] In some embodiments, step S20 further includes:

[0040] Based on the time interval between two similar peaks or troughs in a sub-segment, the time required to collect a preset number of peaks, or the frequency components in the spectrum, determine whether the sub-segment has long-period or short-period characteristics.

[0041] Specifically, the time interval between two peaks or troughs with similar peak values ​​can be used to determine whether the extracted sub-segment belongs to a long period or a short period.

[0042] Alternatively, the time required to collect a preset number of peaks can be used to determine whether the extracted sub-segment belongs to a long period or a short period.

[0043] Alternatively, spectral analysis can be used to extract the periodicity of sub-segments. By performing a Fourier transform on the sub-segment, its spectrum can be obtained. Long-period characteristics are typically represented by lower frequency components in the spectrum, while short-period characteristics are represented by higher frequency components. Therefore, by analyzing the frequency distribution of the spectrum, it can be determined whether a sub-segment belongs to a long-period or short-period segment.

[0044] In some embodiments, step S40 further includes:

[0045] Step S41: Slice the second sub-segment to obtain multiple slices;

[0046] Step S42: When the similarity between any two slices in the plurality of slices is less than or equal to a first preset value, the second sub-segment is discarded; otherwise, several slices are selected from the plurality of slices for feature extraction to obtain the second human voice feature.

[0047] Specifically, slicing refers to dividing the second sub-segment into smaller segments, i.e., slices. These slices are usually shorter and contain smaller sound events, making feature extraction more refined. In step S42, the multiple slices obtained in step S41 are compared one by one, and the similarity between any two slices is calculated. For example, the similarity between any two slices can be compared in terms of feature parameters such as energy, peaks, troughs, and frequency. Different weights are assigned to the similarity comparison results of energy, peaks, troughs, and frequency, and a weighted sum is performed to obtain the similarity between any two slices. If the similarity between any two slices is less than or equal to a first preset value, it indicates that the second sub-segment may be noise, and this second sub-segment is discarded. Otherwise, it indicates that there is a certain similarity between the multiple slices, and several slices are selected from the multiple slices for feature extraction to obtain the second human voice features.

[0048] In some embodiments, step S40 further includes:

[0049] Step S43: When the similarity between any two slices in the plurality of slices is greater than the first preset value and less than or equal to the second preset value, the second sub-segment is selected for feature extraction to obtain the second human voice feature.

[0050] Specifically, if the similarity between any two slices in the plurality of slices is greater than a first preset value and less than or equal to a second preset value, it indicates that the plurality of slices have a certain degree of similarity, but the similarity is relatively low. In this case, feature extraction needs to be performed on the complete second sub-segment.

[0051] Step S44: When the similarity between any two slices in the plurality of slices is greater than the second preset value and less than or equal to the third preset value, select the first k odd-numbered slices or the first k even-numbered slices in the plurality of slices for feature extraction to obtain the second human voice feature.

[0052] Specifically, if the similarity between any two slices in the plurality of slices is greater than the second preset value and less than or equal to the third preset value, it indicates that the plurality of slices have a certain degree of similarity and the similarity is relatively high. Then, the second sub-segment is compressed, and the first k odd-numbered slices or the first k even-numbered slices in the plurality of slices are selected for feature extraction to obtain the second voice feature. For example, the second sub-segment is cut into 10 slices, and the numbers of the 10 slices are 1 to 10 in sequence, and k is a preset value. Assuming k = 5, the slices with numbers 1, 3, 5, 7, and 9 are selected, or the slices with numbers 2, 4, 6, 8, and 10 are selected.

[0053] Step S45: When the similarity between any two slices in the plurality of slices is greater than the third preset, select the first k slices in the plurality of slices for feature extraction to obtain the second human voice feature.

[0054] Specifically, if the similarity between any two slices in the plurality of slices is greater than the third preset value, it indicates that the similarity between the plurality of slices is relatively high. Then, the second sub-segment is compressed, and the first k slices in the plurality of slices are selected for feature extraction to obtain the second human voice feature. For example, the second sub-segment is cut into 10 slices, and the number of the 10 slices is 1 to 10 in sequence, and k is a preset value. Assuming k = 5, then slices with numbers 1 to 5 are selected.

[0055] In some embodiments, both the first voice feature and the second voice feature include at least two features;

[0056] Step S50 further includes:

[0057] The similarity between various features of the first voice feature and the second voice feature is obtained. If the similarity between any feature of the first voice feature and the second voice feature is greater than a fourth preset value, then that feature is taken as a feature of the stable voice feature; wherein, the stable voice feature includes at least one feature.

[0058] Specifically, in this embodiment, both the first and second voice features are multi-dimensional, containing at least two different attributes. The features of each attribute of the first and second voice features are compared, and their similarity is calculated. If the similarity of any attribute of the first and second voice features exceeds a preset threshold (i.e., a fourth preset value), the corresponding attribute feature is considered stable and can be used as one of the stable voice features. The stable voice feature is a set composed of at least one feature, which is selected from the comparison process and is considered a stable feature shared by the first and second voices.

[0059] In some embodiments, the at least two features include at least two of acoustic energy, zero-crossing rate, average amplitude, spectrum, and spectral envelope.

[0060] Specifically, the sound energy can reflect the energy level of a sound segment, which can be obtained by squaring the sound segment and then accumulating the squared results.

[0061] The zero-crossing rate can reflect the frequency characteristics of a sound segment, and can be obtained by calculating the number of zero-crossing points of the sound waveform within a period.

[0062] The average amplitude is a feature used to measure the magnitude of the amplitude of a sound segment. It can be obtained by performing an absolute value operation on the sound segment, accumulating the results of the absolute values, and finally dividing by the length of the period.

[0063] The spectral features refer to the process of converting a sound segment into a spectrum using Fourier transform, and then extracting features from the spectrum, such as spectral energy and spectral average value. Spectral features can be used for spectral analysis of sound, for example, to determine the spectral characteristics of a sound.

[0064] The so-called acoustic spectrum envelope features refer to the acoustic spectrum envelope features obtained by processing sound segments using linear predictive coding or cepstral analysis methods, such as cepstral coefficients and linear predictive coefficients.

[0065] In some embodiments, prior to performing step S40, the method further includes:

[0066] Step S61: Obtain the acoustic energy of the second sub-segment.

[0067] Step S62: Obtain the acoustic energy of ambient sound.

[0068] Step S63: When the difference between the sound energy of the second sub-segment and the sound energy of the cabin environment exceeds a preset threshold, a compensation segment is extracted from the first sub-segment, and sound compensation is performed on the second sub-segment based on the compensation segment; wherein the compensation segment is time-aligned with the second sub-segment.

[0069] Specifically, cabin ambient sound may include various noises, such as engine noise and wind noise. If the sound energy of the second sub-segment is lower than the energy of the cabin ambient sound, the human voice may be drowned out, making it difficult to hear. Through compensation, the relative energy level of the human voice can be increased, making it stand out more in the cabin ambient sound, thereby improving clarity. The sound compensation of the second sub-segment based on the compensation segment can, for example, be achieved by superimposing the compensation segment onto the second sub-segment to increase its sound energy.

[0070] The above embodiments take in-cabin voice recognition as an example. It is worth noting that the voice recognition method can be applied to voice recognition in other scenarios, and no further limitations are made here.

[0071] Another embodiment of this application provides a voice recognition device, including a module for performing the voice recognition method described in the above embodiments. This device can be a hardware device, a software device, or a combination of hardware and software.

[0072] Specifically, this device includes a series of modules capable of performing various steps in the sound recognition method described above. For example, one module is used to perform the steps of acquiring a sound signal inside the cockpit and segmenting the sound signal to obtain sound segments; another module is used to perform the steps of extracting a first sub-segment with long-period characteristics and a second sub-segment with short-period characteristics from the sound segments; another module is used to perform the steps of identifying whether the first sub-segment is a human voice, and if so, extracting features from the first sub-segment to obtain a first human voice feature; another module is used to perform the steps of identifying whether the second sub-segment is noise, and if not, extracting features from the second sub-segment to obtain a second human voice feature; another module is used to perform the steps of obtaining a stable human voice feature that is stable in both long and short periods based on the first and second human voice features; and a third module is used to perform the steps of recognizing the sound signal inside the cockpit that has the stable human voice feature.

[0073] Another embodiment of this application also provides another in-cabin voice recognition device, including:

[0074] A communication interface used for communicating with other electronic devices;

[0075] Memory is used to store computer program instructions;

[0076] A processor is configured to execute the computer program instructions to support the in-cabin voice recognition device in implementing the methods described in the above embodiments.

[0077] In this embodiment, the memory mainly includes a program storage area and a data storage area. The program storage area can store the operating device, applications required for at least one function, etc., and the data storage area can store related data, etc. Furthermore, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), and a flash card, or other volatile solid-state storage devices.

[0078] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can be any conventional processor. The processor is the control center of the in-cabin voice recognition device and uses various interfaces and lines to connect the various parts of the in-cabin voice recognition device.

[0079] This application also provides a computer program product, including computer program instructions, which instruct a computer device to perform operations corresponding to the methods described in the above embodiments.

[0080] Specifically, the computer program product includes a series of computer program instructions that instruct a computer device to execute the voice recognition method described in this application. These instructions are code written in a computer program that defines how to perform specific operations. In this embodiment, these instructions are used to execute the voice recognition method of the above embodiments.

[0081] These program instructions are designed to be loaded onto a computer device and instruct the device to perform specific operations, which refer to the various steps in the sound recognition method described in the above embodiments. For example, step S10: acquiring a sound signal and segmenting the sound signal to obtain sound segments; step S20: extracting a first sub-segment with long-period characteristics and a second sub-segment with short-period characteristics from the sound segments; step S30: identifying whether the first sub-segment is a human voice, and if so, extracting features from the first sub-segment to obtain a first human voice feature; step S40: identifying whether the second sub-segment is noise, and if not, extracting features from the second sub-segment to obtain a second human voice feature; step S50: obtaining a stable human voice feature that is stable in both long and short periods based on the first and second human voice features; and step S60: recognizing the sound signal with the stable human voice feature.

[0082] In this way, the computer program product provides a complete software solution that can run on various computer devices to implement the voice recognition method described in the above embodiments.

[0083] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A voice recognition method characterized by comprising: The method comprises: acquiring a sound signal, and segmenting the sound signal to obtain a sound segment; extracting a first sub-segment with long-period characteristics and a second sub-segment with short-period characteristics from the sound segment; extracting a first human voice feature from the first sub-segment; extracting a second human voice feature from the second sub-segment; the first human voice feature and the second human voice feature each comprise at least two features; obtaining a stable human voice feature that stably exists in both long and short periods according to the first human voice feature and the second human voice feature; recognizing a sound signal with the stable human voice feature; wherein the step of obtaining the stable human voice feature that stably exists in both long and short periods according to the first human voice feature and the second human voice feature comprises: obtaining the similarity of various features of the first human voice feature and the second human voice feature, and if the similarity between any feature of the first human voice feature and the second human voice feature is greater than a fourth preset value, the feature is taken as a feature of the stable human voice feature; wherein the stable human voice feature comprises at least one feature.

2. The method of claim 1, wherein, The step of extracting the first sub-segment with long-period characteristics and the second sub-segment with short-period characteristics from the sound segment further comprises: determining whether a sub-segment has long-period characteristics or short-period characteristics according to the time interval between two similar wave crests or troughs in the sub-segment, the time required to collect a preset number of wave crests, or the frequency components in the frequency spectrum.

3. The method of claim 1, wherein, The step of extracting the second human voice feature from the second sub-segment further comprises: slicing the second sub-segment to obtain a plurality of slices; when the similarity of any two slices in the plurality of slices is less than or equal to a first preset value, discarding the second sub-segment, otherwise selecting a number of slices from the plurality of slices for feature extraction to obtain the second human voice feature.

4. The method of claim 3, wherein, The step of selecting a number of slices from the plurality of slices for feature extraction to obtain the second human voice feature further comprises: when the similarity of any two slices in the plurality of slices is greater than the first preset value and less than or equal to a second preset value, selecting the second sub-segment for feature extraction to obtain the second human voice feature; when the similarity of any two slices in the plurality of slices is greater than the second preset value and less than or equal to a third preset value, selecting the slices with the first k odd numbers or the slices with the first k even numbers in the plurality of slices for feature extraction to obtain the second human voice feature; when the similarity of any two slices in the plurality of slices is greater than the third preset value, selecting the first k slices in the plurality of slices for feature extraction to obtain the second human voice feature; wherein k is a preset value.

5. The method of claim 1, wherein, The at least two features include at least two of sound energy, zero-crossing rate, average amplitude, frequency spectrum, and sound spectrum envelope.

6. The method of claim 1, wherein, Before the step of extracting the second human voice feature from the second sub-segment, the method further comprises: acquiring the sound energy of the second sub-segment; acquiring the sound energy of the environmental sound; When a difference between the acoustic energy of the second sub-fragment and the acoustic energy of the cabin ambient sound exceeds a preset threshold, a compensation sub-fragment is extracted from the first sub-fragment, and the second sub-fragment is compensated for sound according to the compensation sub-fragment; wherein the compensation sub-fragment is aligned in time with the second sub-fragment.

7. A voice recognition apparatus characterized by comprising: comprising means for performing the method according to any one of claims 1 to 6.

8. A voice recognition apparatus characterized by comprising: comprising: a communication interface for communicating with other electronic devices; a memory for storing computer program instructions; a processor for executing the computer program instructions to support the apparatus to implement the method according to any one of claims 1 to 6.

9. A computer program product, characterised in that, comprising computer program instructions instructing a computer device to perform operations corresponding to the method according to any one of claims 1 to 6.

Images