In-vehicle voice response method, device and equipment and storage medium

By pre-processing and VAD processing of the initial voice data of the user in the car, combined with the results of the first and second VAD processing, the response failure problem caused by pauses in the traditional method is solved, and the accurate response of the voice in the car is achieved.

CN118942456APending Publication Date: 2024-11-12ZHEJIANG ZEEKR INTELLIGENT TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410999166.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional in-car voice response methods may pause when the user enters the voice, causing the system to mistakenly believe that the voice has been entered, resulting in the response failure or the response content does not match the expectations.

Method used

By preprocessing and VAD processing of the initial voice data output by the user in the car, it is determined whether the complete voice command is included. If not, it will be detected within the preset time and a second VAD processing is performed, and the processing results are combined to perform voice response.

Benefits of technology

This avoids response failure or response content that is different from expectations caused by pauses when entering voice, and achieves accurate response to voice in the car.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942456A_ABST
    Figure CN118942456A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of vehicle-mounted voice, and discloses an in-vehicle voice response method, device and equipment and a storage medium, and the method comprises the steps: carrying out the preprocessing of initial voice data outputted by an in-vehicle user, and obtaining the processed voice data; performing first VAD processing on the processed voice data, and judging whether the processed voice data contains a complete voice instruction or not based on a processing result; if not, processing whether the user in the vehicle outputs supplementary voice data or not within a preset time, and performing second VAD processing on the supplementary voice data when the supplementary voice data is processed; and performing voice response based on a processing result of the first VAD processing and a processing result of the second VAD processing. Compared with a traditional in-vehicle voice response method, the in-vehicle voice response method has the advantages that the voice response failure or the difference between the response content and the user expectation caused by pause when the user inputs the voice is avoided, and the in-vehicle voice is accurately responded.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted voice technology, and in particular to an in-vehicle voice response method, device, equipment and storage medium. Background Art

[0002] With the development of in-car voice technology, people have higher and higher requirements for in-car voice response. The traditional in-car voice response method has the following problems: some users may pause when recording voice, which will make the in-car voice response system mistakenly believe that the user's voice has been recorded. This will cause the in-car voice response system to respond without reading the user's complete voice, resulting in vehicle response failure or the response content being different from the user's expectations. Therefore, the industry is in urgent need of a method that can accurately respond to in-car voice.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for in-vehicle voice response, aiming to solve the technical problem that the prior art cannot accurately respond to in-vehicle voice.

[0005] To achieve the above object, the present invention provides an in-vehicle voice response method, the method comprising the following steps:

[0006] Preprocessing the initial voice data output by the user in the car to obtain processed voice data;

[0007] Performing a first VAD processing on the processed voice data, and determining whether the processed voice data contains a complete voice command based on the processing result;

[0008] If not, detecting whether the in-vehicle user outputs supplementary voice data within a preset time, and performing a second VAD process on the supplementary voice data when the supplementary voice data is detected;

[0009] A voice response is performed based on a processing result of the first VAD processing and a processing result of the second VAD processing.

[0010] Optionally, the step of preprocessing the initial voice data output by the in-vehicle user to obtain processed voice data includes:

[0011] Collecting first environmental sound source data through a microphone array, and determining whether the first environmental sound source data includes initial voice data output by a user in the vehicle;

[0012] If so, audio denoising, audio gain adjustment and audio downsampling are performed on the initial voice data to obtain processed voice data.

[0013] Optionally, the step of determining whether the first environmental sound source data includes initial voice data output by a user in the vehicle includes:

[0014] Performing time-frequency analysis on the first environmental sound source data by short-time Fourier transform to obtain a time-frequency domain corresponding to the first environmental sound source data;

[0015] Extracting features of the first environmental sound source data in the time-frequency domain to obtain audio features, wherein the audio features include pitch features, loudness features, and timbre features;

[0016] It is determined whether the first environmental sound source data includes initial voice data output by a user in the vehicle according to the audio feature.

[0017] Optionally, the step of performing a first VAD processing on the processed voice data and judging whether the processed voice data contains a complete voice command based on the processing result includes:

[0018] Performing a frame operation on the processed voice data to obtain a plurality of voice frames;

[0019] Performing a first VAD process on the processed speech data based on the plurality of speech frames, and obtaining semantic understanding data corresponding to the processed speech data based on the processing result;

[0020] It is determined whether the processed voice data contains a complete voice instruction according to the semantic understanding data.

[0021] Optionally, the step of performing a first VAD processing on the processed voice data based on the plurality of voice frames includes:

[0022] Performing energy calculation on the plurality of speech frames respectively to obtain frame energies corresponding to the plurality of speech frames;

[0023] The frame energy is compared with the preset energy threshold in time sequence. If it is detected that the frame energy is less than the preset energy threshold within a preset time period, the first VAD processing of the processed voice data is terminated, and continuous voice data is generated based on the several voice frames.

[0024] Optionally, if not included, the step of detecting whether the in-vehicle user outputs supplementary voice data within a preset time includes:

[0025] If the processed voice data does not contain a complete voice command, collecting second environmental sound source data within a preset time through a microphone array;

[0026] The second environmental sound source data is classified, and based on the classification result, the voice data output by the in-vehicle user in the second environmental sound source data is set as the supplementary voice data.

[0027] Optionally, the step of performing voice response based on the processing result of the first VAD processing and the processing result of the second VAD processing includes:

[0028] Merging the processing result of the first VAD processing and the processing result of the second VAD processing to obtain merged speech data;

[0029] A voice response is made to the in-vehicle user based on the combined voice data.

[0030] In addition, to achieve the above-mentioned purpose, the present invention further proposes an in-vehicle voice response device, the in-vehicle voice response device comprising:

[0031] A data processing module, used for pre-processing the initial voice data output by the user in the car to obtain processed voice data;

[0032] A first detection module, configured to perform a first VAD processing on the processed voice data, and determine whether the processed voice data contains a complete voice command based on the processing result;

[0033] A second detection module is used for detecting whether the in-vehicle user outputs supplementary voice data within a preset time if the in-vehicle user does not include the supplementary voice data, and performing a second VAD process on the supplementary voice data when the supplementary voice data is detected;

[0034] The voice response module is used to perform voice response based on the processing result of the first VAD processing and the processing result of the second VAD processing.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also proposes an in-vehicle voice response device, which includes: a memory, a processor, and an in-vehicle voice response program stored in the memory and executable on the processor, wherein the in-vehicle voice response program is configured to implement the steps of the in-vehicle voice response method described above.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which an in-vehicle voice response program is stored, and when the in-vehicle voice response program is executed by a processor, the steps of the in-vehicle voice response method described above are implemented.

[0037] The present invention pre-processes the initial voice data output by the in-car user to obtain processed voice data; performs a first VAD process on the processed voice data, and determines whether the processed voice data contains a complete voice command based on the processing result; if not, detects whether the in-car user outputs supplementary voice data within a preset time, and performs a second VAD process on the supplementary voice data when the supplementary voice data is detected; and performs a voice response based on the processing result of the first VAD process and the processing result of the second VAD process. Compared with the traditional in-car voice response method, the above method of the present invention pre-processes the initial voice data output by the in-car user and then performs the first VAD process to determine whether the in-car user outputs a complete voice command, and continuously detects the in-car user when the complete voice command is not output, and performs the second VAD process after detecting the supplementary voice data, so that the first VAD process and the second VAD process can be combined for voice response, avoiding the failure of voice response caused by the pause of the user when recording the voice or the difference between the content of the response and the user's expectation, thereby achieving accurate response to the in-car voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the structure of an in-vehicle voice response device in a hardware operating environment involved in an embodiment of the present invention;

[0039] Figure 2 It is a flowchart of a first embodiment of the in-vehicle voice response method of the present invention;

[0040] Figure 3 It is a flowchart of a second embodiment of the in-vehicle voice response method of the present invention;

[0041] Figure 4 It is a flowchart of a third embodiment of the in-vehicle voice response method of the present invention;

[0042] Figure 5 FIG. 4 is a structural block diagram of a first embodiment of an in-vehicle voice response device according to the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0045] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of an in-vehicle voice response device in the hardware operating environment involved in the embodiment of the present invention.

[0046] like Figure 1 As shown, the in-vehicle voice response device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0047] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the in-vehicle voice response device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0048] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an in-vehicle voice response program.

[0049] exist Figure 1 In the in-vehicle voice response device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the in-vehicle voice response device of the present invention can be set in the in-vehicle voice response device, and the in-vehicle voice response device calls the in-vehicle voice response program stored in the memory 1005 through the processor 1001, and executes the in-vehicle voice response method provided by the embodiment of the present invention.

[0050] The embodiment of the present invention provides a method for in-vehicle voice response, referring to Figure 2 , Figure 2 FIG. 4 is a flow chart of a first embodiment of an in-vehicle voice response method according to the present invention.

[0051] In this embodiment, the in-vehicle voice response method includes the following steps:

[0052] Step S10: pre-processing the initial voice data output by the user in the car to obtain processed voice data.

[0053] It should be noted that the execution subject of the method of this embodiment can be a terminal device with VAD processing, data processing and program running functions, such as a smart phone, a computer, etc., or an electronic device with the same or similar functions, such as the above-mentioned in-vehicle voice response device. The following takes the in-vehicle voice response device (hereinafter referred to as the response device) as an example to illustrate this embodiment and the following embodiments.

[0054] It is understandable that the above-mentioned initial voice data may refer to the voice data output by the user in the car and not processed.

[0055] In a specific implementation, the preprocessing of the above-mentioned initial voice data can be achieved through noise reduction (the in-vehicle environment may contain engine noise, road noise, etc., for example, spectral subtraction is used to reduce the impact of environmental noise), speech endpoint detection (identifying the starting and ending points of the initial voice data and removing the non-voice part), speech frame extraction (segmenting the initial voice data into short-time frames, usually using a window function such as a Hanning window to extract the features of each frame), feature extraction (extracting features from each voice frame, such as Mel-frequency cepstral coefficients, filter group features, etc.), speech enhancement (for example, beamforming, spectrum enhancement or deep learning models to improve the quality and clarity of the initial voice data), data enhancement (enhancing the initial voice data by speed conversion, pitch conversion, adding noise, etc.) or other methods, and this embodiment is not limited to this.

[0056] Step S20: performing a first VAD process on the processed voice data, and determining whether the processed voice data contains a complete voice command based on the processing result.

[0057] It should be noted that the above-mentioned VAD (Voice Activity Detection) processing is a technology for identifying active and inactive parts in a speech signal.

[0058] It should be understood that the above-mentioned complete voice command may refer to a voice command that the vehicle can recognize and execute, such as "open the driver's half of the window", "adjust the air conditioning temperature to 25 degrees", etc.

[0059] In a specific implementation, the set complete voice command can be stored in a database in advance, and then searched in the database based on the keywords in the above processing results. If it can be retrieved, it is judged that the processed voice data contains the complete voice command. If it cannot be retrieved, it is judged that the processed voice data does not contain the complete voice command. In addition, the complete voice commands in the database can be updated regularly.

[0060] Step S30: If not included, detecting whether the in-vehicle user outputs supplementary voice data within a preset time, and performing a second VAD process on the supplementary voice data when the supplementary voice data is detected.

[0061] It should be noted that the above-mentioned supplementary voice data may refer to the voice data output by the same in-car user after outputting the above-mentioned initial voice data. Taking an actual scenario as an example, suppose that after a user outputs the initial voice data "Navigate to...", he thinks about the destination to go to within a preset time, and then outputs the voice data "The park closest to me", which is the above-mentioned supplementary voice data. In particular, the above-mentioned preset time can be defined by the user, such as 2 seconds, 5 seconds, 8 seconds, etc., and this embodiment does not limit this.

[0062] Step S40: performing a voice response based on the processing result of the first VAD processing and the processing result of the second VAD processing.

[0063] In a specific implementation, a complete voice instruction can be identified by combining the processing result of the first VAD processing and the processing result of the second VAD processing, so that a voice response can be made based on the complete voice instruction.

[0064] In this embodiment, the initial voice data output by the in-car user is preprocessed to obtain processed voice data; the processed voice data is subjected to a first VAD process, and based on the processing result, it is determined whether the processed voice data contains a complete voice command; if not, whether the in-car user outputs supplementary voice data is detected within a preset time, and the supplementary voice data is subjected to a second VAD process when the supplementary voice data is detected; and a voice response is performed based on the processing result of the first VAD process and the processing result of the second VAD process. Compared with the traditional in-car voice response method, the above method of this embodiment performs preprocessing on the initial voice data output by the in-car user, and then performs the first VAD process to determine whether the in-car user outputs a complete voice command, and continuously detects the in-car user when the complete voice command is not output, and performs the second VAD process after detecting the supplementary voice data, so that the first VAD process and the second VAD process can be combined for voice response, thereby avoiding the failure of voice response caused by the user's pause when recording the voice or the difference between the content of the response and the user's expectation, thereby achieving accurate response to the in-car voice.

[0065] refer to Figure 3 , Figure 3 FIG. 4 is a flow chart of a second embodiment of the in-vehicle voice response method of the present invention.

[0066] Based on the above first embodiment, in this embodiment, in order to obtain clearer voice data of the user in the car, the step S10 may include:

[0067] Step S101: collecting first environmental sound source data through a microphone array, and determining whether the first environmental sound source data includes initial voice data output by a user in the vehicle.

[0068] It should be noted that the microphone array is a collection of multiple microphones, usually arranged in a specific geometric shape or layout. This combination enables the microphone array to accurately capture sound signals in space and provide more information than a single microphone.

[0069] It should be understood that due to the certain complexity of the environment inside and outside the vehicle, the first environmental sound source data collected by the microphone array may include but is not limited to road traffic noise data, pedestrian sound data, vehicle engine sound data, and in-vehicle occupant sound data.

[0070] Step S102: If yes, perform audio denoising, audio gain adjustment and audio downsampling on the initial voice data to obtain processed voice data.

[0071] In a specific implementation, the initial voice data may be subjected to audio denoising processing by means of frequency domain filtering (e.g., low-pass filtering, band-pass filtering, etc.), time domain filtering (e.g., median filtering, mean filtering, etc.), wavelet transform, spectral subtraction, etc., and the audio gain adjustment processing may be performed on the initial voice data by means of amplitude adjustment, equalizer adjustment, sliding window averaging, etc., and the audio downsampling processing may be performed on the initial voice data by means of linear interpolation, polynomial interpolation, etc.

[0072] Further, in this embodiment, in order to accurately determine whether the first ambient sound source data contains the initial voice data output by the user in the car, the step S101 may include:

[0073] Step S1011: performing time-frequency analysis on the first ambient sound source data by short-time Fourier transform to obtain the time-frequency domain corresponding to the first ambient sound source data.

[0074] In a specific implementation, the time-frequency analysis of the above-mentioned first environmental sound source data can be performed based on the following steps to obtain the time-frequency domain corresponding to the first environmental sound source data. The first step is to determine the window length and the moving step size. The window length determines the frequency domain resolution. The longer the window, the higher the frequency domain resolution. The moving step size determines the time resolution. The smaller the step size, the higher the time resolution. The second step is to divide the above-mentioned first environmental sound source data into multiple short-time windows based on the window length and the moving step size. The third step is to perform Fourier transform on multiple short-time windows to obtain the spectrum corresponding to each short-time window. The fourth step is to combine the spectrum corresponding to each short-time window with the time information to obtain the time-frequency domain corresponding to the above-mentioned first environmental sound source data.

[0075] Step S1012: extracting features of the first environmental sound source data in the time-frequency domain to obtain audio features, where the audio features include pitch features, loudness features, and timbre features.

[0076] It should be noted that the above-mentioned pitch feature may refer to the frequency perception of the auditory sense, which determines the high and low pitch of the sound, and is usually expressed in Hertz (Hz). The above-mentioned loudness feature may refer to the relative intensity or volume of the sound, and is usually expressed in phon. The above-mentioned timbre feature may refer to the texture or color of the sound, which makes different sound sources sound different at the same pitch and loudness.

[0077] Step S1013: determining whether the first environmental sound source data includes initial voice data output by the user in the vehicle according to the audio features.

[0078] In a specific implementation, the sound source position of the first environmental sound source data can be determined based on the above audio features. If the sound source position is determined to be inside the car, it indicates that the first environmental sound source data contains the initial voice data output by the user inside the car; if the sound source position is determined to be outside the car, it indicates that the first environmental sound source data does not contain the initial voice data output by the user inside the car.

[0079] Based on the above first embodiment, in this embodiment, in order to more accurately identify whether the processed voice data contains a complete voice command, the step S20 may include:

[0080] Step S201: Framing the processed voice data to obtain a number of voice frames. In a specific implementation, the following steps can be used to perform framing on the processed voice data. The first step is to determine the frame length and frame shift. The frame length refers to the length of each voice frame, which is usually 20-30 milliseconds; the frame shift refers to the time interval between two adjacent voice frames, which is usually 10-20 milliseconds. The second step is to perform framing on the processed voice data according to the determined frame length and frame shift to obtain a number of voice frames.

[0081] Step S202: performing a first VAD processing on the processed speech data based on the plurality of speech frames, and obtaining semantic understanding data corresponding to the processed speech data based on the processing result.

[0082] In a specific implementation, after the processed voice data is subjected to the first VAD processing based on the above-mentioned several voice frames, a continuous voice segment of the user in the car can be obtained. At this time, the continuous voice segment can be semantically understood by the speech recognition (Automatic Speech Recognition, ASR) technology, thereby obtaining the semantic understanding data corresponding to the processed voice data.

[0083] Step S203: determining whether the processed voice data contains a complete voice instruction based on the semantic understanding data.

[0084] In this embodiment, first environmental sound source data is collected through a microphone array, and time-frequency analysis is performed on the first environmental sound source data through short-time Fourier transform to obtain a time-frequency domain corresponding to the first environmental sound source data; feature extraction is performed on the first environmental sound source data in the time-frequency domain to obtain audio features, and the audio features include pitch features, loudness features and timbre features; according to the audio features, it is determined whether the first environmental sound source data contains initial voice data output by the user in the car; if so, audio denoising, audio gain adjustment and audio downsampling are performed on the initial voice data to obtain processed voice data; the processed voice data is framed to obtain a number of voice frames; the processed voice data is subjected to a first VAD processing based on the several voice frames, and semantic understanding data corresponding to the processed voice data is obtained based on the processing results; according to the semantic understanding data, it is determined whether the processed voice data contains a complete voice command. Compared with the traditional in-car voice response method, the above method of this embodiment performs time-frequency analysis on the first environmental sound source data through short-time Fourier transform, so that it can accurately determine whether the first environmental sound source data contains the initial voice data output by the in-car user based on the time-frequency domain obtained after the time-frequency analysis.

[0085] refer to Figure 4 , Figure 4 FIG. 4 is a flow chart of a third embodiment of the in-vehicle voice response method of the present invention.

[0086] Based on the above embodiments, in this embodiment, in order to more accurately determine whether the in-vehicle user stops outputting voice data, the step S202 may include:

[0087] Step S2021: performing energy calculation on the plurality of speech frames respectively to obtain frame energies corresponding to the plurality of speech frames.

[0088] In a specific implementation, the frame energy can be calculated based on the following steps: First, calculate the amplitude spectrum of each frame. Second, square each element of the amplitude spectrum to obtain the power spectrum. Third, sum all elements on the power spectrum to obtain the frame energy corresponding to the speech frame.

[0089] Step S2022: Compare the frame energy with the preset energy threshold in sequence. If it is detected that the frame energy is less than the preset energy threshold within the preset time period, end the first VAD processing of the processed voice data and generate continuous voice data based on the multiple voice frames.

[0090] Based on the above embodiments, in this embodiment, in order to avoid a response error caused by the vehicle making a voice response when the user in the vehicle has not finished speaking, the step S30 may include:

[0091] Step S301: If the processed voice data does not contain a complete voice command, the second environmental sound source data within a preset time is collected through a microphone array.

[0092] It should be noted that the second environmental sound source data may include road traffic noise data, pedestrian sound data, vehicle engine sound data, in-vehicle occupant sound data, etc., which is not limited in this embodiment.

[0093] Step S302: classify the second environmental sound source data, and set the voice data output by the in-vehicle user in the second environmental sound source data as supplementary voice data based on the classification result.

[0094] In a specific implementation, the second environment sound source data can be classified based on the pitch characteristics, loudness characteristics and timbre characteristics of each sound source in the second environment sound source data, and the pitch characteristics, loudness characteristics and timbre characteristics corresponding to each sound source after classification are compared with the initial voice data output by the user in the car, so as to determine whether the sound source belongs to the same sound source as the user in the car. If they belong to the same sound source, it can be indicated that the sound source is also output by the user in the car, that is, the voice data corresponding to the sound source is the above-mentioned supplementary voice data.

[0095] Based on the above embodiments, in this embodiment, in order to more accurately respond to the voice of the user in the car, the step S40 may include:

[0096] Step S401: Merging the processing result of the first VAD processing and the processing result of the second VAD processing to obtain merged speech data.

[0097] For example, assuming that in a scenario of an in-car voice response, the semantic data corresponding to the processing result of the first VAD processing is "navigate to", and the semantic data corresponding to the processing result of the second VAD processing is "South Gate of the Sports Center", then the merged voice data corresponding to the in-car voice response is "Navigate to the South Gate of the Sports Center".

[0098] Step S402: providing a voice response to the in-vehicle user based on the combined voice data.

[0099] In this embodiment, the frame energies corresponding to the several voice frames are obtained by respectively calculating the energies of the several voice frames; the frame energies are compared with a preset energy threshold in sequence; if it is detected that the frame energies are less than the preset energy threshold within a preset time period, the first VAD processing of the processed voice data is terminated, and continuous voice data is generated based on the several voice frames; if the processed voice data does not contain a complete voice command, the second ambient sound source data within a preset time is collected through a microphone array; the second ambient sound source data is classified, and based on the classification result, the voice data output by the in-vehicle user in the second ambient sound source data is set as supplementary voice data; the processing result of the first VAD processing and the processing result of the second VAD processing are merged to obtain merged voice data; and a voice response is made to the in-vehicle user based on the merged voice data. Compared with the traditional in-car voice response method, the above method of this embodiment determines the end time of the first VAD processing by the frame energy corresponding to the voice frame, so as to more accurately judge whether the in-car user stops outputting voice data; at the same time, this embodiment also merges the processing results of the first VAD processing and the processing results of the second VAD processing, so as to more accurately respond to the voice of the in-car user.

[0100] In addition, an embodiment of the present invention further provides a storage medium, on which an in-vehicle voice response program is stored. When the in-vehicle voice response program is executed by a processor, the steps of the in-vehicle voice response method described above are implemented.

[0101] Reference Figure 5 , Figure 5 FIG. 4 is a structural block diagram of a first embodiment of an in-vehicle voice response device according to the present invention.

[0102] like Figure 5 As shown, the in-vehicle voice response device proposed in the embodiment of the present invention includes:

[0103] The data processing module 501 is used to pre-process the initial voice data output by the user in the car to obtain processed voice data;

[0104] A first detection module 502, configured to perform a first VAD process on the processed voice data, and determine whether the processed voice data contains a complete voice command based on the processing result;

[0105] A second detection module 503 is used for detecting whether the in-vehicle user outputs supplementary voice data within a preset time if the supplementary voice data is not included, and performing a second VAD process on the supplementary voice data when the supplementary voice data is detected;

[0106] The speech response module 504 is used to perform speech response based on the processing result of the first VAD processing and the processing result of the second VAD processing.

[0107] In this embodiment, the initial voice data output by the in-car user is preprocessed to obtain processed voice data; the processed voice data is subjected to a first VAD process, and based on the processing result, it is determined whether the processed voice data contains a complete voice command; if not, whether the in-car user outputs supplementary voice data is detected within a preset time, and the supplementary voice data is subjected to a second VAD process when the supplementary voice data is detected; and a voice response is performed based on the processing result of the first VAD process and the processing result of the second VAD process. Compared with the traditional in-car voice response method, the above method of this embodiment performs preprocessing on the initial voice data output by the in-car user, and then performs the first VAD process to determine whether the in-car user outputs a complete voice command, and continuously detects the in-car user when the complete voice command is not output, and performs the second VAD process after detecting the supplementary voice data, so that the first VAD process and the second VAD process can be combined for voice response, thereby avoiding the failure of voice response caused by the user's pause when recording the voice or the difference between the content of the response and the user's expectation, thereby achieving accurate response to the in-car voice.

[0108] Based on the first embodiment of the in-vehicle voice response device of the present invention, a second embodiment of the in-vehicle voice response device of the present invention is proposed.

[0109] In this embodiment, the data processing module 501 is also used to collect the first environmental sound source data through the microphone array, and determine whether the first environmental sound source data contains the initial voice data output by the user in the car; if so, the initial voice data is subjected to audio denoising, audio gain adjustment and audio downsampling to obtain processed voice data.

[0110] Furthermore, the data processing module 501 is also used to perform time-frequency analysis on the first environmental sound source data through short-time Fourier transform to obtain the time-frequency domain corresponding to the first environmental sound source data; perform feature extraction on the first environmental sound source data in the time-frequency domain to obtain audio features, and the audio features include pitch features, loudness features and timbre features; and determine whether the first environmental sound source data contains initial voice data output by the user in the car based on the audio features.

[0111] Furthermore, the first detection module 502 is also used to perform frame operations on the processed voice data to obtain a number of voice frames; perform a first VAD processing on the processed voice data based on the number of voice frames, and obtain semantic understanding data corresponding to the processed voice data based on the processing results; and determine whether the processed voice data contains a complete voice command based on the semantic understanding data.

[0112] Furthermore, the first detection module 502 is also used to perform energy calculations on the several voice frames respectively to obtain frame energies corresponding to the several voice frames; the frame energies are compared with a preset energy threshold in sequence, and if it is detected that the frame energy is less than the preset energy threshold within a preset time period, the first VAD processing of the processed voice data is terminated, and continuous voice data is generated based on the several voice frames.

[0113] Furthermore, the second detection module 503 is also used to collect second environmental sound source data within a preset time through a microphone array if the processed voice data does not contain a complete voice command; classify the second environmental sound source data, and set the voice data output by the in-vehicle user in the second environmental sound source data as supplementary voice data based on the classification result.

[0114] Furthermore, the voice response module 504 is also used to merge the processing result of the first VAD processing and the processing result of the second VAD processing to obtain merged voice data; and to make a voice response to the in-vehicle user based on the merged voice data.

[0115] Other embodiments or specific implementations of the in-vehicle voice response device of the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0116] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0117] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0119] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for in-vehicle voice response, characterized in that: The method comprises the following steps: Preprocessing the initial voice data output by the user in the car to obtain processed voice data; Performing a first VAD processing on the processed voice data, and determining whether the processed voice data contains a complete voice command based on the processing result; If not, detecting whether the in-vehicle user outputs supplementary voice data within a preset time, and performing a second VAD process on the supplementary voice data when the supplementary voice data is detected; A voice response is performed based on a processing result of the first VAD processing and a processing result of the second VAD processing.

2. The in-vehicle voice response method according to claim 1, characterized in that: The step of preprocessing the initial voice data output by the user in the vehicle to obtain processed voice data includes: Collecting first environmental sound source data through a microphone array, and determining whether the first environmental sound source data includes initial voice data output by a user in the vehicle; If so, audio denoising, audio gain adjustment and audio downsampling are performed on the initial voice data to obtain processed voice data.

3. The in-vehicle voice response method according to claim 2, characterized in that: The step of determining whether the first environmental sound source data includes initial voice data output by a user in the vehicle comprises: Performing time-frequency analysis on the first environmental sound source data by short-time Fourier transform to obtain a time-frequency domain corresponding to the first environmental sound source data; Extracting features of the first environmental sound source data in the time-frequency domain to obtain audio features, wherein the audio features include pitch features, loudness features, and timbre features; It is determined whether the first environmental sound source data includes initial voice data output by a user in the vehicle according to the audio feature.

4. The in-vehicle voice response method according to claim 1, characterized in that: The step of performing a first VAD processing on the processed voice data and judging whether the processed voice data contains a complete voice command based on the processing result includes: Performing a frame operation on the processed voice data to obtain a plurality of voice frames; Performing a first VAD process on the processed speech data based on the plurality of speech frames, and obtaining semantic understanding data corresponding to the processed speech data based on the processing result; It is determined whether the processed voice data contains a complete voice instruction according to the semantic understanding data.

5. The in-vehicle voice response method according to claim 4, characterized in that: The step of performing a first VAD process on the processed voice data based on the plurality of voice frames comprises: Performing energy calculation on the plurality of speech frames respectively to obtain frame energies corresponding to the plurality of speech frames; The frame energy is compared with the preset energy threshold in time sequence. If it is detected that the frame energy is less than the preset energy threshold within a preset time period, the first VAD processing of the processed voice data is terminated, and continuous voice data is generated based on the several voice frames.

6. The in-vehicle voice response method according to claim 1, characterized in that: If not, the step of detecting whether the in-vehicle user outputs supplementary voice data within a preset time includes: If the processed voice data does not contain a complete voice command, collecting second environmental sound source data within a preset time through a microphone array; The second environmental sound source data is classified, and based on the classification result, the voice data output by the in-vehicle user in the second environmental sound source data is set as the supplementary voice data.

7. The in-vehicle voice response method according to any one of claims 1 to 6, characterized in that: The step of performing voice response based on the processing result of the first VAD processing and the processing result of the second VAD processing includes: Merging the processing result of the first VAD processing and the processing result of the second VAD processing to obtain merged speech data; A voice response is made to the in-vehicle user based on the combined voice data.

8. An in-vehicle voice response device, characterized in that: The in-vehicle voice response device comprises: A data processing module, used for pre-processing the initial voice data output by the user in the car to obtain processed voice data; A first detection module, configured to perform a first VAD processing on the processed voice data, and determine whether the processed voice data contains a complete voice command based on the processing result; A second detection module is used for detecting whether the in-vehicle user outputs supplementary voice data within a preset time if the in-vehicle user does not include the supplementary voice data, and performing a second VAD process on the supplementary voice data when the supplementary voice data is detected; The voice response module is used to perform voice response based on the processing result of the first VAD processing and the processing result of the second VAD processing.

9. An in-vehicle voice response device, characterized in that: The device comprises: a memory, a processor, and an in-vehicle voice response program stored in the memory and executable on the processor, wherein the in-vehicle voice response program is configured to implement the steps of the in-vehicle voice response method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores an in-vehicle voice response program, and when the in-vehicle voice response program is executed by the processor, the steps of the in-vehicle voice response method according to any one of claims 1 to 7 are implemented.