Vehicle-mounted voice control system and method

Abstract

The invention is suitable for the technical field of vehicle-mounted voice control, and provides a vehicle-mounted voice control system and method, and the method is applied to the vehicle-mounted voice control system. The system comprises a system detection module, a voice wake-up module, a feature extraction module, a voice recognition module, a vehicle-mounted operation module, a wireless communication module, an alarm, a memory, a processing center and a smart phone. The system detection module, the voice wake-up module, the feature extraction module, the voice recognition module, the vehicle-mounted operation module, the wireless communication module, the alarm and the memory are respectively connected with the processing center; the smart phone is in automatic networking connection with the wireless communication module within the range of a wireless network or the Internet; the invention further provides a vehicle-mounted voice control system. Functional abnormity of the system can be automatically detected at any time, any person in the vehicle can carry out vehicle-mounted voice control, control by a specially-assigned person is not needed, and remote control is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle-mounted voice control, and more specifically, relates to a vehicle-mounted voice control system and method. Background Art

[0002] With the continuous advancement of technology and the growth of market demand, the intelligentization and electrification of vehicle voice systems are the future development trends. The existing well-known "A vehicle voice control method and system based on sound source positioning (publication number: CN113689852)" discloses a competitive voice command for the driver and passengers in the car, which determines the position of the voice source in the car through the time difference method, and chooses whether to execute according to the settings. While ensuring driving safety performance, it enriches the human-computer interaction functions and scenarios, realizes the function of designated personnel controlling designated vehicles, and increases the safety of vehicle use. If the designated person drives or travels with the car for some reason, the vehicle voice system cannot be used, so it lacks practicality. Summary of the invention

[0003] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a vehicle-mounted voice control system and method. By setting a system detection module, a voice wake-up module, a feature extraction module, a voice recognition module, and a vehicle-mounted operation module, the system function abnormalities can be automatically detected at any time, and anyone in the car can perform vehicle-mounted voice control without the need for dedicated control. As long as the vehicle can be started, the vehicle-mounted voice control system can be used, thereby increasing the scope of use of the vehicle-mounted system, solving the problem of "the vehicle-mounted system cannot be used if the dedicated person does not travel", and realizing remote control.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0005] A vehicle-mounted voice control method is applied to a vehicle-mounted voice control system, comprising a system detection module, a voice wake-up module, a feature extraction module, a voice recognition module, a vehicle-mounted operation module, a wireless communication module, an alarm, a memory, a processing center, and a smart phone; the system detection module, the voice wake-up module, the feature extraction module, the voice recognition module, the vehicle-mounted operation module, the wireless communication module, the alarm, and the memory are respectively connected to the processing center; the smart phone is automatically networked with the wireless communication module within the range of a wireless network or the Internet;

[0006] The wireless communication module is provided with a wireless network unit, which is responsible for sending and receiving wireless network signals and automatically networking with a smart phone within the range of a wireless network or the Internet;

[0007] The alarm compares the vehicle system test data with the vehicle system test standards pre-stored in the memory, and automatically sounds an alarm and notifies the user to repair if the standards are not met; and compares the obtained confidence with the probability confidence threshold preset in the memory, and automatically sounds an alarm and notifies the user that the wake-up is invalid and needs to be re-awakened if the confidence is lower than the threshold;

[0008] The memory is responsible for storing information of the system detection module, voice wake-up module, feature extraction module, voice recognition module, vehicle operation module, wireless communication module, alarm, and vehicle system test standards and probability confidence thresholds;

[0009] The processing center is responsible for information transmission among the system detection module, voice wake-up module, feature extraction module, voice recognition module, vehicle operation module, wireless communication module, alarm, and memory. It is the hub of the system and compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if the standards are met, the vehicle system can be used; if the standards are not met, the data is transmitted to the alarm and the user is notified for repair; and the obtained confidence is compared with the probability confidence threshold preset in the memory: if the confidence exceeds the threshold, the voice request is valid to notify the voice assistant to wake up; if the confidence is lower than the threshold, the data is transmitted to the alarm and the user is notified that the wake-up is invalid and needs to be re-awakened;

[0010] The system detection module includes a response test unit, a complete test unit, a stability test unit, a compatibility test unit, a safety test unit, and an accuracy test unit. The system obtains the address navigation, vehicle control, multimedia operation, and telephone dialing function test data of the vehicle voice recognition system through the set vehicle automatic diagnostic system (OBD) to ensure the accuracy of the voice recognition system and transmits it to the processing center.

[0011] The voice wake-up module includes a model building unit, a voice collection unit, a voice emphasis unit, a windowing and framing unit, a time domain conversion unit, a filtering and smoothing unit, and a compression and data acquisition unit. Through a specific wake-up word or button, the user activates the voice control system to meet the user's needs for enjoying in-car music, and transmits it to the feature extraction module;

[0012] The feature extraction module includes a compression data acquisition unit, a spectrum conversion unit, a first-order difference unit, and a second-order difference unit, which are responsible for extracting dynamic features of the spectrum, energy, and cepstrum of the first-order difference and second-order difference of the converted time domain signal, so that the cepstrum coefficients change over time and are transmitted to the speech recognition module;

[0013] The speech recognition module includes a probability calculation unit, a confidence evaluation unit, a model matching unit, and a test optimization unit. It recognizes specific instructions or commands through speech wake-up probability calculation, confidence evaluation, and matching with the training model, and transmits them to the vehicle operation module;

[0014] The vehicle operation module includes a semantic understanding unit, a location navigation unit, a music playing unit, and a temperature control unit, and is responsible for semantic understanding of the recognized text, and performing location navigation, music playing, and temperature adjustment to ensure the sound quality of the vehicle system.

[0015] The present invention provides a vehicle-mounted voice control method, comprising the following steps:

[0016] S10, before the vehicle is used, the system detection module obtains the various function test data of the vehicle voice recognition system through the set vehicle automatic diagnosis system (OBD) and transmits it to the processing center;

[0017] S20, the processing center compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if the standards are met, the vehicle system can be used; if the standards are not met, the alarm is transmitted to the alarm and the user is notified for maintenance;

[0018] S30, when the vehicle is running, the voice wake-up module enables the user to activate the voice control system through a specific wake-up word or button to meet the user's demand for enjoying the in-vehicle music, and transmits it to the feature extraction module;

[0019] S40, the feature extraction module extracts dynamic features of the spectrum, energy, and cepstrum of the converted time domain signal by first-order difference and second-order difference, so that the cepstrum coefficients change over time, and transmits them to the speech recognition module;

[0020] S50, the speech recognition module recognizes specific instructions or commands by calculating the speech wake-up probability, conducting confidence evaluation, and matching with the training model, and transmits the commands to the vehicle operation module;

[0021] S60, the vehicle operation module performs semantic understanding of the recognized text, and performs location navigation, music playback, and temperature adjustment to ensure the sound quality of the vehicle system.

[0022] Further, the step S10 includes the following steps:

[0023] S11, the response test unit tests the response time data required for the vehicle system from receiving the voice command to completing the execution to ensure that it does not affect driving safety and user time, and passes it to the complete test unit;

[0024] S12, the complete test unit obtains various functional test data by testing the navigation, music playing, phone dialing, message sending and other functions of the vehicle system, and transmits the data to the stable test unit;

[0025] S13, the stability test unit tests the stability test data of various performances of the vehicle system under different temperature and humidity conditions, and transmits the test data to the compatibility test unit;

[0026] S14. The compatibility test unit obtains compatibility test data by testing whether the vehicle system can connect and interact with mobile phones, smart devices, etc. of different models and brands, and transmits the compatibility test data to the safety test unit;

[0027] S15, the safety test unit tests whether the vehicle system complies with the privacy protection principle when processing user voice data, and passes the test data to the accuracy test unit;

[0028] S16. The accuracy test unit tests the vehicle system's ability to recognize voice commands in different driving environments to ensure that the system accurately recognizes various commands.

[0029] Further, the step S30 includes the following steps:

[0030] S31, the model building unit builds a simulated voice wake-up model for different languages, accents, and functions required by the user in different driving scenarios to ensure voice collection in different scenarios and transmit it to the voice collection unit;

[0031] S32, the voice acquisition unit obtains the continuous analog signal of the input voice wake-up signal at a sampling rate of 16KHz and converts it into a discrete digital signal to ensure the frequency range coverage, and transmits it to the voice emphasis unit;

[0032] S33, the voice emphasis unit calculates the first-order high-pass filter formula "H(z) = 1-α*Z -1 Or y[n]=x[n]-α*x[n-1], H(z) is the signal sequence after the first-order high-pass filter pre-emphasis processing, α is a constant less than 1, y[n] is the signal sequence after the discrete signal is pre-emphasized, and x[n] is the original speech signal sequence. The pre-emphasized signal sequence is obtained for subsequent windowing and framing, and is passed to the windowing and framing unit;

[0033] S34, the windowing and framing unit performs windowing and framing on the digital signal after the spectrum, divides the digitized voice signal into a number of small frames to capture the basic frequency characteristics of the voice signal, and transmits it to the time domain conversion unit;

[0034] S35, the time domain conversion unit performs discrete Fourier transform on each frame of the windowed signal, converts the time domain signal into the frequency domain, so that subsequent filtering and smoothing provide frequency domain information, and transmits it to the filtering and smoothing unit;

[0035] S36, the filtering and smoothing unit performs Mel filtering on the frequency domain signal to convert the linear frequency into the Mel frequency that the human auditory sense is sensitive to, and smoothes the spectrum to reduce the amount of information.

[0036] Further, the step S40 includes the following steps:

[0037] S41, the compression acquisition unit takes the logarithm of the filtered signal using the log compression dynamic range, maps the intensity of the original signal to the logarithmic domain, reduces the signal dynamic range, and transmits it to the spectrum conversion unit;

[0038] S42, the spectrum conversion unit calculates the time domain signal according to the formula " X[n] is the time domain signal, n is the time index, k is the frequency index, N is the total number of samples, The corresponding time domain signal in the frequency domain is obtained for the complex exponential function and passed to the first-order difference unit;

[0039] S43, the first-order difference unit obtains the first-order difference speech feature according to the first-order difference speech feature calculation formula "Y(k)=X(k+1)-X(k), Z(k) is the dynamic relationship between three adjacent frames, Y(k) is the relationship between the current frame and the previous frame, X(k+1) is the current frame, and X(k) is the previous frame", and passes it to the second-order difference unit;

[0040] S44. The second-order difference unit obtains the second-order difference speech feature according to the second-order difference speech feature calculation formula "Z(k)=X(k+2)-2X(k+1)+X(k), Z(k) is the second-order difference speech feature, X(k+2) is the next frame, X(k+1) is the current frame, and X(k) is the previous frame".

[0041] Further, the step S50 includes the following steps:

[0042] S51, the probability calculation unit obtains the vehicle voice wake-up probability according to the vehicle voice wake-up probability calculation formula "W = argmax (P (O | W) P (W)), W is the voice wake-up probability, P (O | W) is the acoustic model, P (W) is the language model, O is the voice signal, W is the specific text or command", and transmits it to the confidence evaluation unit;

[0043] S52, the confidence assessment unit obtains the probability confidence after smoothing according to the calculation formula of the probability confidence after smoothing "C = P / (1-P), C is the confidence, P is the probability after smoothing", and transmits it to the processing center;

[0044] S53, the processing center compares the obtained confidence with the probability confidence threshold preset in the memory: if the confidence exceeds the threshold, the voice request is valid and notifies the voice assistant to wake up; if the confidence is lower than the threshold, it is transmitted to the alarm and notifies that the wake-up is invalid and needs to be re-awakened;

[0045] S54, the model matching unit matches the acquired speech features with the data in the preset acoustic model and language model to screen out the most appropriate command or text, and transmits it to the test optimization unit;

[0046] S55. The test optimization unit performs punctuation processing and sensitive word detection text post-processing based on the text converted from the processed audio, and performs test optimization to improve the accuracy and stability of the system.

[0047] Further, the step S60 includes the following steps:

[0048] S61, the semantic understanding unit performs semantic extraction, context interaction, and source search on the recognized text to understand the user's intention and transmit it to the location navigation unit;

[0049] S62, the location navigation unit executes the operation execution instruction of navigating to a certain location according to the result of the semantic understanding to reach the predetermined destination so that the user can travel, and transmits it to the music playing unit;

[0050] S63, the music playing unit executes the operation execution instruction of automatically playing the predetermined song and its playing effect according to the result of the semantic understanding, so as to improve the comfortable atmosphere in the car, and transmits it to the temperature control unit;

[0051] S64. The temperature control unit executes an operation execution instruction to adjust the air-conditioning temperature to a predetermined temperature value according to the result of the semantic understanding, so as to ensure a comfortable air environment in the vehicle.

[0052] A vehicle-mounted voice control system provided by the present invention also includes a computer device and a computer-readable storage medium; the computer device includes a memory and various functional modules, the memory stores a computer program, and when the various functional modules execute the computer program, the steps of the vehicle-mounted voice control method described in any one of the above are implemented; the computer-readable storage medium stores a computer program, and when the computer program is executed by the various functional modules, the steps of the vehicle-mounted voice control method described in any one of the above are implemented.

[0053] The present invention also provides a vehicle-mounted voice control system, which is implemented by using the above-mentioned vehicle-mounted voice control method.

[0054] The beneficial effects of the present invention compared with the prior art are as follows:

[0055] By setting up the system detection module, voice wake-up module, feature extraction module, voice recognition module, and vehicle operation module, the system's functional abnormalities can be automatically detected at any time, and anyone in the car can perform vehicle voice control without the need for dedicated control. As long as the vehicle can be started, the vehicle voice control system can be used, which increases the scope of use of the vehicle system, solves the problem of "the vehicle system cannot be used if the dedicated person does not travel", and realizes remote control. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or exemplary technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 It is a schematic diagram of the system module of the present invention;

[0058] Figure 2 It is a schematic diagram of the system detection module of the present invention;

[0059] Figure 3 This is a schematic diagram of a voice wake-up module of the present invention;

[0060] Figure 4 It is a schematic diagram of a feature extraction module of the present invention;

[0061] Figure 5 is a schematic diagram of a speech recognition module of the present invention;

[0062] Figure 6 It is a schematic diagram of the vehicle-mounted operation module of the present invention;

[0063] Figure 7 It is a schematic diagram of the method flow of the present invention;

[0064] Figure 8 This is a schematic diagram of the process decomposition of step S10 in the method flow of the present invention;

[0065] Fig. 9 This is a schematic diagram of the process decomposition of step S30 in the method flow of the present invention;

[0066] Fig.10 This is a schematic diagram of the process decomposition of step S40 in the method flow of the present invention;

[0067] Fig.11 This is a schematic diagram of the process decomposition of step S50 in the method flow of the present invention;

[0068] Fig.12 It is a schematic diagram of the program decomposition of step S60 in the method flow of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0070] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:

[0071] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application more clearly understood, the application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0072] It should be noted that when a module is referred to as being "set" or "set on" another module, it can be directly on the other module or indirectly on the other module. When a module is referred to as being "connected to" or "connected to" another module, it can be directly connected to the other module or indirectly connected to the other module.

[0073] In the description of this application, "multiple" means two or more, unless otherwise clearly and specifically defined. "Several" means one or more, unless otherwise clearly and specifically defined.

[0074] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0075] Please refer to Figure 1As shown, the present invention provides a vehicle-mounted voice control method, which is applied to a vehicle-mounted voice control system, including a system detection module, a voice wake-up module, a feature extraction module, a voice recognition module, a vehicle-mounted operation module, a wireless communication module, an alarm, a memory, a processing center, and a smart phone; the system detection module, the voice wake-up module, the feature extraction module, the voice recognition module, the vehicle-mounted operation module, the wireless communication module, the alarm, and the memory are respectively connected to the processing center; the smart phone is automatically networked with the wireless communication module within the range of a wireless network or the Internet.

[0076] The wireless communication module is provided with a wireless network unit, which is responsible for sending and receiving wireless network signals and automatically networking with a smart phone within the range of a wireless network or the Internet.

[0077] The alarm compares the vehicle system test data with the vehicle system test standards pre-stored in the memory. If the standards are not met, it automatically sounds an alarm and notifies the user to repair; and compares the obtained confidence with the probability confidence threshold preset in the memory. If the confidence is lower than the threshold, it automatically sounds an alarm and notifies that the wake-up is invalid and needs to be re-awakened.

[0078] The memory is responsible for storing information of the system detection module, voice wake-up module, feature extraction module, voice recognition module, vehicle operation module, wireless communication module, and alarm, as well as vehicle system test standards and probability confidence thresholds.

[0079] The processing center is responsible for information transmission among the system detection module, voice wake-up module, feature extraction module, voice recognition module, vehicle operation module, wireless communication module, alarm, and memory. It is the hub center of the system and compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if the standards are met, the vehicle system can be used; if the standards are not met, the information is transmitted to the alarm and the user is notified for repair; and the obtained confidence is compared with the probability confidence threshold preset in the memory: if the confidence exceeds the threshold, the voice request is valid to notify the voice assistant to wake up; if the confidence is lower than the threshold, the information is transmitted to the alarm and the user is notified that the wake-up is invalid and needs to be re-awakened.

[0080] See also Figure 2 As shown, the system detection module includes a response test unit, a complete test unit, a stability test unit, a compatibility test unit, a safety test unit, and an accuracy test unit. The address navigation, vehicle control, multimedia operation, and telephone dialing function test data of the vehicle voice recognition system are obtained through the set vehicle automatic diagnostic system (OBD) to ensure the accuracy of the voice recognition system and transmit it to the processing center.

[0081] Furthermore, the response test unit tests the response time data required for the vehicle-mounted system from receiving a voice command to completing execution, and passes it to the complete test unit; the complete test unit obtains various function test data by testing the navigation, music playback, phone dialing, message sending, and other functions of the vehicle-mounted system, and passes it to the stability test unit; the stability test unit tests the stability test data of various performances of the vehicle-mounted system under different temperature and humidity conditions, and passes it to the compatibility test unit; the compatibility test unit obtains compatibility test data by testing whether the vehicle-mounted system can connect and interact with mobile phones, smart devices, etc. of different models and brands, and passes it to the security test unit; the security test unit obtains test data by testing whether the vehicle-mounted system complies with the privacy protection principle when processing user voice data, and passes it to the accuracy test unit; the accuracy test unit tests the vehicle-mounted system's ability to recognize voice commands under different driving environments.

[0082] See also Figure 3 As shown, the voice wake-up module includes a model building unit, a voice collection unit, a voice emphasis unit, a windowing and framing unit, a time domain conversion unit, a filtering and smoothing unit, and a compression and data acquisition unit. Through a specific wake-up word or button, the user activates the voice control system to meet the user's needs for enjoying in-car music, and transmits it to the feature extraction module.

[0083] Furthermore, the model building unit builds a simulated voice wake-up model for different languages, accents, and functions required by the user in different driving scenarios, and transmits it to the voice acquisition unit; the voice acquisition unit obtains the continuous analog signal of the input voice wake-up signal at a sampling rate of 16KHz and converts it into a discrete digital signal, and transmits it to the voice emphasis unit; the voice emphasis unit calculates the first-order high-pass filter according to the formula "H(z)=1-α*Z -1 or y[n]=x[n]-α*x[n-1]" to obtain a pre-emphasized signal sequence, and pass it to a windowing and framing unit; the windowing and framing unit performs windowing and framing on the digital signal after the spectrum, divides the digitized voice signal into a plurality of small frames, and passes it to a time domain conversion unit; the time domain conversion unit performs discrete Fourier transform on each frame signal after windowing, converts the time domain signal into the frequency domain, and passes it to a filtering and smoothing unit; the filtering and smoothing unit performs Mel filtering on the frequency domain signal, converts the linear frequency into the Mel frequency to which human hearing is sensitive, and smoothes the spectrum to reduce the amount of information.

[0084] See also Figure 4As shown, the feature extraction module includes a compression data acquisition unit, a spectrum conversion unit, a first-order difference unit, and a second-order difference unit, which are responsible for extracting dynamic features of the spectrum, energy, and cepstrum of the first-order difference and second-order difference of the converted time domain signal, so that the cepstrum coefficients change over time and are transmitted to the speech recognition module.

[0085] Further, the compression acquisition unit uses log compression dynamic range to take the logarithm of the filtered signal, maps the intensity of the original signal to the logarithmic domain, reduces the signal dynamic range, and transmits it to the spectrum conversion unit; the spectrum conversion unit calculates the signal according to the time domain signal formula The corresponding time domain signal in the frequency domain is obtained and passed to the first-order difference unit; the first-order difference unit obtains the first-order difference speech feature according to the first-order difference speech feature calculation formula "Y(k)=X(k+1)-X(k)", and passes it to the second-order difference unit; the second-order difference unit obtains the second-order difference speech feature according to the second-order difference speech feature calculation formula "Z(k)=X(k+2)-2X(k+1)+X(k)".

[0086] See also Figure 5 As shown, the speech recognition module includes a probability calculation unit, a confidence assessment unit, a model matching unit, and a test optimization unit. It recognizes specific instructions or commands through speech wake-up probability calculation, confidence assessment, and matching with the training model, and transmits them to the vehicle operation module.

[0087] Furthermore, the probability calculation unit obtains the vehicle-mounted voice wake-up probability according to the voice wake-up probability calculation formula "W=argmax(P(O|W)P(W))" and passes it to the confidence evaluation unit; the confidence evaluation unit obtains the smoothed probability confidence according to the smoothed probability confidence calculation formula "C=P / (1-P)" and passes it to the processing center; the model matching unit matches the acquired voice features with the data in the preset acoustic model and language model to screen out the most appropriate command or text, and passes it to the test optimization unit; the test optimization unit performs punctuation processing and sensitive word detection text post-processing based on the text converted from the processed audio, and performs test optimization to improve the accuracy and stability of the system.

[0088] See also Figure 6 As shown, the vehicle operation module includes a semantic understanding unit, a location navigation unit, a music playing unit, and a temperature control unit, which are responsible for semantic understanding of the recognized text, and performing location navigation, music playing, and temperature adjustment to ensure the sound quality of the vehicle system.

[0089] Furthermore, the semantic understanding unit performs semantic extraction, context interaction, and information source search on the recognized text to understand the user's intention, and transmits it to the location navigation unit; the location navigation unit executes an operation execution instruction to navigate to a certain location based on the result of the semantic understanding to reach the predetermined destination for the user to travel, and transmits it to the music playing unit; the music playing unit executes an operation execution instruction to automatically play the predetermined song and its playback effect based on the result of the semantic understanding to enhance the comfortable atmosphere in the car, and transmits it to the temperature control unit; the temperature control unit executes an operation execution instruction to adjust the air-conditioning temperature to a predetermined temperature value based on the result of the semantic understanding to ensure a comfortable air environment in the car.

[0090] How the system works:

[0091] Before using the system in the vehicle, the system detection module obtains the test data of various functions of the vehicle voice recognition system through the vehicle automatic diagnostic system (OBD) set up, and transmits it to the processing center; then the processing center compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if it meets the standards, the vehicle system can be used; if it does not meet the standards, it is transmitted to the alarm and the user is notified for maintenance; when the vehicle is running, the voice wake-up module enables the user to activate the voice control system through a specific wake-up word or button to meet the user's needs to enjoy the car music, and transmits it to the feature extraction module; the feature extraction module extracts the dynamic features of the spectrum, energy, and cepstrum of the converted time domain signal through the first-order difference and second-order difference, so that the cepstrum coefficients change over time, and transmits it to the voice recognition module; then the voice recognition module recognizes specific instructions or commands through voice wake-up probability calculation, confidence evaluation, and matching with the training model, and transmits it to the vehicle operation module; the vehicle operation module performs semantic understanding of the recognized text, and performs location navigation, music playback, and temperature adjustment to ensure the sound quality of the vehicle system.

[0092] When the car owner or driver is far away or out of town, he or she can use a smart phone to automatically connect to the wireless network or the Internet through the wireless communication module, and can remotely control or monitor the vehicle operation in real time through the mobile phone, thus realizing intelligent control, improving the efficiency of vehicle management, and saving vehicle management time; if the language used by the user is not within the above-mentioned training model, a new language recognition model can be re-recorded before traveling to establish it, thereby increasing the scope of use.

[0093] See also Figure 7 As shown, the present invention provides a vehicle-mounted voice control method, comprising the following steps:

[0094] S10. Before the vehicle is used, the system detection module obtains the test data of the address navigation, vehicle control, multimedia operation, and telephone dialing functions of the vehicle voice recognition system through the vehicle automatic diagnosis system (OBD) to ensure the accuracy of the voice recognition system, and transmits it to the processing center;

[0095] See also Figure 8 As shown, the step S10 includes the following steps:

[0096] S11, the response test unit tests the response time data required for the vehicle system from receiving the voice command to completing the execution to ensure that it does not affect driving safety and user time, and passes it to the complete test unit;

[0097] Further explanation: determine the wake-up words and common commands that the in-vehicle voice recognition system needs to recognize and apply in different driving scenarios and noise environments to ensure the performance of the system under various conditions; select the in-vehicle test environment such as the vehicle model, interior materials, seat position, and different external test environments such as road noise, engine noise, wind noise, etc. when simulating stationary and driving states; collect the time interval from the user's voice command to the system response; and calculate and analyze the system's response time based on the collected response information to ensure its stability and real-time performance in different scenarios; evaluate the response speed of the in-vehicle voice system based on the test results, and optimize and improve if deficiencies are found to improve the user experience, to ensure that it can provide high-quality voice interaction experience in various complex environments.

[0098] S12. The complete test unit obtains various functional test data by testing the navigation, music playback, phone dialing, message sending, and other functions of the vehicle system to ensure that the vehicle system supports all preset voice commands in different scenarios and transmits them to the stable test unit;

[0099] It is further explained that the functional integrity test of the vehicle-mounted system includes verifying whether the vehicle-mounted system supports all preset voice commands and functions, and testing the performance of various functions of the system, including: navigation function test to verify whether the system can correctly execute navigation commands such as "I want to go..." and "How to get there"; telephone function test to test the implementation of functions such as making and answering calls, and sending text messages; multimedia function test to play music, videos, adjust volume, brightness and other operations; system control function test to adjust the temperature in the car, control windows, seat heating, etc.; knowledge quiz function test such as checking the weather and answering questions; combined function test such as playing music, answering and making calls during navigation, so as to ensure that the vehicle-mounted system can work normally in different usage scenarios and provide a convenient driving experience.

[0100] S13, the stability test unit tests the stability test data of various performances of the vehicle system under different temperature and humidity conditions to ensure that the vehicle system can operate stably in various environments, provide high-quality calls and services, and pass it to the compatibility test unit;

[0101] It is further explained that the various performance stabilities include echo cancellation, stability testing, network factors, data redundancy, security protection, distributed construction, as well as environmental noise, user accents, massive POIs, and scenario challenges. The stability of the system is tested under different temperature and humidity conditions to ensure that the system can work normally without performance degradation or failure under long-term use or continuous operation; wherein, the echo cancellation ensures that there is no echo during the call to avoid affecting the call quality. Through long-term running tests, the stability and reliability of the system in different environments are evaluated to ensure that the system can provide services continuously and stably; the network factors include network delay, jitter and packet loss, which affect the stability of the voice system. The use of dedicated lines or VPN connections can effectively improve the stability of the system; the data redundancy can prevent data loss and ensure the stability of the system; the security protection includes intrusion detection, active defense mechanisms and SSL encrypted transmission to protect call and data security; the distributed construction divides the system functions into modules, which are deployed and maintained separately to ensure that the failure of a module will not cause the entire system to crash.

[0102] S14. The compatibility test unit obtains compatibility test data by testing whether the vehicle system can connect and interact with mobile phones and smart devices of different models and brands, so as to verify the compatibility of system functions on different operating system versions to provide high-quality communication services, and passes it to the safety test unit;

[0103] It is further explained that the compatibility test includes: hardware compatibility test of the vehicle system to be able to run normally on different models of vehicle computers, sensors, display screens and other hardware platforms, including Bluetooth connection pairing test, i.e. search, password verification and data synchronization to verify the pairing process of mobile phone and vehicle Bluetooth; operating system compatibility test of the vehicle system on different QNX, Android Automotive, Linux and other operating systems, including call quality test, i.e. checking the sound quality, volume and channel stability during the call, including multi-party call and reconnection stress test, etc.; software compatibility test of the vehicle system to communicate with other electronic control units (ECU) and external devices through CAN, LIN, Ethernet and other communication protocols to ensure that the software can correctly handle these communication protocols, including interactive function test, i.e. verifying the answering and hanging up of calls in music playback, navigation and other scenarios. The smoothness of operation and the stability of receiving and sending voice information when using Bluetooth; functional compatibility testing of all functions of the vehicle system to ensure that they can work properly in different vehicle models and driving environments in different environments, including: music playback synchronization test, which is to test the synchronization effect of music playback, including volume and progress control, and verify the stability of Bluetooth reconnection during playback; voice clarity test, which is to evaluate the clarity and intelligibility of voice signals during calls to ensure that the driver and the caller can clearly hear each other's voices; voice delay test, which is to measure the transmission delay of voice signals during calls to ensure the real-time and smoothness of calls; noise suppression test, which is to test the performance of the system in different noise environments, evaluate the system's ability to suppress noise, and ensure that the call quality is not affected by noise; echo cancellation test, which is to test the system's echo cancellation effect to avoid echoes during calls and affect call quality.

[0104] S15. The security test unit tests whether the vehicle system follows the privacy protection principle when processing user voice data to ensure that the data is not illegally obtained or abused, whether it has necessary security measures such as data encryption and firewalls to prevent malicious attacks, and ensure that it can provide stable and secure services in various usage scenarios, and pass it to the accuracy test unit;

[0105] Further explanation: In order to conduct accurate and reliable security and privacy protection tests on the in-vehicle system, security and privacy tests are carried out in actual driving scenarios where the car is quiet but various noise environments such as wind noise and tire noise are simulated outside the car, including: Security test: Interference test of the stability and reliability of the test system to ensure that the in-vehicle system will not interfere with driving during use, to ensure that it can work normally in various driving environments; Stability test: By testing the operation of the in-vehicle system for a long time, the stability and reliability of the system in different environments are evaluated to ensure that the system can provide services continuously and stably; Data collection test: By testing whether the system has obtained the user's explicit consent when collecting user voice data, and whether the data collection process is transparent and legal; Data storage and transmission test: By testing the encryption of data during storage and transmission, unauthorized access and leakage are prevented; Compliance test: According to, test whether the system complies with relevant privacy regulations and standards such as GDPR or CCPA.

[0106] S16. The accuracy test unit tests the vehicle system's ability to recognize voice commands in different driving environments to test the effects of different accents, speaking speeds and volumes on voice recognition, to ensure that the system accurately recognizes various commands.

[0107] It is further explained that the existing voice acquisition equipment of the system is used to collect simulated voice data from simulated users of different genders, ages, and accents, as well as in various simulated driving scenarios and simulated noise environments, and preprocessing such as digitization, denoising, and extraction of voice features is performed; the preprocessed voice data is then annotated with the locations of keywords and commands, and the data is organized to ensure consistent format for subsequent processing and storage; corresponding test data is obtained through microphone input tests such as Mandarin, speaking speed, dialect, volume, and timbre, as well as functional processing tests such as navigation, telephone, and multimedia control.

[0108] S20, the processing center compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if the standards are met, the vehicle system can be used; if the standards are not met, the alarm is transmitted to the alarm and the user is notified for maintenance;

[0109] It is further explained that the vehicle-mounted system test standards include ITU-T standards to ensure that vehicle-mounted voice communication equipment can provide high-quality voice interaction experience in various environments, including: call quality performance such as delay, loudness, frequency response, TMOS, idle channel noise, out-of-band signal, echo cancellation, two-way call, signal-to-noise ratio, noise reduction capability, etc. during calls meet the standards; microphone performance such as microphone sensitivity, frequency response, overload point, directional characteristics, distortion, and large sound pressure level meet the standards; transmission performance under different audio signals, including audio signal distortion, noise suppression, dynamic range and other audio transmission performance meet the standards; in simulated emergency situations, the device's response speed, call quality and reliability during emergency calls meet the standards for emergency call communication; in simulated or actual vehicle environments, the device's adaptability to in-vehicle environments such as road noise, wind noise, and vehicle vibration during vehicle driving meets the standards for voice communication performance; the interoperability and compatibility between the device and other communication systems or devices meet the standards to ensure that the device can seamlessly connect and communicate with other systems or devices, and ensure that the system can provide high-quality voice interaction experience in various complex environments.

[0110] S30, when the vehicle is running, the voice wake-up module enables the user to activate the voice control system through a specific wake-up word or button to meet the user's demand for enjoying the in-vehicle music, and transmits it to the feature extraction module;

[0111] See also Fig. 9 As shown, the step S30 includes the following steps:

[0112] S31, the model building unit builds a simulated voice wake-up model for different languages, accents, and functions required by the user in different driving scenarios to ensure voice collection in different scenarios and transmit it to the voice collection unit;

[0113] To further illustrate, the training in step S31 is generally carried out before voice collection, and can also be carried out after the vehicle is purchased or installed, or can be carried out at the same time if there is a mismatch during collection; voice samples of men, women, old and young in Chinese, English, Zhuang, Cantonese or various local dialects are used by users who often drive or ride, so as to cover a wide range of language variants and ensure that the system can cope with a variety of voice features, because the voices of the elderly and the young are inconsistent, and the voices of men and women are inconsistent; and voice wake-up simulation training is carried out in different driving scenarios such as different vehicle speeds, different weather conditions, whether music is played, etc., to evaluate the performance of the system under various situations, so as to facilitate subsequent data preprocessing; voice wake-up training in different languages ​​for simulated scenarios is carried out at various dealerships of vehicle manufacturers for the main local languages ​​and their different users, and the car owner can also set up voice wake-up training in different languages ​​for simulated scenarios after purchasing the car, so as to build a voice / text database for various scenarios, so as to facilitate subsequent voice comparison and matching recognition.

[0114] S32, the voice acquisition unit obtains the continuous analog signal of the input voice wake-up signal at a sampling rate of 16KHz and converts it into a discrete digital signal to ensure the frequency range coverage, and transmits it to the voice emphasis unit;

[0115] Further explanation: the user first determines the wake-up words such as "Hey, Mercedes-Benz" or keywords or phrases of common commands such as navigation, frequency modulation, and temperature adjustment that the in-vehicle voice recognition system needs to recognize, so as to determine the collection target; according to the in-vehicle environment such as the model, interior material, and seat position of the vehicle in which the user is located, and in combination with the external environment such as whether the road is noisy, whether the engine noise is loud, or whether the wind noise is loud, the user can select a road position with very low noise, or reduce the engine throttle or turn off the engine to reduce noise, or select no wind or close the window to avoid noise, etc. to select a suitable voice collection environment, so as to ensure the accuracy of the collected data; confirm that the voice collection equipment such as the microphone is in good condition to capture clear sound, which is helpful for noise suppression in the later stage;

[0116] S33, the voice emphasis unit calculates the first-order high-pass filter formula "H(z) = 1-α*Z -1 Or y[n]=x[n]-α*x[n-1], H(z) is the signal sequence after the first-order high-pass filter pre-emphasis processing, α is a constant less than 1, y[n] is the signal sequence after the discrete signal is pre-emphasized, and x[n] is the original speech signal sequence. The pre-emphasized signal sequence is obtained for subsequent windowing and framing, and is passed to the windowing and framing unit;

[0117] Further explanation: the constant α less than 1 is generally between 0.9 and 0.97, and is calculated by the pre-emphasis processing formula "H(z) = 1-α*Z -1 Or y[n]=x[n]-α*x[n-1]” to perform pre-emphasis processing on the digital signal to increase the high-frequency part of the voice signal, make the spectrum smooth, and improve the quality of the voice signal, so as to compensate for the energy attenuation of the voice signal in the high-frequency part, enhance the high-frequency component, and thus make the entire spectrum more balanced, so as to improve the quality of the voice signal, especially when the high-frequency information is easily lost during the transmission process; due to the position of the microphone and the acoustic characteristics inside the vehicle, the high-frequency part of the voice signal may be attenuated in different vehicle-mounted environments, which can improve the quality of the voice signal, so that the voice recognition system can more accurately recognize the driver's instructions, and improve the user experience and safety.

[0118] S34, the windowing and framing unit performs windowing and framing on the digital signal after the spectrum, divides the digitized voice signal into a number of small frames to capture the basic frequency characteristics of the voice signal, and transmits it to the time domain conversion unit;

[0119] It is further explained that the framing is to divide the continuous speech signal into frames of a small time window, each frame is 20ms long, and there is a frame shift of 10ms between two adjacent frames. The short-term stability of the audio signal is used to adapt to the time-varying characteristics of the human speech signal and facilitate subsequent analysis and feature extraction to capture the basic frequency characteristics of the speech signal. There will be partial overlap (frame shift) between adjacent frames, and the overlapping ratio is usually half or one-third of the frame length to ensure continuity and reduce the boundary effect caused by framing; a smooth transition can be introduced at the edge of the frame by adding a window, and directly analyzing the divided frames will cause mutations at the beginning and end of the frame, introduce high-frequency components, and affect the accuracy of subsequent analysis such as spectrum calculation. Each frame of data is multiplied by a window function so that the frame data gradually decays to zero at both ends, so that the frame has better local stability and reduces the degree of mutation between different frames.

[0120] S35, the time domain conversion unit performs discrete Fourier transform on each frame of the windowed signal, converts the time domain signal into the frequency domain, so that subsequent filtering and smoothing provide frequency domain information, and transmits it to the filtering and smoothing unit;

[0121] It is further explained that the discrete Fourier transform (DFT) transforms the sampling of each frame of the windowed time domain signal into a signal in the discrete time Fourier transform (DTFT) frequency domain, which contains a complex sequence of amplitude and phase information; squares the DFT result to obtain the energy of each frequency point, converts the amplitude information into energy information, converts the audio signal from the time domain to the frequency domain, eliminates the influence of harmonics, and highlights the resonance peak of the original speech; the amplitude of the DC signal requires special processing, because the DC component in the DFT result needs to be multiplied by 2 / N and then divided by 2 to obtain the true DC signal amplitude, where the DFT is composed of two conjugate complex sequences, and the effective frequency range is 0 to half of the sampling rate.

[0122] S36, the filtering and smoothing unit performs Mel filtering on the frequency domain signal to convert the linear frequency into the Mel frequency that the human auditory sense is sensitive to, and smoothes the spectrum to reduce the amount of information.

[0123] It is further explained that the spectrum obtained by FFT is converted from linear frequency to Mel frequency through a Mel filter group designed according to the auditory characteristics of the human ear, so as to better simulate the nonlinear perception of the human ear to the sound frequency, and the spectrum is divided into multiple frequency bands, each frequency band corresponds to a filter, so as to achieve smoothing of the spectrum; taking the logarithm and performing discrete cosine transform (DCT) and other cepstrum analysis on the Mel spectrum, and finally obtaining the Mel frequency cepstrum coefficients (MFCC) as the characteristics of speech for subsequent speech recognition processing, so as to effectively convert the linear frequency signal into the Mel frequency signal, and smooth the spectrum, so as to improve the accuracy and robustness of speech recognition.

[0124] S40, the feature extraction module extracts dynamic features of the spectrum, energy, and cepstrum of the converted time domain signal by first-order difference and second-order difference, so that the cepstrum coefficients change over time, and transmits them to the speech recognition module;

[0125] See also Fig.10 As shown, the step S40 includes the following steps:

[0126] S41, the compression acquisition unit takes the logarithm of the filtered signal using the log compression dynamic range, maps the intensity of the original signal to the logarithmic domain, reduces the signal dynamic range, and transmits it to the spectrum conversion unit;

[0127] It is further explained that by performing log compression dynamic range processing on the filtered signal and dynamically adjusting the signal processing method according to actual conditions, the strong signal will be compressed after taking the logarithm, while the weak signal will be enhanced, so that the dynamic range of the entire speech signal is optimized. The dynamic range of the signal can be compressed to a reasonable range, reducing the dynamic range between coefficients, so that the weak signal can be accurately identified and processed, and the strong signal in the signal will not completely cover up the weak signal, so as to improve the signal-to-noise ratio of the signal and enhance the clarity of the speech, while maintaining the signal strength information. Make the signal more stable to adapt to different speech intensities and background noise levels.

[0128] S42, the spectrum conversion unit calculates the time domain signal according to the formula " X[n] is the time domain signal, n is the time index, k is the frequency index, N is the total number of samples, The corresponding time domain signal in the frequency domain is obtained for the complex exponential function and passed to the first-order difference unit;

[0129] Further explanation: for each frequency component in the frequency domain, Calculate the corresponding complex exponential function; and calculate the formula based on the time domain signal The complex exponential function of each frequency component is added to obtain a signal in the time domain, that is, an inverse discrete Fourier transform (IDFT) is performed on each complex number X[k] in the frequency domain to obtain coefficients to smooth the logarithmic power spectrum so as to convert the speech signal in the frequency domain back to the time domain; the time index n, frequency index k, and total number of samples N are obtained respectively through the above steps.

[0130] S43, the first-order difference unit obtains the first-order difference speech feature according to the first-order difference speech feature calculation formula "Y(k)=X(k+1)-X(k), Z(k) is the dynamic relationship between three adjacent frames, Y(k) is the relationship between the current frame and the previous frame, X(k+1) is the current frame, and X(k) is the previous frame", and passes it to the second-order difference unit;

[0131] It is further explained that, through the first-order difference speech feature calculation formula "Y(k)=X(k+1)-X(k)", the difference between two consecutive adjacent terms in the discrete function is used to extract the dynamic changes between frames in the speech signal, and the first-order difference Y(k) of the speech feature sequence X(k) is the first-order difference feature between the kth frame and the k+1th frame, which reflects the relationship between the current frame X(k+1) and the previous frame X(k), which helps to capture the dynamic changes in the speech signal; the X(k) is the speech feature of the kth frame, and the X(k+1) is the speech feature of the k+1th frame.

[0132] S44. The second-order difference unit obtains the second-order difference speech feature according to the second-order difference speech feature calculation formula "Z(k)=X(k+2)-2X(k+1)+X(k), Z(k) is the second-order difference speech feature, X(k+2) is the next frame, X(k+1) is the current frame, and X(k) is the previous frame".

[0133] It is further explained that the second-order differential speech feature calculation formula "Z(k)=X(k+2)-2X(k+1)+X(k)"Y(k)=X(k+1)-X(k) is derived by importing "Z(k)=Y(k+1)-Y(k)" through "Y(k)=X(k+1)-X(k)", and is further differentiated based on the first-order differential sequence to capture more subtle dynamic changes; the latter frame X(k+2), the current frame X(k+1), and the previous frame X(k) are respectively obtained through the above step S43; the Z(k) is the dynamic relationship between the three adjacent frames, that is, the relationship between the previous first-order difference and the latter first-order difference. By further second-order differentiation of the first-order differential result, more subtle dynamic features in the speech signal are extracted, which respectively reflect the rate change of the feature and the rate of change of the rate, which helps to improve the accuracy and robustness of speech recognition.

[0134] S50, the speech recognition module recognizes specific instructions or commands by calculating the speech wake-up probability, conducting confidence evaluation, and matching with the training model, and transmits the commands to the vehicle operation module;

[0135] See also Fig.11 As shown, the step S50 includes the following steps:

[0136] S51, the probability calculation unit obtains the vehicle voice wake-up probability according to the vehicle voice wake-up probability calculation formula "W = argmax (P (O | W) P (W)), W is the voice wake-up probability, P (O | W) is the acoustic model, P (W) is the language model, O is the voice signal, W is the specific text or command", and transmits it to the confidence evaluation unit;

[0137] It is further explained that the in-vehicle voice wake-up probability calculation formula "W = argmax(P(O|W)P(W))" is used to calculate the probability that each frame of speech belongs to each category label using the extracted acoustic features, and each frame of speech is labeled with each category using the extracted acoustic features to ensure the accuracy of wake-up word recognition; the category labels include wake-up words or non-wake-up words such as "Hey, Siri", "OK, AutoNavi", etc. issued by the user, and the wake-up words obtained by the user by automatically pressing the voice assistant button on the center console through the system; the voice signal O, specific text or command W are respectively obtained by extracting acoustic features.

[0138] S52, the confidence evaluation unit obtains the smoothed probability confidence according to the calculation formula of the smoothed probability confidence "C = P / (1-P), C is the confidence, P is the smoothed probability", so as to evaluate the confidence of the entire voice request, determine whether the system responds to the voice request, and transmit it to the processing center;

[0139] It is further explained that the original probability value is adjusted for the smoothed probability by considering historical data or other relevant factors, and the frame-level probability of the fixed window length after the probability calculation is smoothed. When processing small sample data or considering the impact of historical data on the current probability, the stability of the probability can be better estimated by introducing a smoothing factor, which specifically includes: using the moving average method to assume a frame-level probability sequence, selecting a fixed window length, sliding the window on the sequence, and calculating the average value of the probability in each window to obtain the smoothed probability sequence. The window size and sliding step size can be adjusted as needed. By adjusting the smoothing parameters such as the window size and sliding step size, the larger the window, the more obvious the smoothing effect, but some detailed information may be lost. The smaller the step size, the more delicate the smoothing effect, but the greater the amount of calculation. Adjust these parameters according to specific needs to achieve the best effect.

[0140] S53, the processing center compares the obtained confidence with the probability confidence threshold preset in the memory: if the confidence exceeds the threshold, the voice request is valid and notifies the voice assistant to wake up; if the confidence is lower than the threshold, it is transmitted to the alarm and notifies that the wake-up is invalid and needs to be re-awakened;

[0141] To further explain, if the voice assistant is activated, the user can be reminded to speak out the required needs such as playing music, making calls, using navigation, etc.; if it is not activated, the settings are adjusted to ensure accurate recognition, including ensuring whether the in-car environment is suitable, whether the network connection is normal, and whether the microphone is working properly. Some car models may require pre-setting voice commands or speaking a preset password to activate the voice assistant; the probability confidence threshold is adjusted by adjusting the original probability value by considering historical data or other relevant factors to adjust the threshold size.

[0142] S54, the model matching unit matches the acquired speech features with the data in the preset acoustic model and language model to screen out the most appropriate command or text, and transmits it to the test optimization unit;

[0143] It is further explained that the preset model is obtained through step S31, including a voice database and a text database, and voice features are extracted through the voice database to form an acoustic model training, and language model training is performed through the text database to form an acoustic model and a language model respectively, and the audio features are mapped to phonemes or pronunciation units, thereby improving the recognition accuracy, and according to steps S32 to S53, the collected user voice and its extracted features are matched respectively, and the network is combined with the dictionary for voice decoding and search algorithm to obtain the most matching text or command.

[0144] S55. The test optimization unit performs punctuation processing and sensitive word detection text post-processing based on the text converted from the processed audio, and performs test optimization to improve the accuracy and stability of the system.

[0145] To further explain, the above-mentioned acoustic model and language model are decompressed by the PaddleNLP natural language processing development library, confirmed by the PPASR (PaddlePaddle Automatic Speech Recognition) speech recognition tool, and punctuation marks are automatically added to the text by the PunctuationPredictor punctuation predictor. The text is punctuated using the model, and the system is tested to evaluate its performance in different driving scenarios, including different speeds, weather conditions, and music playback, so as to improve the practicality and user experience of the speech recognition system; then professional sensitive word detection software is used to detect sensitive words in the converted text, and the detected sensitive words are optimized by replacement, deletion or blurring to ensure that the content complies with relevant regulations and requirements.

[0146] S60, the vehicle operation module performs semantic understanding of the recognized text, and performs location navigation, music playback, and temperature adjustment to ensure the sound quality of the vehicle system.

[0147] See also Fig.12 As shown, the step S60 includes the following steps:

[0148] S61, the semantic understanding unit performs semantic extraction, context interaction, and source search on the recognized text to understand the user's intention and transmit it to the location navigation unit;

[0149] It is further explained that the recognized text is further processed using natural language processing (NLP) technology to decompose, analyze and determine appropriate operations, and finally respond in a language that humans can understand, and convert the parsed semantic information into structured data that machines can understand and process, so that the voice assistant can better understand the user's intentions and provide more intelligent and personalized services; the continuation, interruption and end of the conversation are handled and the conversation process is flexibly adjusted based on user feedback to make the user experience more natural and convenient, and the user's current needs are inferred based on the previous conversation content to ensure the smoothness of the entire interaction process; based on semantic understanding, the specific data such as map navigation information, news reports, etc. required by the user are further searched on the Internet and provided to the user.

[0150] S62, the location navigation unit executes the operation execution instruction of navigating to a certain location according to the result of the semantic understanding to reach the predetermined destination so that the user can travel, and transmits it to the music playing unit;

[0151] To further explain, after the system obtains accurate recognition instructions through semantic understanding of the user's voice input, the system will issue navigation instructions based on the semantic understanding for processing and route planning so that users can obtain multiple route options such as shortest distance, fastest time, and lowest fee, and select a suitable route according to their needs. After selecting the route, click to start navigation, and the system will start voice prompts to guide the user to the destination; if the user is not satisfied with the system navigation response, the user can describe the expected travel destination in more detail. Once the navigation system successfully understands and locates the user's destination, the system will start planning navigation and display the route to the destination to start executing navigation instructions.

[0152] S63, the music playing unit executes the operation execution instruction of automatically playing the predetermined song and its playing effect according to the result of the semantic understanding, so as to improve the comfortable atmosphere in the car, and transmits it to the temperature control unit;

[0153] To further explain, the system requires the user to clearly say commands such as "play music" or "play my favorite songs" to ensure that the system can accurately recognize user commands; after receiving the voice command, the system will automatically process the user's current command, search and play music in the system, instruct the user to confirm the play list or specific songs, and the system will ask the user to confirm that the system starts playing the corresponding music songs in accordance with the execution of the operation command. If the volume or other settings need to be adjusted, the user can continue to control the system through voice recognition to ensure user satisfaction with the travel and improve the interest and satisfaction of the travel.

[0154] S64. The temperature control unit executes an operation execution instruction to adjust the air-conditioning temperature to a predetermined temperature value according to the result of the semantic understanding, so as to ensure a comfortable air environment in the vehicle.

[0155] To further explain, when the user clearly speaks out temperature adjustment instructions such as "I'm a little cold" or "I'm a little hot", the system will automatically adjust the vehicle's air-conditioning temperature according to these instructions. For example, when you say "I'm a little cold", the temperature will be automatically adjusted to 26°C and the seat heating will be turned on to the highest level. When you say "I'm a little hot", the air-conditioning temperature will quickly drop to 18°C ​​and the wind volume will increase to make the driver feel cool. And through voice commands, you can open or close the windows, adjust the wind volume, and set different temperature adjustment modes.

[0156] It is further explained that the various steps are displayed in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows, unless clearly stated in this document. There is no strict order restriction for the execution of these steps, and these steps can also be executed according to other orders, and some steps can include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed and completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0157] A vehicle-mounted voice control system provided by the present invention also includes a computer device and a computer-readable storage medium; the computer device includes a memory and various functional modules, the memory stores a computer program, and when the various functional modules execute the computer program, the steps of the vehicle-mounted voice control method described in any one of the above are implemented; the computer-readable storage medium stores a computer program, and when the computer program is executed by the various functional modules, the steps of the vehicle-mounted voice control method described in any one of the above are implemented.

[0158] Further explanation, the computer-readable storage medium includes a memory, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the program and instructions corresponding to a vehicle-mounted voice control method in the present invention, including information transmission instructions for each module; the memory executes various functional applications and data processing of each module by running the stored non-volatile software programs and instructions, that is, implements a vehicle-mounted voice control method in the above method embodiment; one or more units are stored in the memory, and when executed by the one or more modules, a vehicle-mounted voice control method in any of the above method embodiments is executed; the computer-readable storage medium stores computer-executable instructions, which are executed by one or more modules and can also be a vehicle-mounted voice control method in any of the above method embodiments.

[0159] Further explanation, the present invention is described according to the content implemented by a software program of an in-vehicle voice control system. For each in-vehicle voice control method, it is divided into several modules or units to implement the software program instructions generated by each step. The software program instructions include the in-vehicle voice control method described above.

[0160] It is further explained that the content described above in the present invention is written with reference to the implementation content of the software copyrights of "Directional Pass Car Navigation Dedicated Multimedia Player with Bluetooth Hands-free (Registration Number: 2018SR469271)", "Directional Pass Car Navigation Edition Dedicated Multimedia Player (Registration Number: 2018SR469254)", "Directional Pass Car Networking (One-key Navigation) Multimedia Player Controller (Registration Number: 2018SR500490)", and "Directional Pass Car Multimedia Player Software with SRS Decoding Sound Effect (Registration Number: 2018SR499385)" applied by our company on June 21, 2018.

[0161] The present invention also provides a vehicle-mounted voice control system, which is implemented by using the above-mentioned vehicle-mounted voice control method.

[0162] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the attached claims rather than the above description, and therefore it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application can be modified or replaced by equivalents, and all of these should be included in the scope of protection of the present application.

Claims

1. A vehicle voice control method, characterized in that : Applied to a vehicle-mounted voice control system, the system includes a system detection module, a voice wake-up module, a feature extraction module, a voice recognition module, a vehicle-mounted operation module, a wireless communication module, an alarm, a memory, a processing center, and a smart phone; the system detection module, the voice wake-up module, the feature extraction module, the voice recognition module, the vehicle-mounted operation module, the wireless communication module, the alarm, and the memory are respectively connected to the processing center; the smart phone is automatically networked with the wireless communication module within the range of a wireless network or the Internet; the method includes the following steps: S10, before the vehicle is used, the system detection module obtains the various function test data of the vehicle voice recognition system through the set vehicle automatic diagnosis system (OBD) and transmits it to the processing center; S20, the processing center compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if the standards are met, the vehicle system can be used; if the standards are not met, the alarm is transmitted to the alarm and the user is notified for maintenance; S30, when the vehicle is running, the voice wake-up module enables the user to activate the voice control system through a specific wake-up word or button to meet the user's demand for enjoying the in-vehicle music, and transmits it to the feature extraction module; S40, the feature extraction module extracts dynamic features of the spectrum, energy, and cepstrum of the converted time domain signal by first-order difference and second-order difference, so that the cepstrum coefficients change over time, and transmits them to the speech recognition module; S50, the speech recognition module recognizes specific instructions or commands by calculating the speech wake-up probability, conducting confidence evaluation, and matching with the training model, and transmits the commands to the vehicle operation module; S60, the vehicle operation module performs semantic understanding of the recognized text, and performs location navigation, music playback, and temperature adjustment to ensure the sound quality of the vehicle system.

2. The vehicle-mounted voice control method according to claim 1, characterized in that: The system further comprises: The memory is responsible for storing information of the system detection module, voice wake-up module, feature extraction module, voice recognition module, vehicle operation module, wireless communication module, alarm, and vehicle system test standards and probability confidence thresholds; The processing center is responsible for information transmission among the system detection module, voice wake-up module, feature extraction module, voice recognition module, vehicle operation module, wireless communication module, alarm, and memory. It is the hub of the system and compares the vehicle system test data with the vehicle system test standards pre-stored in the memory: if the standards are met, the vehicle system can be used; if the standards are not met, the data is transmitted to the alarm and the user is notified for repair; and the obtained confidence is compared with the probability confidence threshold preset in the memory: if the confidence exceeds the threshold, the voice request is valid to notify the voice assistant to wake up; if the confidence is lower than the threshold, the data is transmitted to the alarm and the user is notified that the wake-up is invalid and needs to be re-awakened; The system detection module includes a response test unit, a complete test unit, a stability test unit, a compatibility test unit, a safety test unit, and an accuracy test unit. The system obtains the address navigation, vehicle control, multimedia operation, and telephone dialing function test data of the vehicle voice recognition system through the set vehicle automatic diagnostic system (OBD) to ensure the accuracy of the voice recognition system and transmits it to the processing center. The voice wake-up module includes a model building unit, a voice collection unit, a voice emphasis unit, a windowing and framing unit, a time domain conversion unit, a filtering and smoothing unit, and a compression and data acquisition unit. Through a specific wake-up word or button, the user activates the voice control system to meet the user's needs for enjoying in-car music, and transmits it to the feature extraction module; The feature extraction module includes a compression data acquisition unit, a spectrum conversion unit, a first-order difference unit, and a second-order difference unit, which are responsible for extracting dynamic features of the spectrum, energy, and cepstrum of the first-order difference and second-order difference of the converted time domain signal, so that the cepstrum coefficients change over time and are transmitted to the speech recognition module; The speech recognition module includes a probability calculation unit, a confidence evaluation unit, a model matching unit, and a test optimization unit. It recognizes specific instructions or commands through speech wake-up probability calculation, confidence evaluation, and matching with the training model, and transmits them to the vehicle operation module; The vehicle operation module includes a semantic understanding unit, a location navigation unit, a music playing unit, and a temperature control unit, and is responsible for semantic understanding of the recognized text, and performing location navigation, music playing, and temperature adjustment to ensure the sound quality of the vehicle system.

3. The vehicle-mounted voice control method according to claim 1, characterized in that: The step S10 comprises the following steps: S11, the response test unit tests the response time data required for the vehicle system from receiving the voice command to completing the execution to ensure that it does not affect driving safety and user time, and passes it to the complete test unit; S12, the complete test unit obtains various functional test data by testing the navigation, music playing, phone dialing, message sending and other functions of the vehicle system, and transmits the data to the stable test unit; S13, the stability test unit tests the stability test data of various performances of the vehicle system under different temperature and humidity conditions, and transmits the test data to the compatibility test unit; S14. The compatibility test unit obtains compatibility test data by testing whether the vehicle system can connect and interact with mobile phones, smart devices, etc. of different models and brands, and transmits the compatibility test data to the safety test unit; S15, the safety test unit tests whether the vehicle system complies with the privacy protection principle when processing user voice data, and passes the test data to the accuracy test unit; S16. The accuracy test unit tests the vehicle system's ability to recognize voice commands in different driving environments to ensure that the system accurately recognizes various commands.

4. The vehicle-mounted voice control method according to claim 1, characterized in that: The step S30 comprises the following steps: S31, the model building unit builds a simulated voice wake-up model for different languages, accents, and functions required by the user in different driving scenarios to ensure voice collection in different scenarios and transmit it to the voice collection unit; S32, the voice acquisition unit obtains the continuous analog signal of the input voice wake-up signal at a sampling rate of 16KHz and converts it into a discrete digital signal to ensure the frequency range coverage, and transmits it to the voice emphasis unit; S33, the voice emphasis unit calculates the formula "H(z) = 1-αZ according to the first-order high-pass filter -1 Or y[n]=x[n] -αx[n-1], H(z) is the signal sequence after the first-order high-pass filter pre-emphasis processing, α is a constant less than 1, y[n] is the signal sequence after the discrete signal is pre-emphasized, and x[n] is the original speech signal sequence" to obtain the pre-emphasized signal sequence for subsequent windowing and framing, and pass it to the windowing and framing unit; S34, the windowing and framing unit performs windowing and framing on the digital signal after the spectrum, divides the digitized voice signal into a number of small frames to capture the basic frequency characteristics of the voice signal, and transmits it to the time domain conversion unit; S35, the time domain conversion unit performs discrete Fourier transform on each frame of the windowed signal, converts the time domain signal into the frequency domain, so that subsequent filtering and smoothing provide frequency domain information, and transmits it to the filtering and smoothing unit; S36, the filtering and smoothing unit performs Mel filtering on the frequency domain signal to convert the linear frequency into the Mel frequency that the human auditory sense is sensitive to, and smoothes the spectrum to reduce the amount of information.

5. The vehicle-mounted voice control method according to claim 1, characterized in that: The step S40 comprises the following steps: S41, the compression acquisition unit takes the logarithm of the filtered signal using the log compression dynamic range, maps the intensity of the original signal to the logarithmic domain, reduces the signal dynamic range, and transmits it to the spectrum conversion unit; S42, spectrum conversion unit calculates the time domain signal according to the formula" X[n] is the time domain signal, n is the time index, k is the frequency index, N is the total number of samples, The corresponding time domain signal in the frequency domain is obtained for the complex exponential function and passed to the first-order difference unit; S43, the first-order difference unit obtains the first-order difference speech feature according to the first-order difference speech feature calculation formula "Y(k)=X(k+1)-X(k), Z(k) is the dynamic relationship between three adjacent frames, Y(k) is the relationship between the current frame and the previous frame, X(k+1) is the current frame, and X(k) is the previous frame", and passes it to the second-order difference unit; S44. The second-order difference unit obtains the second-order difference speech feature according to the second-order difference speech feature calculation formula "Z(k)=X(k+2)-2X(k+1)+X(k), Z(k) is the second-order difference speech feature, X(k+2) is the next frame, X(k+1) is the current frame, and X(k) is the previous frame".

6. The vehicle-mounted voice control method according to claim 1, characterized in that: The step S50 comprises the following steps: S51, the probability calculation unit obtains the vehicle voice wake-up probability according to the vehicle voice wake-up probability calculation formula "W = argmax (P (O | W) P (W)), W is the voice wake-up probability, P (O | W) is the acoustic model, P (W) is the language model, O is the voice signal, W is the specific text or command", and transmits it to the confidence evaluation unit; S52, the confidence assessment unit obtains the probability confidence after smoothing according to the calculation formula of the probability confidence after smoothing "C = P / (1-P), C is the confidence, P is the probability after smoothing", and transmits it to the processing center; S53, the processing center compares the obtained confidence with the probability confidence threshold preset in the memory: if the confidence exceeds the threshold, the voice request is valid and notifies the voice assistant to wake up; if the confidence is lower than the threshold, it is transmitted to the alarm and notifies that the wake-up is invalid and needs to be re-awakened; S54, the model matching unit matches the acquired speech features with the data in the preset acoustic model and language model to screen out the most appropriate command or text, and transmits it to the test optimization unit; S55. The test optimization unit performs punctuation processing and sensitive word detection text post-processing based on the text converted from the processed audio, and performs test optimization to improve the accuracy and stability of the system.

7. The vehicle-mounted voice control method according to claim 1, characterized in that: The step S60 comprises the following steps: S61, the semantic understanding unit performs semantic extraction, context interaction, and source search on the recognized text to understand the user's intention and transmit it to the location navigation unit; S62, the location navigation unit executes the operation execution instruction of navigating to a certain location according to the result of the semantic understanding to reach the predetermined destination so that the user can travel, and transmits it to the music playing unit; S63, the music playing unit executes the operation execution instruction of automatically playing the predetermined song and its playing effect according to the result of the semantic understanding, so as to improve the comfortable atmosphere in the car, and transmits it to the temperature control unit; S64. The temperature control unit executes an operation execution instruction to adjust the air-conditioning temperature to a predetermined temperature value according to the result of the semantic understanding, so as to ensure a comfortable air environment in the vehicle.

8. A vehicle-mounted voice control method according to claims 1-7, characterized in that: The system also includes a computer device and a computer-readable storage medium; the computer device includes a memory and various functional modules, the memory stores a computer program, and when the various functional modules execute the computer program, the steps of the vehicle-mounted voice control method described in any one of claims 1 to 7 are implemented; the computer-readable storage medium stores a computer program, and when the computer program is executed by the various functional modules, the steps of the vehicle-mounted voice control method described in any one of claims 1 to 7 are implemented.

9. A vehicle-mounted voice control system, characterized in that: The method is implemented by using a vehicle-mounted voice control method as described in any one of claims 1 to 8 above.

Images