New energy automobile vehicle-mounted voice assistant system based on off-line lightweight architecture

Abstract

The invention discloses a new energy automobile vehicle-mounted voice assistant system based on an off-line lightweight architecture, and the system comprises a vehicle terminal which is internally provided with a voice recognition control module and an AI artificial intelligence module. The voice recognition control module comprises a voice recognition module, a voice control module, a natural language processing module and a setting module. An off-line scheduling platform is adopted for creating, scheduling and managing off-line tasks at the vehicle-mounted terminal, and the vehicle-mounted terminal is composed of a voice operating system, a local dialogue system, a training system and a storage system. According to the invention, the built-in voice recognition control module and the AI artificial intelligence module are adopted, and meanwhile, the vehicle terminal also adopts a lightweight architecture which comprises a lightweight database, a lightweight training model and lightweight task scheduling, so that not only can quick data access and processing be realized, but also the model can be compressed, thereby reducing the calculation amount and the storage amount, and improving the reliability of the system. And the method has good application in the field of light weight.

Description

Technical Field

[0001] The present invention relates to the technical field of in-vehicle voice assistants, and in particular to an in-vehicle voice assistant system for new energy vehicles based on an offline lightweight architecture. Background Art

[0002] As people's demands for automobile safety, intelligence, and comfort continue to increase, voice recognition technology is being used more and more widely in automobiles. In-car voice assistant is a system that enables drivers to control automobile functions and equipment through voice interaction. It can convert the driver's voice commands into commands that the computer can understand through voice recognition technology and perform corresponding operations. In-car voice assistant can provide a variety of functions, such as navigation, music playback, phone calls, adjusting the temperature in the car, etc. The driver can easily complete these operations through voice commands without having to distract attention to find and operate the car's control panel.

[0003] There are many types of voice assistants on the market with various functions, including networked voice assistants based on cloud database applications, algorithms and computing power, and car-based ones based on local databases. In recent years, voice recognition technology has completely changed the way consumers interact with cars. From personalized voice interaction to hands-free operation that improves safety and overall user experience, voice technology has become a catalyst for automotive innovation.

[0004] With the advent of the intelligent era, our car computers are becoming more and more intelligent, but intelligent car computers often require network support. Without network support, no matter how intelligent the car computer is, it will be useless. When traveling, we will inevitably go to places with weak or even no signal. At this time, the network-based intelligent voice cannot play its due role. In order to make the car computer play its voice function well in places without network, an offline in-vehicle voice assistant system is needed. Different from the offline type on the market, it needs to be further improved in function response and architecture. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the deficiencies of the prior art, the present invention adopts an offline scheduling platform to create, schedule and manage offline tasks. The vehicle-side adopts a lightweight architecture, which reduces the system's computing and storage capacity while achieving fast data access and processing.

[0007] (II) Technical solution

[0008] To achieve the above object, the present invention provides the following technical solutions: a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture, comprising a vehicle terminal, wherein the vehicle terminal has a built-in voice recognition control module and an AI artificial intelligence module;

[0009] The voice recognition control module includes:

[0010] Speech recognition module: the speech recognition engine recognizes the user's speech according to the current command vocabulary and obtains the recognition result;

[0011] Voice control module, which executes given commands and implements voice control such as in-vehicle entertainment, navigation, security and intelligent interaction functions;

[0012] The natural language processing module analyzes the text content and determines the user's specific needs through word segmentation, part-of-speech tagging, and named entity recognition. Specifically, it transcribes the speech in the audio file into text, uses Spacy to parse the text and extract entities, processes the user's intention according to the text content and forms a response, synthesizes the response text into speech, and saves it as an MP3 file;

[0013] Settings module, custom voice commands, voice control methods, and sound and visual feedback settings;

[0014] The AI ​​artificial intelligence module performs offline machine learning model training based on model training and the extracted features, and applies the trained model to the offline in-vehicle voice assistant engine, including acoustic model and language model;

[0015] The acoustic model is responsible for analyzing the acoustic features of the speech signal and matching these features with known speech units, and the language model is used to evaluate the probability of text sequences and help the system understand the relationship between words and grammatical rules.

[0016] As a preferred solution, the vehicle terminal uses an offline scheduling platform to create, schedule and manage offline tasks, and is composed of a voice operating system, a local dialogue system, a training system and a storage system, wherein:

[0017] Voice operating system, based on the wake-up module, uses voice recognition and semantic understanding technology to achieve voice control;

[0018] The local dialogue system provides a series of interfaces for the dialogue controller to call, and its interactive entry is the wake-up module;

[0019] The training system collects audio and text data and performs offline machine learning model training based on the extracted features;

[0020] The storage system is used to store local databases. All user voices are processed locally and the voice signals are encrypted for transmission.

[0021] As a preferred solution, the vehicle-mounted computer adopts a lightweight architecture, which includes: a lightweight database, which achieves fast data access and processing by optimizing the storage structure, reducing redundant data and simplifying the query syntax; a lightweight training model, which optimizes the trained model and compresses the model to reduce the amount of calculation and storage; lightweight task scheduling, which integrates the data management functions in big data analysis through an integrated lightweight management component to achieve lightweight deployment and application of a single application and a single process.

[0022] As a preferred solution, the offline scheduling platform is equipped with a wake-up module, which is a receiving module for vehicle audio signals.

[0023] As a preferred solution, the offline scheduling platform is also equipped with a TTS module and a database, and the TTS module includes:

[0024] A language processing unit for analyzing input text sentences, identifying phrases, words, and special names, converting them into phonetic notation and adding control symbols;

[0025] The speech processing unit is used to search for appropriate speech instances in the database, adjust the sound length, rhythmic features and pause time according to the semantics of the control symbols, and finally output a complete speech data stream.

[0026] As a preferred solution, the speech recognition module is based on a built-in offline speech recognition chip, which integrates a speech recognition algorithm and a memory. It receives speech signals through a microphone and then converts them into digital signals. It then performs feature extraction and comparison through the built-in speech recognition algorithm and finally outputs the recognition results.

[0027] As a preferred solution, the speech recognition module includes:

[0028] Signal acquisition unit, where the sound signal is captured in the form of electrical signal through a microphone;

[0029] The pre-processing unit includes noise removal, echo cancellation, and noise reduction processing. At the same time, sampling and quantization are performed to convert the continuous analog signal into a discrete digital signal;

[0030] The feature extraction unit converts the speech signal into a representative feature vector through an algorithm. The feature vector can capture the key information in the speech signal, such as pitch, timbre and syllable;

[0031] The matching unit matches the extracted feature vector with the vocabulary in the predefined database.

[0032] As a preferred solution, the AI ​​artificial intelligence module is connected to the main control terminal of the vehicle and is provided with a power monitoring unit. When the power of the new energy vehicle is lower than the preset threshold, the wake-up module is triggered and an active voice reminder is given; the AI ​​artificial intelligence module is also provided with a perception unit, which includes capturing the driver's facial expression state, judging whether the driver is fatigued driving based on AI, triggering the wake-up module for active voice reminders, and playing music.

[0033] As a preferred solution, the response process of the vehicle voice assistant system is as follows:

[0034] 1. The user issues a voice command to the voice assistant, which collects the user's voice command through a microphone and converts the sound wave signal into an electrical signal, including: noise filtering, echo cancellation and sound enhancement;

[0035] Second, the pre-processed voice data is converted into text through speech recognition technology, and then the natural language processing module is used to analyze and operate the text;

[0036] 3. Input the exported text into the database engine, compare it with the predetermined vocabulary, find matching words or instructions, and convert the computer's response into voice output through speech synthesis technology.

[0037] (III) Beneficial effects

[0038] Compared with the prior art, the present invention provides a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture, which has the following beneficial effects:

[0039] 1. The present invention adopts a built-in voice recognition control module and an AI artificial intelligence module, and combines an offline scheduling platform to create, schedule and manage offline tasks. It analyzes text content and determines the specific needs of users through word segmentation, part-of-speech tagging and named entity recognition, so that they can respond to instructions in a timely manner without the need for Internet access, thus avoiding problems caused by network delays or instability.

[0040] 2. The vehicle-mounted end of the present invention also adopts a lightweight architecture, which includes a lightweight database, a lightweight training model and a lightweight task scheduling. It can not only realize fast data access and processing, but also compress the model to reduce the amount of calculation and storage, and has a good application in the field of lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the system module of the present invention;

[0042] Figure 2 This is a schematic diagram of the AI ​​artificial intelligence module of the system of the present invention;

[0043] Figure 3This is a schematic diagram of the voice input process principle of the system of the present invention;

[0044] Figure 4 It is a schematic diagram of the system response process of the present invention. DETAILED DESCRIPTION

[0045] In order to better understand the purpose, structure and function of the present invention, and to better use the in-vehicle voice assistant on new energy vehicles, the present invention adopts an offline lightweight architecture to achieve fast data access, response and processing. The following will further illustrate the in-vehicle voice assistant system for new energy vehicles based on an offline lightweight architecture in combination with the accompanying drawings and specific embodiments.

[0046] Example 1

[0047] refer to Figure 1-3 , the present invention: a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture, comprising a vehicle terminal, the vehicle terminal having a built-in voice recognition control module and an AI artificial intelligence module;

[0048] Specifically, the voice recognition control module is the recognition chip of the local voice trigger engine, and its recognition method is offline voice recognition. In terms of voice technology, it has achieved high-reliability wake-up recognition rate, longer-distance wake-up, lower false wake-up rate, richer number of voice control instructions, stronger anti-noise ability, and faster response recognition time. The module is single-sided mounted with pins on both sides. The module supports single microphone input and can be directly connected to the module; it also supports real-person voice response and can directly drive the speaker.

[0049] The voice recognition control module includes:

[0050] Speech recognition module: The speech recognition engine recognizes the user's speech according to the current command vocabulary and obtains the recognition result. It is based on the offline speech recognition chip and integrates the speech recognition engine, the word rule engine and the NLP natural language processing engine;

[0051] More specifically, the speech recognition module is based on a built-in offline speech recognition chip. The offline speech recognition chip integrates a speech recognition algorithm and a memory. It receives speech signals through a microphone, converts them into digital signals, and then performs feature extraction and comparison through a built-in speech recognition algorithm, and finally outputs the recognition results.

[0052] Furthermore, the speech recognition module includes:

[0053] Signal acquisition unit, where the sound signal is captured in the form of electrical signal through a microphone;

[0054] The pre-processing unit includes noise removal, echo cancellation, and noise reduction processing. At the same time, sampling and quantization are performed to convert the continuous analog signal into a discrete digital signal;

[0055] The feature extraction unit converts the speech signal into a representative feature vector through an algorithm. The feature vector can capture the key information in the speech signal, such as pitch, timbre and syllable;

[0056] The matching unit matches the extracted feature vector with the vocabulary in the predefined database.

[0057] In the above description, the speech recognition control module of the present invention also includes:

[0058] The voice control module executes the given commands to realize voice control such as in-vehicle entertainment, navigation, security and intelligent interaction functions. It adopts far-field recognition, dual-microphone noise reduction and fixed-frequency noise reduction technology. In the near-field and far-field environment with noise interference, it suppresses various interference sounds outside the command beam and can intelligently distinguish environmental noise and recognize human voices.

[0059] The natural language processing module analyzes the text content and determines the user's specific needs through word segmentation, part-of-speech tagging, and named entity recognition. Specifically, it transcribes the speech in the audio file into text, uses Spacy to parse the text and extract entities, processes the user's intention according to the text content and forms a response, synthesizes the response text into speech, and saves it as an MP3 file;

[0060] Settings module, customize voice commands, voice control methods, and sound and visual feedback settings. For example, you can change the wake-up word or the response method of the voice assistant.

[0061] In the vehicle-mounted voice system of the present invention, the offline dispatching platform is equipped with a wake-up module, which is a receiving module for the vehicle-mounted audio signal.

[0062] Example 2

[0063] refer to Figure 4 , the response process of the vehicle voice assistant system of the present invention is:

[0064] 1. The user issues a voice command to the voice assistant, which collects the user's voice command through a microphone and converts the sound wave signal into an electrical signal, including: noise filtering, echo cancellation and sound enhancement;

[0065] Second, the pre-processed voice data is converted into text through speech recognition technology, and then the natural language processing module is used to analyze and operate the text;

[0066] 3. Input the exported text into the database engine, compare it with the predetermined vocabulary, find matching words or instructions, and convert the computer's response into voice output through speech synthesis technology.

[0067] Specifically, the vehicle terminal of the present invention is responsible for the central control module of the voice dialogue, that is, the vehicle terminal, which is responsible for coordinating the overall process of the vehicle terminal. Other modules of the vehicle terminal are called by the voice assistant (audio service, local dialogue service, TTS module and application program, etc.). In this embodiment, the vehicle terminal adopts an offline scheduling platform to create, schedule and manage offline tasks, and is composed of a voice operating system, a local dialogue system, a training system and a storage system, wherein:

[0068] Voice operating system, based on the wake-up module, uses voice recognition and semantic understanding technology to achieve voice control;

[0069] The local dialogue system provides a series of interfaces for the dialogue controller to call, and its interactive entry is the wake-up module;

[0070] The training system collects audio and text data and performs offline machine learning model training based on the extracted features;

[0071] The storage system is used to store local databases. All user voices are processed locally and the voice signals are encrypted for transmission.

[0072] Furthermore, the local dialogue system is responsible for executing dialogue processing that needs to be performed on the vehicle computer, such as dialogue functions in offline state, and dialogue processing related to navigation or music search based on the local SDK.

[0073] Furthermore, the offline scheduling platform is also equipped with a TTS module and a database. The TTS module includes:

[0074] A language processing unit for analyzing input text sentences, identifying phrases, words, and special names, converting them into phonetic notation and adding control symbols;

[0075] The speech processing unit is used to search for appropriate speech instances in the database, adjust the sound length, rhythmic features and pause time according to the semantics of the control symbols, and finally output a complete speech data stream.

[0076] Among them, the TTS module is called by the voice assistant and is responsible for converting text into voice broadcast. The results are stored in the database for call by the dialogue system.

[0077] Example 3

[0078] refer to Figure 1-4In the present invention, the vehicle terminal adopts a lightweight architecture, which reduces the amount of calculation and storage of the system while achieving fast data access and processing, and includes:

[0079] Lightweight database, which achieves fast data access and processing by optimizing storage structure, reducing redundant data and simplifying query syntax;

[0080] Lightweight training model, optimize the trained model, compress the model to reduce the amount of calculation and storage;

[0081] Lightweight task scheduling integrates data management functions in big data analysis through integrated lightweight management components to achieve lightweight deployment and application of single application and single process.

[0082] Specifically, database lightweighting means optimizing the database architecture, reducing redundant data, rationally allocating storage space, and improving data processing efficiency, so that the database can achieve higher operating efficiency and lower storage costs while maintaining data integrity and accuracy.

[0083] The in-vehicle voice assistant system of the present invention has an AI artificial intelligence module connected to the vehicle main control terminal and is provided with a power monitoring unit. When the power of the new energy vehicle is lower than the preset threshold, the wake-up module is triggered and an active voice reminder is given; the AI ​​artificial intelligence module is also provided with a perception unit, which includes capturing the driver's facial expression state, judging whether the driver is driving fatigued based on AI, triggering the wake-up module for active voice reminders, and playing music. Specifically, the AI ​​artificial intelligence module is based on model training, and according to the extracted features, offline machine learning model training is performed, and the trained model is applied to the offline in-vehicle voice assistant engine, including acoustic models and language models;

[0084] The acoustic model is responsible for analyzing the acoustic features of the speech signal and matching these features with known speech units. The language model is used to evaluate the probability of text sequences and help the system understand the relationship between words and grammatical rules.

[0085] The present invention uses a built-in voice recognition control module and an AI artificial intelligence module, and combines it with an offline scheduling platform to create, schedule and manage offline tasks, so that it can respond to instructions in a timely manner, and does not need to be connected to the Internet, thus avoiding problems caused by network delays or instability. It also uses a lightweight architecture, which can not only achieve fast data access and processing, but also compress the model to reduce the amount of calculation and storage, and has a good application in the field of lightweight.

[0086] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture, comprising a vehicle terminal, characterized in that: The vehicle terminal is equipped with a voice recognition control module and an AI artificial intelligence module; The voice recognition control module includes: Speech recognition module: the speech recognition engine recognizes the user's speech according to the current command vocabulary and obtains the recognition result; Voice control module, which executes given commands and implements voice control such as in-vehicle entertainment, navigation, security and intelligent interaction functions; The natural language processing module analyzes the text content and determines the user's specific needs through word segmentation, part-of-speech tagging, and named entity recognition. Specifically, it transcribes the speech in the audio file into text, uses Spacy to parse the text and extract entities, processes the user's intention according to the text content and forms a response, synthesizes the response text into speech, and saves it as an MP3 file; Settings module, custom voice commands, voice control methods, and sound and visual feedback settings; The AI ​​artificial intelligence module performs offline machine learning model training based on model training and the extracted features, and applies the trained model to the offline in-vehicle voice assistant engine, including acoustic model and language model; The acoustic model is responsible for analyzing the acoustic features of the speech signal and matching these features with known speech units, and the language model is used to evaluate the probability of text sequences and help the system understand the relationship between words and grammatical rules.

2. According to claim 1, a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture is characterized in that: The vehicle terminal uses an offline scheduling platform to create, schedule and manage offline tasks, and is composed of a voice operating system, a local dialogue system, a training system and a storage system, wherein: Voice operating system, based on the wake-up module, uses voice recognition and semantic understanding technology to achieve voice control; The local dialogue system provides a series of interfaces for the dialogue controller to call, and its interactive entry is the wake-up module; The training system collects audio and text data and performs offline machine learning model training based on the extracted features; The storage system is used to store local databases. All user voices are processed locally and the voice signals are encrypted for transmission.

3. According to claim 2, a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture is characterized in that: The vehicle-mounted computer adopts a lightweight architecture, which includes: a lightweight database, which achieves fast data access and processing by optimizing the storage structure, reducing redundant data and simplifying the query syntax; a lightweight training model, which optimizes the trained model and compresses the model to reduce the amount of calculation and storage; lightweight task scheduling, which integrates the data management functions in big data analysis through an integrated lightweight management component to achieve single-application, single-process lightweight deployment and application.

4. According to claim 2, a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture is characterized in that: The offline dispatching platform is equipped with a wake-up module, which is a receiving module for vehicle audio signals.

5. According to claim 2, a new energy vehicle in-vehicle voice assistant system based on an offline lightweight architecture is characterized in that: The offline scheduling platform is also equipped with a TTS module and a database, and the TTS module includes: A language processing unit for analyzing input text sentences, identifying phrases, words, and special names, converting them into phonetic notation and adding control symbols; The speech processing unit is used to search for appropriate speech instances in the database, adjust the sound length, rhythmic features and pause time according to the semantics of the control symbols, and finally output a complete speech data stream.

6. The in-vehicle voice assistant system for new energy vehicles based on an offline lightweight architecture according to claim 1, characterized in that: The speech recognition module is based on a built-in offline speech recognition chip, which integrates a speech recognition algorithm and a memory. It receives speech signals through a microphone, converts them into digital signals, extracts and compares features through the built-in speech recognition algorithm, and finally outputs the recognition results.

7. The in-vehicle voice assistant system for new energy vehicles based on an offline lightweight architecture according to claim 6 is characterized in that: The speech recognition module comprises: Signal acquisition unit, where the sound signal is captured in the form of electrical signal through a microphone; The pre-processing unit includes noise removal, echo cancellation, and noise reduction processing. At the same time, sampling and quantization are performed to convert the continuous analog signal into a discrete digital signal; The feature extraction unit converts the speech signal into a representative feature vector through an algorithm. The feature vector can capture the key information in the speech signal, such as pitch, timbre and syllable; The matching unit matches the extracted feature vector with the vocabulary in the predefined database.

8. The in-vehicle voice assistant system for new energy vehicles based on an offline lightweight architecture according to claim 1, characterized in that: The AI ​​artificial intelligence module is connected to the main control end of the vehicle computer and is provided with a power monitoring unit. When the power of the new energy vehicle is lower than the preset threshold, the wake-up module is triggered and an active voice reminder is given; the AI ​​artificial intelligence module is also provided with a perception unit, which includes capturing the driver's facial expression state, judging whether the driver is driving fatigued based on AI, triggering the wake-up module for active voice reminders, and playing music.

9. The in-vehicle voice assistant system for new energy vehicles based on an offline lightweight architecture according to claim 1, characterized in that: The response process of the vehicle voice assistant system is as follows:

1. The user issues a voice command to the voice assistant, which collects the user's voice command through a microphone and converts the sound wave signal into an electrical signal, including: noise filtering, echo cancellation and sound enhancement; Second, the pre-processed voice data is converted into text through speech recognition technology, and then the natural language processing module is used to analyze and operate the text; 3. Input the exported text into the database engine, compare it with the predetermined vocabulary, find matching words or instructions, and convert the computer's response into voice output through speech synthesis technology.

Images