A translation device for an intelligent vehicle-mounted system and its control system

Abstract

The present invention provides a translation device for an intelligent vehicle-mounted system and its control system. A vehicle-mounted external translation device providing multi-language translation services is constructed through a service data terminal and a microphone. The device translates the driver and passengers based on NLP technology, and then wakes up the voice assistant of the vehicle-mounted voice control system by playing the translated voice, and issues control instructions to the voice assistant to complete vehicle control in non-default languages, thereby enhancing the intelligence level of the vehicle. Even if the driver and passengers do not speak the same language, they can still realize some functions of the vehicle through voice control, which significantly improves driving safety. The translation device can also translate and play the response voice of the voice assistant, realizing intelligent interactive dialogue between the driver and passengers and the vehicle-mounted voice control system. At the same time, the translation device supports both offline and networked working modes, and can complete voice translation tasks even without a network.

Description

Technical Field

[0001] The present invention relates to the technical field of speech translation processing, and in particular to a translation device of an intelligent vehicle-mounted system and a control system thereof. Background Art

[0002] With the development of automobile technology and economy, automobiles have become popular and have become one of the important ways for people to travel. With the development of artificial intelligence, automobile control has become more and more intelligent. By waking up the voice assistant through the car's on-board voice control system, the vehicle can be controlled contactlessly, effectively improving the safety of car driving.

[0003] With economic development, international communication has become increasingly close, and more and more people are traveling and working across borders. However, existing on-board voice control systems of automobiles mainly use the official language of the location where the automobile is located as the default language. Drivers and passengers need to wake up the voice assistant of the on-board voice control system in the default language, which causes difficulties for international tourists and workers in daily travel in addition to daily communication difficulties. Although the existing automobile control panel display text can complete language switching by selection, international tourists and workers cannot independently complete the language switching setting due to the text barrier. Moreover, due to technical limitations, most voice assistants can only recognize voices in one or a few fixed languages. As a result, even if international tourists and workers complete the language switching on the control panel, they may not be able to use the car's on-board voice control system to control the vehicle normally, reducing driving safety. Therefore, the present invention proposes a translation device and a control system for an intelligent on-board system. Summary of the Invention

[0004] The present invention provides a translation device for an intelligent vehicle-mounted system and a control system thereof, so as to solve the above-mentioned problems.

[0005] The present invention provides a translation device for an intelligent vehicle-mounted system, comprising: a service data terminal, a microphone, and a loudspeaker;

[0006] The service data terminal is electrically connected to the microphone and the speaker respectively;

[0007] The microphone is used to collect real-time voice in the vehicle and send the real-time voice to the service data terminal;

[0008] Among them, real-time voice includes the driver's voice and the vehicle voice assistant's response voice;

[0009] The service data terminal recognizes and translates the real-time voice, generates translated voice, and sends it to the speaker for playback;

[0010] The translated speech includes the target language control speech and the translated response speech.

[0011] Preferably, in a translation device of an intelligent vehicle-mounted system, two three-color indicator lights are provided on the service data terminal for displaying the device status;

[0012] Among them, the red light means that the device is starting or abnormal;

[0013] The green light indicates that the device is operating normally and the network status is good;

[0014] A yellow light indicates the device is offline or waiting to connect.

[0015] The present invention provides a control system for a translation device of an intelligent vehicle-mounted system, comprising:

[0016] The voice collection module is used to control the microphone to collect real-time voice inside the vehicle when confirming that the vehicle is in the starting state;

[0017] The speech recognition module is used to control the service data terminal. It uses NLP technology to perform language and semantic recognition on the received real-time speech, determine whether there are vehicle control wake-up keywords in the real-time speech, and obtain the speech to be translated;

[0018] The speech translation module is used to perform semantic recognition based on the speech to be translated, obtain the translated text, and generate the target language control speech based on the preset vehicle control speech template;

[0019] The voice interaction module is used to control the service data terminal to translate the response voice of the vehicle voice assistant to obtain the translated response voice;

[0020] The playback control module is used to control the loudspeaker to play the target language control voice or the translated response voice.

[0021] Preferably, in a control system of a translation device of an intelligent vehicle-mounted system, the voice acquisition module includes:

[0022] The vehicle status detection unit is used to control the automatic start of the translation device after the door is opened, and monitor the sound inside the vehicle. When the sound of the vehicle engine starting is detected, it determines that the current vehicle is in the starting state;

[0023] The collection control unit is used to generate a human voice collection control instruction after confirming that the vehicle is started, and send it to the human voice collection unit;

[0024] The voice collection unit is used to control the microphone array to switch to the voice collection mode after receiving the voice control command, and collect real-time voice inside the vehicle.

[0025] Preferably, in a control system of a translation device of an intelligent vehicle system, the speech recognition module includes:

[0026] The real-time language recognition unit is used to input the real-time speech received by the service data terminal into the speech recognition center. The speech recognition center is equipped with a multi-language language recognition model to recognize the language of the input real-time speech and determine the current language;

[0027] a speech comparison unit, configured to obtain a preset wake-up speech of a corresponding language from a preset wake-up speech database based on the current language, determine whether the current language is a non-target language, and if so, compare the real-time speech with the preset wake-up speech to determine whether a vehicle control wake-up keyword exists in the real-time speech; if so, determine that the real-time speech is a vehicle wake-up speech, and use the vehicle wake-up speech as the starting speech for human-computer interaction and the sentence to be translated;

[0028] At the same time, a temporary timing clock is added to the starting voice of human-computer interaction;

[0029] A real-time semantic recognition unit, configured to recognize the initial speech of the human-computer interaction based on a multilingual semantic recognition model set by the speech recognition center, and obtain the semantics of the driver and passenger;

[0030] a command voice identification unit, configured to identify a first real-time voice of the driver or passenger based on a multilingual semantic recognition model before the temporary timer reaches a preset time; if the first real-time voice is a control voice of the current vehicle, delete the current temporary timer, use the control voice as the voice to be translated, and generate a new temporary timer based on the latest control voice of the current vehicle;

[0031] If the multilingual semantic recognition model does not detect any control voice before the temporary timing clock reaches the preset time, the temporary timing clock is deleted when the temporary timing clock reaches the preset time, and it is determined that the current round of wake-up control is completed.

[0032] Preferably, in a control system of a translation device of an intelligent vehicle system, the speech translation module includes:

[0033] The speech translation unit is used to generate the translated text based on the semantic recognition results corresponding to the speech to be translated and the language logic corresponding to the target language;

[0034] A text standardization unit is used to determine the driver's control purpose based on the speech recognition results, obtain a corresponding speech control template based on the control purpose, and standardize the translated text based on the speech control instruction module to obtain the control instruction text in the target language;

[0035] The text conversion unit is used to convert the target language control instruction text into the target language control speech based on the preset timbre and the intonation of the speech to be translated.

[0036] Preferably, in a control system of a translation device of an intelligent vehicle system, the voice interaction module includes:

[0037] A response recognition unit is used to perform semantic recognition on the real-time speech corresponding to the vehicle voice assistant to obtain a response speech recognition result;

[0038] The interactive translation unit is used to generate a translated response text corresponding to the current language based on the response voice recognition result and the language logic corresponding to the current language, and to generate a translated response voice based on the text conversion unit.

[0039] Preferably, in a control system of a translation device of an intelligent vehicle-mounted system, the playback control module includes:

[0040] The first playback unit is used to control the speaker to play the target language control voice to the vehicle voice assistant according to the target playback volume;

[0041] The second playback unit is used to control the speaker to play the translated response voice to the vehicle voice assistant according to the target playback volume;

[0042] The playback control unit is used to automatically adjust the target playback volume of the speaker based on the current microphone position parameters and the current environmental noise data.

[0043] Preferably, in a control system of a translation device of an intelligent vehicle-mounted system, the playback control unit includes:

[0044] An intelligent test subunit, configured to, after the translation device is installed, automatically test the voice to send location confirmation instructions and volume control instructions to the vehicle voice assistant, and determine the relative position of the translation device and the vehicle console based on the test response voice of the vehicle voice assistant;

[0045] Determining the relative distance between the translation device and the vehicle console based on sound energy changes of the multiple test response voices received by the translation device;

[0046] Based on the relative orientation and relative distance, combined with the vehicle interior space model, determining the installation position of the translation device, and generating position parameters of the translation device and sending them to the database for storage;

[0047] An intelligent monitoring subunit, which interprets the device's built-in sound sensor to obtain ambient noise data of the current vehicle interior;

[0048] The intelligent control sub-unit is used to determine whether the current translation device is in an online state. If not, the current environmental interference level is determined based on the current environmental noise level, and the playback volume corresponding to the current environmental interference level is determined based on a preset offline control table, and the playback volume is used as the target playback volume.

[0049] Preferably, in a control system of a translation device of an intelligent vehicle-mounted system, the intelligent control subunit is further used to:

[0050] When the current translation device is connected to the Internet, the noise environment data and the location parameters of the transfer device are sent to the cloud;

[0051] Comparing the location parameters of the translation device with the corresponding location parameters of the default installation area to obtain an installation location difference rate, obtaining an interference-free playback volume based on the default volume corresponding to the default installation area and the location installation difference rate, and comparing the vehicle isolation noise threshold with the current noise threshold to obtain a noise difference rate;

[0052] Based on the vehicle data received from the cloud, determine whether the current vehicle has a window open behavior;

[0053] If not, the undisturbed playback volume is corrected based on the noise difference rate to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume;

[0054] If it exists, then determine the window position and size of the current vehicle based on the vehicle data, combine them with the position parameters of the translation device, and judge the degree of influence of each window position on the sound played by the translation device;

[0055] Based on the influence degree of each window position, the sound loss rate is obtained. Based on the sound loss rate and the noise difference rate, the interference-free playback volume is corrected to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention uses the multi-language translation service of the translation device to translate for the driver and passengers based on NLP technology, then wakes up the voice assistant of the vehicle voice control system by playing the translated voice, and issues control instructions to the voice assistant to complete vehicle control in non-default languages, thereby enhancing the intelligence level of the vehicle, so that the driver and passengers can realize some functions of the vehicle through voice control even if they do not speak the same language. The game significantly improves driving safety. The translation device can also translate and play the response voice of the voice assistant, realizing intelligent interactive dialogue between the driver and passengers and the vehicle voice control system. At the same time, the translation device supports both offline and networked modes of operation, and can complete voice translation tasks even without a network.

[0058] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 This is a schematic structural diagram of a translation device for an intelligent vehicle-mounted system according to the present invention;

[0062] Figure 2 This is a structural diagram of a control system of a translation device of an intelligent vehicle-mounted system of the present invention;

[0063] Figure 3 This is a structural diagram of a control system voice acquisition module of a translation device of an intelligent vehicle-mounted system of the present invention;

[0064] Figure 4 This is a structural diagram of a control system speech recognition module of a translation device of an intelligent vehicle-mounted system of the present invention;

[0065] Figure 5 This is a structural diagram of a control system speech translation module of a translation device of an intelligent vehicle-mounted system of the present invention;

[0066] Figure 6 This is a structural diagram of a control system voice interaction module of a translation device of an intelligent vehicle-mounted system of the present invention;

[0067] Figure 7 This is a structural diagram of a control system playback control module of a translation device of an intelligent vehicle-mounted system according to the present invention. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0069] Example 1:

[0070] The present invention provides a translation device for an intelligent vehicle-mounted system, such as Figure 1 As shown, it includes: a service data terminal, a microphone and a speaker;

[0071] The service data terminal is electrically connected to the microphone and the speaker respectively;

[0072] The microphone is used to collect real-time voice in the vehicle and send the real-time voice to the service data terminal;

[0073] Among them, real-time voice includes the driver's voice and the vehicle voice assistant's response voice;

[0074] The service data terminal recognizes and translates the real-time voice, generates translated voice, and sends it to the speaker for playback;

[0075] The translated speech includes the target language control speech and the translated response speech;

[0076] The service data terminal shown is provided with two three-color indicator lights for displaying the device status;

[0077] Among them, the red light means that the device is starting or abnormal;

[0078] The green light indicates that the device is operating normally and the network status is good;

[0079] A yellow light indicates the device is offline or waiting to connect.

[0080] In this embodiment, the translation device can be placed or installed at any location inside the vehicle.

[0081] The beneficial effects of the above technical solution: the present invention constructs an on-vehicle external translation terminal that can complete translation of multiple languages ​​through a service data terminal, a microphone and a speaker, and the service data terminal is electrically connected to the microphone and the speaker respectively; the microphone is used to collect real-time voice in the vehicle and send the real-time voice to the service data terminal; the service data terminal recognizes and translates the real-time voice, generates translated voice, and sends it to the speaker for playback, completing the translation of non-target languages, helping drivers and passengers overcome the problem of being unable to use the on-board voice control system normally due to language barriers. At the same time, the translation equipment can work both in an online state and in an offline state, and can be used normally even in an environment with poor signals, providing convenience for people who do not understand the language and are driving.

[0082] Example 2:

[0083] The present invention provides a control system for a translation device of an intelligent vehicle-mounted system, as shown in FIG2, including:

[0084] The audio collection module is used to control the microphone to collect real-time audio inside the vehicle when the vehicle is confirmed to be in the starting state;

[0085] The speech recognition module is used to control the service data terminal. It uses NLP technology to perform language and semantic recognition on the received real-time speech, determine whether there are vehicle control wake-up keywords in the real-time speech, and obtain the speech to be translated;

[0086] The speech translation module is used to perform semantic recognition based on the speech to be translated, obtain the translated text, and generate the target language control speech based on the preset vehicle control speech template;

[0087] The voice interaction module is used to control the service data terminal to translate the response voice of the vehicle voice assistant to obtain the translated response voice;

[0088] The playback control module is used to control the loudspeaker to play the target language control voice or the translated response voice.

[0089] The beneficial effects of the above technical solution: when confirming that the vehicle is in the starting state, the present invention controls the microphone to collect real-time voice inside the vehicle through the voice collection module, and then controls the service data terminal through the voice recognition module to perform language and semantic recognition on the received real-time voice based on NLP technology, and judge whether there is a vehicle control wake-up keyword in the real-time voice to obtain the voice to be translated, and then performs semantic recognition based on the voice to be translated through the voice translation module to obtain the translated text, and generates the target language control voice in combination with the preset vehicle control voice template. The service data terminal can also be controlled by the voice interaction module to translate the response voice of the vehicle voice assistant to obtain the translated response voice, and finally the speaker is controlled by the playback control module to play the target language control voice or the translated response voice, breaking the language restriction, completing the interaction between the driver and the on-board voice control system, and overcoming the problem that the driver and the passengers cannot use the on-board voice control system normally due to language barriers. The present invention uses the multi-language translation service of the translation device to translate for the driver and passengers based on NLP technology, then wakes up the voice assistant of the vehicle voice control system by playing the translated voice, and issues control instructions to the voice assistant to complete vehicle control in non-default languages, thereby enhancing the intelligence level of the vehicle, so that the driver and passengers can realize some functions of the vehicle through voice control even if they do not speak the same language, significantly improving driving safety. The translation device can also translate and play the response voice of the voice assistant, realizing intelligent interactive dialogue between the driver and passengers and the vehicle voice control system. At the same time, the translation device supports both offline and networked modes of operation, and can complete voice translation tasks even without a network.

[0090] Example 3:

[0091] On the basis of Example 2, the voice collection module, such as Figure 3 Shown, including:

[0092] The vehicle status detection unit is used to control the automatic start of the translation device after the door is opened, and monitor the sound inside the vehicle. When the sound of the vehicle engine starting is detected, it determines that the current vehicle is in the starting state;

[0093] The collection control unit is used to generate a human voice collection control instruction after confirming that the vehicle is started, and send it to the human voice collection unit;

[0094] The voice collection unit is used to control the microphone array to switch to the voice collection mode after receiving the voice control command, and collect real-time voice inside the vehicle.

[0095] The beneficial effects of the above technical solution: The present invention controls the translation device to automatically start at the moment the car door is opened through the vehicle status detection unit, effectively ensuring that every control command of the driver and passenger can be captured, and after confirming that the vehicle is started, generates a human voice collection control instruction through the collection control unit and sends it to the human voice collection unit to start human voice collection, realizing automatic filtering of noise, avoiding interference of noise in the low translation process, and effectively improving the translation accuracy.

[0096] Example 4:

[0097] Based on Example 2, the speech recognition module, such as Figure 4 Shown, including:

[0098] The real-time language recognition unit is used to input the real-time speech received by the service data terminal into the speech recognition center. The speech recognition center is equipped with a multi-language language recognition model to recognize the language of the input real-time speech and determine the current language;

[0099] a speech comparison unit, configured to obtain a preset wake-up speech of a corresponding language from a preset wake-up speech database based on the current language, determine whether the current language is a non-target language, and if so, compare the real-time speech with the preset wake-up speech to determine whether a vehicle control wake-up keyword exists in the real-time speech; if so, determine that the real-time speech is a vehicle wake-up speech, and use the vehicle wake-up speech as the starting speech for human-computer interaction and the sentence to be translated;

[0100] At the same time, a temporary timing clock is added to the starting voice of human-computer interaction;

[0101] A real-time semantic recognition unit, configured to recognize the initial speech of the human-computer interaction based on a multilingual semantic recognition model set by the speech recognition center, and obtain the semantics of the driver and passenger;

[0102] a command voice identification unit, configured to identify a first real-time voice of the driver or passenger based on a multilingual semantic recognition model before the temporary timer reaches a preset time; if the first real-time voice is a control voice of the current vehicle, delete the current temporary timer, use the control voice as the voice to be translated, and generate a new temporary timer based on the latest control voice of the current vehicle;

[0103] If the multilingual semantic recognition model does not detect any control voice before the temporary timing clock reaches the preset time, the temporary timing clock is deleted when the temporary timing clock reaches the preset time, and it is determined that the current round of wake-up control is completed.

[0104] In this embodiment, the multilingual language recognition model is obtained by training a large amount of multiple voice data and their labeled results, and can recognize the language corresponding to the user's input voice. During use, new multiple voice data is continuously acquired through big data technology and deep learning training is performed on the multilingual language recognition model to complete the automatic update of the multilingual language recognition model, so that the multilingual language recognition model can complete the recognition of multiple languages.

[0105] In this embodiment, the multilingual semantic recognition model is obtained by training with a large amount of data on multiple dialects corresponding to different languages. During use, new dialect speech data of different languages ​​is continuously acquired through big data technology to perform deep learning training on the multilingual semantic recognition model, and the multilingual semantic recognition model is automatically updated, so that the multilingual semantic recognition model can complete the recognition of multiple languages.

[0106] In this embodiment, the target language is the language used by the voice assistant, and the current language is the actual language corresponding to the real-time voice.

[0107] In this embodiment, the multi-language includes both multiple languages ​​and multiple dialects contained in the multiple languages.

[0108] The beneficial effects of the above technical solution: the present invention first recognizes the real-time speech through the multilingual language recognition model by the real-time language recognition unit to determine the language of the real-time speech. When the real-time speech is a non-target language, the speech comparison unit determines whether there is a wake-up keyword in the real-time speech through the preset wake-up speech corresponding to the current language, thereby judging whether the driver or passenger has the intention to voice-control the vehicle, and uses the actual speech with the wake-up keyword as the starting speech of human-computer interaction, and adds a temporary timing clock to the starting speech of human-computer interaction to provide a timing basis for determining the effective waiting time of a single wake-up speech; then, a real-time semantic recognition unit is used to recognize the starting speech of human-computer interaction based on the multilingual semantic recognition model set by the speech recognition center to obtain the semantics of the driver or passenger; finally, the command speech identification unit recognizes the first real-time speech of the driver or passenger based on the multilingual semantic recognition model before the temporary timing clock reaches the preset time. If the first If the real-time voice is the control voice of the current vehicle, the current temporary timing clock is deleted to realize the automatic update of the effective waiting time and the number of wake-up times during the use of the effective voice assistant. As long as the user issues an instruction again within the effective waiting time, even if the wake-up instruction is not said (for example, "Hi, Xiaoai!", "Hello, Xiaoxin", etc.), the transfer device can complete the recognition and translation of the vehicle voice control instruction, and use the control voice as the voice to be translated to complete the translation of the driver's voice control instruction, so that the driver can realize some functions of the vehicle through voice control even if they do not speak the language, which significantly improves driving safety; and generates a new temporary timing clock based on the latest control voice of the current vehicle; if the multilingual semantic recognition model does not detect any control voice before the temporary timing clock reaches the preset time, then when the temporary timing clock reaches the preset time, the temporary timing clock is deleted, and it is determined that the current round of wake-up control is completed, effectively avoiding the voice assistant from waiting too long.

[0109] Example 5:

[0110] Based on Example 2, the speech translation module, such as Figure 5 Shown, including:

[0111] The speech translation unit is used to generate the translated text based on the semantic recognition results corresponding to the speech to be translated and the language logic corresponding to the target language;

[0112] A text standardization unit is used to determine the driver's control purpose based on the speech recognition results, obtain a corresponding speech control template based on the control purpose, and standardize the translated text based on the speech control instruction module to obtain the control instruction text in the target language;

[0113] The text conversion unit is used to convert the target language control instruction text into the target language control speech based on the preset timbre and the intonation of the speech to be translated.

[0114] The beneficial effects of the above technical solution: the present invention first generates a translated text based on the semantic recognition result corresponding to the speech to be translated by the speech translation unit, combined with the language logic corresponding to the target language, completes the language conversion of the real-time speech, and ensures that the translated text conforms to the usage logic of the target language. Then, the text standardization unit determines the control purpose of the driver and passenger based on the speech recognition result, obtains the corresponding speech control template based on the control purpose, and standardizes the translated text based on the speech control instruction module to obtain the target language control instruction text. The text after the language conversion is completed is standardized to ensure that the output translated text conforms to the instruction logic of vehicle control. After hearing the output speech, the voice assistant can quickly recognize the voice control instruction and respond, thereby improving the accuracy of vehicle control. Finally, the text conversion unit converts the target language control instruction text into the target language control speech based on the preset timbre and the intonation of the speech to be translated, providing a basis for realizing intelligent dialogue between the driver and the vehicle voice control system.

[0115] Example 6:

[0116] Based on Example 5, the voice interaction module, such as Figure 6 Shown, including:

[0117] A response recognition unit is used to perform semantic recognition on the real-time speech corresponding to the vehicle voice assistant to obtain a response speech recognition result;

[0118] The interactive translation unit is used to generate a translated response text corresponding to the current language based on the response voice recognition result and the language logic corresponding to the current language, and to generate a translated response voice based on the text conversion unit.

[0119] The beneficial effects of the above technical solution: The present invention performs semantic recognition on the real-time voice corresponding to the vehicle voice assistant through the response recognition unit to obtain the response voice recognition result. At the same time, the interactive translation unit can also generate a translated response text corresponding to the current language based on the response voice recognition result and the language logic corresponding to the current language, and generate a translated response voice based on the text conversion unit, completing the translation of the voice assistant's response voice, and providing a basis for realizing intelligent interactive dialogue between the driver and the vehicle-mounted voice control system.

[0120] Example 7:

[0121] Based on embodiment 2, the playback control module, such as Figure 7 Shown, including:

[0122] The first playback unit is used to control the speaker to play the target language control voice to the vehicle voice assistant according to the target playback volume;

[0123] The second playback unit is used to control the speaker to play the translated response voice to the vehicle voice assistant according to the target playback volume;

[0124] The playback control unit is used to automatically adjust the target playback volume of the speaker based on the current microphone position parameters and the current environmental noise data.

[0125] The beneficial effects of the above technical solution: the present invention controls the playback volume of the target language control voice and the translated response voice of the speaker through the first playback unit and the second playback unit, thereby enhancing the intelligence level of the vehicle, so that the driver and passengers can realize some functions of the vehicle through voice control even if they do not speak the language, significantly improving driving safety, and realizing intelligent interactive dialogue between the driver and the vehicle-mounted voice control system. In addition, during the playback of the target language control voice or the translated response voice, the playback control unit automatically adjusts the target playback volume of the speaker based on the current microphone position parameters and the current environmental noise data. On the one hand, it ensures that the driver and passengers and the voice assistant can hear the translated voice in a noisy environment, effectively breaking through the environmental use restrictions of the translation equipment. On the other hand, due to the intelligent control of the playback sound, the installation location of the translation translator is no longer restricted.

[0126] Example 8:

[0127] Based on embodiment 7, the playback control unit includes:

[0128] An intelligent test subunit, configured to, after the translation device is installed, automatically test the voice to send location confirmation instructions and volume control instructions to the vehicle voice assistant, and determine the relative position of the translation device and the vehicle console based on the test response voice of the vehicle voice assistant;

[0129] Determining the relative distance between the translation device and the vehicle console based on sound energy changes of the multiple test response voices received by the translation device;

[0130] Based on the relative orientation and relative distance, combined with the vehicle interior space model, determining the installation position of the translation device, and generating position parameters of the translation device and sending them to the database for storage;

[0131] An intelligent monitoring subunit, which interprets the device's built-in sound sensor to obtain ambient noise data of the current vehicle interior;

[0132] An intelligent control subunit is configured to determine whether the current translation device is connected to the Internet. If not, the intelligent control subunit determines the current environmental interference level based on the current environmental noise level, and determines the playback volume corresponding to the current environmental interference level based on a preset offline control table, and uses the playback volume as the target playback volume.

[0133] When the current translation device is connected to the Internet, the noise environment data and the location parameters of the transfer device are sent to the cloud;

[0134] Comparing the location parameters of the translation device with the corresponding location parameters of the default installation area to obtain an installation location difference rate, obtaining an interference-free playback volume based on the default volume corresponding to the default installation area and the location installation difference rate, and comparing the vehicle isolation noise threshold with the current noise threshold to obtain a noise difference rate;

[0135] Based on the vehicle data received from the cloud, determine whether the current vehicle has a window open behavior;

[0136] If not, the undisturbed playback volume is corrected based on the noise difference rate to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume;

[0137] If it exists, then determine the window position and size of the current vehicle based on the vehicle data, combine them with the position parameters of the translation device, and judge the degree of influence of each window position on the sound played by the translation device;

[0138] Based on the influence degree of each window position, the sound loss rate is obtained. Based on the sound loss rate and the noise difference rate, the interference-free playback volume is corrected to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume.

[0139] In this embodiment, the degree of influence of each window position on the sound played by the translation device is determined based on the distance between the window position and the installation position of the translation device and the distance between the window position and the vehicle console. The sound heard from the driver's seat is accurate, and the sound loss rate will vary depending on the vehicle model and speed.

[0140] Taking a two-row car as an example, the sound loss rate of a single rear window is between (0, 0.15]; the sound loss rate of a single front window is between (0, 0.25]; and the sound loss rate of a sunroof is between (0, 0.2].

[0141] In this embodiment, the response volume of the voice assistant is adjusted based on the difference between the location parameters and the location parameters corresponding to the preset default installation area.

[0142] The beneficial effects of the above technical solution are as follows: the present invention automatically determines the position parameters of the translation device through the test data of the intelligent testing sub-unit, then realizes the detection of the noise data of the use environment through the intelligent monitoring sub-unit, and finally makes different sound control strategies according to the network status of the translation device through the intelligent control sub-unit, thereby effectively improving the intelligence level of the translation device, so that the translation device supports both offline and networked modes of operation, and can complete the voice translation task even in the absence of a network.

[0143] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A translation device for an intelligent vehicle system, characterized in that: include: service data terminal, microphone, and speaker; The service data terminal is electrically connected to the microphone and the speaker respectively; The microphone is used to collect real-time voice in the vehicle and send the real-time voice to the service data terminal; Among them, real-time voice includes the driver's voice and the vehicle voice assistant's response voice; The service data terminal recognizes and translates the real-time voice, generates translated voice, and sends it to the speaker for playback; The translated speech includes the target language control speech and the translated response speech; The speaker plays the target language control voice or the translated response voice to the vehicle voice assistant at the target playback volume; The target playback volume of the speaker is automatically adjusted based on the position parameters of the current microphone and the current ambient noise data. The specific method includes: After the translation device is installed, the automatic test voice sends location confirmation instructions and volume control instructions to the vehicle voice assistant. Based on the test response voice of the vehicle voice assistant, the relative position of the translation device and the vehicle console is determined; Determining the relative distance between the translation device and the vehicle console based on sound energy changes of the multiple test response voices received by the translation device; Based on the relative orientation and relative distance, combined with the vehicle interior space model, determining the installation position of the translation device, and generating position parameters of the translation device and sending them to the database for storage; The built-in sound sensor of the translation device obtains the ambient noise data of the current vehicle environment; Determining whether the current translation device is connected to the Internet; if not, determining the current environmental interference level based on the current environmental noise level, and determining the playback volume corresponding to the current environmental interference level based on a preset offline control table, and setting the playback volume as the target playback volume; When the current translation device is connected to the Internet, the noise environment data and the location parameters of the transfer device are sent to the cloud; Comparing the location parameters of the translation device with the corresponding location parameters of the default installation area to obtain an installation location difference rate, obtaining an interference-free playback volume based on the default volume corresponding to the default installation area and the location installation difference rate, and comparing the vehicle isolation noise threshold with the current noise threshold to obtain a noise difference rate; Based on the vehicle data received from the cloud, determine whether the current vehicle has a window open behavior; If not, the undisturbed playback volume is corrected based on the noise difference rate to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume; If it exists, then determine the window position and size of the current vehicle based on the vehicle data, combine them with the position parameters of the translation device, and judge the degree of influence of each window position on the sound played by the translation device; Based on the influence degree of each window position, the sound loss rate is obtained. Based on the sound loss rate and the noise difference rate, the interference-free playback volume is corrected to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume.

2. The translation device of the intelligent vehicle system according to claim 1, characterized in that: The service data terminal is equipped with two three-color indicator lights to display the device status; Among them, the red light means that the device is starting or abnormal; The green light indicates that the device is operating normally and the network status is good; A yellow light indicates the device is offline or waiting to connect.

3. A control system for a translation device of an intelligent vehicle system, characterized in that: include: The voice collection module is used to control the microphone to collect real-time voice inside the vehicle when confirming that the vehicle is in the starting state; The speech recognition module is used to control the service data terminal. It uses NLP technology to perform language and semantic recognition on the received real-time speech, determine whether there are vehicle control wake-up keywords in the real-time speech, and obtain the speech to be translated; The speech translation module is used to perform semantic recognition based on the speech to be translated, obtain the translated text, and generate the target language control speech based on the preset vehicle control speech template; The voice interaction module is used to control the service data terminal to translate the response voice of the vehicle voice assistant to obtain the translated response voice; The playback control module is used to control the loudspeaker to play the target language control voice or the translated response voice; The playback control module includes: The first playback unit is used to control the speaker to play the target language control voice to the vehicle voice assistant according to the target playback volume; The second playback unit is used to control the speaker to play the translated response voice to the vehicle voice assistant according to the target playback volume; A playback control unit, configured to automatically adjust the target playback volume of the speaker based on the current microphone position parameters and the current ambient noise data; The playback control unit includes: An intelligent test subunit, configured to, after the translation device is installed, automatically test the voice to send location confirmation instructions and volume control instructions to the vehicle voice assistant, and determine the relative position of the translation device and the vehicle console based on the test response voice of the vehicle voice assistant; Determining the relative distance between the translation device and the vehicle console based on sound energy changes of the multiple test response voices received by the translation device; Based on the relative orientation and relative distance, combined with the vehicle interior space model, determining the installation position of the translation device, and generating position parameters of the translation device and sending them to the database for storage; An intelligent monitoring subunit, which interprets the device's built-in sound sensor to obtain ambient noise data of the current vehicle interior; An intelligent control subunit is configured to determine whether the current translation device is connected to the Internet. If not, the intelligent control subunit determines the current environmental interference level based on the current environmental noise level, and determines the playback volume corresponding to the current environmental interference level based on a preset offline control table, and uses the playback volume as the target playback volume. When the current translation device is connected to the Internet, the noise environment data and the location parameters of the transfer device are sent to the cloud; Comparing the location parameters of the translation device with the corresponding location parameters of the default installation area to obtain an installation location difference rate, obtaining an interference-free playback volume based on the default volume corresponding to the default installation area and the location installation difference rate, and comparing the vehicle isolation noise threshold with the current noise threshold to obtain a noise difference rate; Based on the vehicle data received from the cloud, determine whether the current vehicle has a window open behavior; If not, the undisturbed playback volume is corrected based on the noise difference rate to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume; If it exists, then determine the window position and size of the current vehicle based on the vehicle data, combine them with the position parameters of the translation device, and judge the degree of influence of each window position on the sound played by the translation device; Based on the influence degree of each window position, the sound loss rate is obtained. Based on the sound loss rate and the noise difference rate, the interference-free playback volume is corrected to obtain the optimal playback volume, and the optimal playback volume is used as the target playback volume.

4. The control system of the translation device of the intelligent vehicle system according to claim 3, characterized in that: Voice acquisition module, including: The vehicle status detection unit is used to control the automatic start of the translation device after the door is opened, and monitor the sound inside the vehicle. When the sound of the vehicle engine starting is detected, it determines that the current vehicle is in the starting state; The collection control unit is used to generate a human voice collection control instruction after confirming that the vehicle is started, and send it to the human voice collection unit; The voice collection unit is used to control the microphone array to switch to the voice collection mode after receiving the voice control command, and collect real-time voice inside the vehicle.

5. The control system of the translation device of the intelligent vehicle system according to claim 3, characterized in that: Speech recognition module, including: The real-time language recognition unit is used to input the real-time speech received by the service data terminal into the speech recognition center. The speech recognition center is equipped with a multi-language language recognition model to recognize the language of the input real-time speech and determine the current language; a speech comparison unit, configured to obtain a preset wake-up speech of a corresponding language from a preset wake-up speech database based on the current language, determine whether the current language is a non-target language, and if so, compare the real-time speech with the preset wake-up speech to determine whether a vehicle control wake-up keyword exists in the real-time speech; if so, determine that the real-time speech is a vehicle wake-up speech, and use the vehicle wake-up speech as the starting speech for human-computer interaction and the sentence to be translated; At the same time, a temporary timing clock is added to the starting voice of human-computer interaction; A real-time semantic recognition unit, configured to recognize the initial speech of the human-computer interaction based on a multilingual semantic recognition model set by the speech recognition center, and obtain the semantics of the driver and passenger; a command voice identification unit, configured to identify a first real-time voice of the driver or passenger based on a multilingual semantic recognition model before the temporary timer reaches a preset time; if the first real-time voice is a control voice of the current vehicle, delete the current temporary timer, use the control voice as the voice to be translated, and generate a new temporary timer based on the latest control voice of the current vehicle; If the multilingual semantic recognition model does not detect any control voice before the temporary timing clock reaches the preset time, the temporary timing clock is deleted when the temporary timing clock reaches the preset time, and it is determined that the current round of wake-up control is completed.

6. The control system of the translation device of the intelligent vehicle system according to claim 3, characterized in that: Speech translation module, including: The speech translation unit is used to generate the translated text based on the semantic recognition results corresponding to the speech to be translated and the language logic corresponding to the target language; A text standardization unit is used to determine the driver's control purpose based on the speech recognition results, obtain a corresponding speech control template based on the control purpose, and standardize the translated text based on the speech control instruction module to obtain the control instruction text in the target language; The text conversion unit is used to convert the target language control instruction text into the target language control speech based on the preset timbre and the intonation of the speech to be translated.

7. The control system of the translation device of the intelligent vehicle system according to claim 6, characterized in that: Voice interaction module, including: A response recognition unit is used to perform semantic recognition on the real-time speech corresponding to the vehicle voice assistant to obtain a response speech recognition result; The interactive translation unit is used to generate a translated response text corresponding to the current language based on the response voice recognition result and the language logic corresponding to the current language, and to generate a translated response voice based on the text conversion unit.

Images