Vehicle machine voice recognition method and device, electronic equipment and vehicle

Abstract

The invention relates to a vehicle voice recognition method and device, electronic equipment and a vehicle, and relates to the technical field of data processing, and the method comprises the steps: obtaining first feature information corresponding to user voice information, and the first feature information comprises audio information and corresponding recognition text information; acquiring second feature information in a current page of a vehicle-mounted terminal screen, wherein the second feature information comprises vehicle-mounted terminal text information and / or vehicle-mounted terminal picture information in the current page; inputting the first feature information and the second feature information into a recognition model for matching calculation to obtain target text information and / or target picture information in the current page; and starting a vehicle control corresponding to the target text information and / or the target picture information. By applying the technical scheme of the invention, the intention of the user for controlling the screen of the vehicle-mounted terminal can be accurately identified by using multiple modes such as audio, pictures and texts, and the voice request of the user can be timely responded to start the corresponding vehicle-mounted terminal control.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a vehicle-mounted voice recognition method, device, electronic equipment and vehicle. Background Art

[0002] With the rapid development of artificial intelligence, artificial intelligence is increasingly being used in the automotive field. As a form of artificial intelligence, voice recognition is also increasingly being used in car computers, providing users with a more convenient way to control car computers.

[0003] At present, when the car computer receives user voice information, it obtains all the pictures in the page of the car computer screen, and uses optical character recognition (OCR) to recognize the text in the picture, and then matches the text on these pictures with the user request text obtained by automatic speech recognition (ASR), and finally selects the picture corresponding to the OCR-recognized text with the highest similarity to the user request text, and then clicks the car computer control corresponding to the picture.

[0004] However, when the page of the vehicle computer screen contains not only pictures but also text descriptions, the existing solution cannot identify the vehicle computer control corresponding to the text description. Summary of the invention

[0005] In view of this, the present application provides a vehicle-computer voice recognition method, device, electronic device and vehicle, the main purpose of which is to improve the current technical problem that when the page of the vehicle-computer screen contains not only pictures but also text descriptions, the existing scheme cannot recognize the vehicle-computer control corresponding to the text description.

[0006] In a first aspect, the present application provides a vehicle machine speech recognition method, the method comprising:

[0007] Acquire first feature information corresponding to the user voice information, where the first feature information includes audio information and corresponding recognition text information;

[0008] Acquire second characteristic information in the current page of the vehicle screen, where the second characteristic information includes vehicle text information and / or vehicle picture information in the current page;

[0009] Inputting the first feature information and the second feature information into a recognition model to perform matching calculation, and obtaining target text information and / or target image information in the current page;

[0010] Start the vehicle computer control corresponding to the target text information and / or the target image information.

[0011] In a second aspect, the present application provides a vehicle-mounted voice recognition device, comprising:

[0012] The acquisition module is configured to acquire first feature information corresponding to the user voice information, wherein the first feature information includes audio information and corresponding recognized text information; acquire second feature information in the current page of the vehicle screen, wherein the second feature information includes vehicle text information and / or vehicle picture information in the current page;

[0013] A matching module is configured to input the first feature information and the second feature information into a recognition model to perform a matching calculation, and obtain target text information and / or target image information in the current page;

[0014] The start module is configured to start the vehicle computer control corresponding to the target text information and / or the target image information.

[0015] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the vehicle-mounted voice recognition method described in the first aspect is implemented.

[0016] In a fourth aspect, the present application provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the vehicle-machine voice recognition method described in the first aspect when executing the computer program.

[0017] In a fifth aspect, the present application provides a vehicle, comprising: the device as described in the second aspect or the electronic device as described in the fourth aspect.

[0018] By means of the above technical scheme, the present application provides a vehicle-machine voice recognition method, device, electronic device and vehicle. Compared with the current existing technology, the present application can recognize and activate the vehicle-machine control corresponding to the picture and / or text in the page when there are not only pictures but also text descriptions in the page. Specifically, firstly, the first feature information corresponding to the user's voice information is obtained, and the first feature information includes the audio information of the user's voice information and the corresponding recognition text information; and the second feature information in the current page of the vehicle-machine screen is obtained, and the second feature information includes the vehicle-machine text information and / or vehicle-machine picture information in the current page; then the first feature information and the second feature information are input into the recognition model for matching calculation to obtain the target text information and / or target picture information in the current page; and then the vehicle-machine control corresponding to the target text information and / or target picture information is activated. By applying the technical scheme of the present application, it is possible to accurately identify the user's intention to control the vehicle-machine screen by using multiple modes such as audio, pictures and text, and can promptly respond to the user's voice request to activate the corresponding vehicle-machine control, specifically including the vehicle-machine control corresponding to the picture and / or text description, etc., so as to meet more user needs, thereby effectively improving the user's use experience.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A schematic diagram of a process flow of a vehicle machine voice recognition method provided by an embodiment of the present disclosure is shown;

[0023] Figure 2 A schematic diagram showing a flow chart of another vehicle machine voice recognition method provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram showing a model example provided by an embodiment of the present disclosure is shown;

[0025] Figure 4A structural schematic diagram of a vehicle-mounted voice recognition device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0027] This embodiment provides a vehicle machine voice recognition method, such as Figure 1 As shown, the method includes:

[0028] Step 101: Obtain first feature information corresponding to user voice information.

[0029] The execution subject of this embodiment may be a device or equipment for vehicle machine voice recognition, which may be configured on the vehicle machine end or the server end, and the server end may be a local vehicle server end or a cloud end, etc. The execution subject of this embodiment may realize the vehicle's see-and-speak function by executing the process shown in steps 101 to 104, that is, for visible and clickable controls on the screen, such as buttons, playlists, videos, etc., the user may make a voice request for the text or picture information displayed on the page, and the vehicle machine end may click the corresponding control after recognition.

[0030] The first feature information may include: audio information and corresponding recognition text information.

[0031] When the user needs to perform voice control on the car screen and clicks on the car control, a corresponding voice can be issued. In this embodiment, the voice request signal issued by the user can be received through a microphone device arranged in the car to obtain the user's voice information, which can be a piece of audio information. In this embodiment, based on the ASR technology, the audio information can be voice recognized to obtain the corresponding recognized text information. For example, the ASR technology can be used to recognize the user's words in the audio information as a string of text or the like.

[0032] Step 102: Obtain the second characteristic information in the current page of the vehicle screen.

[0033] The second characteristic information includes: the vehicle machine text information and / or vehicle machine picture information in the current page of the vehicle machine screen. The vehicle machine text information and / or vehicle machine picture information may be the text information and / or picture information corresponding to the vehicle machine APP, such as pictures and texts in the vehicle machine APP. Among them, the vehicle machine picture information may include the text and / or image in the vehicle machine picture.

[0034] For example, when there is a playlist on the car machine page and the playlist text is described as "hot songs". In addition, it can also include recommended pictures of hot singers.

[0035] Step 103: input the first feature information and the second feature information into a recognition model to perform matching calculations to obtain target text information and / or target image information in the current page.

[0036] In this embodiment, a recognition model can be pre-trained using a machine learning algorithm based on training sample data. The recognition model can be used to calculate the matching degree between the first feature information and the second feature information, and then determine the text and / or picture that the user needs to click on the current page, that is, the target text information and / or target picture information, based on the matching degree.

[0037] For example, by calculating the recognition model, the audio information and the recognized text information can be matched with the vehicle text information and / or vehicle picture information in the current page to determine the text and / or picture that the user needs to click on in the current page.

[0038] It should be noted that the recognition model may include a speech recognition module, such as an ASR module. In addition to inputting the recognition text information corresponding to the audio information into the recognition model, this embodiment also inputs the audio information into the recognition model for matching calculations. This can effectively avoid the situation where the final visible execution error is caused by the early speech recognition error.

[0039] Step 104: Start the vehicle computer control corresponding to the target text information and / or the target image information.

[0040] For example, when there is a playlist in the current page of the car screen and the playlist text is described as "hot songs", when the user speaks the voice information of "hot songs" or "open hot songs", the microphone device of the vehicle can receive the voice request signal sent by the user and obtain the user voice information, which can be a piece of audio information. Based on ASR technology, voice recognition can be performed on the audio information to obtain recognition text information containing the content of "hot songs" or "open hot songs". And the car text information and / or car picture information in the current page can be obtained. Then the audio information, recognition text information, and the car text information and / or car picture information are input into the recognition model for matching calculation. According to the matching degree, the "hot songs" that the user needs to click on the current page are determined, and then the song list corresponding to the "hot songs" can be executed.

[0041] For another example, when there is a video on the current page of the car screen and there is a little bear in the video picture, and the user says the voice information of "I want to see the little bear", the microphone device of the vehicle can receive the voice request signal sent by the user and obtain the user voice information, which can be specifically a piece of audio information. Based on the ASR technology, the audio information can be voice recognized to obtain the recognized text information containing the content of "I want to see the little bear". And the car text information and / or car picture information in the current page can be obtained. Then the audio information, the recognized text information, and the car text information and / or car picture information are input into the recognition model for matching calculation. According to the matching degree, it is determined that the user needs to click on the picture with the little bear in the current video picture, and then the video with the little bear in the picture can be clicked.

[0042] Compared with the current existing technology, this embodiment can identify and activate the vehicle control corresponding to the picture and / or text in the page when the page contains not only pictures but also text descriptions. By applying the technical solution of this application, it is possible to accurately identify the user's intention to control the vehicle screen using multiple modes such as audio, pictures and text, and to promptly respond to the user's voice request to activate the corresponding vehicle control, specifically including the vehicle control corresponding to the picture and / or text description, etc., thereby meeting more user needs and effectively improving the user experience.

[0043] Further, as a refinement and extension of the above embodiment, in order to fully illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following Figure 2 The specific method shown comprises the following steps:

[0044] Step 201: Obtain training sample data.

[0045] The training sample data includes: sample text information and / or sample picture information corresponding to different car machine APPs, sample audio information used to describe the content of the sample text information and / or sample picture information, and sample recognition text information corresponding to the sample audio information.

[0046] For example, in order to build a recognition model, it is necessary to obtain training sample data, such as obtaining graphic information such as pictures and texts in various APPs in the car computer as sample pictures, sample texts, etc. For each graphic information, 10, 15 or more audios with different statements are recorded to describe the content in the graphic, such as: intercepting part of the text string, the text in the picture, the image in the picture, etc., as the corresponding sample audio; and annotate the sample audio with its corresponding sample recognition text. The obtained data have a corresponding relationship, such as sample picture a1, corresponding to sample audio b1, sample recognition text c1; sample picture a2, corresponding to sample audio b2, sample recognition text c2, etc.

[0047] Step 202: Create a training set and a test set based on the training sample data, and use the training set and the test set to train a recognition model.

[0048] This embodiment may adopt multiple iterative training of the model, wherein the training set can be used for model training, and the test set can be used to detect whether the recognition model meets the standard after each iterative training. After multiple rounds of model training, a recognition model that meets the standard can be obtained.

[0049] For example, in the present embodiment, multimodal information such as text and pictures can be added based on the ASR model to construct a model such as Figure 3 The model shown. The ASR module for audio data is a classic Conformer structure; the two text feature extractors (such as the first text feature extractor and the second text feature extractor) for the audio corresponding recognition text and the APP corresponding sample text are both Bert structures; the image feature extractor for the audio corresponding sample image is a ResNet structure, and the final arbitration module is a Linear+sigmoid structure, forming a multimodal binary classification model.

[0050] During the training process, a recorded audio and its corresponding text and image are used, and other texts and images are added as sub-samples; the matching degree of the text and image corresponding to the audio is 1, and the matching degree of other texts and images with the audio is 0; the audio is sent to the ASR module, its annotation is sent to text encoder1, the text obtained from the page (including the text and sub-sample corresponding to the audio) is sent to text encoder2, and the image obtained from the page (including the image and sub-sample corresponding to the audio) is sent to the vision encoder. Finally, the training annotation of the arbitration module (predictor) is the matching degree of the audio and each text and image (the corresponding text and image of the audio is 1, and the non-corresponding text and image is 0).

[0051] Step 203: When receiving user voice information, obtain first feature information corresponding to the user voice information, and obtain second feature information in the current page of the vehicle screen.

[0052] The first characteristic information includes audio information and corresponding recognition text information, and the second characteristic information includes vehicle text information and / or vehicle picture information in the current page of the vehicle screen.

[0053] Step 204: input the first feature information and the second feature information into a recognition model to perform matching calculations to obtain target text information and / or target image information in the current page.

[0054] In some embodiments, during the matching calculation process of the recognition model, the correspondence between the sample features will be referenced to determine the degree of matching between the current features. For example, the correspondence between the first sample feature information and the second sample feature information will be referenced to match the first feature information and the second feature information, wherein the first sample feature information includes: sample audio information and sample recognition text information, and the second sample feature information includes: sample text information and / or sample picture information; then, the vehicle-machine text information and / or vehicle-machine picture information whose matching degree with the audio information and the recognition text information is greater than a preset threshold (which can be pre-set according to actual needs) is obtained as the target text information and / or target picture information, that is, the text and / or picture that the user needs to click on the current page.

[0055] In some examples, the above-mentioned matching of the first feature information and the second feature information includes: obtaining an audio vector representation corresponding to the audio information, and a first text vector representation corresponding to the recognition text information, and a second text vector representation corresponding to the vehicle computer text information, and a picture vector representation corresponding to the vehicle computer picture information; inputting the audio vector representation, the first text vector representation, and the second text vector representation and / or the picture vector representation into the arbitration module of the recognition model for matching, and obtaining the matching degree between the audio vector representation and the first text vector representation, and the second text vector representation and / or the picture vector representation.

[0056] Optionally, the above-mentioned acquisition of the audio vector representation corresponding to the audio information, the first text vector representation corresponding to the recognition text information, the second text vector representation corresponding to the vehicle computer text information, and the picture vector representation corresponding to the vehicle computer picture information may specifically include: extracting the audio information to the corresponding audio vector representation through the ASR module (Conformer structure) of the recognition model; and, extracting the recognition text information to the corresponding first text vector representation through the first text feature extractor (Bert structure) of the recognition model; and, extracting the vehicle computer text information to the corresponding second text vector representation through the second text feature extractor (Bert structure) of the recognition model; and, extracting the vehicle computer picture information to the corresponding picture vector representation through the picture feature extractor (ResNet structure) of the recognition model.

[0057] Optionally, the above-mentioned acquisition of vehicle-computer text information and / or vehicle-computer picture information having a matching degree greater than a preset threshold with the audio information and the recognized text information as target text information and / or target picture information may specifically include: acquiring a second text vector representation and / or picture vector representation having a matching degree greater than a preset threshold with the audio vector representation and the first text vector representation; determining the vehicle-computer text information corresponding to the second text vector representation having a matching degree greater than the preset threshold as the target text information; and / or determining the vehicle-computer picture information corresponding to the picture vector representation having a matching degree greater than a preset threshold as the target picture information.

[0058] For example, when the user needs to perform voice control on the car screen and click on the car control, a corresponding voice can be issued. In this embodiment, the voice request signal issued by the user can be received by the microphone device arranged in the car, and sent to the ASR module of the recognition model to obtain the audio vector representation speech embedding; the text recognized by the ASR module is sent to textencoder 1 to obtain the vector representation text embedding1 of the recognized text; after the user request is completed, all the texts and pictures on the current page of the car are obtained, and each text on the car page is sent to text encoder 2 to obtain the vector representation text embedding2 of each text on the car page, and each string of text is obtained. Each picture on the car page is sent to vision encoder to obtain the vector representation vision encoding of each picture on the car page, and each picture is obtained. A vector representation; the user request representation speech embedding and text embedding1 and the representation text embedding2 of multiple texts on the page and the representation vision embedding of multiple pictures on the page are sent to the arbitration module (predictor) to obtain the matching degree between the user request and each text and each picture. Then, the text and / or picture that the user needs to click on in the current page, ie, the target text information and / or the target picture information, can be determined based on the matching degree.

[0059] In some embodiments, the above-mentioned acquisition of vehicle-machine text information and / or vehicle-machine picture information whose matching degree with the audio information and the recognized text information is greater than a preset threshold as the target text information and / or target picture information may specifically include: if there are multiple vehicle-machine text information and / or vehicle-machine picture information whose matching degree is greater than a preset threshold, then acquiring the vehicle-machine text information and / or vehicle-machine picture information with the greatest matching degree from the multiple vehicle-machine text information and / or vehicle-machine picture information as the target text information and / or target picture information; if there is no vehicle-machine text information and / or vehicle-machine picture information whose matching degree is greater than the preset threshold, then sending the recognized text information corresponding to the user voice information to the semantic recognition model, such as performing semantic recognition on the recognized text information corresponding to the user voice information, and determining the target control according to the semantic recognition result, and the target control can be used to control the device function in the vehicle.

[0060] For example, if the matching degree between the user's voice request and a text or picture on the current page is greater than a threshold, the car computer control corresponding to the text or picture is clicked; if there are more than one matching degree greater than the threshold, the text or picture with the highest matching degree is obtained; if all matching degrees are less than the threshold, the ASR recognition result is sent to the semantic model for routine understanding operations, such as turning on the lights, closing the windows, etc. according to the user's voice control instructions.

[0061] Step 205: Start the vehicle computer control corresponding to the target text information and / or the target image information.

[0062] In some embodiments, after obtaining the recognition text information corresponding to the user voice information, the method of this embodiment may further include: outputting the recognition text information so that the user can understand whether the content of the voice recognition is correct, and if the recognition is wrong, the user can speak the voice information again; if the corrected user voice information is received, the process shown in steps 203 to 205 is performed again according to the corrected user voice information. In this way, it is convenient for the user to correct the voice control instructions he said in time, which can improve the user experience.

[0063] Compared with the current existing technology, this embodiment can identify and activate the vehicle control corresponding to the picture and / or text in the page when the page contains not only pictures but also text descriptions. By applying the technical solution of this application, it is possible to accurately identify the user's intention to control the vehicle screen using multiple modes such as audio, pictures and text, and to promptly respond to the user's voice request to activate the corresponding vehicle control, specifically including the vehicle control corresponding to the picture and / or text description, etc., thereby meeting more user needs and effectively improving the user experience.

[0064] Further, as Figure 1 and Figure 2The specific implementation of the method shown in the embodiment provides a vehicle-mounted voice recognition device, such as Figure 4 As shown, the device includes: an acquisition module 31, a matching module 32, and a starting module 33.

[0065] The acquisition module 31 is configured to acquire first feature information corresponding to the user voice information, wherein the first feature information includes audio information and corresponding recognized text information; acquire second feature information in the current page of the vehicle screen, wherein the second feature information includes vehicle text information and / or vehicle picture information in the current page;

[0066] A matching module 32 is configured to input the first feature information and the second feature information into a recognition model to perform a matching calculation, and obtain target text information and / or target image information in the current page;

[0067] The starting module 33 is configured to start the vehicle computer control corresponding to the target text information and / or the target image information.

[0068] In some embodiments, the acquisition module 31 is also configured to acquire training sample data, wherein the training sample data includes: sample text information and / or sample picture information corresponding to different vehicle-mounted APPs, and sample audio information used to describe the content of the sample text information and / or sample picture information, and sample recognition text information corresponding to the sample audio information; based on the training sample data, create a training set and a test set; and use the training set and the test set to train and obtain the recognition model.

[0069] In some embodiments, the matching module 32 is specifically configured to match the first feature information and the second feature information with reference to the correspondence between the first sample feature information and the second sample feature information, wherein the first sample feature information includes: sample audio information and sample recognition text information, and the second sample feature information includes: sample text information and / or sample picture information; obtain the vehicle-machine text information and / or vehicle-machine picture information whose matching degree with the audio information and the recognition text information is greater than a preset threshold as the target text information and / or the target picture information.

[0070] In some embodiments, the matching module 32 is further configured to obtain the vehicle-machine text information and / or vehicle-machine picture information with the greatest matching degree from the multiple vehicle-machine text information and / or vehicle-machine picture information as the target text information and / or the target picture information if there are multiple vehicle-machine text information and / or vehicle-machine picture information with a matching degree greater than a preset threshold; if there is no vehicle-machine text information and / or vehicle-machine picture information with a matching degree greater than the preset threshold, perform semantic recognition on the recognized text information corresponding to the user voice information, and determine the target control based on the semantic recognition result.

[0071] In some embodiments, the matching module 32 is further configured to obtain an audio vector representation corresponding to the audio information, a first text vector representation corresponding to the recognized text information, a second text vector representation corresponding to the vehicle text information, and a picture vector representation corresponding to the vehicle picture information; the audio vector representation, the first text vector representation, and the second text vector representation and / or picture vector representation are input into the arbitration module of the recognition model for matching, and the matching degree between the audio vector representation and the first text vector representation and the second text vector representation and / or picture vector representation is obtained.

[0072] The matching module 32 is further specifically configured to obtain a second text vector representation and / or a picture vector representation whose matching degree with the audio vector representation and the first text vector representation is greater than a preset threshold; determine the vehicle-computer text information corresponding to the second text vector representation whose matching degree is greater than the preset threshold as the target text information; and / or determine the vehicle-computer picture information corresponding to the picture vector representation whose matching degree is greater than the preset threshold as the target picture information.

[0073] In some embodiments, the matching module 32 is further configured to extract the audio information into a corresponding audio vector representation through the ASR module of the recognition model; and, extract the recognized text information into a corresponding first text vector representation through the first text feature extractor of the recognition model; and, extract the vehicle-computer text information into a corresponding second text vector representation through the second text feature extractor of the recognition model; and, extract the vehicle-computer picture information into a corresponding picture vector representation through the picture feature extractor of the recognition model.

[0074] In some embodiments, the startup module 33 is also configured to output the recognition text information after acquiring the recognition text information corresponding to the user voice information; if corrected user voice information is received, the vehicle-computer voice recognition method is executed again according to the corrected user voice information.

[0075] It should be noted that for other corresponding descriptions of the functional units involved in the vehicle machine voice recognition device provided in this embodiment, reference can be made to Figure 1 and Figure 2 The corresponding description in will not be repeated here.

[0076] Based on the above Figure 1 and Figure 2 The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 The method shown.

[0077] Based on this understanding, the technical solution of this embodiment can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each implementation scenario of this embodiment.

[0078] Based on the above Figure 1 and Figure 2 The method shown, and Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present disclosure also provides an electronic device, which can be configured on the vehicle (such as a new energy vehicle or a traditional vehicle) side, and the device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 2 The method shown.

[0079] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0080] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0081] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0082] Based on the above electronic device, the embodiment of the present disclosure further provides a vehicle, which may specifically include: Figure 4 The vehicle voice recognition device shown, or the above electronic device. The vehicle can be a new energy vehicle or a traditional vehicle.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the solution of the embodiment of the present disclosure, multimodal information of audio, text, and pictures can be used to determine whether the user's voice request requires clicking on the content on the page. By applying the technical solution of this embodiment, it is possible to use multimodal information such as audio, pictures, and text to accurately identify the user's intention to control the car screen, and to respond to the user's voice request in a timely manner to start the corresponding car controls, including car controls corresponding to pictures and / or text descriptions, etc., thereby meeting more user needs, thereby effectively improving the user experience.

[0084] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0085] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

Claims

1. A vehicle machine voice recognition method, characterized in that: The method comprises: Acquire first feature information corresponding to the user voice information, where the first feature information includes audio information and corresponding recognition text information; Acquire second characteristic information in the current page of the vehicle screen, where the second characteristic information includes vehicle text information and / or vehicle picture information in the current page; Inputting the first feature information and the second feature information into a recognition model for matching calculation to obtain target text information and / or target image information in the current page; Start the vehicle computer control corresponding to the target text information and / or the target image information.

2. The method according to claim 1, characterized in that The process of creating the recognition model includes: Acquire training sample data, the training sample data including: sample text information and / or sample picture information corresponding to different vehicle machine APPs, sample audio information used to describe the content in the sample text information and / or sample picture information, and sample recognition text information corresponding to the sample audio information; Based on the training sample data, create a training set and a test set; The recognition model is trained using the training set and the test set.

3. The method according to claim 2, characterized in that The step of inputting the first feature information and the second feature information into a recognition model for performing matching calculation to obtain target text information and / or target image information in the current page includes: Referring to the correspondence between the first sample feature information and the second sample feature information, the first feature information and the second feature information are matched, wherein the first sample feature information includes: sample audio information and sample identification text information, and the second sample feature information includes: sample text information and / or sample picture information; The vehicle computer text information and / or vehicle computer picture information whose matching degree with the audio information and the recognized text information is greater than a preset threshold is obtained as the target text information and / or the target picture information.

4. The method according to claim 3, characterized in that The acquiring, as the target text information and / or the target image information, the vehicle computer text information and / or the vehicle computer image information whose matching degree with the audio information and the recognized text information is greater than a preset threshold, includes: If there are multiple vehicle-machine text information and / or vehicle-machine picture information with a matching degree greater than a preset threshold, the vehicle-machine text information and / or vehicle-machine picture information with the greatest matching degree is obtained from the multiple vehicle-machine text information and / or vehicle-machine picture information as the target text information and / or the target picture information; If there is no vehicle computer text information and / or vehicle computer picture information with a matching degree greater than a preset threshold, semantic recognition is performed on the recognized text information corresponding to the user voice information, and the target control is determined according to the semantic recognition result.

5. The method according to claim 3, characterized in that: Matching the first characteristic information with the second characteristic information includes: Obtaining an audio vector representation corresponding to the audio information, a first text vector representation corresponding to the recognized text information, a second text vector representation corresponding to the vehicle-computer text information, and a picture vector representation corresponding to the vehicle-computer picture information; Inputting the audio vector representation, the first text vector representation, and the second text vector representation and / or the picture vector representation into the arbitration module of the recognition model for matching, and obtaining the matching degree between the audio vector representation and the first text vector representation, and the second text vector representation and / or the picture vector representation; The acquiring, as the target text information and / or the target image information, the vehicle computer text information and / or the vehicle computer image information whose matching degree with the audio information and the recognized text information is greater than a preset threshold, includes: Acquire a second text vector representation and / or picture vector representation whose matching degree with the audio vector representation and the first text vector representation is greater than a preset threshold; The vehicle machine text information corresponding to the second text vector representation whose matching degree is greater than a preset threshold is determined as the target text information; and / or, The vehicle computer picture information corresponding to the picture vector whose matching degree is greater than a preset threshold is determined as the target picture information.

6. The method according to claim 5, characterized in that The obtaining of the audio vector representation corresponding to the audio information, the first text vector representation corresponding to the recognized text information, the second text vector representation corresponding to the vehicle computer text information, and the picture vector representation corresponding to the vehicle computer picture information includes: Extracting the audio information into a corresponding audio vector representation through the ASR module of the recognition model; and Extracting the recognized text information into a corresponding first text vector representation through a first text feature extractor of the recognition model; and Extracting the vehicle machine text information into a corresponding second text vector representation through the second text feature extractor of the recognition model; and, The vehicle computer image information is extracted into a corresponding image vector representation through the image feature extractor of the recognition model.

7. The method according to any one of claims 1 to 6, characterized in that After obtaining the recognized text information corresponding to the user voice information, the method further includes: Outputting the identified text information; If the corrected user voice information is received, the vehicle machine voice recognition method is executed again according to the corrected user voice information.

8. A vehicle-mounted voice recognition device, characterized in that: The device comprises: The acquisition module is configured to acquire first feature information corresponding to the user voice information, wherein the first feature information includes audio information and corresponding recognized text information; acquire second feature information in the current page of the vehicle screen, wherein the second feature information includes vehicle text information and / or vehicle picture information in the current page; A matching module is configured to input the first feature information and the second feature information into a recognition model to perform a matching calculation, and obtain target text information and / or target image information in the current page; The start module is configured to start the vehicle computer control corresponding to the target text information and / or the target image information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: include: The device as claimed in claim 8, or the electronic device as claimed in claim 10.

Images