Voice system wake-up method and device, equipment, medium and program product

Abstract

The invention discloses an awakening method, device and equipment of a voice system, a medium and a program product, and relates to the technical field of vehicles. The method comprises the steps of firstly obtaining user driving information of a target user, then identifying the user driving information, and determining current driving behavior characteristics of the target user; and inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result. And finally, generating a voice response instruction under the condition that the intention detection result is recognized to be that the voice wake-up intention exists. Therefore, the vehicle-mounted voice assistant can directly enter the voice interaction mode under the condition of predicting that the target user has the voice wake-up intention according to the driving behavior characteristics determined by the multi-modal data. Therefore, the wake-up modes of the vehicle-mounted voice system can be enriched, and the wake-up accuracy of the voice system in a loud-noise environment can be improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a method, device, equipment, medium and program product for waking up a voice system. Background Art

[0002] Currently, the voice system installed in the vehicle's central control device generally uses a wake-up method of waking up the system by touching the voice wake-up control, or waking up the system through a fixed voice wake-up word.

[0003] The above wake-up method is relatively fixed, which reduces the user experience of the voice assistant. In addition, when the vehicle-mounted equipment is in a noisy environment, such as a complex driving environment that is not conducive to voice recognition, such as a large number of people in the car, heavy rainfall outside the car, etc., it is easy to miss the wake-up through fixed voice wake-up words, thereby reducing the wake-up accuracy of the voice system. Summary of the invention

[0004] In view of the technical problems that the current vehicle-mounted voice wake-up system has a fixed wake-up method and low wake-up accuracy in a complex and changeable driving environment, the present invention is proposed to provide a voice system wake-up method, device, equipment, medium and program product that overcome the above problems or at least partially solve the above problems.

[0005] Based on the first aspect of the present invention, a method for waking up a voice system is provided, the method comprising:

[0006] Acquire user driving information of the target user, wherein the user driving information includes at least travel information, an image of the user in the car, and a voice operation record;

[0007] Performing information analysis on the user driving information to determine driving behavior characteristics associated with the user driving information;

[0008] Inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result;

[0009] When it is identified that the intention detection result is a voice wake-up intention, a voice response instruction is generated and sent to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters a voice interaction mode to wait for the detection function instruction.

[0010] In an optional invention, the step of analyzing the user driving information to determine the driving behavior characteristics associated with the user driving information includes:

[0011] Inputting the itinerary information into the itinerary analysis model to perform itinerary identification, and determining the destination features corresponding to the current itinerary of the target user;

[0012] Inputting the in-vehicle user image into a motion analysis model for motion recognition to determine the motion features of the target user;

[0013] Inputting the voice operation record into a record analysis model for record recognition to determine the preferred wake-up feature of the target user;

[0014] The destination feature, action feature and preference awakening feature are used as the current driving behavior feature of the target user.

[0015] An optional invention content, the intention detection model includes a multi-feature fusion network and a classification network;

[0016] The step of inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result includes:

[0017] Inputting the driving behavior features into a multi-feature fusion network to perform feature fusion to obtain a joint behavior feature of the driving behavior features;

[0018] The joint behavior feature is input into the classification network for classification to obtain an intention detection result, wherein the intention detection result includes whether there is a voice wake-up intention and whether there is no voice wake-up intention.

[0019] In an optional invention, the method further includes a first training step of the intent detection model, wherein the first training step includes:

[0020] Obtain multiple sample driving behavior characteristics of target users;

[0021] Inputting each of the sample driving behavior features into the intention detection model to perform intention recognition and predict the intention detection result;

[0022] According to the loss function value between each predicted intention detection result and the real intention detection result corresponding to each sample driving behavior feature, the model parameters of the intention detection model are adjusted, and the trained intention detection model is determined.

[0023] An optional invention content, inputting the driving behavior features into a multi-feature fusion network to perform feature fusion to obtain a joint behavior feature of the driving behavior features, includes:

[0024] The destination features, action features and preference awakening features are spliced ​​through the multi-feature fusion network to obtain the joint behavior features of the driving behavior features.

[0025] In an optional invention, the motion analysis model includes a target detection network and a target classification network;

[0026] The step of inputting the in-vehicle user image into a motion analysis model for motion recognition to determine the motion features of the target user includes:

[0027] Inputting the in-car user image into the target detection network for target recognition, and determining a user action image of the target user, wherein the user action image includes at least one of the following: a facial image and a hand image;

[0028] The user action image is input into the target classification network for classification and recognition to determine the action features of the target user.

[0029] In an optional invention, the method further includes a second training step of the target detection network, wherein the second training step includes:

[0030] Acquire multiple sample user images of the user when performing voice wake-up, wherein the sample user images include at least one of the following: a target user's face and a target user's hand;

[0031] Input each of the sample user images into a target detection network to be trained for target recognition, predict the first coordinate information of the target user's face, and / or predict the second coordinate information of the target user's hand;

[0032] Based on the loss function value between each of the first coordinate information and the actual first coordinate information in the corresponding sample user image, and / or based on the loss function value between the second coordinate information and the actual second coordinate information in the corresponding sample user image, the model parameters of the target detection network are adjusted, and the trained target detection network is determined.

[0033] In an optional invention, the method further includes a third training step of the target classification network, wherein the third training step includes:

[0034] Acquire multiple sample user action images of the target user when performing voice wake-up, wherein the sample user action images include at least one of the following: a face image and a hand image. The sample user action images may be the multiple sample user images, which are input into the trained target detection network for recognition;

[0035] Inputting each of the sample user action images into a target classification network to be trained for classification and recognition, predicting the facial action of the target user, and / or predicting the hand action of the target user;

[0036] Based on the loss function value between each predicted facial action and the actual facial action in the corresponding sample user action image, and / or based on the loss function value between each predicted hand action and the actual hand action in the corresponding sample user action image, the model parameters of the target classification network are adjusted, and the trained target classification network is determined.

[0037] In an optional invention, the method further includes a fourth training step of the travel analysis model, wherein the fourth training step includes:

[0038] Acquire trip sample data, where the trip sample data includes a plurality of frequently used travel routes of target users;

[0039] Inputting the commonly used travel routes into the itinerary analysis model to be trained to perform itinerary recognition, and determining the predicted destination corresponding to the current itinerary of the target user;

[0040] According to the loss function value between the predicted destination and the real destination in the common travel route, the model parameters of the travel analysis model are adjusted, and the trained travel analysis model is determined.

[0041] In an optional invention, the method further includes a fifth training step of the record analysis model, wherein the fifth training step includes:

[0042] Obtain multiple common voice operation records of target users;

[0043] Inputting each of the commonly used voice operation records into the record analysis model to be trained for record recognition, and predicting the commonly used function instructions of the target user;

[0044] According to the loss function value between the predicted common function instructions and the actual common function instructions, the model parameters of the record analysis model are adjusted, and the trained record analysis model is determined.

[0045] In an optional embodiment of the invention, the method further comprises:

[0046] When it is identified that the intention detection result is that there is no voice wake-up intention, the user driving information is reacquired.

[0047] Based on the second aspect of the present invention, a wake-up device for a voice system is also provided, the wake-up device comprising:

[0048] An information acquisition module, used to acquire user driving information of a target user, wherein the user driving information includes at least travel information, an image of the user in the vehicle, and voice operation records;

[0049] A feature determination module, used to analyze the user driving information and determine the driving behavior features associated with the user driving information;

[0050] An intention recognition module, used for inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result;

[0051] The response judgment module is used to generate a voice response instruction when it is recognized that the intention detection result is a voice wake-up intention, and send it to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters the voice interaction mode to wait for the detection function instruction.

[0052] According to the third aspect of the present invention, there is also provided an electronic device, including:

[0053] one or more processors;

[0054] Memory;

[0055] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute any of the methods described in the above invention.

[0056] Based on the fourth aspect of the present invention, a computer-readable storage medium is also provided, which stores a computer program used in conjunction with an electronic device, characterized in that the computer program can be executed by a processor to perform any of the methods described in the above invention content.

[0057] Based on the fifth aspect of the present invention, a computer program product is also provided, including a computer program / computer executable instructions, characterized in that the computer program / computer executable instructions, when executed by a processor in an electronic device, implements any method described in the above invention contents.

[0058] Compared with the prior art, the present invention includes first obtaining the user driving information of the target user, then performing information analysis on the user driving information, and determining the driving behavior characteristics associated with the user driving information. The driving behavior characteristics are input into a preset intention detection model for intention recognition to obtain an intention detection result. Finally, when it is identified that the intention detection result is a voice wake-up intention, a voice response instruction is generated and sent to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters the voice interaction mode to wait for the detection function instruction. Therefore, based on the driving behavior characteristics determined by the multimodal data, it is possible to predict that the target user has a voice wake-up intention, so that the in-vehicle voice assistant directly enters the voice interaction mode. In this way, the wake-up method of the in-vehicle voice system can be enriched, which helps to improve the user experience and improve the wake-up accuracy in a noisy environment.

[0059] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0061] In the attached picture:

[0062] Figure 1 This is a schematic flow chart of the steps of a method for waking up a voice system provided by an embodiment of the present invention;

[0063] Figure 2 is a schematic flow chart of steps of another method for waking up a voice system provided by an embodiment of the present invention;

[0064] Figure 3 The present invention is a schematic block diagram of a wake-up structure of a voice system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0066] Currently, the voice system installed in the vehicle's central control device generally uses a wake-up method of waking up the system by touching the voice wake-up control, or waking up the system through a fixed voice wake-up word.

[0067] The above wake-up method is relatively fixed, which reduces the user experience of the voice assistant. In addition, when the vehicle-mounted equipment is in a noisy environment, such as a complex driving environment that is not conducive to voice recognition, such as a large number of people in the car, heavy rainfall outside the car, etc., it is easy to miss the wake-up through fixed voice wake-up words, thereby reducing the wake-up accuracy of the voice system.

[0068] Based on the above technical problems, an embodiment of the present invention is proposed. The embodiment of the present invention may include first obtaining the user driving information of the target user, and then performing information analysis on the user driving information to determine the driving behavior characteristics associated with the user driving information. The driving behavior characteristics are input into a preset intention detection model for intention recognition to obtain an intention detection result. Finally, when it is identified that the intention detection result is that there is a voice wake-up intention, a voice response instruction is generated and sent to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters the voice interaction mode to wait for the detection function instruction. Therefore, based on the driving behavior characteristics determined by the multimodal data, it is possible to predict that the target user has a voice wake-up intention, so that the in-vehicle voice assistant directly enters the voice interaction mode. In this way, the wake-up method of the in-vehicle voice system can be enriched, which helps to improve the user experience and improve the wake-up accuracy in a noisy environment.

[0069] Reference Figure 1 , shows a voice system wake-up method provided by an embodiment of the present invention, which is applied to a vehicle-mounted terminal. The method may include:

[0070] S101: Acquire driving information of a target user.

[0071] In an embodiment of the present invention, the target user can be understood as the driver of the current vehicle. The user driving information includes at least itinerary information, an in-vehicle user image and a voice operation record, wherein the itinerary information is detected based on a vehicle positioning device, and the in-vehicle user image is captured based on an in-vehicle camera device. For example, the in-vehicle camera device can be installed in front of the driving position, so that the in-vehicle user image can include the face of the target user, or the hand of the target user, so that the facial movements (or facial expressions) and / or hand movements of the target user can be obtained through the in-vehicle user image. The voice operation record can be generated within a preset time period based on the time when the voice function instruction is issued (or the execution time of the voice function instruction), the function items to be executed and other information.

[0072] In some other implementations, the user driving information may also include driving voice information, wherein the driving voice information may be understood as the voice information of the target user acquired within the current information acquisition time interval.

[0073] S102: Analyze the user driving information to determine driving behavior characteristics associated with the user driving information.

[0074] In an embodiment of the present invention, the information analysis may include analysis of the trip information, analysis of the in-vehicle user image, and analysis of the voice operation record, so that the analysis results of the trip information, the analysis results of the in-vehicle user image, and the analysis results of the voice operation record can be combined to obtain the driving behavior characteristics of the target user, wherein the driving behavior characteristics can be multimodal behavior characteristics that accurately characterize the user in the current driving state.

[0075] In other optional embodiments, if the user driving information also includes driving voice information, the information analysis step may also include analysis of the driving voice information, so that the analysis results of the trip information, the analysis results of the in-vehicle user image, the analysis results of the voice operation record and the analysis results of the driving voice information can be combined to obtain the driving behavior characteristics of the target user.

[0076] S103: Input the driving behavior characteristics into a preset intention detection model to perform intention recognition and obtain an intention detection result.

[0077] In an embodiment of the present invention, the intention detection model is a binary classification model, and the driving behavior features are input into the intention detection model, so that the intention detection model can be used to identify whether the user's current driving state has a voice wake-up intention. Correspondingly, the intention detection result may include one of the following: the presence of a voice wake-up intention and the absence of a voice wake-up intention.

[0078] S104: When it is identified that the intention detection result is a voice wake-up intention, a voice response instruction is generated and sent to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters a voice interaction mode to wait for a detection function instruction.

[0079] In an embodiment of the present invention, when the vehicle-mounted terminal identifies that the intention detection result is a voice wake-up intention and predicts that the user wants to wake up the voice assistant, a voice response instruction can be generated and sent to the vehicle-mounted voice assistant. In response to the voice response instruction, the vehicle-mounted voice assistant directly switches from standby mode to voice interaction mode, so that the user's function instructions can be collected through the vehicle-mounted microphone integrated in the vehicle-mounted voice assistant. In this way, the wake-up method of the vehicle-mounted voice system can be enriched, which helps to improve the user's experience. In addition, in a noisy driving environment, the problem of missed response of the voice system caused by the microphone missing the collection or failing to recognize the wake-up word can be avoided, and the wake-up accuracy in a noisy environment can be improved.

[0080] Reference Figure 2 , shows another voice system wake-up method provided by an embodiment of the present invention, the method may include:

[0081] S201. Obtain user driving information of a target user, where the user driving information at least includes travel information, an image of the user in the vehicle, and voice operation records.

[0082] In the embodiment of the present invention, the description of step S201 refers to the description of the above S101.

[0083] S202: Input the itinerary information into a itinerary analysis model to perform itinerary identification, and determine the destination features corresponding to the current itinerary of the target user.

[0084] In an embodiment of the present invention, the trip information may include at least a departure place and a trip route. For another example, the trip information may also include a destination, and the trip analysis model may be a classification model. Thus, the trip information may be input into the trip analysis model, and the destination of the target user's driving this time may be identified through the trip analysis model, thereby obtaining an expression feature vector corresponding to the destination and using it as a destination feature. In an embodiment of the present invention, when the trip information includes a destination, the trip information is input into the trip analysis model in order to obtain the destination feature, thereby improving the accuracy of subsequent intention detection.

[0085] After the trip information is input into the trip analysis module, the trip analysis module can extract features of the departure place and the driving route formed from the departure place to the current moment, so as to obtain the trip information features corresponding to the target user, and calculate the trip similarity based on the trip information features and multiple preset trip information features. When the trip similarity is greater than or equal to the trip threshold, the destination with the highest corresponding trip similarity is determined, and it is predicted as the destination of the target user's driving route this time, and the expression feature vector corresponding to the destination is output.

[0086] In some implementations, the method further includes a second training step of the trip analysis model, and the second training step may include:

[0087] Acquire trip sample data, where the trip sample data may include: a user's frequently used travel routes. A frequently used travel route may be a travel route that has been used by the user for a number of times that reaches a first threshold. The first threshold may be preset, for example, the first threshold may be 5 times, 6 times, 7 times, etc., which is not limited in detail herein.

[0088] The user's common travel routes are distributed in time series, for example, according to departure place-travel route-destination, departure place-travel route, respectively, and input into the itinerary analysis model to be trained for itinerary identification, and the predicted destination corresponding to each common travel route is obtained. Based on the loss function value between the predicted destination and the actual destination of each common travel route, the model parameters of the itinerary analysis model are adjusted to reduce the loss function value corresponding to the itinerary analysis model, and when the reduction amplitude of the loss function value meets the amplitude change condition, it is determined that the training of the itinerary analysis model is completed.

[0089] S203: Input the in-vehicle user image into a motion analysis model to perform motion recognition, and determine the motion features of the target user.

[0090] In an embodiment of the present invention, the motion analysis model is used to identify the user's hand motions and / or facial motions in the in-vehicle user image, thereby obtaining an expression feature vector corresponding to the target user's motion and using it as the motion feature of the target user.

[0091] In some optional embodiments of the invention, the motion analysis model may include a target detection network and a target classification network. The target detection network is used to perform target recognition on the in-vehicle user image and detect the user's facial image and / or the user's hand image from the in-vehicle user image. The user motion image is then input into the target classification network for classification and recognition to determine the motion features of the target user.

[0092] In one example, when the user action image only includes the user's facial image, the action feature of the target user is the expression feature vector corresponding to the user's facial action. In other examples, when the user action image only includes the user's hand image, the action feature of the target user is the expression feature vector corresponding to the user's hand action. In still other examples, when the user action image includes the user's hand image and the user's facial image, the action feature of the target user is the concatenation of the expression feature vector corresponding to the user's hand action and the expression vector corresponding to the user's facial action.

[0093] For example, when the user action image is input into the target classification network for classification and recognition, the user action corresponding to the user action image can be obtained, and the expression feature vector associated with the user action can be output synchronously. The user action is determined by the expression feature vector. For example, the user's facial actions may include but are not limited to smiling, sad, opening the mouth, closing the eyes, and other actions. The user's hand actions may include but are not limited to clenching a fist, reaching toward the central control device, and other actions.

[0094] In some optional implementations, the wake-up method may further include a second training step of the target detection network, and the second training step may include the following steps:

[0095] Acquire multiple sample user images of the user when performing voice wake-up, wherein the sample user images include at least one of the following: a target user's face and a target user's hand.

[0096] Each of the sample user images is input into the target detection network to be trained for target recognition, and the first coordinate information of the target user's face is predicted, and / or the second coordinate information of the target user's hand is predicted.

[0097] According to the loss function value between each of the first coordinate information and the actual first coordinate information in the corresponding sample user image, and / or according to the loss function value between the second coordinate information and the actual second coordinate information in the corresponding sample user image, the model parameters of the target detection network are adjusted, and the trained target detection network is determined.

[0098] In some optional implementations, the awakening method may further include a third training step of the target classification network, and the third training step may include:

[0099] Acquire multiple sample user action images of the target user when performing voice wake-up, wherein the sample user action images include at least one of the following: a face image and a hand image. The sample user action image can be the multiple sample user images, which are input into the trained target detection network for recognition.

[0100] Inputting each of the sample user action images into a target classification network to be trained for classification and recognition, predicting the facial action of the target user, and / or predicting the hand action of the target user;

[0101] Based on the loss function value between each predicted facial action and the actual facial action in the corresponding sample user action image, and / or based on the loss function value between each predicted hand action and the actual hand action in the corresponding sample user action image, the model parameters of the target classification network are adjusted, and the trained target classification network is determined.

[0102] S204: Input the voice operation record into a record analysis model for record recognition to determine the preferred wake-up feature of the target user.

[0103] In an embodiment of the present invention, the record analysis model may be a classification model, and the voice operation record may be recorded and identified by the record analysis model to obtain the preferred function instruction corresponding to the target user, thereby obtaining the expression feature vector corresponding to the preferred function instruction and serving as the preferred wake-up feature of the target user. For example, the preferred function instruction may include but is not limited to navigation, playing music, and the like.

[0104] In some implementations, the awakening method may further include a fifth training step of the record analysis model, the fifth training step including:

[0105] A plurality of common voice operation records of the target user are obtained, wherein the common voice operation records may include information such as the issuing time of the voice function instruction (or the execution time of the voice function instruction) and function items within a preset time period.

[0106] Inputting each of the commonly used voice operation records into the record analysis model to be trained for record recognition, and predicting the commonly used function instructions of the target user;

[0107] According to the loss function value between the predicted common function instructions and the actual common function instructions, the model parameters of the record analysis model are adjusted, and the trained record analysis model is determined.

[0108] S205: Using the destination feature, action feature, and preference awakening feature as the current driving behavior feature of the target user.

[0109] In the embodiment of the present invention, the destination feature, action feature and preference awakening feature are respectively used to characterize the behavior features of the target user in different modes of the current driving state, so that the destination feature, action feature and preference awakening feature can be combined as the current driving behavior feature of the target user. Therefore, the driving behavior feature can be a multi-modal behavior feature that accurately characterizes the user in the current driving state.

[0110] S206: Input the driving behavior characteristics into a preset intention detection model to perform intention recognition and obtain an intention detection result.

[0111] In an embodiment of the present invention, the intention detection model is a binary classification model, and the driving behavior features are input into the intention detection model, so that the intention detection model can be used to identify whether the user's current driving state has a voice wake-up intention. Correspondingly, the intention detection result may include one of the following: the presence of a voice wake-up intention and the absence of a voice wake-up intention.

[0112] In some implementations, inputting the driving behavior feature into a preset intention detection model for intention recognition to obtain an intention detection result may include:

[0113] The driving behavior features are input into a multi-feature fusion network to perform feature fusion to obtain a joint behavior feature of the driving behavior features. The joint behavior features are input into the classification network to perform classification to obtain an intention detection result.

[0114] In an embodiment of the present invention, the intention detection model may include a multi-feature fusion network and a classification network, and the driving behavior features are input into the multi-feature fusion network to perform feature fusion to obtain the joint behavior features of the driving behavior features. Feature fusion can be understood as the sequential splicing between multimodal features. For example, the feature vector corresponding to the destination feature is a 1*N dimensional vector, the feature vector corresponding to the action feature is a 1*N dimensional vector, and the feature vector corresponding to the preference awakening feature is a 1*N dimensional vector. Thus, the feature vector obtained after feature fusion can be a 1*3N dimensional vector, and is determined as the joint behavior feature of the target user.

[0115] Before feature fusion, the feature vector corresponding to the destination feature and the feature vector corresponding to the preference awakening feature may be normalized so that the two feature vectors have a unified scale during feature fusion.

[0116] In the classification network, the feature similarity between the joint behavior feature and the behavior feature corresponding to the detection intention can be calculated. If the feature similarity is greater than or equal to the similarity threshold, the corresponding intention detection result is determined to be the presence of voice wake-up intention. If the feature similarity is less than the similarity threshold, the corresponding intention detection result is determined to be the absence of voice wake-up intention.

[0117] In some optional implementations, the method may further include a first training step of the intent detection model, and the first training step may include:

[0118] Acquire multiple sample driving behavior features of the target user. The sample driving behavior features may include sample destination features, sample action features, and sample preference awakening features. The sample destination features may be identified by a trained itinerary analysis model, the sample action features may be identified by a trained record analysis model, and the preference awakening features may be identified by a trained record analysis model.

[0119] Each of the sample driving behavior features is input into the intention detection model to perform intention recognition and predict an intention detection result. The intention detection result may include one of the following: the presence of voice wake-up intention and the absence of voice wake-up intention.

[0120] According to the loss function value between each predicted intention detection result and the real intention detection result corresponding to each sample driving behavior feature, the model parameters of the intention detection model are adjusted, and the trained intention detection model is determined.

[0121] In the embodiment of the present invention, in the process of adjusting the above model parameters, the model parameters can be adjusted once after each calculation of the loss function value, and when the loss function value meets the preset training conditions, it can be determined that the current model training is completed. For example, the preset training conditions may include: the decrease in the loss function value is less than one thousandth, etc.

[0122] S207: Determine whether the intention detection result indicates that there is a voice wake-up intention.

[0123] In an embodiment of the present invention, if the intention detection result is identified as the presence of a voice wake-up intention, it is determined that the user's current driving behavior characteristics are intended to wake up the voice system. Execute the following step S208; if the intention detection result is identified as the absence of a voice wake-up intention, it is determined that the user's current driving behavior characteristics are irrelevant to waking up the voice system. Repeat the above step S201, that is, re-acquire the user driving information of the target user.

[0124] S208: Generate a voice response instruction and send it to the vehicle-mounted voice assistant, so that the vehicle-mounted voice assistant enters a voice interaction mode to wait for a detection function instruction.

[0125] In some optional implementations, before analyzing the user driving information, the information may be preprocessed, such as denoising the in-car user image or the driving voice data, so as to remove environmental noise.

[0126] In summary, an embodiment of the present invention discloses a method for waking up a voice system, which may include first acquiring user driving information of a target user, then performing information analysis on the user driving information, and determining driving behavior characteristics associated with the user driving information. The driving behavior characteristics are input into a preset intention detection model for intention recognition to obtain an intention detection result. Finally, when it is identified that the intention detection result is that there is a voice wake-up intention, a voice response instruction is generated and sent to the vehicle-mounted voice assistant, so that the vehicle-mounted voice assistant enters a voice interaction mode to wait for the detection function instruction. Thus, based on the driving behavior characteristics determined by the multimodal data, it is possible to predict that the target user has a voice wake-up intention, so that the vehicle-mounted voice assistant directly enters the voice interaction mode. In this way, the wake-up method of the vehicle-mounted voice system can be enriched, which helps to improve the user experience and can improve the wake-up accuracy in a noisy environment.

[0127] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0128] Reference Figure 3 , shows a wake-up device for a voice system provided by an embodiment of the present invention, and the device may include:

[0129] The information acquisition module 301 is used to acquire the user driving information of the target user, and the user driving information at least includes travel information, an image of the user in the car, and voice operation records.

[0130] The feature determination module 302 is used to perform information analysis on the user driving information to determine the driving behavior features associated with the user driving information.

[0131] The intention recognition module 303 is used to input the driving behavior characteristics into a preset intention detection model to perform intention recognition and obtain an intention detection result.

[0132] The response judgment module 304 is used to generate a voice response instruction when it is identified that the intention detection result is a voice wake-up intention, and send it to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters the voice interaction mode to wait for the detection function instruction.

[0133] In an optional embodiment of the invention, the feature determination module 302 may include:

[0134] The first feature determination submodule is used to input the itinerary information into the itinerary analysis model to perform itinerary identification and determine the destination feature corresponding to the current itinerary of the target user.

[0135] The second feature determination submodule is used to input the in-vehicle user image into the motion analysis model for motion recognition to determine the motion features of the target user.

[0136] The third feature determination submodule is used to input the voice operation record into the record analysis model for record recognition, and determine the preferred wake-up feature of the target user.

[0137] The fourth feature determination submodule is used to use the destination feature, action feature and preference awakening feature as the current driving behavior feature of the target user.

[0138] In an optional embodiment of the invention, the intention detection model includes a multi-feature fusion network and a classification network. The intention recognition module 303 may include:

[0139] The feature fusion submodule is used to input the driving behavior features into a multi-feature fusion network to perform feature fusion to obtain a joint behavior feature of the driving behavior features.

[0140] The intention recognition submodule is used to input the joint behavior features into the classification network for classification to obtain an intention detection result, wherein the intention detection result includes the presence of voice wake-up intention and the absence of voice wake-up intention.

[0141] In an optional embodiment of the invention, the device further includes a first training device for training the intention detection model, wherein the first training device is used to:

[0142] Obtain multiple sample driving behavior characteristics of the target user.

[0143] The driving behavior characteristics of each sample are input into the intention detection model to perform intention recognition and predict the intention detection result.

[0144] According to the loss function value between each predicted intention detection result and the real intention detection result corresponding to each sample driving behavior feature, the model parameters of the intention detection model are adjusted, and the trained intention detection model is determined.

[0145] In an optional embodiment of the invention, the first training device is further used for:

[0146] The destination features, action features and preference awakening features are spliced ​​through the multi-feature fusion network to obtain the joint behavior features of the driving behavior features.

[0147] In an optional embodiment of the invention, the action analysis model includes a target detection network and a target classification network. The second feature determination submodule can also be used for:

[0148] The in-vehicle user image is input into the target detection network for target recognition, and a user action image of the target user is determined, wherein the user action image includes at least one of the following: a facial image and a hand image.

[0149] The user action image is input into the target classification network for classification and recognition to determine the action features of the target user.

[0150] In an optional embodiment of the invention, the device may further include a second training device for training the target detection network, and the second training device may be used to:

[0151] Acquire multiple sample user images of the user when performing voice wake-up, wherein the sample user images include at least one of the following: a target user's face and a target user's hand.

[0152] Each of the sample user images is input into the target detection network to be trained for target recognition, and the first coordinate information of the target user's face is predicted, and / or the second coordinate information of the target user's hand is predicted.

[0153] Based on the loss function value between each of the first coordinate information and the actual first coordinate information in the corresponding sample user image, and / or based on the loss function value between the second coordinate information and the actual second coordinate information in the corresponding sample user image, the model parameters of the target detection network are adjusted, and the trained target detection network is determined.

[0154] In an optional embodiment of the invention, the device further includes a third training device for training the target classification network, and the third training device can be used to:

[0155] Acquire multiple sample user action images of the target user when performing voice wake-up, wherein the sample user action images include at least one of the following: a face image and a hand image. The sample user action image can be the multiple sample user images, which are input into the trained target detection network for recognition.

[0156] Each of the sample user action images is input into the target classification network to be trained for classification and recognition, and the facial action of the target user and / or the hand action of the target user are predicted.

[0157] Based on the loss function value between each predicted facial action and the actual facial action in the corresponding sample user action image, and / or based on the loss function value between each predicted hand action and the actual hand action in the corresponding sample user action image, the model parameters of the target classification network are adjusted, and the trained target classification network is determined.

[0158] In an optional embodiment of the invention, the device may further include a fourth training device for training the travel analysis model, and the fourth training device may be used to:

[0159] Acquire trip sample data, where the trip sample data includes a plurality of frequently used travel routes of target users.

[0160] The commonly used travel routes are input into the itinerary analysis model to be trained to perform itinerary identification, and determine the predicted destination corresponding to the current itinerary of the target user.

[0161] According to the loss function value between the predicted destination and the real destination in the common travel route, the model parameters of the travel analysis model are adjusted, and the trained travel analysis model is determined.

[0162] In an optional embodiment of the invention, the device may further include a fifth training device for training the record analysis model, and the fifth training device may be used to:

[0163] Get multiple common voice operation records of the target user.

[0164] Each of the commonly used voice operation records is input into the record analysis model to be trained for record recognition, and the commonly used function instructions of the target user are predicted.

[0165] According to the loss function value between the predicted common function instructions and the actual common function instructions, the model parameters of the record analysis model are adjusted, and the trained record analysis model is determined.

[0166] In an optional embodiment of the invention, the response determination module is further used to:

[0167] When it is identified that the intention detection result is that there is no voice wake-up intention, the user driving information is reacquired.

[0168] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0169] It is easy for a person skilled in the art to think that any combination of the above embodiments is feasible, so any combination of the above embodiments is an implementation scheme of the present invention. However, due to space limitations, this specification will not describe them in detail here.

[0170] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0171] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0172] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0173] An electronic device, comprising:

[0174] one or more processors;

[0175] Memory;

[0176] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method described in the above embodiment.

[0177] A computer-readable storage medium stores a computer program used in combination with an electronic device, wherein the computer program can be executed by a processor to implement the method described in the above embodiment.

[0178] A computer program product includes a computer program / computer executable instructions, wherein the computer program / computer executable instructions, when executed by a processor in an electronic device, implements the method described in any one of the above-mentioned embodiments of the invention.

[0179] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0180] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable information processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable information processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable information processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0182] These computer program instructions can also be loaded onto a computer or other programmable information processing terminal device, so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0183] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0184] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0185] The above is a detailed introduction to a voice system wake-up method and a voice system wake-up device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for waking up a voice system, characterized in that: Applied to a vehicle-mounted terminal, the method comprises: Acquire user driving information of the target user, wherein the user driving information includes at least travel information, an image of the user in the car, and a voice operation record; Performing information analysis on the user driving information to determine driving behavior characteristics associated with the user driving information; Inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result; When it is identified that the intention detection result is a voice wake-up intention, a voice response instruction is generated and sent to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters a voice interaction mode to wait for the detection function instruction.

2. The voice system wake-up method according to claim 1, characterized in that: The performing information analysis on the user driving information to determine the driving behavior characteristics associated with the user driving information includes: Inputting the itinerary information into the itinerary analysis model to perform itinerary identification, and determining the destination features corresponding to the current itinerary of the target user; Inputting the in-vehicle user image into a motion analysis model for motion recognition to determine the motion features of the target user; Inputting the voice operation record into a record analysis model for record recognition to determine the preferred wake-up feature of the target user; The destination feature, action feature and preference awakening feature are used as the current driving behavior feature of the target user.

3. The voice system wake-up method according to claim 2, characterized in that: The intention detection model includes a multi-feature fusion network and a classification network; The step of inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result includes: Inputting the driving behavior features into a multi-feature fusion network to perform feature fusion to obtain a joint behavior feature of the driving behavior features; The joint behavior feature is input into the classification network for classification to obtain an intention detection result, wherein the intention detection result includes whether there is a voice wake-up intention and whether there is no voice wake-up intention.

4. The voice system wake-up method according to claim 3, characterized in that: The method further comprises a first training step of the intent detection model, wherein the first training step comprises: Obtain multiple sample driving behavior characteristics of target users; Inputting each of the sample driving behavior features into the intention detection model to perform intention recognition and predict the intention detection result; According to the loss function value between each predicted intention detection result and the real intention detection result corresponding to each sample driving behavior feature, the model parameters of the intention detection model are adjusted, and the trained intention detection model is determined.

5. The method for waking up a voice system according to claim 3, characterized in that: The step of inputting the driving behavior features into a multi-feature fusion network to perform feature fusion to obtain a joint behavior feature of the driving behavior features includes: The destination features, action features and preference awakening features are spliced ​​through the multi-feature fusion network to obtain the joint behavior features of the driving behavior features.

6. The wake-up method of the voice system according to claim 2, wherein the action analysis model comprises a target detection network and a target classification network; The step of inputting the in-vehicle user image into a motion analysis model for motion recognition to determine the motion features of the target user includes: Inputting the in-car user image into the target detection network for target recognition, and determining a user action image of the target user, wherein the user action image includes at least one of the following: a facial image and a hand image; The user action image is input into the target classification network for classification and recognition to determine the action features of the target user.

7. The method for waking up a voice system according to claim 6, characterized in that: The method further comprises a second training step of the target detection network, wherein the second training step comprises: Acquire multiple sample user images of the user when performing voice wake-up, wherein the sample user images include at least one of the following: a target user's face and a target user's hand; Input each of the sample user images into a target detection network to be trained for target recognition, predict the first coordinate information of the target user's face, and / or predict the second coordinate information of the target user's hand; Based on the loss function value between each of the first coordinate information and the actual first coordinate information in the corresponding sample user image, and / or based on the loss function value between the second coordinate information and the actual second coordinate information in the corresponding sample user image, the model parameters of the target detection network are adjusted, and the trained target detection network is determined.

8. The method for waking up a voice system according to claim 7, characterized in that: The method further comprises a third training step of the target classification network, wherein the third training step comprises: Acquire multiple sample user action images of the target user when performing voice wake-up, wherein the sample user action images include at least one of the following: a face image and a hand image, and the sample user action images are the multiple sample user images, which are input into the trained target detection network for recognition; Inputting each of the sample user action images into a target classification network to be trained for classification and recognition, predicting the facial action of the target user, and / or predicting the hand action of the target user; Based on the loss function value between each predicted facial action and the actual facial action in the corresponding sample user action image, and / or based on the loss function value between each predicted hand action and the actual hand action in the corresponding sample user action image, the model parameters of the target classification network are adjusted, and the trained target classification network is determined.

9. The voice system wake-up method according to claim 2, characterized in that: The method further comprises a fourth training step of the trip analysis model, wherein the fourth training step comprises: Acquire trip sample data, where the trip sample data includes a plurality of frequently used travel routes of target users; Inputting the commonly used travel routes into the itinerary analysis model to be trained to perform itinerary recognition, and determining the predicted destination corresponding to the current itinerary of the target user; According to the loss function value between the predicted destination and the real destination in the common travel route, the model parameters of the travel analysis model are adjusted, and the trained travel analysis model is determined.

10. The voice system wake-up method according to claim 2, characterized in that: The method further comprises a fifth training step of the record analysis model, wherein the fifth training step comprises: Obtain multiple common voice operation records of target users; Inputting each of the commonly used voice operation records into the record analysis model to be trained for record recognition, and predicting the commonly used function instructions of the target user; According to the loss function value between the predicted common function instructions and the actual common function instructions, the model parameters of the record analysis model are adjusted, and the trained record analysis model is determined.

11. The voice system wake-up method according to claim 1, characterized in that: The method further comprises: When it is identified that the intention detection result is that there is no voice wake-up intention, the user driving information is reacquired.

12. A wake-up device for a voice system, characterized in that: Applied to a vehicle-mounted terminal, the device comprises: An information acquisition module, used to acquire user driving information of a target user, wherein the user driving information includes at least travel information, an image of the user in the vehicle, and voice operation records; A feature determination module, used to analyze the user driving information and determine the driving behavior features associated with the user driving information; An intention recognition module, used for inputting the driving behavior characteristics into a preset intention detection model for intention recognition to obtain an intention detection result; The response judgment module is used to generate a voice response instruction when it is recognized that the intention detection result is a voice wake-up intention, and send it to the in-vehicle voice assistant, so that the in-vehicle voice assistant enters the voice interaction mode to wait for the detection function instruction.

13. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program for use in conjunction with an electronic device, characterized in that: The computer program can be executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program / computer executable instructions, characterized in that: The computer program / computer executable instructions, when executed by a processor in an electronic device, implement the method according to any one of claims 1 to 11.

Images