A vehicle machine voice assistant interaction method, device, equipment and storage medium
By acquiring multimodal data to determine the user's emotional state, the image and voice information of the in-vehicle voice assistant are adjusted, solving the problem of the single dialogue mode of existing in-vehicle voice assistants and realizing personalized emotional interaction and interactive experience.
Patent Information
- Application Number
- CN202510101564.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2026-07-24
AI Technical Summary
Existing in-vehicle voice assistants have a limited dialogue style, lack deep emotional interaction and personalized experience with users, and cannot adjust accordingly based on different emotional states of users.
By acquiring multimodal data, including facial expressions, gestures, voice data, and music style data, the system comprehensively judges the user's emotional state and adjusts the image and voice information of the in-vehicle voice assistant according to the emotional state, providing a personalized interaction method.
It achieves accurate judgment of the user's emotional state and adjusts the image and voice information of the vehicle's voice assistant, providing a more intelligent and considerate interactive experience and meeting the user's needs for deeper emotional interaction.
Smart Images

Figure CN122454964A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of vehicle voice processing technology, specifically to a vehicle-mounted voice assistant interaction method, device, equipment, and storage medium. Background Technology
[0002] A car infotainment system's voice assistant is an intelligent voice interaction system integrated into the vehicle's infotainment system. It allows drivers to control certain vehicle functions via voice commands, thereby reducing driver distraction and manual operation, and improving driving safety.
[0003] With the development of artificial intelligence technology, in-vehicle voice assistants are becoming increasingly powerful, providing drivers with a more convenient and safer driving experience. At the same time, in-vehicle voice assistants are also an important component of the development of intelligent connected vehicles, helping to drive the digital transformation of the automotive industry.
[0004] However, the voice assistants currently installed in new energy vehicles generally suffer from limited dialogue options and a lack of deeper emotional interaction and personalized experiences. They cannot adjust to different emotional states of users, making it difficult to meet users' needs for more intelligent and considerate interaction. Summary of the Invention
[0005] This application provides a vehicle-mounted voice assistant interaction method, device, equipment, and storage medium, which can solve the problems of existing vehicle-mounted voice assistants having a single dialogue mode and insufficient emotional expression.
[0006] In a first aspect, embodiments of this application provide a vehicle-mounted voice assistant interaction method, the vehicle-mounted voice assistant interaction method comprising the following steps:
[0007] Acquire multimodal data that influences user emotions and comprehensively assess user emotional state;
[0008] Based on the user's emotional state, the appearance and voice information of the in-vehicle voice assistant are adjusted.
[0009] In conjunction with the first aspect, in one implementation, acquiring multimodal data affecting user emotions and comprehensively judging the user's emotional state includes:
[0010] Collect multimodal data that influences user emotions, and set the weight of each data type, as well as the scores for various scenarios under each data type;
[0011] Calculate the user's overall sentiment score based on the weight of each data category and the score of the corresponding situation;
[0012] Multiple emotional state types are set up, and the corresponding emotional state is determined based on the user's comprehensive emotional score.
[0013] In conjunction with the first aspect, in one implementation, the multimodal data influencing user emotions includes facial expression and posture data, voice data, and / or music style data.
[0014] In conjunction with the first aspect, in one implementation, facial expression and posture data is acquired via a camera, the facial expression and posture data including facial expressions, sitting posture, and gesture information;
[0015] Voice data is acquired through a microphone, and the voice data includes pitch, timbre, and tone information.
[0016] In conjunction with the first aspect, in one implementation, the emotional state type includes a pleasant state and a sad state;
[0017] The step of adjusting the image and voice information of the vehicle's voice assistant based on the user's emotional state judgment results includes:
[0018] When the user is in a pleasant state, the image of the in-vehicle voice assistant becomes more lively, and a high-pitched, bright tone and cheerful manner are adopted;
[0019] When a user is sad, the car's voice assistant will adopt a gentler tone and a softer image.
[0020] In conjunction with the first aspect, in one implementation, it further includes:
[0021] Based on the user's emotional state, a recommendation matching degree is set for each smart cockpit recommendation type;
[0022] Based on the user's overall sentiment score and the recommendation matching degree, a recommendation score is calculated for each smart cockpit recommendation type;
[0023] The recommended combination of smart cockpits is determined based on the total score of all smart cockpit recommendation types and the weight of each smart cockpit recommendation type.
[0024] In conjunction with the first aspect, in one implementation, the intelligent cockpit recommendation types include music recommendations, seat adjustment recommendations, massage adjustment recommendations, air conditioning recommendations, aromatherapy recommendations, ambient lighting recommendations, and / or travel recommendations.
[0025] Secondly, embodiments of this application provide a vehicle-mounted voice assistant interaction device, the vehicle-mounted voice assistant interaction device comprising:
[0026] The data acquisition module is used to acquire multimodal data that affects users' emotions and to comprehensively judge the user's emotional state.
[0027] The interactive control module adjusts the appearance and voice information of the vehicle's voice assistant based on the user's emotional state.
[0028] Thirdly, this application provides a vehicle-mounted voice assistant interaction device, which includes a processor, a memory, and a vehicle-mounted voice assistant interaction program stored in the memory and executable by the processor. When the vehicle-mounted voice assistant interaction program is executed by the processor, it implements the steps of the above-described vehicle-mounted voice assistant interaction method.
[0029] Fourthly, a computer-readable storage medium storing a vehicle-mounted voice assistant interaction program, wherein when the vehicle-mounted voice assistant interaction program is executed by a processor, it implements the steps of the above-described vehicle-mounted voice assistant interaction method.
[0030] The beneficial effects of the technical solutions provided in this application include at least the following:
[0031] The in-vehicle voice assistant interaction method in this application acquires multimodal data affecting user emotions to comprehensively judge the user's emotional state; based on the judgment result of the user's emotional state, it adjusts the image and voice information of the in-vehicle voice assistant. Thus, on the one hand, it analyzes user emotions from multiple dimensions, reflecting the user's emotional state more objectively and accurately; on the other hand, based on the judgment result of the user's emotional state, it adjusts both the image and voice information of the in-vehicle voice assistant, effectively solving the problems of existing in-vehicle voice assistants having a single dialogue method and insufficient emotional expression. Attached Figure Description
[0032] Figure 1 This is a flowchart illustrating an embodiment of the in-vehicle voice assistant interaction method of this application;
[0033] Figure 2 This is a flowchart of step S1 of this application;
[0034] Figure 3 The flowchart recommended for the intelligent cockpit in this application;
[0035] Figure 4 This is a structural block diagram of an embodiment of the in-vehicle voice assistant interaction device of this application;
[0036] Figure 5 This is a schematic diagram of the hardware structure of the in-vehicle voice assistant interaction device involved in the embodiments of this application. Detailed Implementation
[0037] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0038] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0039] In a first aspect, embodiments of this application provide a method for interacting with a vehicle-mounted voice assistant.
[0040] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the in-vehicle voice assistant interaction method of this application. Figure 1 As shown, the in-vehicle voice assistant interaction methods include:
[0041] S1. Obtain multimodal data that influences user emotions and comprehensively judge the user's emotional state;
[0042] Specifically, see Figure 2 As shown, step S1 includes:
[0043] S11. Collect multimodal data that affects users' emotions, and set the weight of each type of data, as well as the scores for various situations under each type of data;
[0044] Specifically, in this embodiment, the multimodal data affecting user emotions mainly includes facial expression and posture data, voice data, and / or music style data. Using facial recognition technology, voice analysis technology, and other techniques, multimodal recognition is performed on the user's facial expressions, tone of voice, timbre, and intonation to accurately determine the user's emotional state.
[0045] In practice, multimodal data can be collected in the following ways:
[0046] Camera recognition: The system continuously collects the user's facial expressions, posture, and gestures using in-vehicle cameras, and then analyzes and recognizes these information using deep learning algorithms. For example, a smiling facial expression and relaxed posture may indicate a happy state; a downward head or a frustrated expression may indicate a sad state.
[0047] Microphone recognition: Captures user speech via microphone and analyzes pitch, timbre, and tone features. High pitch, bright timbre, and cheerful tone may indicate pleasant emotions; low pitch, deep timbre, and dejected tone may indicate sad emotions.
[0048] Music style analysis: Analyze the music style played by users while driving. For example, upbeat pop music may be associated with a pleasant state, while soothing classical music may be associated with a relaxed or gentle state.
[0049] Then, by analyzing the user's facial expressions, posture, voice features, and music style, a formula for judging the user's emotional level is formed:
[0050] First, determine the weights of the three judgment factors: facial expression and posture, voice features, and music style.
[0051] For example, we can set the weight of facial expression and posture as W1, the weight of speech features as W2, and the weight of music style as W3, and W1+W2+W3=1.
[0052] For each judgment factor, different scores are assigned based on the different states.
[0053] Facial expression and posture score: set as S1. Facial expression and posture fully consider different facial expressions and sitting postures. For example, facial expressions are divided into smiling, frowning and expressionless, and sitting postures are divided into standard sitting posture, forward-leaning sitting posture, and backward-leaning sitting posture. The corresponding facial expression and posture score is given based on the user's actual situation.
[0054] Speech feature score: set to S2, for example, set different scores for high pitch, bright tone and cheerful tone; low pitch, deep tone and depressed tone, etc.
[0055] Music style score: Set to S3, for example, set corresponding scores for upbeat rhythm, soothing rhythm, exciting rhythm, etc.
[0056] S12. Calculate the user's overall sentiment score based on the weight of each type of data and the score of the corresponding situation;
[0057] S13. Set multiple emotional state types and determine the corresponding emotional state based on the user's comprehensive emotional score.
[0058] Specifically, the formula for calculating the user's overall emotion score is: E=W1×S1+W2×S2+W3×S3.
[0059] At the same time, based on the user's overall emotional score, multiple score ranges of emotional state types will be set, such as score ranges corresponding to happy and sad states. After obtaining the user's overall emotional score, their emotional state type can be determined.
[0060] S2. Based on the user's emotional state, adjust the image and voice information of the vehicle's voice assistant.
[0061] In this embodiment, the emotional state types include a joyful state and a sad state. When the user is in a joyful state, the vehicle's voice assistant becomes more lively, using a high-pitched, bright tone and cheerful manner; when the user is in a sad state, the voice assistant becomes more gentle, using a gentle tone. It is understood that other emotional state types can be set as needed, such as an angry state or a calm state; this embodiment does not impose any limitations on this.
[0062] It is worth noting that in order to adjust the image of the in-vehicle voice assistant according to the user's emotional state, this embodiment will perform image model training, that is, collect a large amount of image data of different styles, including color tone, movements, etc., and train it through machine learning algorithms to obtain image models under different emotional states and the most suitable interactive feedback.
[0063] Meanwhile, during voice conversations, the system selects an appropriate character model based on the user's emotional state. For example, when a pleasant emotion is detected, a lively character model is selected, making the in-vehicle intelligent assistant's image more vibrant and its movements more cheerful.
[0064] To enable the in-vehicle voice assistant to adjust its voice information based on the user's emotional state, this embodiment will perform timbre model training. This involves collecting various timbre data, classifying and training them to obtain timbre models suitable for different emotional states; it will also establish a corpus, which involves organizing rich data, including words of blessing, words of encouragement and comfort, etc., and classifying them according to emotional states.
[0065] During voice conversations, the system selects an appropriate voice model and corpus content based on the user's emotional state assessment. For example, when sadness is detected, a gentle voice model is selected, and encouraging and comforting phrases are extracted from the corpus to respond.
[0066] The following describes the workflow of the in-vehicle voice assistant interaction in this implementation:
[0067] The system monitors real-time voice conversations between users and the in-vehicle infotainment system. When a user activates the voice assistant, it immediately acquires multimodal data influencing the user's emotions to comprehensively assess their emotional state. Based on the emotion recognition results, it quickly calculates the voice assistant's presentation in terms of image, tone, and corpus, and adjusts the voice assistant's image and voice information accordingly. This ensures that the adjusted voice assistant can interact with users promptly and accurately, providing considerate service. Furthermore, it trains an AI model using extensive user state and emotion data, as well as in-vehicle response data, to establish a correlation between user states and emotions and the emotional responses of the in-vehicle voice assistant.
[0068] As a preferred implementation method, after determining the user's overall emotional score, a series of intelligent cockpit recommendations will be made based on the specific situation, such as music, seat adjustment, massage adjustment, air conditioning, aromatherapy, ambient lighting, and travel recommendations.
[0069] See Figure 3 As shown, the recommended smart cockpit includes the following steps:
[0070] S41. Based on the user's emotional state, set the recommendation matching degree for each smart cockpit recommendation type;
[0071] S42. Calculate the recommendation score for each smart cockpit recommendation type based on the user's overall emotion score and recommendation matching degree;
[0072] S43. Determine the recommended combination of smart cockpits based on the total score of all smart cockpit recommendation types and the weight of each smart cockpit recommendation type.
[0073] In practice, it can be executed in the following ways:
[0074] (1) Music recommendation score
[0075] The matching degree M corresponding to the current mood of the music recommendation 当前 The matching degree of music recommendation can be set as M1 for pleasant mood, M2 for tense mood, and M3 for relaxed mood, etc. Different matching degrees can determine whether music should be recommended in the end.
[0076] Music Recommendation Score MRS = E × M 当前 E represents the user's overall emotional score.
[0077] (2) Seat Adjustment Recommendation Score
[0078] The matching degree S corresponding to the current mood of the seat adjustment recommendation 当前 The seat adjustment matching degree can be set to S1 when under tension and S2 when under relaxation, etc.
[0079] Seat Adjustment Recommended Score SRS = E × S 当前 .
[0080] (3) Recommended score for massage therapy
[0081] The matching degree (MA) to the current mood recommended by the massage therapy. 当前 The matching degree of massage regulation under fatigue can be set as MA1, and the matching degree of massage regulation under tension can be set as MA2, etc.
[0082] Massage therapy recommendation score MARS≡E×N 当前 .
[0083] (4) Air Conditioner Recommendation Score
[0084] The matching degree C corresponding to the current mood of the air conditioner recommendation 当前 Different air conditioning adjustment matching degrees are determined according to different emotions and environmental conditions. For example, the air conditioning matching degree is C1 when the mood is pleasant in hot weather.
[0085] Air Conditioner Recommended Score (CRS) = E × C 当前 .
[0086] (6) Aromatherapy Recommendation Score
[0087] The matching degree F corresponding to the current mood of the aromatherapy recommendation 当前 The matching degree of aromatherapy can be set as F1 for pleasant mood and F2 for relaxed mood, etc.
[0088] Aromatherapy Recommendation Score FRS≡E×F 当前 .
[0089] (7) Ambient Lighting Recommendation Score
[0090] The matching degree L corresponding to the current mood of the ambient light recommendation 当前 The matching degree of ambient lighting can be set to L1 for pleasant mood and L2 for tense mood, etc.
[0091] Ambient lighting recommendation score LRS = E × L 当前 .
[0092] (8) Travel recommendation score
[0093] The matching degree T corresponding to the current mood of the travel recommendation 当前 The matching degree of travel recommendations under a pleasant mood can be set as T1, and the matching degree of travel recommendations under a relaxed mood can be set as T2, etc.
[0094] Trip Recommendation Score (TRS) = E × T 当前 .
[0095] (9) Comprehensive Recommendation Score Formula
[0096] The overall recommendation score (CRS total) is calculated as follows: CRS total = MRS + SRS + MARS + CRS + FRS + LRS + TRS. Based on the overall recommendation score and the weight of each smart cockpit recommendation type, the final smart cockpit recommendation combination can be determined to provide users with personalized services.
[0097] In summary, the in-vehicle voice assistant interaction method in this application acquires multimodal data influencing user emotions to comprehensively determine the user's emotional state; based on the user's emotional state determination, it adjusts the image and voice information of the in-vehicle voice assistant. Thus, on the one hand, it analyzes user emotions from multiple dimensions, providing a more objective and accurate reflection of the user's emotional state; on the other hand, based on the user's emotional state determination, it adjusts both the image and voice information of the in-vehicle voice assistant, effectively solving the problems of existing in-vehicle voice assistants having a single dialogue method and insufficient emotional expression.
[0098] Secondly, embodiments of this application also provide a vehicle-mounted voice assistant interaction device.
[0099] In one embodiment, reference is made to Figure 4 , Figure 4 This is a schematic diagram of the functional modules of an embodiment of the in-vehicle voice assistant interaction device of this application. Figure 4 As shown, the in-vehicle voice assistant interaction device includes a data acquisition module and an interaction control module.
[0100] The data acquisition module is used to acquire multimodal data that affects users' emotions and comprehensively judge the user's emotional state; the interaction control module adjusts the image and voice information of the vehicle's voice assistant based on the judgment result of the user's emotional state.
[0101] Furthermore, in one embodiment, the data acquisition module acquires multimodal data affecting user emotions and comprehensively judges the user's emotional state, including:
[0102] Collect multimodal data that influences user emotions, and set the weight of each data type, as well as the scores for various scenarios under each data type;
[0103] Calculate the user's overall sentiment score based on the weight of each data category and the score of the corresponding situation;
[0104] Multiple emotional state types are set up, and the corresponding emotional state is determined based on the user's comprehensive emotional score.
[0105] Furthermore, in one embodiment, the multimodal data influencing user emotions includes facial expression and posture data, voice data, and / or music style data.
[0106] Furthermore, in one embodiment, the data acquisition module is used for:
[0107] The facial expression and posture data are acquired through a camera, including facial expressions, sitting posture, and gesture information.
[0108] Voice data is acquired through a microphone, and the voice data includes pitch, timbre, and tone information.
[0109] Furthermore, in one embodiment, the emotional state types include a pleasant state and a sad state;
[0110] The interactive control module adjusts the image and voice information of the vehicle's voice assistant based on the user's emotional state judgment, including:
[0111] When the user is in a pleasant state, the image of the in-vehicle voice assistant becomes more lively, and a high-pitched, bright tone and cheerful manner are adopted;
[0112] When a user is sad, the car's voice assistant will adopt a gentler tone and a softer image.
[0113] Furthermore, in one embodiment, it also includes a smart cockpit recommendation module, which is used to:
[0114] Based on the user's emotional state, a recommendation matching degree is set for each smart cockpit recommendation type;
[0115] Based on the user's overall sentiment score and the recommendation matching degree, a recommendation score is calculated for each smart cockpit recommendation type;
[0116] The recommended combination of smart cockpits is determined based on the total score of all smart cockpit recommendation types and the weight of each smart cockpit recommendation type.
[0117] Furthermore, in one embodiment, the intelligent cockpit recommendation types include music recommendations, seat adjustment recommendations, massage adjustment recommendations, air conditioning recommendations, aromatherapy recommendations, ambient lighting recommendations, and / or travel recommendations.
[0118] The functions of each module in the above-mentioned vehicle voice assistant interaction device correspond to the steps in the above-mentioned vehicle voice assistant interaction method embodiment, and their functions and implementation processes will not be described in detail here.
[0119] Thirdly, embodiments of this application provide a vehicle-mounted voice assistant interaction device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.
[0120] Reference Figure 5 , Figure 5 This is a schematic diagram of the hardware structure of the in-vehicle voice assistant interaction device involved in the embodiments of this application. In this embodiment, the in-vehicle voice assistant interaction device may include a processor, a memory, a communication interface, and a communication bus.
[0121] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.
[0122] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used to interconnect components within the in-vehicle voice assistant interaction device, as well as interfaces used to interconnect the in-vehicle voice assistant interaction device with other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user devices can be displays, keyboards, etc.
[0123] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0124] The processor can be a general-purpose processor, which can call the vehicle-mounted voice assistant interaction program stored in the memory and execute the vehicle-mounted voice assistant interaction method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the vehicle-mounted voice assistant interaction program is called can be referred to in the various embodiments of the vehicle-mounted voice assistant interaction method of this application, and will not be repeated here.
[0125] Those skilled in the art will understand that Figure 5 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0126] Fourthly, embodiments of this application also provide a readable storage medium.
[0127] The present application has a readable storage medium storing a vehicle-mounted voice assistant interaction program, wherein when the vehicle-mounted voice assistant interaction program is executed by a processor, it implements the steps of the vehicle-mounted voice assistant interaction method described above.
[0128] The method implemented when the vehicle-mounted voice assistant interaction program is executed can be referred to in various embodiments of the vehicle-mounted voice assistant interaction method of this application, and will not be repeated here.
[0129] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0131] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0132] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0133] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0134] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0135] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
[0136] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A method for interacting with a vehicle-mounted voice assistant, characterized in that, The in-vehicle voice assistant interaction method includes: Acquire multimodal data that influences user emotions and comprehensively assess user emotional state; Based on the user's emotional state, the appearance and voice information of the in-vehicle voice assistant are adjusted.
2. The in-vehicle voice assistant interaction method as described in claim 1, characterized in that, The acquisition of multimodal data affecting user emotions, and the comprehensive judgment of user emotional state, includes: Collect multimodal data that influences user emotions, and set the weight of each data type, as well as the scores for various scenarios under each data type; Calculate the user's overall sentiment score based on the weight of each data category and the score of the corresponding situation; Multiple emotional state types are set up, and the corresponding emotional state is determined based on the user's comprehensive emotional score.
3. The in-vehicle voice assistant interaction method as described in claim 2, characterized in that: The multimodal data that influences user emotions includes facial expression and posture data, voice data, and / or music style data.
4. The in-vehicle voice assistant interaction method as described in claim 3, characterized in that: The facial expression and posture data are acquired through a camera, including facial expressions, sitting posture, and gesture information. Voice data is acquired through a microphone, and the voice data includes pitch, timbre, and tone information.
5. The in-vehicle voice assistant interaction method as described in claim 2, characterized in that: The emotional state types include a pleasant state and a sad state; The step of adjusting the image and voice information of the vehicle's voice assistant based on the user's emotional state judgment results includes: When the user is in a pleasant state, the image of the in-vehicle voice assistant becomes more lively, and a high-pitched, bright tone and cheerful manner are adopted; When a user is sad, the car's voice assistant will adopt a gentler tone and a softer image.
6. The in-vehicle voice assistant interaction method as described in claim 2, characterized in that, Also includes: Based on the user's emotional state, a recommendation matching degree is set for each smart cockpit recommendation type; Based on the user's overall sentiment score and the recommendation matching degree, a recommendation score is calculated for each smart cockpit recommendation type; The recommended combination of smart cockpits is determined based on the total score of all smart cockpit recommendation types and the weight of each smart cockpit recommendation type.
7. The in-vehicle voice assistant interaction method as described in claim 6, characterized in that: The intelligent cockpit recommendation types include music recommendations, seat adjustment recommendations, massage adjustment recommendations, air conditioning recommendations, aromatherapy recommendations, ambient lighting recommendations, and / or travel recommendations.
8. A vehicle-mounted voice assistant interaction device, characterized in that, The in-vehicle voice assistant interaction device includes: The data acquisition module is used to acquire multimodal data that affects users' emotions and to comprehensively judge the user's emotional state. The interactive control module adjusts the appearance and voice information of the vehicle's voice assistant based on the user's emotional state.
9. A vehicle-mounted voice assistant interaction device, characterized in that, The vehicle-mounted voice assistant interaction device includes a processor, a memory, and a vehicle-mounted voice assistant interaction program stored in the memory and executable by the processor, wherein when the vehicle-mounted voice assistant interaction program is executed by the processor, it implements the steps of the vehicle-mounted voice assistant interaction method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a vehicle-mounted voice assistant interaction program, wherein when the vehicle-mounted voice assistant interaction program is executed by a processor, it implements the steps of the vehicle-mounted voice assistant interaction method as described in any one of claims 1 to 7.
