Vehicle user evaluation method and electronic equipment
By collecting in-vehicle user emotions and vehicle data, combined with micro-expression and audio analysis, the problem of lag and subjectivity in vehicle user experience evaluation in existing technologies has been solved, enabling accurate user experience evaluation and dynamic adjustment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-23
- Publication Date
- 2026-04-07
AI Technical Summary
Existing vehicle user experience evaluation methods suffer from lag and strong subjectivity, failing to accurately capture genuine user feedback, especially since micro-expressions and deliberate concealment by users have a significant impact.
By collecting in-vehicle user emotion evaluation data, including in-vehicle video and audio, extracting facial micro-expressions and keywords, and combining vehicle signal and operating condition data, the vehicle attributes to be evaluated are determined, and user emotions are aligned with vehicle attributes for objective evaluation.
It achieves accurate capture of user experience through natural interaction, provides accurate and objective evaluation, and can dynamically adjust vehicle functions to improve user experience.
Smart Images
Figure CN121810355A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of vehicle, and particularly relates to a vehicle user evaluation method, an electronic device, a storage medium and a computer program product. BACKGROUND
[0002] In today's increasingly competitive automobile industry, user experience has become a key factor in determining product success or failure.
[0003] The prior art mainly adopts questionnaire survey, user interview and focus group for the user experience evaluation method. However, the questionnaire survey evaluation depends on the user's subjective recall and is easily affected by cognitive bias. Macro label recognition cannot capture the fleeting micro-expression (duration 0.04-0.5 seconds) and is easily disturbed by the user's deliberate concealment.
[0004] Therefore, the vehicle user experience evaluation method of the prior art has obvious lag, strong subjectivity and "what the mouth says is not what the heart thinks", and the like. The user may not be able to provide real and effective feedback due to memory bias, social desirability effect or inability to accurately describe feelings. SUMMARY
[0005] Therefore, it is necessary to provide a vehicle user evaluation method, an electronic device, a storage medium and a computer program product to solve the technical problem that the prior art cannot provide real and effective feedback for the user experience evaluation of the vehicle.
[0006] The present application provides a vehicle user evaluation method, comprising: collecting a plurality of in-vehicle user emotional evaluation data in a test time and collecting a plurality of vehicle evaluation data in the test time; determining in-vehicle user emotions based on the in-vehicle user emotional evaluation data; determining vehicle evaluation attributes based on the vehicle evaluation data; dividing the test time into a plurality of time periods, and determining the evaluation results of the vehicle evaluation attributes in the same time period based on the in-vehicle user emotions.
[0007] Further, the in-vehicle user emotional evaluation data includes in-vehicle video, and the determination of the in-vehicle user emotions based on the in-vehicle user emotional evaluation data comprises: extracting facial micro-expression from the in-vehicle video, and determining in-vehicle user emotions based on the facial micro-expression.
[0008] Further, the in-vehicle user emotional evaluation data includes in-vehicle audio, and the determination of the in-vehicle user emotions based on the in-vehicle user emotional evaluation data comprises: extracting a keyword, a key sentence, and / or an acoustic feature from the in-vehicle audio, determining an in-vehicle user emotion according to the keyword and the acoustic feature.
[0009] Further, the vehicle to be evaluated data comprises in-vehicle video and vehicle signal, the attribute to be evaluated comprises a part to be evaluated and / or a function to be evaluated, and the determining the attribute to be evaluated of the vehicle based on the vehicle to be evaluated data comprises: extracting a user action from the in-vehicle video; if the user action is contacting a vehicle signal triggering part, obtaining the vehicle signal in the same time period, and determining the function to be evaluated according to the vehicle signal; if the user action is not contacting the vehicle signal triggering part, taking the part to be evaluated indicated by the user action as the part to be evaluated.
[0010] Further, the vehicle to be evaluated data comprises in-vehicle audio, the attribute to be evaluated comprises a part to be evaluated and / or a function to be evaluated, and the determining the attribute to be evaluated of the vehicle based on the vehicle to be evaluated data comprises: extracting a vehicle part keyword or a vehicle function keyword from the in-vehicle audio, taking the part to be evaluated indicated by the extracted vehicle part keyword as the part to be evaluated, and taking the function to be evaluated indicated by the extracted vehicle function keyword as the function to be evaluated.
[0011] Further, the vehicle to be evaluated data comprises vehicle working condition data, the attribute to be evaluated comprises a working condition to be evaluated, and the determining the attribute to be evaluated of the vehicle based on the vehicle to be evaluated data comprises: taking the working condition indicated by the vehicle working condition data as the working condition to be evaluated.
[0012] Further, the method further comprises: if the in-vehicle user emotion is a negative emotion and the attribute to be evaluated of the vehicle in the same time period is a function to be evaluated, reducing the display frequency of the function to be evaluated and / or displaying prompt information of the function to be evaluated; if the in-vehicle user emotion is a positive emotion and the attribute to be evaluated of the vehicle in the same time period is a function to be evaluated, displaying other functions related to the function to be evaluated.
[0013] The present application provides an electronic device, comprising: at least one processor; and, a memory in communication connection with the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the vehicle user evaluation method as described above.
[0014] The present application provides a storage medium storing computer instructions for performing all steps of the vehicle user evaluation method as described above when the computer executes the computer instructions.
[0015] The present application provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the vehicle user evaluation method as described above.
[0016] The present application collects in-vehicle user emotional evaluation data and vehicle to-be-evaluated data, determines in-vehicle user emotions based on the in-vehicle user emotional evaluation data, determines vehicle to-be-evaluated attributes based on the vehicle to-be-evaluated data, and then aligns the in-vehicle user emotions with the vehicle to-be-evaluated attributes to determine the evaluation results of the vehicle to-be-evaluated attributes in the same time period based on the in-vehicle user emotions. The present application does not require a wearable device, maintains natural interaction, can capture experience evaluation in a natural state of the user, and accurately matches the in-vehicle user emotions with the vehicle to-be-evaluated attributes, thus providing accurate and objective evaluation. BRIEF DESCRIPTION OF DRAWINGS
[0017] Figure 1 FIG. 1 is a workflow diagram of a vehicle user evaluation method according to an embodiment of the present application; Figure 2 FIG. 2 is a workflow diagram of a vehicle user evaluation method according to another embodiment of the present application; Figure 3 FIG. 3 is a system schematic diagram of a vehicle user evaluation system according to a preferred embodiment of the present application; Figure 4 FIG. 4 is a workflow diagram of a vehicle user evaluation method according to a preferred embodiment of the present application; Figure 5 FIG. 5 is a workflow diagram of signal emotion mapping according to a preferred embodiment of the present application; Figure 6 FIG. 6 is a hardware structure schematic diagram of an electronic device according to the present application. DETAILED DESCRIPTION
[0018] The specific embodiments of the present application will be further described below with reference to the accompanying drawings. Identical parts are denoted by identical reference numerals in the drawings. It should be noted that the words "front", "back", "left", "right", "up" and "down" used in the following description refer to the directions in the drawings, and the words "inner" and "outer" refer to the directions towards or away from the geometric center of a particular part.
[0019] As Figure 1 FIG. 1 is a workflow diagram of a vehicle user evaluation method according to an embodiment of the present application, which includes: Step S101, collecting a plurality of in-vehicle user emotional evaluation data in a test time and a plurality of vehicle to-be-evaluated data in the test time; Step S102: Determine the in-vehicle user's emotions based on the in-vehicle user emotion evaluation data; Step S103: Determine the vehicle attributes to be evaluated based on the vehicle data to be evaluated; Step S104: Divide the test time into multiple time periods, and determine the evaluation results of the vehicle's attributes to be evaluated within the same time period based on the emotions of the users inside the vehicle.
[0020] Specifically, this invention can be applied to electronic devices with processing capabilities, such as vehicle controllers. For example, the Electronic Control Unit (ECU) of a vehicle.
[0021] First, perform step S101 to collect multiple in-vehicle user emotion evaluation data and multiple vehicle evaluation data during the test period.
[0022] Specifically, after the test begins, data is collected using the acquisition device within the acquisition time.
[0023] The in-vehicle user emotion evaluation data is used to determine the emotions of in-vehicle users, including but not limited to: in-vehicle video, in-vehicle audio, etc.
[0024] The vehicle data to be evaluated is used to determine the vehicle's attributes to be evaluated, including but not limited to vehicle signals, in-vehicle video, in-vehicle audio, and vehicle operating condition data.
[0025] For example, such as Figure 3 As shown, multimodal data can be collected through the in-vehicle multimodal receiving layer 31. For example, facial micro-expression information of the driver and front passenger can be collected by the in-vehicle infrared camera as in-vehicle user emotion evaluation data, or in-vehicle audio can be recorded by the in-vehicle audio acquisition device as in-vehicle user emotion evaluation data. At the same time, key nodes of the vehicle are marked, and the user's operation information or the vehicle's operating condition data at this moment are recorded as vehicle evaluation data.
[0026] Then, step S102 is executed to determine the in-vehicle user's emotions based on the in-vehicle user emotion evaluation data.
[0027] Specifically, after data collection is completed, such as Figure 3 As shown, the in-vehicle system captures key facial information (such as downturned corners of the mouth) of the customer through the micro-expression feature extraction layer 32, and then determines the in-vehicle user's emotion based on the facial micro-expressions through the emotion mapping layer 33. Alternatively, it can extract keywords from in-vehicle audio and determine the in-vehicle user's emotion based on the keywords.
[0028] Then, step S103 is executed to determine the vehicle attributes to be evaluated based on the vehicle evaluation data.
[0029] Specifically, the vehicle data to be evaluated is used to determine the vehicle attributes to be evaluated, which include, but are not limited to: components to be evaluated, functions to be evaluated, and operating conditions to be evaluated.
[0030] The components to be evaluated include, but are not limited to, vehicle components such as steering wheels, seats, trim panels, and ambient lighting. The functions to be evaluated include, but are not limited to, vehicle infotainment systems and ambient lighting effects. The operating conditions to be evaluated include, but are not limited to, constant speed driving, rapid acceleration, emergency braking, and sharp turning.
[0031] For example, the vehicle's infotainment system can be identified based on vehicle function trigger information, and the functions triggered by these functions can be used as the functions to be evaluated.
[0032] Finally, step S104 is executed to divide the test time into multiple time periods, and the evaluation results of the vehicle's attributes to be evaluated within the same time period are determined based on the emotions of the users inside the vehicle.
[0033] Specifically, the entire data collection period is divided into multiple time segments. The in-vehicle user emotions identified by the in-vehicle user emotion evaluation data collected within the same time segment are linked to the vehicle attributes to be evaluated identified by the vehicle evaluation data collected within the same time segment. In-vehicle user emotions can include both negative and positive emotions, thus evaluating the vehicle attributes to be evaluated within that time segment based on the in-vehicle user emotions within that segment. The evaluated vehicle attributes can then be synchronized to the cloud backend, where they are assigned scores according to vehicle function and categorized based on different age groups. For example... Figure 3 As shown, vehicle evaluation can be fed back through the vehicle feedback mapping layer 34.
[0034] This invention collects in-vehicle user emotion evaluation data and vehicle evaluation data. It determines the in-vehicle user's emotion based on the in-vehicle emotion evaluation data and the vehicle's evaluation attributes based on the vehicle evaluation data. Then, it aligns the in-vehicle user's emotion with the vehicle's evaluation attributes to determine the evaluation result of the vehicle's evaluation attributes within the same time period. This invention requires no wearable devices, maintains natural interaction, captures the user's experience evaluation in a natural state, and accurately matches the in-vehicle user's emotion with the vehicle's evaluation attributes, resulting in accurate and objective evaluation.
[0035] like Figure 2 The diagram shown is a flowchart of a vehicle user evaluation method according to another embodiment of the present invention, including: Step S201: Collect multiple in-vehicle user emotion evaluation data and multiple vehicle evaluation data during the test period. The in-vehicle user emotion evaluation data includes in-vehicle video and / or in-vehicle audio. The vehicle evaluation data includes in-vehicle video, vehicle signals, in-vehicle audio, and / or vehicle operating condition data. The attributes to be rated include: components to be evaluated, functions to be evaluated, and / or operating conditions to be evaluated.
[0036] Step S202: Extract facial micro-expressions from the in-vehicle video, and determine the emotions of the users in the vehicle based on the facial micro-expressions; and / or Keywords, key phrases, and / or acoustic features are extracted from the in-vehicle audio, and the emotions of the in-vehicle users are determined based on the keywords and acoustic features.
[0037] Step S203: Extract user actions from the in-vehicle video; If the user action is to touch the vehicle signal triggering part, then the vehicle signals within the same time period are acquired, and the function to be evaluated is determined based on the vehicle signals. If the user's action does not contact the vehicle signal trigger point, the vehicle component indicated by the user's action will be considered the component to be evaluated. And / or Step S204: Extract vehicle component keywords or vehicle function keywords from the in-vehicle audio. The vehicle components represented by the extracted vehicle component keywords are designated as components to be evaluated, and the vehicle functions represented by the extracted vehicle function keywords are designated as functions to be evaluated. And / or Step S205: The operating condition represented by the vehicle operating condition data is taken as the operating condition to be evaluated.
[0038] Step S206: Divide the test time into multiple time periods, and determine the evaluation results of the vehicle's attributes to be evaluated within the same time period based on the emotions of the users inside the vehicle.
[0039] Step S207: If the in-vehicle user's emotion is negative and the vehicle's attribute to be evaluated at the same time period is a function to be evaluated, then reduce the display frequency of the function to be evaluated and / or display the prompt information of the function to be evaluated. If the in-vehicle user's emotion is positive, and the vehicle's attribute to be evaluated at the same time period is a function to be evaluated, then other functions related to the function to be evaluated will be displayed.
[0040] Micro-expressions are involuntary, rapid facial expressions that last only 1 / 25 to 1 / 5 of a second. They can realistically reflect a person's inner emotional state (such as pleasure, disgust, fear, surprise, anger, sadness, and contempt). Applying micro-expression analysis technology to a real-world vehicle environment can obtain objective and authentic user feedback; enable dynamic and continuous experience evaluation; and uncover deeper user needs.
[0041] This embodiment is a method for detecting users' feelings about vehicle evaluation based on facial micro-expression recognition technology. By capturing and analyzing the facial micro-expressions generated by users when they come into contact with the vehicle in real time, it automatically identifies their potential emotional evaluation of the vehicle's design, functions, or experience.
[0042] Specifically, step S201 is executed first to collect multiple in-vehicle user emotion evaluation data and multiple vehicle evaluation data during the test period. The in-vehicle user emotion evaluation data includes in-vehicle video and / or in-vehicle audio. The vehicle evaluation data includes in-vehicle video, vehicle signals, in-vehicle audio, and / or vehicle operating condition data. The attributes to be rated include: components to be evaluated, functions to be evaluated, and / or operating conditions to be evaluated.
[0043] Specifically, step S201 is the data acquisition stage, which involves collecting in-vehicle data, including but not limited to: 1) Use an in-vehicle infrared camera (ensuring stability in low light) to capture in-vehicle video in real time. Preferably, the in-vehicle video is a video stream of the user's face; 2) Vehicle signals used to indicate the vehicle status when an event occurs (such as the moment of touch screen operation, seat adjustment node, etc.). 3) Record in-vehicle audio using an in-vehicle audio recording device; 4) Record vehicle operating data, including but not limited to: vehicle speed, acceleration, deceleration, steering, etc.
[0044] Then, step S202 is executed to extract facial micro-expressions from the in-vehicle video, and to determine the emotions of the in-vehicle user based on the facial micro-expressions; and / or Keywords, key phrases, and / or acoustic features are extracted from the in-vehicle audio, and the emotions of the in-vehicle users are determined based on the keywords and acoustic features.
[0045] Specifically, step S202 is the feature extraction step. Extraction includes, but is not limited to: 1) A temporal segmentation model is used to locate micro-expression regions, and the micro-expression extraction module extracts micro-expression features with reference to FACS encoding.
[0046] 2) Audio data extraction: AI identifies and extracts the voice content, and outputs the topic corresponding to the time point.
[0047] One approach involves extracting facial micro-expressions from in-vehicle videos. Time-series segmentation can be performed to align the micro-expression data with the vehicle's evaluation data along the timeline. An analysis method based on multi-dimensional test data conjugate display using time-series scales can then be employed to identify micro-expression changes caused by the evaluation conditions, components, and / or functions. For example, analyzing the intensity and duration of driver micro-expressions during rapid acceleration can assess acceleration smoothness.
[0048] Existing technologies, such as facial behavior coding systems, can be used to extract facial micro-expressions. Specifically, multiple sets of data quantification index combinations can be set to determine whether a face meets the corresponding data quantification index combination, and the emotional state corresponding to the data quantification index combination is taken as the emotion of the user in the car.
[0049] Table 1. Mapping Relationship between FACS Facial 3D Features, Motion Units, and Emotions
[0050] According to the Facial Behavior Coding System (FACS), the mapping relationship between facial 3D features, motion units, and emotions is shown in Table 1. For example, when the displacement vector of the corner of the mouth, the rate of change of the nasolabial fold depth, and the area of the 3D folds at the corners of the eyes all meet the preset threshold range, it indicates that the facial expression is the corresponding pleasant / satisfied emotion.
[0051] In some embodiments, a "differential privacy module" (such as real-time facial feature desensitization) is also provided to meet regulatory requirements.
[0052] On the other hand, the emotions of people can be determined based on the collected in-vehicle audio. Specifically, emotions can be determined from the in-vehicle audio based on acoustic features, keywords, and key sentence structures.
[0053] Acoustic characteristics include, but are not limited to, pitch, speech rate, volume, and frequency distribution. Table 2 shows the mapping relationship between the collected audio and the emotions of the person.
[0054] Table 2. Mapping relationship between collected audio and human emotions
[0055] In some embodiments, the step of extracting facial micro-expressions from the in-vehicle video and determining the in-vehicle user's emotion based on the facial micro-expressions; and / or extracting keywords, key sentence structures, and / or acoustic features from the in-vehicle audio and determining the in-vehicle user's emotion based on the keywords and the acoustic features includes: Extract facial micro-expressions from the in-vehicle video, and determine the emotions of the users in the vehicle based on the facial micro-expressions; or Keywords, key phrases, and / or acoustic features are extracted from the in-vehicle audio; based on the keywords and acoustic features, the emotions of the in-vehicle user are determined; or Facial micro-expressions are extracted from the in-vehicle video. Based on the facial micro-expressions, the emotion of the first in-vehicle user is determined. Keywords, key phrases, and / or acoustic features are extracted from the in-vehicle audio. Based on the keywords and acoustic features, the emotion of the second in-vehicle user is determined. When the emotions of the first and second in-vehicle users are different, a first priority corresponding to the emotion of the first in-vehicle user and a second priority corresponding to the emotion of the second in-vehicle user are obtained. Based on the priority relationship between the first and second priorities, either the emotion of the first or the emotion of the second in-vehicle user is determined as the in-vehicle user emotion. When the emotions of the first and second in-vehicle users are the same, either the emotion of the first or the emotion of the second in-vehicle user is determined as the in-vehicle user emotion.
[0056] Specifically, the emotions of users inside the vehicle can be determined based on facial micro-expressions, in-vehicle audio, or a combination of facial micro-expressions and in-vehicle audio.
[0057] When determining the emotions of a user inside the vehicle based on both facial micro-expressions and in-vehicle audio, there may be discrepancies between the first user emotion determined by facial micro-expressions and the second user emotion determined by in-vehicle audio. In this case, to resolve emotional conflicts and determine a unique and reasonable in-vehicle emotion, embodiments of this application determine the final user emotion based on a preset priority assigned to the first and second user emotions.
[0058] Specifically, when the emotions of the first user in the car and the emotions of the second user in the car are different, a predefined priority rule will be queried (i.e., the first priority corresponding to the emotion of the first user in the car and the second priority corresponding to the emotion of the second user in the car will be obtained, and the priority relationship between the two will be compared), and the emotion of the first user in the car or the emotion of the second user in the car with higher priority will be selected as the final emotion of the user in the car; conversely, when the emotions of the first user in the car and the emotions of the second user in the car are the same, the consistent emotion of the user in the car can be directly determined as the final emotion of the user in the car without priority arbitration.
[0059] Because stronger emotions better reflect user intent, they should be addressed more dynamically. Therefore, as an example, stronger emotions can be prioritized. For instance, anger is stronger than disgust, and disgust is stronger than pleasure. Therefore, anger could be prioritized over disgust, and disgust over pleasure.
[0060] The vehicle evaluation data can be of various types to meet the needs of different types of vehicle evaluation attributes. Therefore, steps S203, S204, or S205 can be executed according to different types of vehicle evaluation attributes to determine the vehicle evaluation attributes based on the vehicle evaluation data.
[0061] In step S203, the function or component to be evaluated is determined based on the in-vehicle video, and the attribute to be evaluated is either the function or the component to be evaluated. In step S203, user actions are extracted from the in-vehicle video. If the user action is to touch the vehicle signal triggering part, then the vehicle signals within the same time period are acquired, and the function to be evaluated is determined based on the vehicle signals. If the user's action does not touch the vehicle signal triggering part, the vehicle part indicated by the user's action will be the part to be evaluated.
[0062] Specifically, user actions are extracted from in-vehicle video using existing image recognition methods. If the user's action involves touching a vehicle signal trigger point, such as with a finger, it indicates that the user is operating a vehicle function. Based on vehicle signals within the same time period, the vehicle function triggered by the user can be identified as the function to be evaluated. Vehicle signal trigger points include, but are not limited to: in-vehicle infotainment screen buttons, and various buttons on the vehicle (including physical and virtual buttons).
[0063] In some embodiments, the step of acquiring vehicle signals within the same time period if the user's action is to touch the vehicle signal triggering part includes: acquiring vehicle signals of the component where the vehicle signal triggering part is located within the same time period if the user's action is to touch the vehicle signal triggering part.
[0064] For example, if a user's action is to tap an in-vehicle infotainment app, the system will acquire signals from other in-vehicle infotainment systems that were triggered within the same time period to determine the vehicle function activated by the user. Similarly, if a user's action is to press an ambient light switch, the system will acquire light signals from other vehicles within the same time period to determine the vehicle function activated by the user.
[0065] If the user's action does not touch the vehicle signal triggering part, the user's action instruction is identified. The vehicle part that the user's finger touches or points to during the action is the vehicle part indicated by the user's action, and that vehicle part is taken as the part to be evaluated.
[0066] Step S204 determines the component or function to be evaluated based on the in-vehicle audio, where the attribute to be evaluated is the component or function to be evaluated. Specifically, in step S204, vehicle component keywords or vehicle function keywords are extracted from the in-vehicle audio. The vehicle component represented by the extracted vehicle component keywords is taken as the component to be evaluated, and the vehicle function represented by the extracted vehicle function keywords is taken as the function to be evaluated.
[0067] Specifically, keywords are extracted from in-vehicle audio using existing audio analysis methods. These keywords can be vehicle component keywords or vehicle function keywords. More specifically, multiple vehicle component keywords and multiple vehicle function keywords can be pre-set. If the text extracted from the in-vehicle audio matches any of the vehicle component keywords or vehicle function keywords, then the matching vehicle component keyword or vehicle function keyword is used as the extracted vehicle component keyword or vehicle function keyword from the in-vehicle audio.
[0068] In step S205, the operating condition represented by the vehicle operating condition data is taken as the operating condition to be evaluated. The attribute to be evaluated is the operating condition to be evaluated.
[0069] Specifically, vehicle operating condition data is used to represent the current operating condition of the vehicle. Vehicle operating conditions include, but are not limited to: constant speed driving, rapid acceleration, sudden braking, sharp turning, etc. Vehicle operating condition data includes, but is not limited to: vehicle speed, acceleration, deceleration, steering, etc. By comparing the vehicle operating condition data with a preset range of operating conditions, the operating condition corresponding to the current vehicle operating condition data is determined as the operating condition to be evaluated.
[0070] Then, step S206 is executed, dividing the test time into multiple time periods, and determining the evaluation results of the vehicle's attributes to be evaluated within the same time period based on the emotions of the users inside the vehicle.
[0071] Specifically, step S206 performs emotion mapping to determine the evaluation results of the vehicle's attributes to be evaluated within the same time period based on the emotions of the users inside the vehicle. Table 3 shows an example of time alignment.
[0072] Table 3 Time Alignment Examples
[0073] Wherein, AU(tn) is the micro-expression combination information within the time period tn, V(tn) is the audio information within the time period tn, A(tn) is the touch or pointing information other than the human face within the time period tn, and P(tn) is the vehicle signal within the time period tn.
[0074] If the occupants in the vehicle are experiencing negative emotions, the corresponding vehicle attributes to be evaluated will be evaluated negatively. If the occupants are experiencing positive emotions, the corresponding vehicle attributes to be evaluated will be evaluated positively. For example: 1) Construct a vehicle scene-micro-expression emotion mapping table, such as: "Dislike / Dissatisfaction" (e.g., "corner of the mouth pulls down + eyelid tightens") + function to be evaluated → "Dissatisfaction with the operation logic"; "Dislike / Dissatisfaction" (e.g., "corners of the mouth downturned + eyelids tightened") + part to be evaluated → "Dissatisfaction with the part"; "Surprise / Alertness" (e.g., "eyes widen instantly + eyebrows rise") + function to be evaluated → "surprise at the novel function".
[0075] 2) Construct a voice emotion keyword-emotion mapping, such as: "xx is not easy to use" + function to be evaluated → "Dissatisfied with the function"; "Discomfort" + rapid acceleration condition → "Dissatisfaction with acceleration".
[0076] Then, real-time feedback is provided, including: 1) Mark high-frequency touchpoints that trigger negative micro-expressions and output dynamic optimization suggestions to the mobile app (showroom mode app) or backend server, etc. For example, if clicking on the in-vehicle app triggers frequent disgust expressions, it is recommended to optimize the app's loading logic / response time.
[0077] 2) Component data is archived in the cloud and categorized according to the advantages and disadvantages of different vehicle models.
[0078] 3) Provide on-board function guidance for parts that users are interested in or highly rate; Finally, step S207 can be executed: if the in-vehicle user's emotion is negative and the vehicle's attribute to be evaluated at the same time period is a function to be evaluated, then reduce the display frequency of the function to be evaluated and / or display the prompt information of the function to be evaluated. If the in-vehicle user's emotion is positive, and the vehicle's attribute to be evaluated at the same time period is a function to be evaluated, then other functions related to the function to be evaluated will be displayed.
[0079] Specifically, step S207 involves dynamic recovery of the functions to be evaluated on the vehicle side. For functions that users dislike, dynamic recovery measures are implemented, such as text prompts. For functions to be evaluated that evoke negative emotions in the vehicle user, the display frequency is reduced, or the function is prompted with text. For example, a shortcut to the function to be evaluated is provided to the user to reduce user dissatisfaction. For functions to be evaluated that evoke positive emotions in the vehicle user, other functions related to the function to be evaluated can be displayed. Specifically, other functions related to the function to be evaluated can be similar to other functions.
[0080] In some embodiments, it also includes: Optimize the emotional interaction of the smart cockpit, and dynamically adjust ambient lighting / fragrance based on the user's emotions; Guide the conversation to points of dislike, such as comparing it to car model XX, which is already XXX.
[0081] This embodiment requires no wearable devices, maintains natural interaction, and can capture the user's experience evaluation in a natural state. It accurately determines the in-vehicle user's emotions through micro-expressions or in-vehicle voice, and by dividing the test time, it achieves precise matching between in-vehicle user emotions and vehicle events (error <100ms). Simultaneously, it determines the vehicle's evaluable attributes through various types of vehicle data, thus enabling accurate evaluation of vehicle attributes generated by vehicle events based on in-vehicle emotions. This embodiment has strong scalability and adaptability, fitting multiple scenarios such as cockpits, display vehicles, and virtual cockpits, and can perform comparative mapping in complex scenarios such as in-vehicle cabins.
[0082] like Figure 3 The diagram shown is a system principle diagram of a vehicle user evaluation system according to the preferred embodiment of the present invention, including: an in-vehicle multimodal receiving layer 31, a feature extraction layer 32, an emotion mapping layer 33, and a vehicle feedback mapping layer 34.
[0083] Among them, the in-vehicle multimodal receiving layer 31 records real-vehicle data through real-time data recording of the vehicle's infotainment system; captures human faces and in-vehicle actions through an in-vehicle infrared camera (located on the steering wheel or above the center console); and records and recognizes in-vehicle audio through audio processing.
[0084] The feature extraction layer 32 performs key data capture on real vehicle data records, extracts micro-expression features from faces, and performs artificial intelligence (AI) recognition on audio recordings.
[0085] The emotion mapping layer 33 combines facial features and maps emotions to obtain the emotions of the users in the car. The vehicle feedback mapping layer 34 evaluates and provides feedback on the vehicle based on the emotions of the users inside the vehicle. This feedback is then sent to the cloud server, adjusted in real time on the vehicle's infotainment system, or sent to the vehicle display mode app.
[0086] like Figure 4 The diagram shown is a flowchart of a vehicle user evaluation method according to a preferred embodiment of the present invention, including: Step S401: User adjusts vehicle settings / vehicle default display status; Step S402, begin the experience evaluation; Step S403, in-vehicle data collection, including: Collect in-vehicle video stream data, extract video data, locate micro-expression regions and extract key features, and perform micro-expression evaluation mapping; Vehicle function trigger, vehicle function data recorded; Audio acquisition, AI recognition and analysis; Step S404: Associating vehicle components (body parts) with emotional data; Step S405: Evaluation data is organized, real-time optimization prompts are output, and cloud-based component data is categorized for new model development evaluation; Step S406: Dynamic recovery on the vehicle side. If the function evaluation is positive, similar functions will be prominently displayed; otherwise, such displays will be avoided and some textual reminders will be added.
[0087] like Figure 5 The diagram shown illustrates the workflow of signal emotion mapping according to the preferred embodiment of the present invention, including: Step S501: Extract audio keywords; In step S502, if the device includes directional components (such as a navigation / steering wheel), proceed to step S503; otherwise, proceed to step S507. Step S503: If emotional evaluation is included, the first part experience output method is adopted, and the evaluation of the directional part is output based on the emotional evaluation contained in the audio keywords; otherwise, step S507 is executed. Step S504: Obtain facial micro-expression data (AU) and record the timestamp of expression change (t); Step S505: Capture video information; Step S506: If the user touches the vehicle signal triggering part, extract the actual vehicle signal (such as vehicle infotainment function / steering wheel buttons, etc.) and confirm the vehicle function information (such as a certain application / function); otherwise, extract the vehicle component information (such as the surface of the center console, etc.). Step S507, Signal-Emotion Mapping Summary, adopts the second component experience output method, based on the emotional evaluation determined by facial micro-expressions and the vehicle component or vehicle function captured by video information.
[0088] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0089] like Figure 6 The diagram shown is a hardware structure schematic of an electronic device according to the present invention, comprising: At least one processor 601; and, A memory 602 is communicatively connected to at least one of the processors 601; wherein, The memory 602 stores instructions that can be executed by at least one of the processors to enable the at least one of the processors to perform the vehicle user evaluation method as described above.
[0090] Figure 6 Take the 601 processor as an example.
[0091] The electronic device may also include an input device 603 and a display device 604.
[0092] The processor 601, memory 602, input device 603 and display device 604 can be connected by a bus or other means. The figure shows an example of connection by a bus.
[0093] The memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the vehicle user evaluation method in the embodiments of this application, for example, Figure 1 , Figure 2 The method flow is shown. The processor 601 executes various functional applications and data processing by running non-volatile software programs, instructions, and modules stored in the memory 602, thereby implementing the vehicle user evaluation method in the above embodiments.
[0094] The memory 602 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the vehicle user evaluation method, etc. Furthermore, the memory 602 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 602 may optionally include memory remotely located relative to the processor 601, and these remote memories may be connected via a network to the apparatus performing the vehicle user evaluation method. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0095] The input device 603 can receive user clicks and generate signal inputs related to user settings and function control for vehicle user evaluation methods. The display device 604 may include a display screen or other display equipment.
[0096] When one or more modules are stored in the memory 602, and are run by one or more processors 601, the vehicle user evaluation method in any of the above method embodiments is executed.
[0097] This invention collects in-vehicle user emotion evaluation data and vehicle evaluation data. It determines the in-vehicle user's emotion based on the in-vehicle emotion evaluation data and the vehicle's evaluation attributes based on the vehicle evaluation data. Then, it aligns the in-vehicle user's emotion with the vehicle's evaluation attributes to determine the evaluation result of the vehicle's evaluation attributes within the same time period. This invention requires no wearable devices, maintains natural interaction, captures the user's experience evaluation in a natural state, and accurately matches the in-vehicle user's emotion with the vehicle's evaluation attributes, resulting in accurate and objective evaluation.
[0098] One embodiment of the present invention provides a storage medium that stores computer instructions, which, when executed by a computer, are used to perform all the steps of the vehicle user evaluation method as described above.
[0099] In the context of this disclosure, a storage medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The storage medium can be a machine-readable signal medium or a machine-readable storage medium. Optionally, the storage medium can be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), compact disc ROM (CD-ROM), magnetic tape, floppy disk, and optical data storage device.
[0100] One embodiment of the present invention provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements the vehicle user evaluation method as described above.
[0101] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A method for evaluating vehicle users, characterized in that, include: Collect emotional evaluation data of multiple in-vehicle users during the test period and collect evaluation data of multiple vehicles during the test period; The in-vehicle user's emotional state is determined based on the in-vehicle user emotional evaluation data. The vehicle attributes to be evaluated are determined based on the vehicle data to be evaluated. The test time is divided into multiple time periods, and the evaluation results of the vehicle's attributes to be evaluated within the same time period are determined based on the emotions of the users inside the vehicle.
2. The vehicle user evaluation method according to claim 1, characterized in that, The in-vehicle user emotion evaluation data includes in-vehicle video, and the step of determining the in-vehicle user emotion based on the in-vehicle user emotion evaluation data includes: Facial micro-expressions are extracted from the in-vehicle video, and the emotions of the users in the vehicle are determined based on the facial micro-expressions.
3. The vehicle user evaluation method according to claim 1, characterized in that, The in-vehicle user emotion evaluation data includes: in-vehicle audio; determining the in-vehicle user emotion based on the in-vehicle user emotion evaluation data includes: Keywords, key phrases, and / or acoustic features are extracted from the in-vehicle audio, and the emotions of the in-vehicle users are determined based on the keywords and acoustic features.
4. The vehicle user evaluation method according to claim 1, characterized in that, The vehicle data to be evaluated includes in-vehicle video and vehicle signals. The attributes to be rated include: components to be evaluated and / or functions to be evaluated. Determining the vehicle attributes to be evaluated based on the vehicle data to be evaluated includes: Extract user actions from the in-vehicle video; If the user action is to touch the vehicle signal triggering part, then the vehicle signals within the same time period are acquired, and the function to be evaluated is determined based on the vehicle signals. If the user's action does not touch the vehicle signal triggering part, the vehicle part indicated by the user's action will be the part to be evaluated.
5. The vehicle user evaluation method according to claim 1, characterized in that, The vehicle data to be evaluated includes in-vehicle audio, and the attributes to be rated include: components to be evaluated and / or functions to be evaluated. Determining the vehicle attributes to be evaluated based on the vehicle data to be evaluated includes: Vehicle component keywords or vehicle function keywords are extracted from the in-vehicle audio. The vehicle components represented by the extracted vehicle component keywords are used as components to be evaluated, and the vehicle functions represented by the extracted vehicle function keywords are used as functions to be evaluated.
6. The vehicle user evaluation method according to claim 1, characterized in that, The vehicle data to be evaluated includes vehicle operating condition data, and the attributes to be rated include: operating conditions to be evaluated. Determining the vehicle attributes to be evaluated based on the vehicle data to be evaluated includes: The operating conditions represented by the vehicle operating condition data are taken as the operating conditions to be evaluated.
7. The vehicle user evaluation method according to claim 1, characterized in that, Also includes: If the in-vehicle user's emotion is negative, and the vehicle's attribute to be evaluated at the same time period is a function to be evaluated, then reduce the display frequency of the function to be evaluated and / or display the prompt information of the function to be evaluated. If the in-vehicle user's emotion is positive, and the vehicle's attribute to be evaluated at the same time period is a function to be evaluated, then other functions related to the function to be evaluated will be displayed.
8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by at least one of the processors, which enable the at least one processor to perform the vehicle user evaluation method as described in any one of claims 1 to 7.
9. A storage medium, characterized in that, The storage medium stores computer instructions, which, when executed by the computer, are used to perform all the steps of the vehicle user evaluation method as described in any one of claims 1 to 7.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the vehicle user evaluation method as described in any one of claims 1 to 7.