Vehicle contextual model recommendation method and system
By acquiring vehicle status and multimodal data, a chain-like association graph is constructed for scenario pattern recommendation, which solves the problem that existing vehicle scenario pattern recommendation methods cannot adapt to dynamic changes in real time, thus improving the accuracy of recommendations and user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-03
AI Technical Summary
Existing vehicle scenario mode recommendation methods are unable to adapt to dynamic changes during driving in real time, resulting in low recommendation accuracy and failing to meet users' actual needs.
By acquiring vehicle status data, multimodal data of in-vehicle users, and external environment data, driving behavior and user emotions are classified, and a chain-like relationship graph is constructed between environmental factors, behavioral classifications, and emotion classifications. Based on this, scenario-based pattern recommendations are made.
It achieves good adaptability to dynamic changes during driving, and the scenario mode recommendation is highly accurate, reducing the number of times users need to manually switch scenario modes and improving the user experience.
Smart Images
Figure CN121786574A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and system for recommending vehicle scenario patterns. Background Technology
[0002] With the continuous development of intelligent vehicles, vehicle scenario modes, such as child mode, nap mode, and commuting mode, have become important functions to enhance the driving and riding experience. When users switch scenario modes, vehicle parameter configurations (such as seat position, air conditioning temperature, ambient lighting color, and audio settings) change accordingly to meet user needs in different scenarios.
[0003] In related technologies, scenario-based recommendation methods typically employ static rule-based approaches, relying on pre-defined fixed conditions. For example, scenario-based recommendations might be triggered based on a single dimension such as time (e.g., morning, noon, night) or location (e.g., home, office, shopping mall). However, this approach has several limitations. For instance, it cannot adapt to dynamic changes during driving, such as sudden weather changes or traffic congestion, leading to a disconnect between the recommended scenario-based recommendations and the user's actual needs, thus degrading the user experience. Summary of the Invention
[0004] This application provides a vehicle scenario mode recommendation method and system to solve the technical problems of scenario mode recommendation methods in related technologies, such as difficulty in adapting to dynamic changes during driving in real time and low accuracy.
[0005] This application provides a vehicle scenario mode recommendation method, the method comprising: acquiring vehicle status data, multimodal data of users inside the vehicle, and external environment data of the vehicle; Based on the vehicle status data, driving behavior is classified to obtain the behavior classification results; Based on the multimodal data, user emotions are classified to obtain the final emotion classification result; By filtering the external environment data using keywords, the environmental factor filtering results are obtained; Construct a chain-like relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results; Based on the aforementioned chain-like association graph, scenario pattern recommendations are made.
[0006] In one embodiment of this application, the vehicle status data includes the vehicle's speed, acceleration, and steering angle; The behavior classification results include sudden braking, sudden acceleration, and sharp turns.
[0007] In one embodiment of this application, the multimodal data includes: the user's facial image, voice data, physiological data, and steering wheel grip force data; The process of classifying user emotions based on the multimodal data to obtain the final emotion classification result includes: Feature extraction is performed on the facial image to obtain facial features; feature extraction is performed on the speech data to obtain speech features; feature extraction is performed on the physiological data to obtain physiological features; feature extraction is performed on the grip strength data to obtain grip strength features; The facial features, voice features, physiological features, and grip strength features are concatenated to obtain a concatenated feature sequence; temporal dependencies are extracted from the concatenated feature sequence to obtain a fused feature, which carries the temporal information of the concatenated feature sequence. Based on the facial features, user emotions are classified to obtain a first emotion classification result; based on the voice features, user emotions are classified to obtain a second emotion classification result; based on the physiological features, user emotions are classified to obtain a third emotion classification result; based on the grip strength features, user emotions are classified to obtain a fourth emotion classification result; based on the fusion features, user emotions are classified to obtain a fifth emotion classification result. A weighted voting mechanism is used to integrate the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result to obtain the final emotion classification result.
[0008] In one embodiment of this application, the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result all contain predicted probabilities for multiple emotion categories; the step of integrating the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result using a weighted voting mechanism to obtain the final emotion classification result includes: Any emotion category is identified as the target emotion category; the product of the predicted probability of the target emotion category in the first emotion classification result and a first preset weight is determined as a first value; the product of the predicted probability of the target emotion category in the second emotion classification result and a second preset weight is determined as a second value; the product of the predicted probability of the target emotion category in the third emotion classification result and a third preset weight is determined as a third value; the product of the predicted probability of the target emotion category in the fourth emotion classification result and a fourth preset weight is determined as a fourth value; the product of the predicted probability of the target emotion category in the fifth emotion classification result and a fifth preset weight is determined as a fifth value; the fifth preset weight is greater than any one of the first preset weight, the second preset weight, the third preset weight, and the fourth preset weight. The sum of the first value, the second value, the third value, the fourth value, and the fifth value is determined as the final probability value of the target emotion category; the remaining emotion categories are traversed to obtain the final probability value of each emotion category; based on the final probability values of multiple emotion categories, the final emotion classification result is obtained.
[0009] In one embodiment of this application, the final emotion classification result includes at least one final emotion category, and obtaining the final emotion classification result based on the final probability values of multiple emotion categories includes: If the final probability value of the current emotion category is greater than or equal to the preset emotion probability threshold, then the current emotion category is determined as the final emotion category.
[0010] In one embodiment of this application, constructing a chain-like relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results includes: The causal relationship of the environmental factor screening results, the behavioral classification results, and the final emotion classification results is quantified to obtain the chain association graph. The chain association graph includes the environmental factor screening results, the behavioral classification results, the final emotion classification results, a first side connecting the environmental factor screening results and the behavioral classification results, and a second side connecting the behavioral classification results and the final emotion classification results. The first side is labeled with a first path coefficient, and the second side is labeled with a second path coefficient. The first path coefficient characterizes the causal effect strength between the environmental factor screening results and the behavioral classification results, and the second path coefficient characterizes the causal effect strength between the behavioral classification results and the final emotion classification results.
[0011] In one embodiment of this application, the step of recommending scenario patterns based on the chain-like association graph includes: The chain-like association graph is matched with the matching conditions of multiple scenario patterns in the preset scenario pattern library to obtain the corresponding target scenario pattern. Based on the target scenario pattern, scenario pattern recommendation is completed.
[0012] In one embodiment of this application, the step of recommending a scenario pattern based on the target scenario pattern includes: The target scenario mode is sent to the vehicle's infotainment system to instruct the system to generate target scenario mode recommendation information and push it to the display device within the system. Feedback from the user is received; if the feedback indicates acceptance of the recommendation, the vehicle's current scenario mode is switched to the target scenario mode; if the feedback indicates rejection of the recommendation, an optimization instruction is sent to the vehicle scenario mode recommendation system, which is the system running the vehicle scenario mode recommendation method. Upon receiving the optimization instruction, the preset emotion classification model is updated based on the optimization instruction. The emotion classification model is used to classify user emotions based on the multimodal data to obtain the final emotion classification result.
[0013] In one embodiment of this application, the vehicle status data further includes the remaining charge of the vehicle's power battery; After performing scenario pattern recommendation based on the aforementioned chain-like association graph, the process further includes: The target power is obtained based on the remaining power and the preset power mapping relationship, wherein the power mapping relationship is the mapping relationship between the power battery power and the air conditioner power. The target power is sent to the vehicle's infotainment system to instruct the system to adjust the air conditioning power to the target power.
[0014] This application also provides a vehicle scenario mode recommendation system, the system comprising: The data acquisition module is used to acquire vehicle status data, multimodal data of users inside the vehicle, and external environmental data of the vehicle. The behavior classification module is used to classify driving behavior based on the vehicle status data and obtain the behavior classification result; The emotion classification module is used to classify user emotions based on the multimodal data and obtain the final emotion classification result. The environmental factor screening module is used to obtain environmental factor screening results by filtering the external environment data using keywords; The causal relationship construction module is used to construct a chain relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results; The scenario pattern recommendation module is used to recommend scenario patterns based on the chain-like association graph.
[0015] The beneficial effects of this application are as follows: This application proposes a vehicle scenario pattern recommendation method and system. The method includes: acquiring vehicle state data, multimodal data of users within the vehicle, and external environmental data of the vehicle; classifying driving behavior based on the vehicle state data to obtain behavior classification results; classifying user emotions based on the multimodal data to obtain a final emotion classification result; obtaining environmental factor filtering results by filtering external environmental data using keywords; constructing a chain-like association graph among the environmental factor filtering results, behavior classification results, and final emotion classification results; and performing scenario pattern recommendation based on the chain-like association graph. This method can adapt well to dynamic changes during driving, has high accuracy in scenario pattern recommendation, and is low in cost. Attached Figure Description
[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0017] In the attached diagram: Figure 1 This is a flowchart illustrating a vehicle scenario mode recommendation method provided in one embodiment of this application. Figure 2 This is a flowchart illustrating a specific embodiment of the vehicle scenario mode recommendation method provided in this application. Figure 3 This is a schematic diagram of the structure of a vehicle scenario mode recommendation system provided in one embodiment of this application; Figure 4 This is an example of an architecture diagram of a vehicle scenario mode recommendation system provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation
[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.
[0021] Please see Figure 1 , Figure 1 This is a flowchart illustrating a vehicle scenario mode recommendation method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes: S110: Acquire vehicle status data, multimodal data of users inside the vehicle, and external environmental data of the vehicle.
[0022] In some examples of this embodiment, vehicle status data may include the vehicle's CAN (Controller Area Network) bus data stream (such as vehicle speed, acceleration, and steering angle), the remaining charge of the power battery, and the vehicle's air conditioning status.
[0023] In some examples of this embodiment, the multimodal data of the user in the vehicle refers to the image, voice, physiological and behavioral data of the user (driver or passenger) in the vehicle collected by different types of sensors, such as the user's facial image, voice data and physiological data.
[0024] In some examples of this embodiment, the vehicle's external environment data includes map data (such as traffic congestion 2 kilometers ahead), weather data (such as sunny days, heavy rain, etc.), traffic event data (such as a car accident ahead), and environmental brightness, etc.
[0025] In some examples of this embodiment, vehicle status data, multimodal data of users inside the vehicle, and external environment data of the vehicle can be acquired simultaneously or sequentially.
[0026] S120: Based on the vehicle status data, classify driving behavior to obtain the behavior classification result.
[0027] In some examples of this embodiment, the behavior classification results are such as sudden braking, sudden acceleration, etc.
[0028] S130: Based on the multimodal data, perform user emotion classification to obtain the final emotion classification result.
[0029] In some examples of this embodiment, the final emotion classification results are such as anxiety, happiness, and anger.
[0030] Understandably, using multimodal data for user emotion classification can help improve the accuracy of emotion classification. Compared to emotion classification based on single-modal data, such as relying solely on user images (which is easily affected by environmental interference, such as strong light affecting the camera, leading to reduced accuracy), it effectively improves the accuracy of emotion classification and avoids misclassification due to environmental interference.
[0031] S140: By filtering the external environment data using keywords, the environmental factor screening results are obtained.
[0032] In some examples of this embodiment, a preset keyword filtering network can be used to filter external environmental data using keywords, resulting in environmental factor filtering results. These results include multiple keywords, such as heavy rain, traffic congestion, traffic accidents ahead, and highways at night. It is understood that this embodiment, by filtering external environmental data using keywords, facilitates subsequent scenario-based recommendations.
[0033] S150: Construct a chain-like relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results.
[0034] In some examples of this embodiment, the chain-like relationship graph quantifies the causal relationship between the environmental factor screening results, behavioral classification results, and the final emotion classification results, which helps improve the accuracy of subsequent scenario pattern recommendations. For example: traffic congestion → sudden braking → anxiety, etc. Compared to simply classifying emotions, this embodiment, by constructing a chain-like relationship graph between the environmental factor screening results, behavioral classification results, and the final emotion classification results, can easily identify the causes of emotions, such as effectively distinguishing between "anxiety due to traffic congestion" and "irritability due to fatigue," thereby providing more accurate data support for subsequent scenario pattern recommendations and increasing the user's acceptance rate (adoption rate) of the recommended scenario patterns.
[0035] S160: Based on the chain-like association graph, perform scenario pattern recommendation.
[0036] Understandably, by adopting the above methods, the accuracy of vehicle scenario mode recommendations can be significantly improved, and the system can better adapt to dynamic changes during driving (such as sudden rainstorms), providing scenario mode recommendations that meet user needs. Furthermore, by employing these methods, the number of times users need to manually switch scenario modes can be effectively reduced, thus enhancing the user experience.
[0037] In some embodiments, the vehicle status data includes the vehicle's speed, acceleration, and steering angle; the behavior classification results include emergency braking, rapid acceleration, and sharp turns.
[0038] In some examples of this embodiment, existing behavior classification models can be used to classify driving behavior in vehicle state data, thereby improving the accuracy of driving behavior classification.
[0039] In some embodiments, the multimodal data includes: the user's facial image, voice data, physiological data, and steering wheel grip force data. In some examples of this embodiment, the user's facial image can be acquired by a camera preset in the vehicle cabin. Voice data can be acquired by a voice acquisition device (such as a microphone) preset in the vehicle cabin. Physiological data can be acquired by a PPG (Photoplethysmography) sensor, heart rate monitoring seat, etc., preset in the vehicle cabin. Steering wheel grip force data can be acquired by a pressure sensor preset on the steering wheel.
[0040] In some embodiments, the step of classifying user emotions based on the multimodal data to obtain a final emotion classification result includes: S1301: Extract features from the facial image to obtain facial features; extract features from the speech data to obtain speech features; extract features from the physiological data to obtain physiological features; extract features from the grip strength data to obtain grip strength features.
[0041] In some examples of this embodiment, facial images can be input into a preset facial landmark detection network, such as YOLOv8 (an object detection framework), to perform face location recognition and feature extraction, thereby obtaining facial features (such as facial landmark features).
[0042] In some examples of this embodiment, speech data can be input into a preset speech feature extraction network to extract features and obtain speech features output by the speech feature extraction network, such as fundamental frequency, speech rate, energy and other acoustic features.
[0043] In some examples of this embodiment, physiological data (such as photovolume pulse waves collected by a PPG sensor) can be input into a preset physiological data feature extraction network for feature extraction to obtain physiological features, such as heart rate variability (HRV). It is understood that physiological data is related to the user's stress levels; by collecting the user's physiological data and performing feature extraction, the accuracy of subsequent scenario-based pattern recommendations can be improved.
[0044] In some examples of this embodiment, grip strength data can be input into a preset grip strength data feature extraction network to obtain grip strength features, such as grip strength change rate.
[0045] S1302: The facial features, voice features, physiological features, and grip strength features are concatenated to obtain a concatenated feature sequence; temporal dependencies are extracted from the concatenated feature sequence to obtain a fused feature, wherein the fused feature carries the temporal information of the concatenated feature sequence.
[0046] In some examples of this embodiment, the spliced feature sequence can be input into an existing gated recurrent unit (GRU) model to extract temporal dependencies and obtain the fused features output by the gated recurrent unit model.
[0047] Understandably, by adopting the above method, it is possible to achieve better fusion of multiple features, thereby enabling the fused features to contain more useful information, such as temporal information.
[0048] S1303: Based on the facial features, classify the user's emotions to obtain a first emotion classification result; based on the voice features, classify the user's emotions to obtain a second emotion classification result; based on the physiological features, classify the user's emotions to obtain a third emotion classification result; based on the grip strength features, classify the user's emotions to obtain a fourth emotion classification result; based on the fusion features, classify the user's emotions to obtain a fifth emotion classification result.
[0049] In some examples of this embodiment, facial features, voice features, physiological features, grip strength features, and fusion features can be input into a preset emotion classification model to classify emotions and obtain the corresponding emotion classification results.
[0050] S1304: Using a weighted voting mechanism, the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result are integrated to obtain the final emotion classification result.
[0051] Understandably, this embodiment employs a combination of unimodal and multimodal emotion classification, resulting in multiple distinct emotion classification outcomes. Finally, a weighted voting process is used to determine the final emotion classification result, effectively improving the accuracy of emotion classification.
[0052] In some embodiments, the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result all contain predicted probabilities for multiple emotion categories; the integration of the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result using a weighted voting mechanism to obtain the final emotion classification result includes: Any emotion category is identified as the target emotion category; the product of the predicted probability of the target emotion category in the first emotion classification result and a first preset weight is determined as a first value; the product of the predicted probability of the target emotion category in the second emotion classification result and a second preset weight is determined as a second value; the product of the predicted probability of the target emotion category in the third emotion classification result and a third preset weight is determined as a third value; the product of the predicted probability of the target emotion category in the fourth emotion classification result and a fourth preset weight is determined as a fourth value; the product of the predicted probability of the target emotion category in the fifth emotion classification result and a fifth preset weight is determined as a fifth value; the fifth preset weight is greater than any one of the first preset weight, the second preset weight, the third preset weight, and the fourth preset weight. The sum of the first value, the second value, the third value, the fourth value, and the fifth value is determined as the final probability value of the target emotion category; the remaining emotion categories are traversed to obtain the final probability value of each emotion category; based on the final probability values of multiple emotion categories, the final emotion classification result is obtained.
[0053] In some examples of this embodiment, the specific values of the first preset weight, the second preset weight, the third preset weight, the fourth preset weight, and the fifth preset weight can be set according to actual needs, or can be dynamically adjusted according to the environment, as long as the fifth preset weight is the maximum value among the above five preset weights.
[0054] In some examples of this embodiment, the emotion category with the highest final probability value can be determined as the final emotion classification result.
[0055] Understandably, the final emotion classification result obtained by using the above method has a high degree of accuracy.
[0056] Considering the complexity of user emotions—that is, more than one emotion may arise simultaneously, such as both anger and anxiety—this embodiment provides a method for determining the final emotion classification result. Specifically, the final emotion classification result includes at least one final emotion category, and obtaining the final emotion classification result based on the final probability values of multiple emotion categories includes: If the final probability value of the current emotion category is greater than or equal to the preset emotion probability threshold, then the current emotion category is determined as the final emotion category.
[0057] In some examples of this embodiment, the emotion probability threshold can be set according to the actual situation, such as 0.7. It is understood that by adopting the above method, the need for judging complex emotions can be better met, which helps to improve the accuracy of subsequent scenario pattern recommendations.
[0058] In some embodiments, constructing a chain-like relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results includes: S1501: Quantify the causal relationship of the environmental factor screening results, the behavior classification results, and the final emotion classification results to obtain the chain association graph. The chain association graph includes the environmental factor screening results, the behavior classification results, the final emotion classification results, a first side connecting the environmental factor screening results and the behavior classification results, and a second side connecting the behavior classification results and the final emotion classification results. The first side is labeled with a first path coefficient, and the second side is labeled with a second path coefficient. The first path coefficient characterizes the causal effect strength between the environmental factor screening results and the behavior classification results, and the second path coefficient characterizes the causal effect strength between the behavior classification results and the final emotion classification results.
[0059] In some examples of this embodiment, the results of environmental factor screening, behavior classification, and final emotion classification can be input into an existing structural equation model (SEM) to quantify causal relationships and obtain a chain-like association graph output by the structural equation model.
[0060] Understandably, the above method facilitates the construction of a chain-like relationship diagram of "environment → behavior → emotion" with high accuracy.
[0061] In some embodiments, the scenario pattern recommendation based on the chain association graph includes: S1601: Match the chain association graph with the matching conditions of multiple scenario patterns in the preset scenario pattern library to obtain the corresponding target scenario pattern.
[0062] In some examples of this embodiment, the scenario modes in the scenario mode library support custom configuration, such as commuting relaxation mode, focus mode, leisure mode, and energy-saving mode. Each scenario mode has corresponding matching conditions. For example, if the behavior classification result is sudden braking, the final emotion classification result is anxiety, the environmental factor screening result is congestion, and the first path coefficient and the second path in the chain association graph are both greater than the preset coefficient threshold (such as 0.5), then the scenario mode that matches it is determined to be commuting relaxation mode, etc. If the behavior classification result is abnormal speed control, the final emotion classification result is fatigue, the environmental factor screening result is nighttime highway, and the first path coefficient and the second path in the chain association graph are both greater than the preset coefficient threshold (such as 0.5), then the scenario mode that matches it is determined to be focus mode (brighten the HUD (Head-Up Display) and turn on lane keeping assist). If the behavior classification result is normal driving, the final emotion classification result is pleasant, the environmental factor screening result is normal road conditions, and the coefficients of the first and second paths in the chain association graph are both greater than the preset coefficient threshold (e.g., 0.5), then the matching scenario mode is determined to be the leisure mode (playing the user's preferred playlist and adjusting the ambient lighting to a warm color). Understandably, each scenario mode has a corresponding set of parameters, such as screen brightness, air conditioning temperature, and music type.
[0063] In some examples of this embodiment, the matching conditions for the scenario mode can be set according to the actual situation.
[0064] In some examples of this embodiment, a corresponding dynamic weight can be set for each node (environmental factor screening result, behavior classification result, and final emotion classification result) in the chain association graph. For example, the dynamic weight of the environmental factor screening result is 60%, the dynamic weight of the behavior classification result is 10%, and the dynamic weight of the final emotion classification result is 30%. Furthermore, a score lookup table is pre-configured, which contains a first score corresponding to different environmental factor screening results, a second score corresponding to different behavior classification results, and a third score corresponding to different final emotion classification results. When performing scenario pattern recommendation, the first score, second score, and third score can be obtained based on the environmental factor screening result, behavior classification result, final emotion classification result in the chain association graph, and the score lookup table. The first score, the dynamic weight of the environmental factor screening result, and the first path coefficient are multiplied to obtain the first product value. The second score, the dynamic weight of the behavior classification result, and the second path coefficient are multiplied to obtain the second product value. The third score is multiplied by the dynamic weight corresponding to the final emotion classification result to obtain the third product value. The first product value, the second product value, and the third product value are added together to obtain the target value. The target value is matched with the scenario patterns in the preset pattern matching table to obtain the target scenario pattern. Each scenario pattern in the pattern matching table corresponds to a different range of target values.
[0065] S1602: Based on the target scenario mode, complete the scenario mode recommendation.
[0066] Understandably, the above methods can better meet the personalized needs of different users and achieve scenario-based recommendations that are highly aligned with user needs.
[0067] In some embodiments, the step of recommending a scenario pattern based on the target scenario pattern includes: 1. The target scenario mode is sent to the vehicle's infotainment system to instruct the infotainment system to generate target scenario mode recommendation information based on the target scenario mode, and push the target scenario mode recommendation information to the display device in the infotainment system; feedback information from the user is received; if the feedback information is acceptance of the recommendation, the current scenario mode of the vehicle is switched to the target scenario mode; if the feedback information is rejection of the recommendation, an optimization instruction is sent to the vehicle scenario mode recommendation system, which is a system that runs the vehicle scenario mode recommendation method.
[0068] In some examples of this embodiment, the display device is equipped with an "Accept Recommendation" button and a "Reject Recommendation" button. If the user triggers (touches or clicks, etc.) the "Accept Recommendation" button, the vehicle system automatically generates feedback information indicating acceptance of the recommendation; if the user triggers the "Reject Recommendation" button, the vehicle system automatically generates feedback information indicating rejection of the recommendation.
[0069] 2. Receive the optimization instruction, and update the preset emotion classification model based on the optimization instruction. The emotion classification model is used to classify user emotions based on the multimodal data to obtain the final emotion classification result.
[0070] In some examples of this embodiment, upon receiving the optimization instruction, the emotion classification model can be updated through online learning to adapt to user habits.
[0071] In some embodiments, the vehicle status data may also include the remaining charge of the vehicle's power battery.
[0072] In some embodiments, after performing scenario pattern recommendation based on the chain-like association graph, the method further includes: 1. Based on the remaining power and the preset power mapping relationship, the target power is obtained, wherein the power mapping relationship is the mapping relationship between the power battery power and the air conditioner power.
[0073] 2. The target power is sent to the vehicle's infotainment system to instruct the infotainment system to adjust the air conditioning power to the target power.
[0074] Understandably, by using the above methods, it is possible to achieve accurate recommendations for scenario modes while also optimizing the air conditioning power based on the remaining charge of the power lithium battery, thereby achieving intelligent energy saving and reducing the overall vehicle energy consumption.
[0075] In some embodiments, the user's acceptance rate of the recommended scenario pattern can also be counted. If the number of times a user rejects a certain recommended scenario pattern exceeds a preset threshold, such as 5 times, the scenario pattern recommendation strategy is optimized to reduce the number of times the scenario pattern is recommended.
[0076] The following explanation uses the chain-like relationship of "congestion → sudden braking → anxiety" as an example to illustrate the process of the vehicle scenario mode recommendation method in the above embodiments. Please refer to [link / reference]. Figure 2 First, acquire external environmental data of the vehicle, and then perform keyword filtering based on this data. For example, obtain map data (such as congestion 2 kilometers ahead for 15 minutes) from map software through a pre-defined traffic API (Application Programming Interface). Then, perform keyword filtering on this map data to obtain environmental factor filtering results, such as congestion.
[0077] Secondly, driving behavior is classified. Specifically, vehicle state data is acquired, including vehicle speed (e.g., 0-30 km / h), acceleration (e.g., -0.8g (lasting 2 seconds), where g represents gravitational acceleration), and steering angle (e.g., 40°). Based on this vehicle state data, driving behavior is classified to obtain the behavior classification results, such as emergency braking.
[0078] Next, emotion classification is performed. Specifically, the user's facial images, voice data, physiological data, and steering wheel grip strength data are acquired. Based on these data, emotion classification is performed to obtain the final emotion classification result, such as anxiety.
[0079] Next, a chain-relationship graph is constructed. Specifically, the chain-relationship graph includes three nodes: environmental factor screening result (congestion) → behavioral classification result (sudden braking) → final emotion classification result (anxiety). The first side between congestion and sudden braking is labeled with a first path coefficient, such as 0.7, and the second side between sudden braking and anxiety is labeled with a second path coefficient, such as 0.9. The confidence level of the chain-relationship graph is 92%.
[0080] Finally, scenario mode recommendation is performed. Specifically, the behavior classification results, the final emotion classification results, the environmental factor screening results, and the chain association graph are matched with the matching conditions of multiple scenario modes in a preset scenario mode library to obtain the corresponding target scenario mode, such as a commuting relaxation mode. This commuting relaxation mode is sent to the vehicle's infotainment system to instruct the system to generate commuting relaxation mode recommendation information and push it to the display device in the infotainment system. User feedback is received; if the feedback is acceptance of the recommendation, the vehicle's current scenario mode is switched to commuting relaxation mode (the corresponding actions include increasing the air conditioning fan speed, playing soft music, and activating seat massage, etc.); if the feedback is rejection of the recommendation, an optimization instruction is sent to the vehicle scenario mode recommendation system. Simultaneously, the vehicle's air conditioning power is optimized and adjusted based on the remaining battery charge.
[0081] The vehicle scenario mode recommendation system in the above embodiments can be applied to fields such as smart cockpits (combined with AR-HUD (Augmented Reality Head-Up Display) and multi-screen linkage functions to enhance the immersive experience), autonomous driving (in L3+ autonomous driving, dynamically adjusting the control takeover strategy according to the driver's emotional state), and vehicle network services (linking with third-party services (such as music platforms and navigation software) to push scenario-based content (such as recommending music during traffic jams)). It has a wide range of applications and good application prospects.
[0082] Please refer to Figure 3 This embodiment also provides a vehicle scenario mode recommendation system, the system comprising: The data acquisition module 310 is used to acquire vehicle status data, multimodal data of users inside the vehicle, and external environmental data of the vehicle. The behavior classification module 320 is used to classify driving behavior based on the vehicle status data and obtain the behavior classification result. The emotion classification module 330 is used to classify user emotions based on the multimodal data to obtain the final emotion classification result; The environmental factor screening module 340 is used to obtain environmental factor screening results by performing keyword screening on the external environment data; The causal relationship construction module 350 is used to construct a chain relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results. The scenario pattern recommendation module 360 is used to recommend scenario patterns based on the chain-like association graph. The vehicle scenario pattern recommendation system in this embodiment achieves the technical effects of the vehicle scenario pattern recommendation method in the above embodiments, and will not be elaborated further here.
[0083] The vehicle scenario pattern recommendation system in this embodiment can be deployed on the vehicle's edge computing device, effectively reducing response latency. The measured response latency is ≤200ms (milliseconds).
[0084] It should be noted that the vehicle scenario mode recommendation method and the vehicle scenario mode recommendation system provided in the above embodiments belong to the same concept. The specific methods of execution of each module have been described in detail in the method embodiments and will not be repeated here. In practical applications, the vehicle scenario mode recommendation system provided in the above embodiments can assign the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above. This is not a limitation here.
[0085] Figure 4 For an example architecture diagram of the vehicle scenario mode recommendation system provided in one embodiment of this application, please refer to... Figure 4 , Figure 4The vehicle scenario mode recommendation system is divided into three functional layers: data acquisition, analysis and reasoning, and recommendation execution. The data acquisition layer includes the collection of user multimodal data, vehicle status data, and external environment data. User multimodal data collection is primarily handled by a visual module, an auditory module, a tactile module, and a physiological module. The visual module collects facial images, the auditory module collects voice data, the tactile module collects steering wheel grip force data, and the physiological module collects user physiological data. Vehicle status data collection mainly includes CAN bus data streams (such as vehicle speed, acceleration, and steering angle) and battery status (such as SOC (State of Charge)). External environment data collection primarily includes a high-precision map module (which has a traffic API to obtain map data from map software), a weather data module (for collecting weather data such as temperature, humidity, and precipitation probability), and a traffic event module (for collecting traffic event data such as accidents and construction). The analysis and reasoning layer mainly includes a data processing module (used to classify driving behavior based on vehicle status data to obtain behavior classification results; classify user emotions based on multimodal data to obtain final emotion classification results; and obtain environmental factor screening results by filtering keywords from external environmental data) and a causal relationship construction module. The execution recommendation layer includes a dynamic recommendation engine (which has a scenario pattern library containing custom / predefined scenario patterns, uses IF-THEN rules for scenario pattern matching, and has a reinforcement learning strategy) and a feedback optimization module (which has "accept recommendation" and "reject recommendation" buttons on the display device and updates the model through online learning). Throughout the process, data collection → data processing → causal relationship construction → scenario pattern recommendation → vehicle system execution → feedback to the data processing stage for model updates forms a complete closed loop.
[0086] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a vehicle scenario mode recommendation method. This method includes: acquiring vehicle status data, multimodal data of users within the vehicle, and external environmental data of the vehicle; classifying driving behavior based on the vehicle status data to obtain behavior classification results; classifying user emotions based on the multimodal data to obtain a final emotion classification result; filtering external environmental data using keywords to obtain environmental factor filtering results; constructing a chain-like association graph among the environmental factor filtering results, behavior classification results, and final emotion classification results; and performing scenario mode recommendation based on the chain-like association graph.
[0087] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0088] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the vehicle scenario pattern recommendation method provided by the above methods. This method includes: acquiring vehicle state data, multimodal data of users inside the vehicle, and external environment data of the vehicle; classifying driving behavior based on the vehicle state data to obtain behavior classification results; classifying user emotions based on the multimodal data to obtain a final emotion classification result; obtaining environmental factor filtering results by filtering keywords from the external environment data; constructing a chain-like association graph among the environmental factor filtering results, behavior classification results, and final emotion classification results; and performing scenario pattern recommendation based on the chain-like association graph.
[0089] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0091] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.
Claims
1. A vehicle scenario mode recommendation method, characterized in that, The method includes: Acquire vehicle status data, multimodal data of users inside the vehicle, and external environmental data of the vehicle; Based on the vehicle status data, driving behavior is classified to obtain the behavior classification results; Based on the multimodal data, user emotions are classified to obtain the final emotion classification result; By filtering the external environment data using keywords, the environmental factor filtering results are obtained; Construct a chain-like relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results; Based on the aforementioned chain-like association graph, scenario pattern recommendations are made.
2. The vehicle scenario mode recommendation method according to claim 1, characterized in that, The vehicle status data includes the vehicle's speed, acceleration, and steering angle; The behavior classification results include sudden braking, sudden acceleration, and sharp turns.
3. The vehicle scenario mode recommendation method according to claim 1, characterized in that, The multimodal data includes: the user's facial image, voice data, physiological data, and steering wheel grip force data; The process of classifying user emotions based on the multimodal data to obtain the final emotion classification result includes: Feature extraction is performed on the facial image to obtain facial features; feature extraction is performed on the speech data to obtain speech features; feature extraction is performed on the physiological data to obtain physiological features; feature extraction is performed on the grip strength data to obtain grip strength features; The facial features, voice features, physiological features, and grip strength features are concatenated to obtain a concatenated feature sequence; temporal dependencies are extracted from the concatenated feature sequence to obtain a fused feature, which carries the temporal information of the concatenated feature sequence. Based on the facial features, user emotions are classified to obtain a first emotion classification result; based on the voice features, user emotions are classified to obtain a second emotion classification result; based on the physiological features, user emotions are classified to obtain a third emotion classification result; based on the grip strength features, user emotions are classified to obtain a fourth emotion classification result; based on the fusion features, user emotions are classified to obtain a fifth emotion classification result. A weighted voting mechanism is used to integrate the first emotion classification result, the second emotion classification result, the third emotion classification result, the fourth emotion classification result, and the fifth emotion classification result to obtain the final emotion classification result.
4. The vehicle scenario mode recommendation method according to claim 3, characterized in that, The first, second, third, fourth, and fifth emotion classification results all contain predicted probabilities for multiple emotion categories. The weighted voting mechanism integrates the first, second, third, fourth, and fifth emotion classification results to obtain the final emotion classification result, including: Any emotion category is identified as the target emotion category; the product of the predicted probability of the target emotion category in the first emotion classification result and a first preset weight is determined as a first value; the product of the predicted probability of the target emotion category in the second emotion classification result and a second preset weight is determined as a second value; the product of the predicted probability of the target emotion category in the third emotion classification result and a third preset weight is determined as a third value; the product of the predicted probability of the target emotion category in the fourth emotion classification result and a fourth preset weight is determined as a fourth value; the product of the predicted probability of the target emotion category in the fifth emotion classification result and a fifth preset weight is determined as a fifth value; the fifth preset weight is greater than any one of the first preset weight, the second preset weight, the third preset weight, and the fourth preset weight. The sum of the first value, the second value, the third value, the fourth value, and the fifth value is determined as the final probability value of the target emotion category; the remaining emotion categories are traversed to obtain the final probability value of each emotion category; based on the final probability values of multiple emotion categories, the final emotion classification result is obtained.
5. The vehicle scenario mode recommendation method according to claim 4, characterized in that, The final emotion classification result includes at least one final emotion category. The final emotion classification result is obtained based on the final probability values of multiple emotion categories, including: If the final probability value of the current emotion category is greater than or equal to the preset emotion probability threshold, then the current emotion category is determined as the final emotion category.
6. The vehicle scenario mode recommendation method according to claim 1, characterized in that, The construction of the chain-like relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results includes: The causal relationship of the environmental factor screening results, the behavioral classification results, and the final emotion classification results is quantified to obtain the chain association graph. The chain association graph includes the environmental factor screening results, the behavioral classification results, the final emotion classification results, a first side connecting the environmental factor screening results and the behavioral classification results, and a second side connecting the behavioral classification results and the final emotion classification results. The first side is labeled with a first path coefficient, and the second side is labeled with a second path coefficient. The first path coefficient characterizes the causal effect strength between the environmental factor screening results and the behavioral classification results, and the second path coefficient characterizes the causal effect strength between the behavioral classification results and the final emotion classification results.
7. The vehicle scenario mode recommendation method according to claim 1, characterized in that, The scenario pattern recommendation based on the chain association graph includes: The chain-like association graph is matched with the matching conditions of multiple scenario patterns in the preset scenario pattern library to obtain the corresponding target scenario pattern. Based on the target scenario pattern, scenario pattern recommendation is completed.
8. The vehicle scenario mode recommendation method according to claim 7, characterized in that, The process of recommending scenarios based on the target scenario pattern includes: The target scenario mode is sent to the vehicle's infotainment system to instruct the system to generate target scenario mode recommendation information and push it to the display device within the system. Feedback from the user is received; if the feedback indicates acceptance of the recommendation, the vehicle's current scenario mode is switched to the target scenario mode; if the feedback indicates rejection of the recommendation, an optimization instruction is sent to the vehicle scenario mode recommendation system, which is the system running the vehicle scenario mode recommendation method. Upon receiving the optimization instruction, the preset emotion classification model is updated based on the optimization instruction. The emotion classification model is used to classify user emotions based on the multimodal data to obtain the final emotion classification result.
9. The vehicle scenario mode recommendation method according to claim 2, characterized in that, The vehicle status data also includes the remaining charge of the vehicle's power battery; After performing scenario pattern recommendation based on the aforementioned chain-like association graph, the process further includes: The target power is obtained based on the remaining power and the preset power mapping relationship, wherein the power mapping relationship is the mapping relationship between the power battery power and the air conditioner power. The target power is sent to the vehicle's infotainment system to instruct the system to adjust the air conditioning power to the target power.
10. A vehicle scenario mode recommendation system, characterized in that, The system includes: The data acquisition module is used to acquire vehicle status data, multimodal data of users inside the vehicle, and external environmental data of the vehicle. The behavior classification module is used to classify driving behavior based on the vehicle status data and obtain the behavior classification result. The emotion classification module is used to classify user emotions based on the multimodal data and obtain the final emotion classification result. The environmental factor screening module is used to obtain environmental factor screening results by filtering the external environment data using keywords; The causal relationship construction module is used to construct a chain relationship diagram among the environmental factor screening results, the behavior classification results, and the final emotion classification results; The scenario pattern recommendation module is used to recommend scenario patterns based on the chain-like association graph.