User driving behavior modeling method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of user portraits, and discloses a user driving behavior modeling method, device and equipment and a storage medium, the method comprises the following steps: in response to a user state change event, shooting vehicle internal and external environments and a user to obtain vehicle internal and external environment data and user state change data, and sending the vehicle internal and external environment data and the user state change data to a cloud; performing preliminary intention analysis on the cloud to obtain an initial user driving intention model; in response to a received user intention verification instruction sent by the cloud, collecting vehicle control behavior data and vehicle state change data of a user, and performing intention analysis based on the vehicle control behavior data and the vehicle state change data to obtain an intention analysis result; and the intention analysis result is sent to the cloud, so that the cloud performs intention analysis again, and a verified target user driving intention model is obtained. By applying the technical scheme of the invention, the driving habits and preferences of the user can be better understood and adapted.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of user portrait, in particular to a user driving behavior modeling method, device, equipment and storage medium. BACKGROUND

[0002] The current intelligent vehicle can recommend entertainment information such as music, news and video for users through user portrait. These functions have improved the user's entertainment experience, but there is still a significant deficiency in the intelligentization and personalization of the vehicle control function. Users hope that the vehicle can not only understand simple voice instructions, but also deeply analyze and learn their driving habits and preferences, so as to realize adaptive recommendation of the vehicle control function in the use process. SUMMARY

[0003] In view of the above problems, the embodiment of the present application provides a user driving behavior modeling method, device, equipment and storage medium, which is used to solve the problem of insufficient intelligentization and personalization of the vehicle control function in the prior art.

[0004] According to one aspect of the embodiment of the present application, a user driving behavior modeling method is provided, which comprises:

[0005] In response to the occurrence of a user state change event, vehicle internal and external environment data and user state change data obtained by photographing the vehicle internal and external environment and the user are sent to the cloud, so that the cloud inputs the vehicle internal and external environment data and the user state change data into a pre-trained user intent recognition model for preliminary intent analysis, to obtain an initial user driving intent model, the user state change event being a case where the user's body posture, facial expression or interactive object with the vehicle changes in the user's interaction with the vehicle;

[0006] In response to receiving a user intent verification instruction sent by the cloud, user control behavior data and vehicle state change data are collected, and intent analysis is performed based on the control behavior data and the vehicle state change data to obtain an intent analysis result, the user intent verification instruction being generated by the cloud based on the initial user driving intent model;

[0007] The intent analysis result is sent to the cloud, so that the cloud inputs the intent analysis result, the vehicle internal and external environment data and the user state change data into the user intent recognition model for further intent analysis, to obtain a verified target user driving intent model, the target user driving intent model describing the user's driving decision-making process in the corresponding driving scene.

[0008] According to another aspect of the embodiment of the present application, a user driving behavior modeling device is provided, which comprises:

[0009] The first sending module is configured to, in response to occurrence of a user state change event, send vehicle internal and external environment data and user state change data obtained by photographing the vehicle internal and external environment and the user to the cloud, so that the cloud inputs the vehicle internal and external environment data and the user state change data into a pre-trained user intention recognition model to perform preliminary intention analysis, and obtains an initial user driving intention model, wherein the user state change event refers to a case where a user's body posture, facial expression or an interaction object with the vehicle changes during the user's interaction with the vehicle.

[0010] The verification module is configured to, in response to receiving a user intention verification instruction sent by the cloud, collect user vehicle control behavior data and vehicle state change data, and perform intention analysis based on the user vehicle control behavior data and the vehicle state change data to obtain an intention analysis result, wherein the user intention verification instruction is generated by the cloud based on the initial user driving intention model.

[0011] The second sending module is configured to send the intention analysis result to the cloud, so that the cloud inputs the intention analysis result, the vehicle internal and external environment data and the user state change data into the user intention recognition model to perform intention analysis again, and obtains a verified target user driving intention model, wherein the target user driving intention model describes a driving decision-making process of the user in a corresponding driving scenario.

[0012] According to another aspect of the embodiment of the present application, a user driving behavior modeling device is provided, which comprises a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface complete communication with each other through the communication bus.

[0013] The memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform the operations of the user driving behavior modeling method as described above.

[0014] According to still another aspect of the embodiment of the present application, a computer readable storage medium is provided, and the storage medium stores at least one executable instruction, and the executable instruction causes the user driving behavior modeling device / apparatus to perform the operations of the user driving behavior modeling method as described above.

[0015] The embodiment of the present application can capture the vehicle internal and external environment data and user state change data by responding to the user state change event, and input them into the pre-trained user intention recognition model for preliminary analysis. After receiving the user intention verification instruction sent by the cloud, the user's vehicle control behavior data and vehicle state change data are further collected. These data are used for more in-depth intention analysis to ensure that the user intention recognition is more accurate and reliable. By inputting the intention analysis result and the vehicle internal and external environment data and user state change data into the user intention recognition model again, a verified target user driving intention model can be generated. Through preliminary intention analysis and subsequent intention verification and validation, the user driving intention model can be continuously updated and optimized. This adaptive learning capability enables better understanding and prediction of user driving habits and preferences. Through the above process, the vehicle can better understand and adapt to the user's driving habits and preferences, thereby providing more personalized and intelligent vehicle control functions.

[0016] The above description is only a summary of the technical solutions of the embodiments of the present application. In order to enable one skilled in the art to better understand the technical means of the embodiments of the present application, the contents of the specification can be implemented, and in order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not considered as limiting the present application. Moreover, the same reference signs are used to represent the same parts throughout the drawings. In the drawings:

[0018] Figure 1 A flowchart of a first embodiment of a user driving behavior modeling method provided by the present application is shown;

[0019] Figure 2 A flowchart of a second embodiment of a user driving behavior modeling method provided by the present application is shown;

[0020] Figure 3 An interaction process diagram of a user driving behavior modeling system provided by the present application is shown;

[0021] Figure 4 Another interaction process diagram of a user driving behavior modeling system provided by the present application is shown;

[0022] Figure 5 A structural diagram of a first embodiment of a user driving behavior modeling device provided by the present application is shown;

[0023] Figure 6 A structural diagram of an embodiment of a user driving behavior modeling device provided by the present application is shown;

[0024] Figure 7 A structural schematic diagram of an embodiment of a vehicle provided by the present application is shown. DETAILED DESCRIPTION

[0025] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0026] Figure 1 A flowchart of a first embodiment of a user driving behavior modeling method provided by the present application is shown, which can be executed by a domain controller in a vehicle. As shown in the figure, the method comprises the following steps: Figure 1

[0027] Step 110: In response to a user state change event, vehicle internal and external environment data and user state change data obtained by photographing the vehicle internal and external environment and the user are sent to the cloud, so that the cloud inputs the vehicle internal and external environment data and the user state change data into a pre-trained user intent recognition model for preliminary intent analysis, to obtain an initial user driving intent model, the user state change event being a case where the user's body posture, facial expression or interactive object with the vehicle changes during the user's interaction with the vehicle.

[0028] In this step, in response to a user state change event, data collection is started, the user state change event being a case where the user's body posture, facial expression or interactive object with the vehicle changes during the user's interaction with the vehicle. To capture these changes, the vehicle internal and external environment and the user can be photographed by a camera or other sensors to obtain corresponding data. These data include vehicle internal and external environment data and user state change data. Then, these data are sent to the cloud, which inputs them into a pre-trained user intent recognition model for preliminary intent analysis, thereby generating an initial user driving intent model. This model provides a preliminary understanding of the user's current driving intent.

[0029] Step 120: In response to receiving a user intent verification instruction sent by the cloud, user control behavior data and vehicle state change data are collected, and intent analysis is performed based on the control behavior data and the vehicle state change data to obtain an intent analysis result, the user intent verification instruction being generated by the cloud based on the initial user driving intent model.

[0030] ​The user intent verification instruction is generated based on the initial user driving intent model and aims to further verify or refine the user intent. After receiving the user intent verification instruction sent by the cloud, the user's control behavior data (e.g., steering wheel rotation, acceleration or braking, etc.) and vehicle state change data (e.g., speed change, lane deviation, etc.) are collected. Based on these data, intent analysis is performed to confirm and correct possible inaccuracies in the initial model.

[0031] Step 130: Send the intent analysis result to the cloud to make the cloud input the intent analysis result, vehicle internal and external environment data, and user state change data into the user intent recognition model for further intent analysis, obtaining a verified target user driving intent model that describes the user's driving decision-making process in the corresponding driving scenario.

[0032] In this step, the intent analysis result is sent to the cloud. The cloud inputs this result together with previously collected vehicle internal and external environment data and user state change data into the user intent recognition model for analysis again. Through this re-analysis process, a verified target user driving intent model can be generated. This model more accurately describes the user's driving decision-making process in a specific driving scenario, reflecting the user's true intent and preferences. The verified model can be used to further optimize the vehicle's control function and user experience. Through the above steps, the user's driving behavior can be dynamically captured and analyzed, providing more intelligent and personalized driving support.

[0033] The embodiment of the present application can capture vehicle internal and external environment data and user state change data by responding to user state change events, and input them into the pre-trained user intent recognition model for preliminary analysis. After receiving the user intent verification instruction sent by the cloud, further collection of user's control behavior data and vehicle state change data is performed. These data are used for more in-depth intent analysis to ensure that the identification of user intent is more accurate and reliable. By inputting the intent analysis result together with vehicle internal and external environment data and user state change data into the user intent recognition model again, a verified target user driving intent model can be generated. Through preliminary intent analysis and subsequent intent verification and validation, the user driving intent model can be continuously updated and optimized. This adaptive learning capability enables better understanding and prediction of user driving habits and preferences. Through the above process, the vehicle can better understand and adapt to user driving habits and preferences, thereby providing more personalized and intelligent control functions.

[0034] Figure 2 A flowchart showing another embodiment of the user driving behavior modeling method of the present application is shown, which can be executed by a domain controller in the vehicle. As shown in FIG. 6, the method includes the following steps: Figure 2As shown, the method comprises the following steps:

[0035] Step 210: In response to the occurrence of a user state change event, vehicle interior and exterior environment data and user state change data obtained by photographing the vehicle interior and exterior environment and the user are sent to the cloud, so that the cloud inputs the vehicle interior and exterior environment data and the user state change data into a pre-trained user intention recognition model to perform preliminary intention analysis, and obtains an initial user driving intention model. The user state change event is a case where the user's body posture, facial expression or interactive object with the vehicle changes during the user's interaction with the vehicle.

[0036] In the embodiment of the application, when it is monitored that the user's body posture, the user's facial expression or the user's interactive object changes, the vehicle interior and exterior environment and the user are photographed. Since some actions of the user have continuity, the image data of the same group can be desensitized and uploaded to the cloud based on user posture continuity or grouping by time period (t0-t1). The image data after desensitization and its related time information are uploaded to the cloud.

[0037] In the cloud, the uploaded image data can be numbered and a user driving intention model can be generated by analysis. The user driving intention model can be composed of the following key components: driving scene, user driving intention in the driving scene, control behavior taken and control target expected to be achieved.

[0038] Specifically, during driving, when the user suddenly notices a large vehicle carrying goods in the adjacent lane, the user chooses to accelerate to overtake the large vehicle to avoid parallel driving with the large vehicle. The user driving intention model can be as follows:

[0039] Driving scene: describes the key elements in the current driving environment. For example, a large vehicle carrying goods is detected in the adjacent lane.

[0040] Driving intention: reflects the user's perception and judgment of the current driving scene. For example, the user thinks that there is a safety risk, and may worry about the large vehicle tipping over, the brake not being timely or the goods falling off.

[0041] Control behavior: describes the specific driving action taken by the user in the driving scene. For example, the user changes lanes and overtakes by controlling the steering wheel to avoid parallel driving with the large vehicle.

[0042] Control target: reveals the user's expectations and needs in the driving scene. For example, the user wants to accelerate in a safe road environment, and the ideal safe road environment is that there is no large vehicle carrying goods near the vehicle.

[0043] In one alternative approach, the cloud is also used to generate a user intent verification instruction when the user intent recognition model determines that the vehicle's internal and external environmental data and user state change data have a definite user intent.

[0044] In this embodiment, when it is determined that the vehicle's internal and external environmental data and user state change data have a clear user intent, a user intent verification instruction is generated. Specifically, after a clear user intent is identified through the user intent recognition model, the image data number and its corresponding time information can be sent to the vehicle. The time information identifies the specific time range of the vehicle's internal and external environment and user state changes, which can be a time period, such as t0-t1.

[0045] Step 220: In response to receiving the user intent verification command sent by the cloud, collect the user's vehicle control behavior data and vehicle status change data, and perform intent analysis based on the vehicle control behavior data and vehicle status change data to obtain the intent analysis results. The user intent verification command is generated by the cloud based on the initial user driving intent model.

[0046] In one alternative approach, in response to receiving a user intent verification command from the cloud, data on the user's vehicle control behavior and vehicle status changes are collected, which may specifically include the following steps:

[0047] In response to receiving a user intent verification command sent from the cloud, the collection time period is extracted from the user intent verification command;

[0048] Collect vehicle control behavior data and vehicle status change data within the collection period.

[0049] In this implementation, the time period t0-t1 for which data needs to be collected is extracted from the user intent verification command. Vehicle control behavior data and vehicle state change data within this time period are collected. Vehicle control behavior data includes user operations on the vehicle, such as acceleration, braking, and steering. Vehicle state change data includes vehicle speed, position, and engine status. Intent analysis is performed based on the collected vehicle control behavior data and vehicle state change data.

[0050] In one alternative approach, intent analysis is performed based on vehicle control behavior data and vehicle state change data to obtain intent analysis results, which may specifically include the following steps:

[0051] Intent analysis is performed based on vehicle control behavior data and vehicle status change data to generate intent description text, which describes the user's vehicle control behavior and the corresponding vehicle status changes.

[0052] In this embodiment, vehicle control behavior data and vehicle state change data can be used as inputs, and appropriate analysis techniques can be applied to perform intent analysis. After the analysis is completed, an intent description text is generated. The main function of this text is to clearly and accurately describe the user's vehicle control behaviors and the resulting changes in vehicle state.

[0053] "The user operates XX (traversing the user's vehicle control actions), and the vehicle state changes from XX to XX (traversing the changes in vehicle state)" is an example template. In actual applications, XX will be replaced by specific vehicle control actions and vehicle state changes. For example, "The user operates the accelerator pedal, and the vehicle speed changes from 0 km / h to 50 km / h"; another example is "The user controls the steering wheel and accelerator pedal to achieve a rapid lane change and overtaking."

[0054] Step 230: Send the intent analysis results to the cloud so that the cloud can input the intent analysis results, vehicle internal and external environment data and user state change data into the user intent recognition model for further intent analysis, and obtain a validated target user driving intent model. The target user driving intent model describes the user's driving decision-making process in the corresponding driving scenario.

[0055] The intent analysis results, along with their corresponding identifiers, can be sent to the cloud. These results contain a description of the user's vehicle control behavior and the resulting changes in vehicle state. The cloud then inputs the received intent analysis results and the corresponding image sequence data (indoor and external environmental data and user state change data) into the user intent recognition model for further intent analysis. This step aims to validate and optimize the initial user driving intent model. The validated target user driving intent model is then obtained. This model describes the user's decision-making process and behavioral patterns in a specific driving scenario. The optimized target user driving intent model can be shown below:

[0056] Driving scenario: A large truck carrying goods is detected in a nearby lane.

[0057] Driving intention: The user believes there is a safety risk, and may be worried about the vehicle overturning, not braking in time, or cargo falling off.

[0058] Vehicle control behavior: Users control the steering wheel and accelerator pedal to change lanes and overtake, in order to avoid driving alongside large vehicles.

[0059] Vehicle control objective: Users want to drive in a safe road environment. The ideal safe road environment is one where there are no large vehicles carrying cargo nearby.

[0060] In one alternative approach, the cloud is also used to generate policy generation instructions based on the target user's driving intent model after obtaining the validated target user driving intent model.

[0061] In this embodiment, after obtaining a verified target user driving intention model, a strategy generation instruction is generated. This instruction is used to request the vehicle to generate specific driving habit strategies and vehicle control strategies to better adapt to the user's personalized needs.

[0062] Step 240: In response to receiving the policy generation instruction sent by the cloud, extract the target user driving intention model from the policy generation instruction. The target user driving intention model includes the driving scenario, the user's driving intention in the driving scenario, the vehicle control behavior taken, and the vehicle control goal to be achieved.

[0063] Step 250: Based on the target user's driving intention model, and combined with vehicle control behavior data and vehicle state change data, generate the user's driving habit strategy and vehicle control strategy. The driving habit strategy includes the mapping relationship between driving intention, user state change data and vehicle internal and external environment data. The user state change data includes one or more of the following: user posture change data, user facial expression change data and user interaction object data. The vehicle control strategy includes the mapping relationship between driving intention and vehicle control commands.

[0064] Driving habit strategies describe a user's behavior and state in a given driving scenario. These strategies consist of driving intentions, user posture change data, user facial expression change data, user interaction object data, and vehicle internal and external environment data.

[0065] User posture change data includes, but is not limited to: both hands gripping the steering wheel, one hand gripping the steering wheel, the user turning the steering wheel, turning the head left and right to check the rearview mirror, one elbow resting on the window, one hand picking up or searching for an object, upper body leaning to the side, and upper body leaning forward.

[0066] User facial expression data includes, but is not limited to: drowsiness, yawning, slight sweating, anger, odor searching, and disgust.

[0067] User interaction object data includes, but is not limited to: mobile phones, cigarettes, food, and beverages.

[0068] Vehicle internal and external environmental data: light intensity, temperature, PM2.5, humidity, various road surfaces (ice and snow, slippery, bumpy mountain roads, rugged and winding roads, steep slopes, potholes, dead ends, narrow roads), road conditions (traffic flow, distance to surrounding dangerous vehicles, congestion, traffic accidents, tunnels, heavy fog, densely populated pedestrian areas), and location information.

[0069] Specifically, the generated driving habit strategy can be as follows:

[0070] Driving intention: The large vehicle may overturn, brake in time, or cargo may fall off. The vehicle must move away from the dangerous vehicle as soon as possible.

[0071] User posture change data: The user turns the steering wheel and tilts their head to the left or right to check the exterior rearview mirror;

[0072] User facial expression change data: anxious;

[0073] User interaction object data: Steering wheel;

[0074] Vehicle internal and external environment data: There is a large vehicle carrying goods in parallel, and a slow-moving vehicle blocking the way ahead.

[0075] Vehicle control strategy describes the vehicle control commands issued by a user under specific driving intentions. The vehicle control strategy consists of driving intentions and vehicle control commands. It can extract key vehicle control information from the target user's driving intention model to generate vehicle control commands, and can combine this with local user control command data on the vehicle to verify and optimize the generated commands, including multi-command orchestration and control parameter refinement.

[0076] Specifically, the generated vehicle control strategy can be as follows:

[0077] Vehicle control strategy 1:

[0078] Driving intent: The vehicle needs to move away from the target dangerous vehicle by changing lanes and accelerating;

[0079] Vehicle control commands: Stp1: Turn on the turn signal and hold for 3 seconds; Stp2: After confirming that the lane to be changed is safe, control the steering to change lanes; Stp3: The vehicle accelerates to change lanes; Stp4: Turn off the turn signal, the vehicle accelerates away, and the dangerous vehicle is no longer within 100 meters.

[0080] Vehicle control strategy 2:

[0081] Driving intention: Slow down as quickly as possible, maintain a safe distance from dangerous vehicles, and then overtake quickly and move away when the opportunity arises.

[0082] Vehicle control commands: Stp1: Decelerate the vehicle until the target dangerous vehicle is no longer within 100m; Stp2: Maintain a safe following distance and plan an overtaking route and overtaking time based on the current road conditions; Stp3: Turn on the turn signal and keep it on for 3 seconds; Stp4: After confirming that the lane change is safe, control the steering to change lanes; Stp5: Turn off the turn signal, accelerate the vehicle away, and the dangerous vehicle is no longer within 100m.

[0083] Reference Figure 3The diagram illustrates the interaction process of the user driving behavior modeling system provided by this invention. This system comprises a vehicle-side in-vehicle and out-of-vehicle perception system, an in-vehicle user intent recognition system, a vehicle control system, a vehicle status monitoring system, and a cloud-based AI model. User vehicle control behavior data is collected by the vehicle control system from user operations, and the vehicle status monitoring system monitors and records the vehicle's motion state, body posture, and the activation status of various vehicle functions. When the in-vehicle camera detects changes in the user's body posture, facial expressions, or new spatial relationships between the user and other objects, it takes photos of the in-vehicle and out-of-vehicle environment, as well as the user. Based on the AI ​​model, the system identifies the user's current intent. Locally, the vehicle generates a simple description of the user's intent based on the user's vehicle control behavior data and vehicle status change data, and simultaneously uploads it to the AI ​​model for correction. The vehicle-side, based on the user driving intent model fed back from the AI ​​model, combines local user vehicle control behavior data and vehicle status change data to generate user driving habit strategies and vehicle control strategies. After multiple verifications and corrections on the vehicle-side, executable user driving habit strategies and vehicle control strategies are generated.

[0084] In an optional embodiment, the user driving behavior modeling method of this invention may further include the following steps:

[0085] In response to changes in user status or vehicle environment, the system takes photos of the vehicle's internal and external environment and the user.

[0086] Feature extraction is performed on the captured images to obtain user environment feature data, which includes one or more of the following: user posture features, user facial expression features, user interaction object features, in-vehicle environment features, and out-of-vehicle environment features.

[0087] The system queries driving habits and strategies to determine driving intentions that match user environmental characteristics data, and uses these driving intentions as the user's current driving intentions.

[0088] The system queries the vehicle control strategy, determines the vehicle control command that matches the current driving intention, and generates assisted driving prompt information based on the vehicle control command. The assisted driving prompt information is used to ask the user whether they need to execute the vehicle control command.

[0089] When a change in user status or vehicle environment is detected, images are taken of the vehicle's internal and external environment, as well as the user. These images are analyzed to extract features and obtain user environment feature data. Predefined driving habit strategies are queried to determine the driving intention matching the current user environment feature data, and this identified driving intention is taken as the user's current driving intention. Vehicle control strategies are queried to find control commands that match the current driving intention, and assisted driving prompts are generated based on these matching commands. The assisted driving prompts are then presented to the user, asking if they wish to execute the suggested control commands.

[0090] When applying driving habit and vehicle control strategies for the first time, the user should be asked whether they wish to execute the suggested vehicle control command. If the user agrees, the corresponding vehicle control command in the vehicle control strategy will be executed. After execution, a completion message will be sent to the user, and the user will be asked whether they authorize the automatic execution of vehicle control commands in the vehicle control strategy without the user's consent in the future. If the user authorizes, the vehicle control commands in the vehicle control strategy will be executed automatically the next time the strategy is triggered.

[0091] When identifying a user's current driving intent by querying driving habit strategies and vehicle control strategies, and determining the vehicle control commands the user might need to execute, the system asks the user whether the identified driving intent is correct and whether they want to execute the suggested vehicle control command. If the user reports that the identified driving intent is correct and agrees to execute the suggested vehicle control command, the system monitors whether the user engages in any other vehicle control behaviors after the command is executed. If other vehicle control behaviors occur, the system captures images of the vehicle's internal and external environment and the user, and uploads the captured images to the cloud to optimize the user's driving intent model. If the user reports that the identified driving intent is incorrect or disagrees with executing the suggested vehicle control command, the system monitors the user's subsequent vehicle control behaviors, i.e., captures images of the vehicle's internal and external environment and the user, and uploads the captured images to the cloud. In addition, the system can also upload the intent description text generated based on user vehicle control behavior data and vehicle state change data, along with rejected driving intents and rejected vehicle control strategies, to the cloud for in-depth learning. The cloud performs in-depth learning on the uploaded data to update the user's driving intent model. The updated model is then distributed to the vehicle to correct user driving habits and vehicle control strategies, better adapting to the user's actual needs and habits. The specific interaction process is as follows: Figure 4 As shown.

[0092] This invention integrates vehicle-side perception capabilities, vehicle status change data, and user vehicle control behavior data, inputting information into a cloud-based AI big data model in a combined text and image format for user intent recognition. The vehicle-side uses a semantic model of user intent (i.e., a user driving intent model) fed back from the cloud-based AI big data model to extract user driving habit strategies and vehicle control strategies based on local vehicle user control behavior data and vehicle status change data. Obtaining the semantic model of user intent through the cloud-based AI big data model solves the high computing power and resource limitations required for locally deployed AI big data models. The user driving habit strategies and vehicle control strategies generated on the vehicle-side combine the universal user intent generated by the cloud-based AI big data model with the actual user data of the local vehicle, thus better aligning with the current user's driving habits. When the vehicle environment or user status triggers a user driving habit strategy, the system will proactively ask the user upon first execution, and automatically execute the corresponding vehicle control strategy after user authorization. This not only makes the vehicle's intelligent functions more intuitive to the user but also allows the user to experience the vehicle's continuous learning and understanding of their intent during use. Furthermore, by collecting user feedback and continuously inputting data, the AI ​​big data model continuously improves its ability to understand user intent.

[0093] This invention, through responses to user state change events, captures data on the vehicle's internal and external environment, as well as user state change data, and inputs this data into a pre-trained user intent recognition model for preliminary analysis. Upon receiving a user intent verification command from the cloud, it further collects user vehicle control behavior data and vehicle state change data. This data is used for more in-depth intent analysis, ensuring more accurate and reliable user intent recognition. By inputting the intent analysis results, along with the vehicle's internal and external environment data and user state change data, back into the user intent recognition model, a validated target user driving intent model can be generated. Through preliminary intent analysis and subsequent intent verification and validation, the user driving intent model can be continuously updated and optimized. This adaptive learning capability enables a better understanding and prediction of user driving habits and preferences. Through this process, the vehicle can better understand and adapt to user driving habits and preferences, thereby providing more personalized and intelligent vehicle control functions.

[0094] Figure 5 A schematic diagram of an embodiment of the user driving behavior modeling device of the present invention is shown. Figure 5 As shown, the device 500 includes: a first transmitting module 510, a verification module 520, and a second transmitting module 530.

[0095] The first sending module is used to respond to the occurrence of a user state change event by sending the vehicle's internal and external environment data and user state change data obtained by taking pictures of the vehicle's internal and external environment and the user to the cloud. The cloud then inputs the vehicle's internal and external environment data and user state change data into a pre-trained user intent recognition model for preliminary intent analysis to obtain an initial user driving intent model. The user state change event is the situation where the user's body posture, facial expression or the interaction object with the vehicle changes during the interaction between the user and the vehicle.

[0096] The verification module is used to respond to the user intent verification command sent from the cloud, collect the user's vehicle control behavior data and vehicle status change data, and perform intent analysis based on the vehicle control behavior data and vehicle status change data to obtain intent analysis results. The user intent verification command is generated by the cloud based on the initial user driving intent model.

[0097] The second sending module is used to send the intent analysis results to the cloud, so that the cloud can input the intent analysis results, vehicle internal and external environment data and user state change data into the user intent recognition model for further intent analysis, and obtain a verified target user driving intent model. The target user driving intent model describes the user's driving decision-making process in the corresponding driving scenario.

[0098] In one alternative embodiment, the user driving behavior modeling device of the present invention is further used for:

[0099] In response to receiving a strategy generation instruction sent from the cloud, the target user driving intention model is extracted from the strategy generation instruction. The target user driving intention model includes the driving scenario, the user's driving intention in the driving scenario, the vehicle control behavior taken, and the vehicle control goal to be achieved.

[0100] Based on the target user's driving intent model, combined with vehicle control behavior data and vehicle state change data, the user's driving habit strategy and vehicle control strategy are generated. The driving habit strategy includes the mapping relationship between driving intent, user state change data and vehicle internal and external environment data. User state change data includes one or more of user posture change data, user facial expression change data and user interaction object data. The vehicle control strategy includes the mapping relationship between driving intent and vehicle control commands.

[0101] In one alternative approach, the verification module is specifically used for:

[0102] In response to receiving a user intent verification command sent from the cloud, the collection time period is extracted from the user intent verification command;

[0103] Collect vehicle control behavior data and vehicle status change data within the collection period.

[0104] In one alternative approach, the verification module is specifically used for:

[0105] Intent analysis is performed based on vehicle control behavior data and vehicle status change data to generate intent description text, which describes the user's vehicle control behavior and the corresponding vehicle status changes.

[0106] In one alternative approach, the cloud is also used to generate a user intent verification instruction when the user intent recognition model determines that the vehicle's internal and external environmental data and user state change data have a definite user intent.

[0107] In one alternative approach, the cloud is also used to generate policy generation instructions based on the target user's driving intent model after obtaining the validated target user driving intent model.

[0108] In one alternative embodiment, the user driving behavior modeling device of the present invention is further used for:

[0109] In response to changes in user status or vehicle environment, the system takes photos of the vehicle's internal and external environment and the user.

[0110] Feature extraction is performed on the captured images to obtain user environment feature data, which includes one or more of the following: user posture features, user facial expression features, user interaction object features, in-vehicle environment features, and out-of-vehicle environment features.

[0111] The system queries driving habits and strategies to determine driving intentions that match user environmental characteristics data, and uses these driving intentions as the user's current driving intentions.

[0112] The system queries the vehicle control strategy, determines the vehicle control command that matches the current driving intention, and generates assisted driving prompt information based on the vehicle control command. The assisted driving prompt information is used to ask the user whether they need to execute the vehicle control command.

[0113] This invention, through responses to user state change events, captures data on the vehicle's internal and external environment, as well as user state change data, and inputs this data into a pre-trained user intent recognition model for preliminary analysis. Upon receiving a user intent verification command from the cloud, it further collects user vehicle control behavior data and vehicle state change data. This data is used for more in-depth intent analysis, ensuring more accurate and reliable user intent recognition. By inputting the intent analysis results, along with the vehicle's internal and external environment data and user state change data, back into the user intent recognition model, a validated target user driving intent model can be generated. Through preliminary intent analysis and subsequent intent verification and validation, the user driving intent model can be continuously updated and optimized. This adaptive learning capability enables a better understanding and prediction of user driving habits and preferences. Through this process, the vehicle can better understand and adapt to user driving habits and preferences, thereby providing more personalized and intelligent vehicle control functions.

[0114] Figure 6 The diagram shows a structural schematic of an embodiment of the user driving behavior modeling device of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the user driving behavior modeling device.

[0115] like Figure 6 As shown, the user driving behavior modeling device may include: a processor 602, a communications interface 604, a memory 606, and a communications bus 608.

[0116] The processor 602, communication interface 604, and memory 606 communicate with each other via communication bus 608. Communication interface 604 is used to communicate with other network elements such as clients or other servers. The processor 602 executes program 610, specifically performing the relevant steps described in the embodiment of the user driving behavior modeling method.

[0117] Specifically, program 610 may include program code, which includes computer-executable instructions.

[0118] Processor 602 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The user driving behavior modeling device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0119] Memory 606 is used to store program 610. Memory 606 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0120] Specifically, program 610 can be called by processor 602 to cause the user driving behavior modeling device to perform the following operations:

[0121] In response to a user state change event, the system captures images of the vehicle's internal and external environment and the user, and sends the resulting data on the vehicle's internal and external environment and user state change to the cloud. The cloud then inputs the data into a pre-trained user intent recognition model for preliminary intent analysis, resulting in an initial user driving intent model. The user state change event refers to a change in the user's body posture, facial expression, or the object of interaction with the vehicle during the interaction process.

[0122] In response to receiving a user intent verification command sent from the cloud, the system collects user vehicle control behavior data and vehicle status change data, and performs intent analysis based on the vehicle control behavior data and vehicle status change data to obtain intent analysis results. The user intent verification command is generated by the cloud based on the initial user driving intent model.

[0123] The intent analysis results are sent to the cloud, so that the cloud can input the intent analysis results, vehicle internal and external environmental data, and user state change data into the user intent recognition model for further intent analysis, thereby obtaining a validated target user driving intent model. The target user driving intent model describes the user's driving decision-making process in the corresponding driving scenario.

[0124] In an alternative manner, program 610 is invoked by processor 602 to cause the user driving behavior modeling device to perform the following operations:

[0125] In response to receiving a strategy generation instruction sent from the cloud, the target user driving intention model is extracted from the strategy generation instruction. The target user driving intention model includes the driving scenario, the user's driving intention in the driving scenario, the vehicle control behavior taken, and the vehicle control goal to be achieved.

[0126] Based on the target user's driving intent model, combined with vehicle control behavior data and vehicle state change data, the user's driving habit strategy and vehicle control strategy are generated. The driving habit strategy includes the mapping relationship between driving intent, user state change data and vehicle internal and external environment data. User state change data includes one or more of user posture change data, user facial expression change data and user interaction object data. The vehicle control strategy includes the mapping relationship between driving intent and vehicle control commands.

[0127] In an alternative manner, program 610 is invoked by processor 602 to cause the user driving behavior modeling device to perform the following operations:

[0128] In response to receiving a user intent verification command sent from the cloud, the collection time period is extracted from the user intent verification command;

[0129] Collect vehicle control behavior data and vehicle status change data within the collection period.

[0130] In an alternative manner, program 610 is invoked by processor 602 to cause the user driving behavior modeling device to perform the following operations:

[0131] Intent analysis is performed based on vehicle control behavior data and vehicle status change data to generate intent description text, which describes the user's vehicle control behavior and the corresponding vehicle status changes.

[0132] In one alternative approach, the cloud is also used to generate a user intent verification instruction when the user intent recognition model determines that the vehicle's internal and external environmental data and user state change data have a definite user intent.

[0133] In one alternative approach, the cloud is also used to generate policy generation instructions based on the target user's driving intent model after obtaining the validated target user driving intent model.

[0134] In an alternative manner, program 610 is invoked by processor 602 to cause the user driving behavior modeling device to perform the following operations:

[0135] In response to changes in user status or vehicle environment, the system takes photos of the vehicle's internal and external environment and the user.

[0136] Feature extraction is performed on the captured images to obtain user environment feature data, which includes one or more of the following: user posture features, user facial expression features, user interaction object features, in-vehicle environment features, and out-of-vehicle environment features.

[0137] The system queries driving habits and strategies to determine driving intentions that match user environmental characteristics data, and uses these driving intentions as the user's current driving intentions.

[0138] The system queries the vehicle control strategy, determines the vehicle control command that matches the current driving intention, and generates assisted driving prompt information based on the vehicle control command. The assisted driving prompt information is used to ask the user whether they need to execute the vehicle control command.

[0139] This invention, through responses to user state change events, captures data on the vehicle's internal and external environment, as well as user state change data, and inputs this data into a pre-trained user intent recognition model for preliminary analysis. Upon receiving a user intent verification command from the cloud, it further collects user vehicle control behavior data and vehicle state change data. This data is used for more in-depth intent analysis, ensuring more accurate and reliable user intent recognition. By inputting the intent analysis results, along with the vehicle's internal and external environment data and user state change data, back into the user intent recognition model, a validated target user driving intent model can be generated. Through preliminary intent analysis and subsequent intent verification and validation, the user driving intent model can be continuously updated and optimized. This adaptive learning capability enables a better understanding and prediction of user driving habits and preferences. Through this process, the vehicle can better understand and adapt to user driving habits and preferences, thereby providing more personalized and intelligent vehicle control functions.

[0140] Figure 7 A structural schematic diagram of an embodiment of the vehicle of the present invention is shown. (As shown) Figure 7 As shown, the vehicle 700 includes: a camera, one or more processors, and a communication interface;

[0141] The camera is used to capture images of the vehicle's internal and external environment and the user to obtain data on the vehicle's internal and external environment and changes in the user's status.

[0142] The processor is used to execute the steps in the above-described user driving behavior modeling method embodiments.

[0143] This invention, through responses to user state change events, captures data on the vehicle's internal and external environment, as well as user state change data, and inputs this data into a pre-trained user intent recognition model for preliminary analysis. Upon receiving a user intent verification command from the cloud, it further collects user vehicle control behavior data and vehicle state change data. This data is used for more in-depth intent analysis, ensuring more accurate and reliable user intent recognition. By inputting the intent analysis results, along with the vehicle's internal and external environment data and user state change data, back into the user intent recognition model, a validated target user driving intent model can be generated. Through preliminary intent analysis and subsequent intent verification and validation, the user driving intent model can be continuously updated and optimized. This adaptive learning capability enables a better understanding and prediction of user driving habits and preferences. Through this process, the vehicle can better understand and adapt to user driving habits and preferences, thereby providing more personalized and intelligent vehicle control functions.

[0144] This invention provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on a user driving behavior modeling device / app, it causes the user driving behavior modeling device / app to perform the user driving behavior modeling method in any of the above method embodiments.

[0145] Specifically, the executable instructions can be used to cause the user driving behavior modeling device / device to perform the following operations:

[0146] In response to a user state change event, the system captures images of the vehicle's internal and external environment and the user, and sends the resulting data on the vehicle's internal and external environment and user state change to the cloud. The cloud then inputs the data into a pre-trained user intent recognition model for preliminary intent analysis, resulting in an initial user driving intent model. The user state change event refers to a change in the user's body posture, facial expression, or the object of interaction with the vehicle during the interaction process.

[0147] In response to receiving a user intent verification command sent from the cloud, the system collects user vehicle control behavior data and vehicle status change data, and performs intent analysis based on the vehicle control behavior data and vehicle status change data to obtain intent analysis results. The user intent verification command is generated by the cloud based on the initial user driving intent model.

[0148] The intent analysis results are sent to the cloud, so that the cloud can input the intent analysis results, vehicle internal and external environmental data, and user state change data into the user intent recognition model for further intent analysis, thereby obtaining a validated target user driving intent model. The target user driving intent model describes the user's driving decision-making process in the corresponding driving scenario.

[0149] In one alternative approach, the executable instructions cause the user driving behavior modeling device / apparatus to perform the following operations:

[0150] In response to receiving a strategy generation instruction sent from the cloud, the target user driving intention model is extracted from the strategy generation instruction. The target user driving intention model includes the driving scenario, the user's driving intention in the driving scenario, the vehicle control behavior taken, and the vehicle control goal to be achieved.

[0151] Based on the target user's driving intent model, combined with vehicle control behavior data and vehicle state change data, the user's driving habit strategy and vehicle control strategy are generated. The driving habit strategy includes the mapping relationship between driving intent, user state change data and vehicle internal and external environment data. User state change data includes one or more of user posture change data, user facial expression change data and user interaction object data. The vehicle control strategy includes the mapping relationship between driving intent and vehicle control commands.

[0152] In one alternative approach, the executable instructions cause the user driving behavior modeling device / apparatus to perform the following operations:

[0153] In response to receiving a user intent verification command sent from the cloud, the collection time period is extracted from the user intent verification command;

[0154] Collect vehicle control behavior data and vehicle status change data within the collection period.

[0155] In one alternative approach, the executable instructions cause the user driving behavior modeling device / apparatus to perform the following operations:

[0156] Intent analysis is performed based on vehicle control behavior data and vehicle status change data to generate intent description text, which describes the user's vehicle control behavior and the corresponding vehicle status changes.

[0157] In one alternative approach, the cloud is also used to generate a user intent verification instruction when the user intent recognition model determines that the vehicle's internal and external environmental data and user state change data have a definite user intent.

[0158] In one alternative approach, the cloud is also used to generate policy generation instructions based on the target user's driving intent model after obtaining the validated target user driving intent model.

[0159] In one alternative approach, the executable instructions cause the user driving behavior modeling device / apparatus to perform the following operations:

[0160] In response to changes in user status or vehicle environment, the system takes photos of the vehicle's internal and external environment and the user.

[0161] Feature extraction is performed on the captured images to obtain user environment feature data, which includes one or more of the following: user posture features, user facial expression features, user interaction object features, in-vehicle environment features, and out-of-vehicle environment features.

[0162] The system queries driving habits and strategies to determine driving intentions that match user environmental characteristics data, and uses these driving intentions as the user's current driving intentions.

[0163] The system queries the vehicle control strategy, determines the vehicle control command that matches the current driving intention, and generates assisted driving prompt information based on the vehicle control command. The assisted driving prompt information is used to ask the user whether they need to execute the vehicle control command.

[0164] This invention, through responses to user state change events, captures data on the vehicle's internal and external environment, as well as user state change data, and inputs this data into a pre-trained user intent recognition model for preliminary analysis. Upon receiving a user intent verification command from the cloud, it further collects user vehicle control behavior data and vehicle state change data. This data is used for more in-depth intent analysis, ensuring more accurate and reliable user intent recognition. By inputting the intent analysis results, along with the vehicle's internal and external environment data and user state change data, back into the user intent recognition model, a validated target user driving intent model can be generated. Through preliminary intent analysis and subsequent intent verification and validation, the user driving intent model can be continuously updated and optimized. This adaptive learning capability enables a better understanding and prediction of user driving habits and preferences. Through this process, the vehicle can better understand and adapt to user driving habits and preferences, thereby providing more personalized and intelligent vehicle control functions.

[0165] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments of this invention are not directed to any particular programming language.

[0166] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. Similarly, for the sake of brevity and to aid in understanding one or more aspects of the invention, in the description of exemplary embodiments of the invention above, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0167] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

[0168] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method of modeling user driving behavior, the method comprising: The method comprises: In response to a user state change event occurring, vehicle internal and external environment data and user state change data obtained by photographing the vehicle internal and external environment and the user are sent to the cloud end, so that the cloud end inputs the vehicle internal and external environment data and the user state change data into a pre-trained user intention recognition model for preliminary intention analysis, obtaining an initial user driving intention model, the user state change event being a case where the user's body posture, facial expression or interactive object with the vehicle changes during the user's interaction with the vehicle; In response to receiving a user intention verification instruction sent by the cloud end, collecting user vehicle control behavior data and vehicle state change data, and performing intention analysis based on the vehicle control behavior data and the vehicle state change data to obtain an intention analysis result, the user intention verification instruction being generated by the cloud end based on the initial user driving intention model; The intention analysis result is sent to the cloud end, so that the cloud end inputs the intention analysis result, the vehicle internal and external environment data and the user state change data into the user intention recognition model for intention analysis again, obtaining a verified target user driving intention model, the target user driving intention model describing the user's driving decision-making process in the corresponding driving scenario.

2. The method of claim 1, wherein, The method further comprises: In response to receiving a strategy generation instruction sent by the cloud end, extracting the target user driving intention model from the strategy generation instruction, the target user driving intention model including a driving scenario, a user driving intention in the driving scenario, a vehicle control behavior taken and a vehicle control target expected to be achieved; Based on the target user driving intention model, the vehicle control behavior data and the vehicle state change data, a user driving habit strategy and a vehicle control strategy are generated, the driving habit strategy including a driving intention, a mapping relationship between user state change data and vehicle internal and external environment data, the user state change data including one or more of user posture change data, user expression change data and user interactive object data; the vehicle control strategy including a mapping relationship between a driving intention and a vehicle control instruction.

3. The method of claim 1, wherein, The response to receiving a user intention verification instruction sent by the cloud end to collect user vehicle control behavior data and vehicle state change data comprises: In response to receiving a user intention verification instruction sent by the cloud end, extracting a collection time period from the user intention verification instruction; Collecting the vehicle control behavior data and the vehicle state change data within the collection time period.

4. The method of claim 1, wherein, The intention analysis based on the vehicle control behavior data and the vehicle state change data to obtain an intention analysis result comprises: Based on the vehicle control behavior data and the vehicle state change data, intention analysis is performed to generate an intention description text, the intention description text describing the user's vehicle control behavior and the corresponding vehicle state change.

5. The method according to any one of claims 1 to 4, characterized in that, The cloud end is further configured to generate the user intention verification instruction when it is determined through the user intention recognition model that the vehicle internal and external environment data and the user state change data have a determined user intention.

6. The method of claim 2, wherein, The cloud is also configured to generate the strategy generation instruction based on the target user driving intention model after obtaining the target user driving intention model.

7. The method according to any one of claims 2-4, characterized in that, The method further comprises: in response to a change in user state or a change in vehicle environment, capturing the user and the environment inside and outside the vehicle; extracting features from the captured images to obtain user environment feature data, the user environment feature data including one or more of user posture features, user expression features, user interactive object features, in-vehicle environment features, and out-of-vehicle environment features; querying the driving habit strategy to determine a driving intention that matches the user environment feature data, and taking the driving intention as the user's current driving intention; querying the vehicle control strategy to determine a vehicle control instruction that matches the current driving intention, and generating an auxiliary driving prompt message based on the vehicle control instruction, the auxiliary driving prompt message being used to ask the user whether the vehicle control instruction needs to be executed.

8. A user driving behavior modeling apparatus, characterized by, The device comprises: a first sending module configured to, in response to a user state change event, send vehicle environment data and user state change data obtained by capturing the user and the environment inside and outside the vehicle to the cloud, so that the cloud inputs the vehicle environment data and the user state change data into a pre-trained user intention recognition model for preliminary intention analysis to obtain an initial user driving intention model, the user state change event being a change in user body posture, facial expression, or interactive object with the vehicle during user interaction with the vehicle; a verification module configured to, in response to receiving a user intention verification instruction sent by the cloud, collect user vehicle control behavior data and vehicle state change data, and perform intention analysis based on the vehicle control behavior data and the vehicle state change data to obtain an intention analysis result, the user intention verification instruction being generated by the cloud based on the initial user driving intention model; a second sending module configured to send the intention analysis result to the cloud, so that the cloud inputs the intention analysis result, the vehicle environment data, and the user state change data into the user intention recognition model for further intention analysis to obtain a target user driving intention model that has been verified, the target user driving intention model describing the user's driving decision-making process in a corresponding driving scenario.

9. A user driving behavior modeling device, characterized by, comprises: a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface being in communication with each other through the communication bus; the memory is configured to store at least one executable instruction, the executable instruction causing the processor to perform the operations of the user driving behavior modeling method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, and the executable instruction, when executed on the user driving behavior modeling device / apparatus, causes the user driving behavior modeling device / apparatus to perform the operations of the user driving behavior modeling method according to any one of claims 1-7.

Images