Driving style quantitative evaluation method based on multi-modal perception and knowledge graph

Abstract

The invention discloses a driving style quantitative evaluation method based on multi-modal perception and a knowledge graph, and the method comprises the steps: employing a multi-modal data source, which comprises various sensor data, vehicle motion data, and physiological reaction and behavior data (such as heart rate change and operation intervention frequency during emergency braking) of a driver and a passenger in a real scene; a user emotion mapping algorithm is constructed by the model, subjective experience is converted into quantifiable numerical indexes (such as a trust index and a comfort score), a driving style deviation index (SDI) is calculated by using an algorithm in combination with a constructed driving behavior knowledge graph, quantitative evaluation of brand driving genes is realized, and a data driving basis is provided for function and algorithm optimization. And a personalized intelligent driving mode is realized. Therefore, the driving style quantitative evaluation method based on the multi-modal perception and the knowledge graph has the advantages of being high in intelligent degree and high in individuation degree.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a driving style quantitative evaluation method based on multimodal perception and knowledge graph with high intelligence and personalization. Background Art

[0002] With the development of autonomous driving technology and the application of end-to-end large models, autonomous driving functions are increasingly dependent on data closed-loop training. The resulting problem is that the intelligent driving style of each company is greatly influenced by the data and driving style of most of its brand customers, which leads to the homogenization of intelligent driving style and the deviation of its data distribution (such as road scenes and driving habits) from the actual scenes of the target users of the OEM (such as complex road conditions in China and local driving styles). As a result, the driving style output by the model (such as conservative, aggressive, sporty, efficient, stable, cautious, etc.) is inconsistent with the brand positioning. In addition, evaluating the user's feelings and trust in the autonomous driving system has become a key issue. The existing technology lacks a systematic method to quantify user feelings, and there is no good methodology to screen and select the required data to train the software according to different driving style requirements.

[0003] The existing autonomous driving evaluation system relies too much on binary “pass / fail” judgments (such as regulatory testing) and lacks refined quantitative measurements of users’ subjective feelings (such as comfort, security, and trust).

[0004] The details are as follows:

[0005] 1. Data-driven style solidification: The existing data collection network of the OEM is limited by the characteristics of the existing user group (such as age distribution and regional driving habits), which causes the distribution of training data to deviate from the real scene of the target market. Specifically, it is manifested as: Road scene deviation: Model training focuses on highway scenes, and the coverage of complex urban scenes (such as non-standard intersections and mixed traffic) is insufficient; Driving behavior deviation: The system inherits the inherent driving characteristics of the user group, which is in significant conflict with the brand's expected control style target value;

[0006] 2. Blind spot in user experience quantification: The current evaluation system is based on the traditional functional safety verification framework, and its core flaws are: binary evaluation dimensions: only focusing on mechanical indicators such as takeover rate and collision avoidance success rate; perceived value decoupling: unable to quantify the subjective cognitive indicators of drivers and passengers (such as comfort, trust, and control expectation matching).

[0007] Industry research shows that 84% of users believe that the existing L2+ system has "lack of anthropomorphism", which is manifested in the failure to predict the tension of the passenger's neck and back muscles during rapid acceleration and the lack of operating rhythm like an experienced driver in cornering control. Therefore, it is necessary to propose an improvement to overcome the defects of the existing technology. Summary of the invention

[0008] The purpose of the present invention is to solve the problems in the prior art and provide a driving style quantitative evaluation method based on multimodal perception and knowledge graph with high intelligence and personalization.

[0009] The technical solution of the present invention is:

[0010] A driving style quantitative evaluation method based on multimodal perception and knowledge graph, comprising the following steps: S1, data acquisition: perform global signal capture and collect vehicle information: vehicle information includes; vehicle bus CAN signal, CSV data, video data, Rosbag data, Ethernet bus data; S2, process the vehicle information and calculate static and dynamic scene features: comprising the following steps: S21, process the vehicle information; S22, determine whether the frame is lost by the size of each file, form the frame loss rate of each file, determine the empty data and dirty data and automatically eliminate them; S23, calculate and extract static scene features and dynamic scene features; S3, quantify the subjective driving experience into objective data for scoring: calculate the physical quantity required for evaluation by identifying the scene, quantify the driving experience into objective data for scoring through a multi-dimensional and multi-layer mapping model, and summarize the scored objective data to form a multimodal data set;

[0011] S4. Establish knowledge graph modeling, associate the multimodal data in the multimodal data set through the knowledge graph, and establish a scene-style association matrix; S5. Driving style classification and driving style deviation index (SDI) calculation: Driving style classification is performed based on the scene-style association matrix established based on the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is:

[0012]

[0013] Among them, P (i) actual is the actual measured value of the i-th driving style parameter, P (i) brand is the ideal value of the preset brand driving mode, P (i) range is the reasonable value range of the parameter, wi is the weight coefficient of the data parameter in the brand driving mode; S6, intelligent driving mode parameter optimization: according to the SDI positioning deviation index, the driving mode training is carried out according to the deviation index to form personalized intelligent driving mode parameters.

[0014] As a preferred technical solution, the global signal capture in step S1 is specifically as follows: a. Collecting the output signals of each sensor: the sensors include cameras, millimeter-wave radars, lidars, ultrasonic radars, and positioning sensors; the signal content includes the original signal and the signal after solution processing; b. Collecting vehicle chassis signals, including control instructions and feedback, kinematic states such as speed and acceleration, states such as wheel speed and wheel angle, power and steering system states, and states of various vehicle body components; c. Collecting biological sign monitoring signals: monitoring heart rate and breathing rate through in-cabin radars and cameras to build a user health portrait; inferring the user's emotional state by combining the biological sign monitoring signals and driving behavior.

[0015] As a preferred technical solution, step S21 vehicle information processing specifically includes the following steps: S211, unstructured data processing: perform visual data stream cleaning, and use a spatiotemporal consistency detection algorithm to repair video frame loss; perform point cloud data optimization, suppress ground reflection noise, and eliminate motion distortion; CAN bus data calibration, and establish signal rationality verification; S212, perform structured data cleaning: fill in null values, remove outliers, and apply the 3σ principle in conjunction with DBSCAN clustering; S213, supplement missing data; S214, merge duplicate data; S215, clean useless data; S216, remove null data values.

[0016] As a preferred technical solution, the calculation and extraction of static scene features in step S23 is specifically as follows: generating static road network feature labels through the VLM large model output layer label system.

[0017] As a further preferred technical solution, the static road network feature labels include road topology information and other road environment elements; the road topology information includes expressways, urban roads, intersections, rural roads, crosswalks, bends, ramps, roundabouts, abnormal intersections, ramps, emergency roads and bus lanes; the other road environment elements include traffic lights, cones and speed bumps.

[0018] As a preferred technical solution, the calculation and extraction of dynamic scene features in step S23 is specifically as follows: through an offline perception enhancement algorithm and a scene slicing algorithm, interactive behavior modeling and a traffic participant kinematic relationship algorithm are constructed, which specifically includes the following steps: S231. Based on the acquired road information, dynamic scene information data is automatically identified and generated; S232. The dynamic scene information data is preprocessed, and the collected data is filtered and denoised to ensure that the dynamic scene information data includes basic information about road infrastructure, vehicle driving data, and other objects and traffic participants that change over time.

[0019] As a preferred technical solution, step S3 quantifies the subjective driving experience into objective data for scoring, and specifically includes the following steps: S31, defining behavior categories, and classifying driving scenarios according to selected reference parameters; S32, dividing individual scenarios into temporally constant sequences, and analyzing these behavior categories in a multidimensional parameter space; S33, clustering the parameter space using mathematical methods, and each sequence is assigned to a behavior category containing a scenario and a corresponding independent cluster member; S34, completing the scoring by progressively mapping the signals and results layer by layer according to the mapping relationship of each evaluation layer.

[0020] As a preferred technical solution, the multimodal data set in step S3 includes a safety index, a comfort index, a reliability index and a passability index.

[0021] As a preferred technical solution, in step S4, associating the multimodal data in the multimodal data set through the knowledge graph specifically involves establishing relationships between the multimodal data through the knowledge graph, and extracting entities, classes and relationships through the knowledge graph.

[0022] As a further preferred technical solution, the entities include a driver entity, a vehicle entity, an environment entity and a behavior entity.

[0023] The present invention is a driving style quantitative evaluation method based on multimodal perception and knowledge graph. Through multimodal data sources, including various sensor data, vehicle motion data, and physiological reactions and behavioral data of drivers and passengers in real scenes (such as heart rate changes and operation intervention frequency during emergency braking), the model constructs a user emotion mapping algorithm, converts subjective experience into quantifiable numerical indicators (such as "trust index" and "comfort score"), combines the constructed driving behavior knowledge graph, and uses an algorithm to calculate the driving style deviation index (SDI), so as to achieve a quantitative evaluation of brand driving genes, provide a data-driven basis for function and algorithm optimization, and realize a personalized intelligent driving mode. Therefore, the present invention is a driving style quantitative evaluation method based on multimodal perception and knowledge graph, which has the advantages of high intelligence and high personalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a flowchart of a specific implementation method of a driving style quantitative evaluation method based on multimodal perception and knowledge graph. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0027] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0028] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0029] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0030] like Figure 1 The figure shows a specific embodiment of a driving style quantitative evaluation method based on multimodal perception and knowledge graph of the present invention. The driving style quantitative evaluation method based on multimodal perception and knowledge graph of the present invention comprises the following steps:

[0031] S1. Data collection: perform global signal capture and collect vehicle information: vehicle information includes vehicle bus CAN signal, CSV data, video data, Rosbag data, and Ethernet bus data;

[0032] S2, processing the vehicle information to calculate static and dynamic scene features: comprising the following steps:

[0033] S21, processing vehicle information;

[0034] S22, judging whether a frame is lost by the size of each file, forming a frame loss rate for each file, judging empty data and dirty data and automatically eliminating them;

[0035] S23, calculating and extracting static scene features and dynamic scene features;

[0036] S3. Quantify subjective driving experience into objective data for scoring: Calculate the physical quantities required for evaluation by identifying the scene, quantify the driving experience into objective data for scoring through a multi-dimensional and multi-layer mapping model, and aggregate the scored objective data to form a multimodal data set;

[0037] S4, establishing a knowledge graph model, associating the multimodal data in the multimodal data set through the knowledge graph, and establishing a scene-style association matrix;

[0038] S5. Driving style classification and driving style deviation index (SDI) calculation: Driving style classification is performed based on the scene-style association matrix established based on the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is:

[0039]

[0040] Among them, P (i) actual is the actual measured value of the i-th driving style parameter, P (i) brand is the ideal value of the preset brand driving mode, P (i) range is the reasonable value range of the parameter, wi is the weight coefficient of the data parameter in the brand driving mode;

[0041] S6. Intelligent driving mode parameter optimization: Based on the SDI positioning deviation index, driving mode training is performed according to the deviation index to form personalized intelligent driving mode parameters.

[0042] Specifically, a driving style quantitative evaluation method based on multimodal perception and knowledge graph in this embodiment is as follows:

[0043] Step S1 is data acquisition: global signal capture is performed to collect vehicle information: vehicle information includes vehicle bus CAN signal, CSV data, video data, Rosbag data, and Ethernet bus data. The data acquisition of this embodiment realizes global signal capture through a three-level hierarchical architecture, and the specific steps are:

[0044] a. Collect the output signals of each sensor: the sensors include cameras, millimeter wave radars, laser radars, ultrasonic radars, and positioning sensors; the signal content includes the original signal and the signal after solution processing;

[0045] b. Collect vehicle chassis signals, including control instructions and feedback, kinematic states such as speed and acceleration, wheel speed and wheel angle, power and steering system states, and vehicle body component states;

[0046] c. Collecting biological sign monitoring signals: Monitor heart rate and breathing rate through in-cabin radar and cameras to build a user health portrait; combine biological sign monitoring signals and driving behavior (sudden acceleration / braking) to infer the user's emotional state and estimate the user's stress / fatigue level.

[0047] Step S2 is to process the vehicle information and calculate static and dynamic scene features, including the following steps:

[0048] S21, processing the vehicle information; specifically comprising the following steps:

[0049] S211, unstructured data processing: perform visual data stream cleaning, use spatiotemporal consistency detection algorithm to repair video frame loss; perform point cloud data optimization, suppress ground reflection noise, and eliminate motion distortion; CAN bus data calibration, establish signal rationality verification;

[0050] S212, perform structured data cleaning: fill in empty values, remove outliers, apply 3σ principle and DBSCAN clustering;

[0051] S213, supplement missing data;

[0052] S214, merge duplicate data;

[0053] S215, clean up useless data;

[0054] S216. Remove null data values.

[0055] S22, judging whether there is frame loss by the size of each file, forming the frame loss rate of each file, judging empty data and dirty data and automatically eliminating them. This step can judge whether there is frame loss by the size of each file, further analyze the timestamp data of the video file, calculate and statistically judge whether the file has frame loss, form the frame loss rate of each file, and determine which is empty data and which is dirty data, and automatically eliminate them, which can identify various data start time asynchrony problems.

[0056] S23, calculate and extract static scene features and dynamic scene features. The main task of this step is to calculate and extract the features of static and dynamic scenes. The purpose of extracting features is 1. to provide information to the automatic evaluation module, which performs corresponding subjective and objective scoring evaluations based on different static and dynamic scene features. 2. This information will be used to provide the required semantic information for the knowledge graph modeling step later.

[0057] In step S23, the calculation and extraction of static scene features are specifically as follows: generating static road network feature labels through the VLM large model output layer label system. That is, through the VLM large model Vision-Language Multimodal Model, the output layer label system includes highways, urban roads, intersections, rural roads, crosswalks, bends, ramps, roundabouts, abnormal intersections, ramps, emergency roads, bus lanes and other road environment elements such as traffic lights, cones, speed bumps, etc.

[0058] After generating the static road network feature labels, these road topology information will be provided to the dynamic scene module. Because if you want to identify, for example, a three-lane overtaking scene, the dynamic module must first know how many lanes there are, the width of the lanes and other information before it can use dynamic information (information about its own vehicle and other mobile physics) to judge the dynamic interaction relationship between its own vehicle and other traffic participants.

[0059] In step S23, the calculation and extraction of dynamic scene features specifically includes: constructing interactive behavior modeling and constructing a kinematic relationship algorithm for traffic participants through an offline perception enhancement algorithm and a scene slicing algorithm, which specifically includes the following steps:

[0060] S231. Automatically identify and generate dynamic scene information data based on acquired road information: The autonomous vehicle (VUT) is equipped with sensors such as radar, lidar, and cameras that collect information during driving. This information is processed by specialized tools that use the recorded data to determine the scenes encountered by the vehicle during recording. The events that occurred are evaluated as critical or interesting events and extracted from the recorded data.

[0061] S232. Preprocess the dynamic scene information data, filter and reduce noise on the collected data, and ensure that the dynamic scene information data includes basic information about road infrastructure, vehicle driving data, and other objects and traffic participants that change over time. Data quality has a great impact on the subsequent steps of extracting the scene. The requirements for equipment and measurement frequency must be carefully defined to minimize measurement errors. To cope with residual noise, the collected data must be filtered and reduced in noise. The data preprocessing step will deal with significant data inaccuracies. The measured parameters must contain basic information about road infrastructure, vehicle driving data, and other objects and traffic participants that change over time. Based on this information, the road infrastructure and the scenes that occur can be reconstructed for simulation. Specifically:

[0062] a. The measured road parameters describe the road geometry (e.g. lanes, elevation) and its topology, i.e. the road network and its elements (intersections, motorway entrances and exits, parking areas, roundabouts, etc.). In addition, if information about objects and traffic signals around the road is available (e.g. traffic signs or pedestrian crossings), this information will help to reconstruct the area. In combination with the vehicle's driving data, this information can be used to generate the road infrastructure using specialized tools. The generated converted road must be similar to the recorded data while ensuring the continuity of the road.

[0063] b. Recording information about the relative positions of surrounding vehicles and pedestrians as well as obstacles and objects of interest is a key process for understanding traffic conditions and subsequently extracting scenes. The first step in this process is object detection, which is the identification of objects on the road based on the collected sensor data. This data annotation is the task of the perception system. There are two possible approaches. We use an offline perception algorithm. Modules after S23 can also be used with an online perception system.

[0064] c. Online perception system: integrated into the vehicle, automatically detects objects and traffic participants around the VUT during driving. However, efficient real-time object annotation requires a lot of computing power and memory, making it expensive to develop and integrate.

[0065] d. Offline perception system: objects are annotated after the fact. The advantage is that the complete data record can be used, providing a wider range of potential algorithm choices. Although traffic participants are not detected in real time, the offline system can access more reference data and achieve higher accuracy. In addition, the offline system is executed in the cloud platform and can be reused for multiple vehicles, which is cheaper and more accurate.

[0066] e. Once the true value information of traffic participants and road objects is labeled, this data will be combined with their position information around the VUT and provided to the scene recognition algorithm. The algorithm can automatically identify the dynamic scenes encountered (for example: following, changing lanes, turning, detouring around obstacles, cutting in, cutting out, ramps, etc.) and the physical quantities in the scene (speed, acceleration, reaction time, pre-collision time, lane change response, etc.) based on the processed data.

[0067] In an identified scene, the calculated physical signals include, for example (Ego refers to the own vehicle, TOF refers to the identified vehicle object): the longitudinal speed of the Ego vehicle, the longitudinal acceleration, the lateral acceleration of the Ego vehicle, the yaw rate of the Ego vehicle, the distance line on the left side of the Ego vehicle, the distance line on the right side of the Ego vehicle, the curvature of the left lane marking of the Ego vehicle, the curvature of the right lane marking of the Ego vehicle, the longitudinal distance between the Ego vehicle and the front TOF, the lateral distance between the Ego vehicle and the front TOF, the longitudinal speed of the front TOF, the longitudinal acceleration of the front TOF, the identifier of the left TOR, the distance between the Ego vehicle and the left TOR The longitudinal distance between the Ego vehicle and the left TOF, the lateral distance between the Ego vehicle and the left TOF, the longitudinal speed of the left TOF, the longitudinal acceleration of the left TOF, the longitudinal distance between the Ego vehicle and the left TOF, the lateral distance between the Ego vehicle and the left TOF, the longitudinal speed of the left TOF, the longitudinal acceleration of the left TOF, the longitudinal distance between the Ego vehicle and the right TOF, the lateral distance between the Ego vehicle and the right TOF, the longitudinal speed of the right TOF, the longitudinal acceleration of the right TOF, the longitudinal distance between the Ego vehicle and the right TOF, the lateral distance between the Ego vehicle and the right TOF, the longitudinal speed of the right TOF, and the longitudinal acceleration of the right TOF.

[0068] Step S3 is to quantify the subjective driving experience into objective data for scoring: the physical quantities required for evaluation are calculated by identifying the scene, and the driving experience is quantified into objective data for scoring through a multi-dimensional and multi-layer mapping model. The objective data after scoring is summarized to form a multimodal data set. The core of step S3 is to calculate the physical quantities required for evaluation through automatically identified scenes, and quantify the subjective driving experience into objective data for scoring through a multi-dimensional and multi-layer mapping model, such as comfort, perceived safety, physical safety, efficiency, etc. The goal of step S3 is to propose an objective description method for complex driving scenarios that consider objective or subjective criteria. The method defines a process for considering complex relationships in a multidimensional parameter space. Step S3 quantifies the subjective driving experience into objective data for scoring, and specifically includes the following steps: S31, defining behavior categories, and classifying driving scenarios according to selected reference parameters; S32, analyzing these behavior categories in a multidimensional parameter space by dividing individual scenarios into temporally constant sequences; S33, clustering the parameter space using mathematical methods, and each sequence is assigned to a behavior category containing a scenario and a corresponding independent cluster member; S34, completing the scoring by progressively mapping the signals and results layer by layer according to the mapping relationship of each evaluation layer.

[0069] For example, when evaluating the intelligent driving function of a vehicle in the face of lane cutting in a high-speed curve, the following signals will be analyzed: response delay, minimum acceleration, minimum TTC (time to collision), minimum distance to the vehicle in front, lane accuracy, lateral control quality, lane utilization, distance to the left lane, distance to the right lane, lane accuracy, lateral control quality, lane utilization, left lane distance, right lane distance, maximum latitude deviation, maximum yaw, maximum torque, maximum acceleration y, minimum distance to the lane, etc. Through calculation, the following signals are obtained: minimum acceleration x, acceleration smoothness, first reaction collision time, critical time gap (critical time gap refers to: assuming that the vehicle in front will stop immediately, evaluate whether the time gap of the vehicle following the vehicle is safe, and whether the EGO vehicle can stop within the remaining distance to avoid a collision. According to the braking distance given by the maximum deceleration and the road coefficient, evaluate the time gap itself and the emergency braking distance required for safety). In this scenario, the safety index score is greater than 6 points (1-10 points evaluation system) in the range of: response delay <1.3[s], minimum acceleration>-2.5[m / s 2 ], minimum TTC>3[s], minimum distance>12[m], maximum latitude deviation<0.35[m], maximum yaw<6[° / s], maximum acceleration y<0.5[m / s 2 ], maximum steering wheel torque <2.5[Nm], minimum distance to the lane <0.3[m].

[0070] The multimodal data set in step S3 includes safety indicators, comfort indicators, reliability indicators and passability indicators. Safety indicators include, for example, response timeliness, speed range, following distance range, following stopping distance, TTC (Time to Collision), takeover response time, automatic start-stop strategy, lane centering characteristic, critical time gap, path following offset, default lane, speed change strategy, dynamic object meeting strategy, obstacle avoidance strategy, obstacle avoidance success rate, etc. Comfort indicators include: Response Timeliness, Driving Smoothness, Longitudinal Acceleration Value and Linear Degree of Acceleration and Deceleration, Steering Wheel Rotation Situation, Lane Change Time, Speed ​​Fluctuation Range, Automatic Start-Stop Duration, Lane Centering Degree, Curve Deceleration Strategy, etc.Reliability indicators include: takeover rate (TakeoverRate), takeover requirement time, average takeover mileage (Average Takeover Mileage), curve passing rate (CurvePassing Rate), autonomous lane change success rate (Autonomous Lane Change Success Rate), traffic light recognition accuracy rate (Traffic Light Recognition Accuracy Rate), following straight passing rate (IntersectionStraight Passing Rate-Following), non-following straight passing rate (Intersection StraightPassing Rate-Not Following), intersection left turn passing rate (Intersection Left Turn PassingRate), intersection right turn passing rate (Intersection Right Turn Passing Rate), intersection U-turn passing rate (Intersection U-turn Passing Rate) and obstacle avoidance success rate (Obstacle Avoidance Success Rate). The passability / continuity indicators include: Driving Time, Human-Machine Co-driving Performance, Path Deviation Rate, Speed ​​Range, Following Distance Range, Following Stop Distance, Effective Lane Change Rate, Deceleration Strategy When Entering an Intersection and Acceleration Strategy When Leaving an Intersection, Meeting Strategy, Obstacle Avoidance Strategy, etc.

[0071] Step S4 is to establish knowledge graph modeling, and the multimodal data in the multimodal data set are associated through the knowledge graph to establish a scene-style association matrix. Knowledge graphs are good at processing structured knowledge and can integrate information from different data sources into an association network. In the Zhijia veteran driver database, driving behavior data, user portraits, environmental information, etc. are all multimodal data. Knowledge graphs can help establish the relationship between these data, such as the association between driving habits and specific scenarios, or the connection between user behavior and safety scores. Through the three basic elements of the knowledge graph, entities, classes and relationships are extracted, for example:

[0072] Driver entity: {attributes: driving style (aggressive / smooth), physiological state (heart rate / fatigue value)}

[0073] Vehicle entity: {attributes: vehicle model, sensor configuration, dynamic parameters}

[0074] Environmental entity: {attributes: weather, road type (highway / city), traffic density, road attributes}

[0075] Behavior entity: {attributes: lane change decision, braking intensity, following distance}

[0076] The relationship is: ("aggressive driver", "on", "dry highway") → ("tends to", "following distance < 1s", "minimum TTC < 2.5s", "minimum following distance < 12m", "maximum latitude deviation < 0.35m", "maximum yaw < 6[° / s]").

[0077] The advantages of knowledge graph modeling are: a. Brand gene encoding technology driven by knowledge graph; b. Define entities, relationships and attributes (such as drivers, vehicles, roads, behavior patterns) to establish knowledge graphs based on parameter feature definitions for different driver styles; c. Semanticization of driving habit portrait feature graphs; d. Construction of a driving style decoupling graph containing scenario factors; e. Brand gene encoding, for example: decomposing abstract positioning such as "sporty" into 90 measurable parameters (such as cornering speed retention rate and following distance gradient); f. Data targeted screening: Establish a scenario-style association matrix based on the spectral clustering algorithm to achieve intelligent optimization of training data.

[0078] Step S5 is driving style classification and driving style deviation index (SDI) calculation: driving style classification is performed based on the scene-style association matrix established based on the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is:

[0079]

[0080] Among them, P (i) actualis the actual measured value of the i-th driving style parameter, P (i) brand is the ideal value of the preset brand driving mode, P (i) range is the reasonable value range of the parameter, and wi is the weight coefficient of the data parameter in the brand driving mode.

[0081] Step 5 is to automatically identify specific driving styles based on the output of data and knowledge graphs on the basis of the previous steps, establish a scene-style association matrix through the knowledge graph, and realize intelligent optimization of training data. In this process, in order to better quantify, the present invention proposes a driving style deviation index (SDI), which is a quantitative indicator to measure the difference between the actual performance of the intelligent driving system and the preset brand driving gene. The construction of SDI is based on multimodal feature space projection and weighted distance measurement, and the calculation formula is:

[0082]

[0083] in:

[0084] P (i) actual is the actual measured value of the i-th driving style parameter, such as lateral acceleration in a curve and following distance;

[0085] P (i) brand The ideal value for the preset brand driving mode, such as setting the cornering acceleration to 0.3g for sports models;

[0086] P (i) range is a reasonable value range for the parameter;

[0087] wi is the weight coefficient of the data parameter in the brand driving mode, which is determined by the hierarchical analysis method and reflects the importance of the parameter (for example, the acceleration weight in a sports car model can reach 0.6).

[0088] SDI can be used to ultimately evaluate brand genetic consistency and differentiation, converting abstract brand driving characteristics (such as "sporty" or "comfortable") into calculable numerical indicators.

[0089] Step S6 is the optimization of intelligent driving mode parameters: according to the deviation index of SDI positioning, the driving mode training is carried out for the deviation index to form personalized intelligent driving mode parameters. Combined with the big data foundation, the model of step S6 can analyze tens of thousands of sliced ​​data and identify the defects of existing functional driving styles. Developers can select the missing driving style suitable for brand strategy in the cloud data closed-loop platform for training based on the analysis results. The present invention solves the problem that large model training relies on data distribution to cause brand driving style deviation, and the problem that the intelligent driving solution of the OEM lacks brand differentiation. Due to the lack of market competitiveness caused by homogeneous functional stacking (such as L2+ / L3 level autonomous driving), the OEM urgently needs to shape the brand label through "personalized" intelligent driving experience (such as Tesla's "radical" style vs. Xiaopeng's "sense of technology"). The present invention proposes a brand personality configuration engine that supports the following customization capabilities: Driving style library: pre-defines multiple style templates (such as "conservative", "sporty", "family-friendly"), covering decision logic (such as following distance), interaction mode (such as voice intonation) and other parameters. It can learn based on user preferences: through online learning (such as feedback data after the user manually adjusts the cruising speed), style parameters are dynamically optimized to achieve a personalized experience of "one thousand faces for one thousand people".

[0090] A driving style quantitative evaluation method based on multimodal perception and knowledge graph in this embodiment provides an intelligent driving mode parameter recognition optimization case:

[0091] 1. Obtain 100 hours of driving data (including biological signs) of the target brand;

[0092] 2. Identify the current style tendency through knowledge graph style classification (aggressive score 78%);

[0093] Technical reason analysis: Lane change aggressiveness index is too high (actual measurement > target value)

[0094] Steering wheel torque fluctuation exceeds the limit by 32%

[0095] Lateral acceleration threshold: >0.3g

[0096] Lane change decision time: <1.2 seconds

[0097] 3. Compare the "luxury and comfortable" template in the knowledge graph and locate multiple deviation indicators. Data examples:

[0098] The average longitudinal jerk is too high (current 0.12 g / s vs. target 0.08 g / s)

[0099] Steering wheel oscillations in curves exceeded the target (currently 5.2 times / km vs target <3 times)

[0100] The frequency of abnormal fluctuations in passenger comfort values ​​is too high (currently 12.3 times / h vs. target <8 times)

[0101] 4. Filter 3,000 scene clips containing target features on the data closed-loop platform;

[0102] 5. Retrain and fine-tune the model;

[0103] 6. Quantification of technical effects: brand style matching improved; user net recommendation value increased; takeover rate decreased; style transfer training efficiency improved; user satisfaction improved.

[0104] The present embodiment provides a driving style quantitative evaluation method based on multimodal perception and knowledge graph, and proposes a multi-dimensional user experience quantification framework. Through multimodal data sources, including various sensor data, vehicle motion data, and physiological reactions and behavioral data of drivers and passengers in real scenarios (such as heart rate changes during emergency braking and frequency of operation intervention), the model constructs a user emotion mapping algorithm to convert subjective experience into quantifiable numerical indicators (such as "trust index" and "comfort score"). Combined with the constructed driving behavior knowledge graph, the algorithm is used to calculate the driving style deviation index (SDI), thereby achieving quantitative evaluation of brand driving genes and providing a data-driven basis for function and algorithm optimization.

[0105] The core innovations of a driving style quantitative evaluation method based on multimodal perception and knowledge graph in this embodiment are summarized as follows: 1. Fusion of multimodal sensor data and passenger subjective feeling sign data; 2. Objective quantitative model of subjective feeling; 3. Intelligent driving knowledge graph modeling based on engineering experience; 4. Quantitative breakthrough in user experience, quantitative evaluation of user subjective feelings through objectivity; 5. Digitalization of intelligent driving brand style: parametric gene coding: decomposing abstract brand positioning into a multi-dimensional evaluation system (safety, comfort, passenger perceived safety, efficiency, reliability) and a measurable parameter database, for example: "Sports" style: cornering speed reduction ratio ≤15% & following distance gradient ≥0.35s / m; "Family" style: longitudinal acceleration standard deviation <0.12g & seat vibration power spectrum density ≤4.5m 2 / s 3; 6. Scene semantic deconstruction and feature engineering: Static scene feature extraction: Vision-Language Multimodal Model is used to build a road topology knowledge base, and the output layer contains more than 100 types of road elements (such as compound roundabouts, five-way intersections) and infrastructure parameters; Dynamic scene feature extraction: Three-layer dynamic scene automatic extraction (atomic layer-behavior layer-task layer), covering more than hundreds of sub-scenes; 7. Semantic knowledge graph library and SDI dynamic optimization mechanism to realize driving style deviation index (SDI) 8. Improve the matching degree of brand target style and shorten the data closed-loop iteration cycle; 9. Improve the effectiveness of engineering implementation: Training efficiency breakthrough: Targeted data screening based on knowledge graph improves the proportion of effective training data and the speed of model convergence; 10. Targeted solution to industry pain points: Regional adaptability: Not only adapt to the styles of various parts of China, but also adapt to different countries and regions around the world; Improve user acceptance of aggressive driving style and improve the pass rate of non-standard intersections; 11. Long-tail scene coverage: Improve the detection rate of low-probability scenes and cross-scene style transfer screening through knowledge graph association reasoning.

[0106] The driving style quantitative evaluation method based on multimodal perception and knowledge graph of the present invention enables the intelligent driving system to achieve three major leaps: 1. From functional availability to trustworthy experience: establishing an autonomous driving trust quantification system that integrates biological signs and multi-sensor data sources; 2. From blind data selection to targeted evolution: through the establishment of knowledge graphs and the SDI index to drive the training efficiency to an order of magnitude improvement; 3. From homogeneity to gene expression: supporting OEMs to form perceptible brand driving DNA at the L2+ stage.

[0107] The present invention is a driving style quantitative evaluation method based on multimodal perception and knowledge graph. Through multimodal data sources, including various sensor data, vehicle motion data, and physiological reactions and behavioral data of drivers and passengers in real scenes (such as heart rate changes and operation intervention frequency during emergency braking), the model constructs a user emotion mapping algorithm, converts subjective experience into quantifiable numerical indicators (such as "trust index" and "comfort score"), combines the constructed driving behavior knowledge graph, and uses an algorithm to calculate the driving style deviation index (SDI), so as to achieve a quantitative evaluation of brand driving genes, provide a data-driven basis for function and algorithm optimization, and realize a personalized intelligent driving mode. Therefore, the present invention is a driving style quantitative evaluation method based on multimodal perception and knowledge graph, which has the advantages of high intelligence and high personalization.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. That is, all equivalent changes and modifications made according to the content of the patent application scope of the present invention should belong to the technical scope of the present invention.

Claims

1. A driving style quantitative evaluation method based on multimodal perception and knowledge graph, characterized in that: The following steps are involved: S1. Data collection: perform global signal capture and collect vehicle information: vehicle information includes vehicle bus CAN signal, CSV data, video data, Rosbag data, and Ethernet bus data; S2, processing the vehicle information to calculate static and dynamic scene features: comprising the following steps: S21, processing vehicle information; S22, judging whether a frame is lost by the size of each file, and forming a frame loss rate of each file, Determine empty data and dirty data and automatically eliminate them; S23, calculating and extracting static scene features and dynamic scene features; S3. Quantify subjective driving experience into objective data for scoring: Calculate the physical quantities required for evaluation by identifying the scene, quantify the driving experience into objective data for scoring through a multi-dimensional and multi-layer mapping model, and aggregate the scored objective data to form a multimodal data set; S4, establishing a knowledge graph model, associating the multimodal data in the multimodal data set through the knowledge graph, and establishing a scene-style association matrix; S5. Driving style classification and driving style deviation index (SDI) calculation: Driving style classification is performed based on the scene-style association matrix established based on the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is: Among them, P (i) actual is the actual measured value of the i-th driving style parameter, P (i) brand is the ideal value of the preset brand driving mode, P (i) range is the reasonable value range of the parameter, wi is the weight coefficient of the data parameter in the brand driving mode; S6. Intelligent driving mode parameter optimization: Based on the SDI positioning deviation index, driving mode training is performed according to the deviation index to form personalized intelligent driving mode parameters.

2. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: The global signal capture in step S1 is specifically as follows: a. Collect the output signals of each sensor: the sensors include cameras, millimeter wave radars, laser radars, ultrasonic radars, and positioning sensors; the signal content includes the original signal and the signal after solution processing; b. Collect vehicle chassis signals, including control instructions and feedback, kinematic states such as speed and acceleration, wheel speed and wheel angle, power and steering system states, and vehicle body component states; c. Collect biological sign monitoring signals: Monitor heart rate and breathing rate through in-cabin radar and cameras to build a user health portrait; combine biological sign monitoring signals and driving behavior to infer the user's emotional state.

3. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: Step S21 vehicle information processing specifically includes the following steps: S211, unstructured data processing: perform visual data stream cleaning, use spatiotemporal consistency detection algorithm to repair video frame loss; perform point cloud data optimization, suppress ground reflection noise, and eliminate motion distortion; CAN bus data calibration, establish signal rationality verification; S212, perform structured data cleaning: fill in empty values, remove outliers, apply 3σ principle and DBSCAN clustering; S213, supplement missing data; S214, merge duplicate data; S215, clean up useless data; S216. Remove null data values.

4. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: The calculation and extraction of static scene features in step S23 specifically includes: generating static road network feature labels through the VLM large model output layer label system.

5. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 4 is characterized by: The static road network feature labels include road topology information and other road environment elements; the road topology information includes expressways, urban roads, intersections, rural roads, crosswalks, bends, ramps, roundabouts, abnormal intersections, ramps, emergency roads and bus lanes; the other road environment elements include traffic lights, cones and speed bumps.

6. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: The calculation and extraction of dynamic scene features in step S23 specifically includes: constructing interactive behavior modeling and constructing a kinematic relationship algorithm for traffic participants through an offline perception enhancement algorithm and a scene slicing algorithm, specifically including the following steps: S231, automatically identifying and generating dynamic scene information data based on the acquired road information; S232. Preprocess the dynamic scene information data, filter and reduce noise on the collected data, and ensure that the dynamic scene information data includes basic information about road infrastructure, vehicle driving data, and other objects and traffic participants that change over time.

7. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: Step S3 quantifies the subjective driving experience into objective data for scoring, and specifically includes the following steps: S31, defining behavior categories and classifying driving scenarios according to selected reference parameters; S32, analyze these behavior categories in a multidimensional parameter space by dividing individual scenes into temporally constant sequences; S33, clustering the parameter space using mathematical methods, each sequence is assigned to the behavior category containing the scene and the corresponding independent cluster member; S34. According to the mapping relationship of each evaluation layer, the signals and results are progressively scored layer by layer.

8. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: The multimodal data set in step S3 includes a safety index, a comfort index, a reliability index and a passability index.

9. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 1, characterized in that: In step S4, the multimodal data in the multimodal data set are associated through the knowledge graph, specifically establishing the relationship between the multimodal data through the knowledge graph, and extracting entities, classes and relationships through the knowledge graph.

10. The driving style quantitative evaluation method based on multimodal perception and knowledge graph according to claim 9, characterized in that: The entities include driver entity, vehicle entity, environment entity and behavior entity.

Images