A driving style quantitative evaluation method based on multi-modal perception and knowledge graph

Abstract

The application discloses a kind of driving style quantitative evaluation method based on multi-modal perception and knowledge graph, through multi-modal data source, which contains various sensor data, vehicle motion data and physiological response and behavior data (such as heart rate change when emergency brake, operation intervention frequency) of driver and passenger in real scene, model constructs user emotion mapping algorithm, subjective experience is converted into quantifiable numerical index (such as "trust index" "comfort score"), combined with the driving behavior knowledge graph constructed, the driving style deviation index (SDI) is calculated using algorithm, the quantitative evaluation of brand driving gene is realized, data-driven basis is provided for function and algorithm optimization, and personalized intelligent driving mode is realized.Therefore, the driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the application has the advantages of high degree of intelligentization and high degree of personalization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a driving style quantitative evaluation method based on multi-modal perception and knowledge graph with high intelligence and high personalization. BACKGROUND

[0002] With the development of automatic driving technology and the application of end-to-end large models, automatic driving functions increasingly rely on data closed-loop training, and the resulting problem is that the intelligent driving style of each enterprise is greatly influenced by the data and driving style of most customers of its brand, which leads to the singleization of the intelligent driving style and the deviation of the data distribution (such as road scenes and driving habits) from the actual scenes of the target users of the host factory (such as complex road conditions in China and local driving styles), resulting in the driving style (such as conservative, aggressive, sporty, efficient, smooth, cautious, etc.) output by the model being inconsistent with the brand positioning. In addition, evaluating the feelings and trust of users towards the automatic driving system becomes a key problem, and the existing technology lacks a systematic method to quantify user feelings and does not have a good methodology to filter and select the required data to train the software according to different driving style requirements.

[0003] The existing automatic driving evaluation system excessively relies on the binary decision of "pass / fail" (such as regulatory testing) and lacks fine quantitative measurement of user subjective feelings (such as comfort, safety, and trust).

[0004] Specifically as follows:

[0005] 1. Data-driven style solidification: The existing data collection network of the host factory is limited by the characteristics of the stock user group (such as age distribution and regional driving habits), resulting in a deviation of the training data distribution from the real scenarios of the target market. The specific manifestations are as follows: road scene deviation: the model training focuses on highway scenarios, and the coverage rate of complex urban scenarios (such as non-standard intersections and mixed traffic) is insufficient; driving behavior deviation: the system inherits the inherent driving characteristics of the user group, which significantly conflicts with the target value of the brand's expected control style;

[0006] 2. User experience quantitative blind area: the existing evaluation system adopts a traditional functional safety verification framework, and its core defects are as follows: binary evaluation dimension: only mechanical indicators such as intervention rate and collision avoidance success rate are concerned; perception value decoupling: it is impossible to quantify subjective cognitive indicators (such as comfort, trust, and control expectation matching degree) of the driver and passengers.

[0007] Industry research shows that 84% of users believe that the existing L2+ system lacks "personification", which is manifested in the lack of pre-judgment of passenger neck muscle tension during sudden acceleration and the lack of old driver-like operation rhythm in curve control. Therefore, it is necessary to propose an improvement to overcome the defects of the existing technology. SUMMARY

[0008] The purpose of the present application is to solve the problems in the prior art and provide a driving style quantitative evaluation method based on multi-modal perception and knowledge graph, which has high intelligence and high personalization.

[0009] The technical scheme of the present application is:

[0010] A driving style quantitative evaluation method based on multi-modal perception and knowledge graph, comprising the following steps: S1, data acquisition: performing global signal capture, collecting vehicle information: the vehicle information includes: vehicle bus CAN signal, CSV data, video data, Rosbag data, Ethernet bus data; S2, performing data processing on the vehicle information, calculating static and dynamic scene features: comprising the following steps: S21, processing the vehicle information; S22, judging whether to lose frames by the size of each file, forming the frame loss rate of each file, judging and automatically eliminating empty data and dirty data; S23, calculating and extracting static scene features and dynamic scene features; S3, quantifying subjective driving experience into objective data for scoring: calculating the required physical quantity for evaluation by recognizing the scene, quantifying the driving experience into objective data for scoring by a multi-dimensional multi-layer mapping model, and the scored objective data is summarized to form a multi-modal data set;

[0011] S4, establishing knowledge graph modeling, associating the multi-modal data in the multi-modal data set through the knowledge graph, and establishing a scene-style correlation matrix; S5, driving style classification and driving style deviation index (SDI) calculation: driving style classification based on the scene-style correlation matrix established by the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is:

[0012]

[0013] Wherein, P (i) actual is the actual measurement value of the i-th driving style parameter, P (i) brand is the ideal value of the preset brand driving mode, P (i) range is the reasonable value range of the parameter, wi is the weight coefficient of the data parameter in the brand driving mode; S6, intelligent driving mode parameter optimization: according to the SDI positioning deviation index, the driving mode is trained according to the deviation index, and the personalized intelligent driving mode parameter is formed.

[0014] As a preferred technical solution, the global signal capturing in step S1 is specifically: a, collecting each sensor output signal: the sensor includes a camera, a millimeter wave radar, a laser radar, an ultrasonic radar, a positioning sensor; the signal content includes raw signals and processed signals; b, collecting vehicle chassis signals, including control instructions and feedback, kinematics states such as speed and acceleration, states such as wheel speed and wheel angle, power and steering system state, vehicle body component state; c, collecting biological sign monitoring signals: monitoring heart rate and breathing rate through cabin radar and camera to construct user health portrait; combining the sign monitoring signals and driving behavior to infer user emotional state.

[0015] As a preferred technical solution, the vehicle information processing in step S21 specifically includes the following steps: S211, unstructured data processing: performing visual data stream cleaning, using a spatiotemporal consistency detection algorithm to repair video frame loss; performing point cloud data optimization, suppressing ground reflection noise, and eliminating motion distortion; CAN bus data calibration, establishing signal rationality verification; S212, performing structured data cleaning: filling in null values, removing outliers, and applying the 3 sigma principle combined with DBSCAN clustering; S213, supplementing missing data; S214, merging duplicate data; S215, cleaning useless data; S216, removing data null values.

[0016] As a preferred technical solution, the calculation and extraction of static scene features in step S23 specifically includes: generating static road network feature labels through the VLM large model output layer label system.

[0017] As a further preferred technical solution, the static road network feature labels include road topology information and other road environment elements; the road topology information includes expressways, urban roads, crossroads, rural roads, pedestrian crossings, curves, slopes, roundabouts, abnormal intersections, ramps, emergency roads, and bus lanes; the other road environment elements include traffic lights, cone barrels, and speed bumps.

[0018] As a preferred technical solution, the calculation and extraction of dynamic scene features in step S23 specifically includes: constructing interactive behavior modeling and constructing traffic participant kinematics relationship algorithms through offline perception enhancement algorithms and scene slicing algorithms, specifically including the following steps: S231, based on the obtained road information, automatically identifying and generating dynamic scene information data; S232, preprocessing the dynamic scene information data, filtering and denoising the collected data, and ensuring that the dynamic scene information data includes road infrastructure, vehicle driving data, and other objects and traffic participants basic information changing over time.

[0019] As a preferred technical solution, the step S3 quantifies the subjective driving experience into objective data for scoring specifically includes the following steps: S31, defining behavior categories, classifying driving scenes according to selected reference parameters; S32, analyzing these behavior categories in a multi-dimensional parameter space by dividing individual scenes into time-invariant sequences; S33, clustering the parameter space using mathematical methods, each sequence is assigned to a behavior category containing the scene and a corresponding independent cluster member; S34, according to the mapping relationship of each evaluation layer, the signal and the result are progressively completed for scoring layer by layer.

[0020] As a preferred technical solution, the multi-modal data set in step S3 includes safety indicators, comfort indicators, reliability indicators and passability indicators.

[0021] As a preferred technical solution, the multi-modal data in the multi-modal data set in step S4 is associated through the knowledge graph, specifically, the relationship between the multi-modal data is established through the knowledge graph, and the entities, classes and relationships are extracted through the knowledge graph.

[0022] As a further preferred technical solution, the entity includes a driver entity, a vehicle entity, an environment entity and a behavior entity.

[0023] The driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the present application, through multi-modal data sources, which contains various sensor data, vehicle motion data and physiological responses and behavior data of drivers and passengers in real scenes (such as heart rate changes when emergency braking, operation intervention frequency), a user emotion mapping algorithm is constructed, subjective experience is converted into quantifiable numerical indicators (such as "trust index" and "comfort score"), combined with the constructed driving behavior knowledge graph, the driving style deviation index (SDI) is calculated by using the algorithm, the quantitative evaluation of brand driving genes is realized, data-driven basis is provided for function and algorithm optimization, and personalized intelligent driving mode is realized. Therefore, the driving style quantitative evaluation method based on multi-modal perception and knowledge graph has the advantages of high intelligent degree and high personalization degree. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The specific implementation process block diagram of the driving style quantitative evaluation method based on multi-modal perception and knowledge graph. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0026] The terms used in the embodiments of the present application are only for the purpose of describing particular embodiments and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two, but does not exclude the case of including at least one.

[0027] It should be understood that the term "and / or" used herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0028] Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (a stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)".

[0029] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that the product or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such product or system. Without more limitations, the element defined by the sentence "including a" does not exclude the presence of other identical elements in the product or system including the element.

[0030] As Figure 1 A specific embodiment of a driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the present application is shown. The driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the embodiment includes the following steps:

[0031] S1, data collection: global signal capture is performed, and vehicle information is collected: the vehicle information includes vehicle bus CAN signals, CSV data, video data, Rosbag data, and Ethernet bus data;

[0032] S2, data processing of the vehicle information, and calculation of static and dynamic scene features: including the following steps:

[0033] S21, processing the vehicle information;

[0034] S22, judging whether frames are lost by the size of each file, forming a frame loss rate of each file, judging empty data and dirty data and automatically eliminating them;

[0035] S23, calculating and extracting static scene features and dynamic scene features;

[0036] S3, quantifying subjective driving experience into objective data for scoring: calculating the required physical quantity for evaluation by recognizing the scene, quantifying the driving experience into objective data for scoring by a multi-dimensional multi-layer mapping model, and forming a multi-modal data set by summarizing the scored objective data;

[0037] S4, establishing a knowledge graph modeling, correlating the multi-modal data in the multi-modal data set through the knowledge graph, and establishing a scene-style correlation matrix;

[0038] S5, driving style classification and driving style deviation index (SDI) calculation: driving style classification based on the scene-style correlation matrix established by the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is:

[0039]

[0040] wherein, P (i) actual is the actual measured value of the i-th driving style parameter, P (i) brand is the ideal value of the preset brand driving mode, P (i) range is the reasonable value range of the parameter, and wi is the weight coefficient of the data parameter in the brand driving mode;

[0041] S6, intelligent driving mode parameter optimization: according to the SDI positioning deviation index, the driving mode is trained for the deviation index, and the personalized intelligent driving mode parameter is formed.

[0042] Specifically, the driving style quantification and evaluation method based on multi-modal perception and knowledge graph of the embodiment:

[0043] Step S1 is data collection: global signal capture is performed, and vehicle information is collected: the vehicle information includes vehicle bus CAN signals, CSV data, video data, Rosbag data, and Ethernet bus data. The data collection of the embodiment realizes global signal capture through a three-level hierarchical architecture, and the specific steps are as follows:

[0044] a. Collecting output signals of various sensors: the sensors include cameras, millimeter wave radars, laser radars, ultrasonic radars, and positioning sensors; the signal content includes original signals and signals after calculation and processing;

[0045] b. Collecting vehicle body chassis signals, including control instructions and feedback, kinematic states such as speed and acceleration, states such as wheel speed and wheel angle, power and steering system states, and vehicle body component states;

[0046] c. Collecting biological sign monitoring signals: heart rate and breathing rate are monitored through cabin radar and camera to construct a user health portrait; user emotional state inference is performed in combination with the sign monitoring signals and driving behavior (sudden acceleration / braking) to infer user stress / fatigue level.

[0047] Step S2 is data processing of the vehicle information to calculate static and dynamic scene features: including the following steps:

[0048] S21, processing the vehicle information; specifically including the following steps:

[0049] S211, unstructured data processing: visual data stream cleaning is performed, a space-time consistency detection algorithm is used to repair video frame loss; point cloud data optimization is performed to suppress ground reflection noise and eliminate motion distortion; CAN bus data calibration is performed to establish signal rationality verification;

[0050] S212, structured data cleaning: missing values are filled, abnormal values are removed, and the 3σ principle is applied in combination with DBSCAN clustering;

[0051] S213, supplementing missing data;

[0052] S214, merging duplicate data;

[0053] S215, cleaning useless data;

[0054] S216, removing data null values.

[0055] S22, determine whether to drop frames by each file size, form the frame loss rate of each file, judge and automatically eliminate empty data and dirty data. This step can determine whether there is frame loss by the size of each file, further analyze the timestamp data of the video file, calculate and statistically judge whether the file has frame loss, form the frame loss rate of each file and which is empty data and which is dirty data, and make automatic elimination, which can identify various data start time synchronization problems.

[0056] S23, calculate and extract static scene features and dynamic scene features. The main task of this step is to calculate and extract the features of static and dynamic scenes. The purpose of extracting features is 1. to provide information for the automatic evaluation module, and the automatic evaluation module performs corresponding subjective and objective score evaluation according to different static and dynamic scene features. 2. These information will be used to provide the required semantic information for the knowledge graph modeling step later.

[0057] In step S23, the calculation and extraction of static scene features are specifically: generating static road network feature labels through the VLM large model output layer label system. That is, through the VLM large model Vision-Language Multimodal Model, the output layer label system, the labels include highway, urban road, crossroad, rural road, pedestrian crossing, curve, slope, roundabout, abnormal intersection, ramp, emergency road, bus lane and other road topological information as well as other road environment elements such as signal lamp, cone barrel, deceleration zone, etc.

[0058] After generating the static road network feature labels, these road topological information will be provided to the dynamic scene module, because if you want to identify a three-lane overtaking scene, the dynamic module first needs to know how many lanes there are in total, the width of the lane, etc. Information to use dynamic information (its own vehicle and other moving physical information) to judge the dynamic interaction relationship between its own vehicle and other road users.

[0059] In step S23, the calculation and extraction of dynamic scene features are specifically: through the offline perception enhancement algorithm and scene slicing algorithm, the interactive behavior modeling and the traffic participant kinematics relationship algorithm are constructed, which specifically includes the following steps:

[0060] S231, based on the obtained road information, automatically identify and generate dynamic scene information data: the autonomous vehicle (VUT) is equipped with sensors that collect information during driving, such as radar, lidar and camera. These information is processed through special tools, using recorded data to determine the scene encountered by the vehicle during recording. The events that occur are evaluated as key or interesting events and are extracted from the recorded data.

[0061] S232, Preprocessing of dynamic scene information data, filtering and denoising of collected data, ensuring that the dynamic scene information data includes the basic information of the road infrastructure, vehicle driving data and other objects and traffic participants changing over time. Data quality has a great influence on the subsequent steps of extracting the scene. The requirements of the equipment and measurement frequency must be carefully defined to minimize measurement errors. To cope with the remaining noise, the collected data must be filtered and denoised. The data preprocessing step will deal with significant data inaccuracies. The measured parameters must contain basic information about the road infrastructure, the driving data of the vehicle, and other objects and traffic participants changing over time. Based on this information, the road infrastructure and the scene that occurred can be reconstructed for simulation. Specifically:

[0062] a. The measured road parameters describe the road geometry (e.g. lanes, elevation) and its topology, i.e. the road network and its elements (intersections, highway exits, parking areas, roundabouts, etc.). In addition, if information about objects and traffic signals around the road (such as traffic signs or pedestrian crossings) is provided, this information will help to reconstruct the area. Combined with the driving data of the vehicle, these information can be used to generate the road infrastructure using specialized tools. The generated transformed road must be similar to the recorded data while ensuring the continuity of the road.

[0063] b. Recording information about the relative position of surrounding vehicles and pedestrians, as well as obstacles and objects of interest, is a critical process for understanding traffic conditions and subsequently extracting the scene. The first step of this process is object detection, i.e. identifying objects on the road based on collected sensor data. This data labeling is the task of perception systems, and there are two possible approaches, we use the offline perception algorithm, the modules after S23 can also be used with online perception systems.

[0064] c. Online perception systems: integrated into vehicles, automatically detecting objects and traffic participants around the VUT during driving. However, efficient real-time labeling of objects requires a lot of computing power and memory, so the development and integration costs are high.

[0065] d. Offline perception systems: labeling objects after the fact. The advantage is that it can use complete data records, providing a wider range of potential algorithm choices. Although traffic participants are not detected in real time, offline systems have access to more reference data, enabling higher accuracy. In addition, offline systems are executed in cloud platforms, which can be reused for multiple vehicles, are less expensive and more accurate.

[0066] e、Once the ground truth labeling of traffic participants and road objects is completed, these data will be combined with their location information around the VUT and provided to the algorithm of scene recognition. The algorithm can automatically recognize the dynamic scene (e.g. following, lane changing, turning, obstacle avoidance, cut-in, cut-out, ramp, etc.) and the physical quantities (speed, acceleration, reaction time, pre-crash time, lane changing response, etc.) in the scene according to the processed data.

[0067] In one recognized scene, the calculated physical signals include, for example (Ego refers to the ego vehicle, TOF refers to the recognized vehicle object): longitudinal speed of the ego vehicle, longitudinal acceleration of the ego vehicle, lateral acceleration of the ego vehicle, yaw rate of the ego vehicle, distance line on the left side of the ego vehicle, distance line on the right side of the ego vehicle, curvature of the left lane marking of the ego vehicle, curvature of the right lane marking of the ego vehicle, longitudinal distance between the ego vehicle and the front TOF, lateral distance between the ego vehicle and the front TOF, longitudinal speed of the front TOF, longitudinal acceleration of the front TOF, identifier of the left TOR, longitudinal distance between the ego vehicle and the left TOR, lateral distance between the ego vehicle and the left TOF, longitudinal speed of the left TOF, longitudinal acceleration of the left TOF, longitudinal distance between the ego vehicle and the left TOF, lateral distance between the ego vehicle and the left TOF, longitudinal speed of the left TOF, longitudinal acceleration of the left TOF, longitudinal distance between the ego vehicle and the right TOF, lateral distance between the ego vehicle and the right TOF, longitudinal speed of the right TOF, longitudinal acceleration of the right TOF, longitudinal distance between the ego vehicle and the right TOF, lateral distance between the ego vehicle and the right TOF, longitudinal speed of the right TOF, longitudinal acceleration of the right TOF.

[0068] Step S3 is to quantify subjective driving experience into objective data for scoring: the required physical quantities for evaluation are calculated by identifying the scene, and the driving experience is quantified into objective data for scoring by a multi-dimensional multi-layer mapping model, and the scored objective data is aggregated to form a multi-modal data set. The core of step S3 is to calculate the required physical quantities for evaluation through automatic identification of the scene, and to quantify the subjective driving experience into objective data for scoring through a multi-dimensional multi-layer mapping model, such as comfort, perceived safety, physical safety, efficiency, etc. The goal of step S3 is to propose an objective description method for complex driving scenes considering objective or subjective criteria. This method defines a process for considering complex relationships in a multi-dimensional parameter space. Step S3 quantifies subjective driving experience into objective data for scoring, which specifically includes the following steps: S31, define behavior categories, classify driving scenes according to selected reference parameters; S32, analyze these behavior categories in a multi-dimensional parameter space by dividing individual scenes into time-invariant sequences; S33, cluster the parameter space using mathematical methods, and each sequence is assigned to a behavior category containing the scene and a corresponding independent cluster member; S34, complete the scoring layer by layer according to the mapping relationship of each evaluation layer.

[0069] For example, when evaluating a car's intelligent driving function in a high-speed curve with a cut-in, the following signals are analyzed: response delay, minimum acceleration, minimum TTC (time to collision), minimum distance to the front vehicle, lane accuracy, lateral control quality, lane utilization rate, distance to the left lane, distance to the right lane, lane accuracy, lateral control quality, lane utilization rate, left lane distance, right lane distance, maximum latitude deviation, maximum yaw, maximum torque, maximum acceleration y, minimum distance to the lane, etc. Through calculation, the following results are obtained: minimum acceleration x, acceleration smoothness, first reaction collision time, critical time gap (critical time gap refers to: assuming that the vehicle in front will stop immediately, evaluate whether the time gap of the following vehicle is safe, and whether the EGO vehicle can stop within the remaining distance to avoid collision. According to the given braking distance based on maximum deceleration and road coefficient, evaluate the time gap itself and the required emergency braking distance for safety). In this scenario, the safety index score is greater than 6 points (1-10 point evaluation system) in the following ranges: response delay <1.3[s], minimum acceleration >-2.5[m / s 2 ], minimum TTC >3[s], minimum distance >12[m], maximum latitude deviation <0.35[m], maximum yaw <6[° / s], maximum acceleration y <0.5[m / s 2 ], maximum steering wheel torque <2.5[Nm], minimum distance to the lane <0.3[m].

[0070] The safety indicators include, for example, response timeliness, speed range, following distance range, following stop distance, TTC (Time to Collision), takeover response time, automatic start-stop strategy, lane centering characteristic, critical time gap, path following offset, default lane, speed change strategy, dynamic object meeting strategy, obstacle avoidance strategy, obstacle avoidance success rate, and the like. The comfort indicators include response timeliness, driving smoothness, longitudinal acceleration value and linear degree of acceleration and deceleration, steering wheel rotation situation, lane change time, speed fluctuation range, automatic start-stop duration, lane centering degree, curve deceleration strategy, and the like.Reliability indicators include: Takeover Rate, Takeover Demand Time, Average Takeover Mileage, Curve Passing Rate, Autonomous Lane Change Success Rate, Traffic Light Recognition Accuracy Rate, Intersection Straight Passing Rate-Following, Intersection Straight Passing Rate-Not Following, Intersection Left Turn Passing Rate, Intersection Right Turn Passing Rate, Intersection U-turn Passing Rate, Obstacle Avoidance Success Rate, etc. Throughput / continuity indicators include: Driving Time, Man-machine Co-piloting Performance, Path Deviation Rate, Speed Range, Following Distance Range, Following Stop Distance, Effective Lane Change Rate, Deceleration Strategy When Entering an Intersection and Acceleration Strategy When Leaving an Intersection, Meeting Strategy, Obstacle Avoidance Strategy, etc.

[0071] Step S4 is to establish a knowledge graph modeling, and the multi-modal data in the multi-modal data set are associated through the knowledge graph to establish a scene-style association matrix. The knowledge graph is good at processing structured knowledge, and can integrate information of different data sources into an association network. In the intelligent driving veteran driver library, driving behavior data, user portrait, environmental information, etc. are multi-modal data, and the knowledge graph can help establish the relationship between these data, such as the association between driving habits and specific scenes, or the connection between user behavior and safety score. Through the basic three elements of the knowledge graph, entities, classes and relationships are extracted, for example:

[0072] Driver entity: {attribute: driving style (aggressive / stable), physiological state (heart rate / fatigue value)}

[0073] Vehicle entity: {attribute: vehicle type, sensor configuration, dynamics parameter}

[0074] Environmental entity: {attribute: weather, road type (highway / urban), traffic density, road attribute}

[0075] Behavior entity: {attribute: lane changing decision, brake intensity, following distance}

[0076] The relationship is: ("aggressive driver", "in", "dry highway") → ("tends to", "following distance <1s", "minimum TTC <2.5s", "minimum distance <12m", "maximum latitude deviation <0.35m", "maximum yaw <6[° / s]").

[0077] The advantages of knowledge graph modeling are: a. Knowledge graph driven brand gene coding technology; b. Define entities, relationships and attributes (such as drivers, vehicles, roads, behavior patterns) to establish a knowledge graph based on parameter characteristics of different driver styles; c. Semanticization of driving habit portrait feature graph; d. Construction of driving style decoupling graph containing scene factors; e. Brand gene coding, for example: abstract positioning such as "sporty" is disassembled into 90 measurable parameters (such as corner speed retention rate, following distance gradient); f. Data targeted screening: based on the scene-style association matrix established by spectral clustering algorithm, the training data is intelligently optimized.

[0078] Step S5 is driving style classification and driving style deviation index (SDI) calculation: driving style classification based on the scene-style association matrix established by the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is:

[0079]

[0080] Where, P (i) actualP is the actual measured value of the i-th driving style parameter (i) brand P is the ideal value of the preset brand driving mode (i) range P is the reasonable value range of the parameter, and wi is the weight coefficient of the data parameter in the brand driving mode.

[0081] Step 5 is based on the previous steps, based on the output of data and knowledge graph, to perform specific driving style automatic recognition, to establish a scene-style association matrix through the knowledge graph, and to realize intelligent optimization of training data. In this process, in order to better quantify, the present application proposes a driving style deviation index (Style Deviation Index, SDI), which is a quantitative index for measuring the difference between the actual performance of the intelligent driving system and the preset brand driving gene. The construction of SDI is based on multi-modal feature space projection and weighted distance measurement, and the calculation formula is:

[0082]

[0083] Among them:

[0084] P (i) actual P is the actual measured value of the i-th driving style parameter, such as lateral acceleration on a curve or distance when following a car.

[0085] P (i) brand P is the ideal value of the preset brand driving mode, such as setting the curve acceleration to 0.3g for a sports car model.

[0086] P (i) range P is the reasonable value range of the parameter;

[0087] wi is the weight coefficient of the data parameter in the brand driving mode, which is determined by the analytic hierarchy process and reflects the importance of the parameter (such as the weight of acceleration in a sports car model can reach 0.6).

[0088] SDI can be used to finally evaluate the consistency and difference of brand genes, and convert abstract brand driving characteristics (such as "sports type" or "comfort type") into calculable numerical indicators.

[0089] Step S6 is intelligent driving mode parameter optimization: according to the SDI positioning deviation index, the driving mode training is carried out for the deviation index, and the personalized intelligent driving mode parameter is formed. Combined with big data basis, the model of step S6 can analyze tens of thousands of slice data and identify the defects of the existing functional driving style. The developer can select the missing driving style suitable for the brand strategy for training in the cloud data closed loop platform according to the analysis result. The present application solves the problem that the big model training depends on the data distribution, which leads to the deviation of the brand driving style, and the problem that the intelligent driving scheme of the host factory lacks brand differentiation. Due to the homogenization of functional stacking (such as L2+ / L3 level automatic driving), the market competitiveness is insufficient, and the host factory urgently needs to shape the brand label through "personalized" intelligent driving experience (such as the "aggressive" style of Tesla vs. the "technological" style of Xiaopeng). The present application proposes a brand personality configuration engine, which supports the following customization capabilities: driving style library: predefine multiple style templates (such as "conservative type", "sporty type", "family-friendly type"), covering parameters such as decision logic (such as following distance) and interaction mode (such as voice tone). User preference learning: dynamically optimize style parameters through online learning (such as feedback data after user manually adjusts cruise speed), realize "thousand faces" personalized experience.

[0090] The driving style quantification evaluation method based on multi-modal perception and knowledge graph of the embodiment provides an intelligent driving mode parameter recognition optimization case:

[0091] 1. Obtain 100 hours of driving data (including biological signs) of the target brand;

[0092] 2. Identify the current style tendency (aggressive type score 78%) through the knowledge graph style classification;

[0093] · Technical reason analysis: lane change aggressive index is too high (actual value > target value)

[0094] · Steering torque fluctuation exceeds 32%

[0095] · Lateral acceleration threshold: > 0.3g

[0096] · Lane change decision time: <1.2 seconds

[0097] 3. Compare the "luxury comfort type" template in the knowledge graph, and locate multiple deviation indexes, data examples:

[0098] · Longitudinal jerk mean value is too high (present 0.12g / s vs. target 0.08g / s)

[0099] · Number of steering wheel oscillations on curves exceeds the standard (present 5.2 times / km vs. target <3 times)

[0100] • High frequency of passenger comfort value abnormal fluctuations (current 12.3 times / h vs. target <8 times)

[0101] 4. Screen 3000 scene clips containing target features on the data closed-loop platform;

[0102] 5. Retrain the fine-tuned model;

[0103] 6. Quantify technical effects: improve brand style matching degree; increase user net recommendation value; reduce takeover rate; improve style transfer training efficiency; and improve user satisfaction.

[0104] The driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the embodiment proposes a multi-dimensional user experience quantitative framework, obtains multi-modal data sources, including various sensor data, vehicle motion data, and physiological responses and behavior data of drivers and passengers in real scenes (such as heart rate changes when suddenly braking, operation intervention frequency), constructs a user emotion mapping algorithm, converts subjective experience into quantifiable numerical indicators (such as "trust index" and "comfort score"), combines the constructed driving behavior knowledge graph, calculates the driving style deviation index (SDI) by using the algorithm, realizes the quantitative evaluation of brand driving genes, and provides data-driven basis for function and algorithm optimization.

[0105] The core innovation points of the driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the embodiment are summarized as follows: 1. Fusion of multi-modal sensor data and passenger subjective feeling sign data; 2. Subjective feeling objective quantification model; 3. Intelligent driving knowledge graph modeling based on engineering experience; 4. User experience quantification breakthrough, quantitatively evaluate the subjective feelings of users by objectification; 5. Intelligent driving brand style digitization: parameterized gene coding: abstract brand positioning is decomposed into a multi-dimensional evaluation system (safety, comfort, passenger perception safety, efficiency, reliability) and a measurable parameter database, for example: "sports type" style: bend speed reduction ratio ≤15% & following distance gradient ≥0.35 s / m; "family type" style: longitudinal acceleration standard deviation <0.12g & seat vibration power spectral density ≤4.5 m / s 2 / s 3;6, scene semantic deconstruction and feature engineering: static scene feature extraction: adopt Vision-Language Multimodal Model to construct road topology knowledge base, and the output layer contains more than 100 types of road elements (such as composite roundabouts and five-way intersections) and infrastructure parameters; dynamic scene feature extraction: three-layer dynamic scene automatic extraction (atomic layer-behavior layer-task layer), covering hundreds of sub-scenes; 7, semantic knowledge graph library and SDI dynamic optimization mechanism realize driving style deviation index (SDI) 8, brand target style matching degree is improved, and the data closed loop iteration cycle is shortened; 9, engineering landing efficiency is improved: training efficiency breakthrough: based on the knowledge graph, the proportion of effective training data is improved, and the model convergence speed is improved; 10, targeted solution to industry pain points: regional adaptability: not only adapt to the style of all places in China, but also adapt to different countries and regions in the world; the acceptance of aggressive driving style users is improved, and the passing rate of non-standard intersections is improved; 11, long-tail scene coverage: through knowledge graph association reasoning, the detection rate of low-probability scenes and cross-scene style migration screening is improved.

[0106] The driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the application realizes three major transitions of intelligent driving system: 1. From function available to experience credible: establish an automatic driving trust quantitative system integrating biological signs and multi-sensor data sources; 2. From data blind selection to targeted evolution: through knowledge graph establishment and SDI index driven training efficiency, the training efficiency is improved by orders of magnitude; 3. From homogeneity to gene expression: support OEMs to form perceptible brand driving DNA at L2+ stage.

[0107] The driving style quantitative evaluation method based on multi-modal perception and knowledge graph of the application, through multi-modal data sources, including various sensor data, vehicle motion data and physiological response and behavior data (such as heart rate change when emergency braking, operation intervention frequency) of drivers and passengers in real scenes, a user emotion mapping algorithm is constructed, subjective experience is converted into quantifiable numerical indicators (such as "trust index" and "comfort score"), combined with the constructed driving behavior knowledge graph, the driving style deviation index (SDI) is calculated by using the algorithm, the quantitative evaluation of brand driving gene is realized, data-driven basis is provided for function and algorithm optimization, and personalized intelligent driving mode is realized. Therefore, the driving style quantitative evaluation method based on multi-modal perception and knowledge graph has the advantages of high intelligent degree and high individualization degree.

[0108] To sum up, only the preferred embodiments of the application are described above, and they are not used to limit the implementation range of the application. Any equivalent changes and modifications made according to the content of the application patent range should belong to the technical scope of the application.

Claims

1. A driving style quantitative evaluation method based on multi-modal perception and knowledge graph, characterized in that, Comprise the following steps: S1, data acquisition: global signal capture is carried out, and vehicle information is collected; The vehicle information includes: vehicle bus CAN signal, CSV data, video data, Rosbag data, Ethernet bus data; S2, data processing is carried out on the vehicle information, and static and dynamic scene features are calculated: comprising the following steps: S21, processing the vehicle information; S22, judging whether to lose frames by the size of each file, forming the frame loss rate of each file, judging empty data and dirty data and automatically eliminating them; S23, calculating and extracting static scene features and dynamic scene features; S3, quantifying subjective driving experience into objective data for scoring: the required physical quantity for evaluation is calculated by identifying the scene, the driving experience is quantified into objective data for scoring by a multi-dimensional multi-layer mapping model, and the scored objective data is summarized to form a multi-modal data set; S4, knowledge graph modeling is established, and the multi-modal data in the multi-modal data set is associated through the knowledge graph to establish a scene-style correlation matrix; S5, driving style classification and driving style deviation index (SDI) calculation: driving style classification is carried out based on the scene-style correlation matrix established by the knowledge graph; SDI is the difference between the actual performance of the intelligent driving system and the preset brand driving mode, and the calculation formula is: wherein P (i) actual is the actual measured value of the ith driving style parameter, P （i） brand is the ideal value of the preset brand driving mode, P （i） range is the reasonable value range of the parameter, and wi is the weight coefficient of the data parameter in the brand driving mode. S6, intelligent driving mode parameter optimization: according to the SDI positioning deviation index, the driving mode is trained for the deviation index to form personalized intelligent driving mode parameters.

2. The method of claim 1, wherein the method is based on multi-modal perception and knowledge graph. The global signal capture in step S1 is specifically: a, collecting the output signals of various sensors: the sensors include cameras, millimeter wave radars, laser radars, ultrasonic radars and positioning sensors; the signal content includes original signals and processed signals; b, collecting vehicle chassis signals, including control instructions and feedback, speed and acceleration kinematics state, wheel speed and wheel angle state, power and steering system state, and vehicle body component state; c, collecting biological sign monitoring signals: heart rate and respiration rate are monitored through cabin radar and camera to construct user health portrait; user emotional state is inferred in combination with sign monitoring signals and driving behavior.

3. The method of claim 1, wherein the method is characterized by: The vehicle information processing in step S21 specifically comprises the following steps: S211, unstructured data processing: visual data stream cleaning is carried out, time-space consistency detection algorithm is used to repair video frame loss; point cloud data optimization is carried out to suppress ground reflection noise and eliminate motion distortion; CAN bus data calibration is carried out to establish signal rationality verification; S212, structured data cleaning: missing values are filled, abnormal values are removed, and 3σ principle is applied in combination with DBSCAN clustering; S213, supplementing missing data; S214, merging repeated data; S215, cleaning useless data; S216, removing data null values.

4. The method of claim 1, wherein the method is characterized by: The calculation and extraction of static scene features in step S23 are specifically: through the VLM large model output layer label system, static road network feature labels are generated.

5. The method of claim 4, wherein: The static road network feature label includes road topological information and other road environment elements; the road topological information includes expressway, urban road, crossroad, rural road, pedestrian crossing, curve, slope, roundabout, abnormal intersection, ramp, emergency road and bus lane; the other road environment elements include signal lamp, cone barrel and deceleration strip.

6. The method of claim 1, wherein: The calculation and extraction of the dynamic scene features in step S23 are specifically: constructing an interactive behavior modeling and constructing a traffic participant kinematics relationship algorithm through an offline perception enhancement algorithm and a scene slicing algorithm, and specifically including the following steps: S231, automatically identifying and generating dynamic scene information data based on the obtained road information; S232, preprocessing the dynamic scene information data, filtering and denoising the collected data, and ensuring that the dynamic scene information data includes basic information of road infrastructure, vehicle driving data and other objects and traffic participants changing over time.

7. The method of claim 1, wherein the method is based on multi-modal perception and knowledge graph. The quantification of the subjective driving experience into objective data for scoring in step S3 specifically includes the following steps: S31, defining a behavior category, and classifying the driving scene according to the selected reference parameters; S32, analyzing the behavior categories in a multi-dimensional parameter space by dividing individual scenes into time-invariant sequences; S33, clustering the parameter space using mathematical methods, and assigning each sequence to a behavior category containing the scene and a corresponding independent cluster member; S34, completing the scoring layer by layer according to the mapping relationship of each evaluation layer.

8. The method of claim 1, wherein the method is based on multi-modal perception and knowledge graph. The multi-modal data set in step S3 includes safety indicators, comfort indicators, reliability indicators and passability indicators.

9. The method of claim 1, wherein: The correlation of the multi-modal data in the multi-modal data set in step S4 through the knowledge graph is specifically: establishing the relationship between the multi-modal data through the knowledge graph, and extracting entities, classes and relationships through the knowledge graph.

10. The method of claim 9, wherein the method is characterized by: The entities include driver entities, vehicle entities, environment entities and behavior entities.

Images