Driving record data extraction method and device, electronic equipment and storage medium

Abstract

The invention discloses a driving record data extraction method and device, electronic equipment and a storage medium, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring driving environment data of a current vehicle in a vehicle driving process; wherein the driving environment data comprises a current driving track; performing feature extraction on the driving environment data to obtain an environment global feature vector; inputting the environment global feature vector into a pre-trained trajectory planning model, and outputting a reference driving trajectory; detecting a track deviation degree between the current driving track and the reference driving track; and when the trajectory deviation degree is greater than a preset trajectory deviation degree, acquiring driving record data in a current time period, and feeding back the driving record data to a green driver for learning by the green driver. According to the technical scheme of the embodiment of the invention, the driving level of a green driver can be quickly improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a driving record data extraction method, device, electronic device and storage medium. Background Art

[0002] Currently, novice drivers often cannot directly cope with actual scenarios after passing the driver's test and applying for a driver's license for the first time. Faced with increasingly complex driving scenarios, novice drivers need to go through a lot of actual road practice to meet the needs of daily driving.

[0003] In the prior art, the vehicle will only send an alarm to the novice driver when the novice driver has a collision or is about to collide. Therefore, when the novice driver first starts driving on the actual road, he may experience one or two collisions or near-collisions. In this process, a large number of safety issues and trial and error costs are bound to arise. Moreover, after experiencing one or two collisions or near-collisions, the novice driver may have certain psychological obstacles, which will lead to a vicious cycle of "the more you practice, the less you dare to drive", which needs to be solved urgently. Summary of the invention

[0004] The present invention provides a driving record data extraction method, device, electronic equipment and storage medium, which can quickly improve the driving level of novice drivers.

[0005] According to one aspect of the present invention, a method for extracting driving record data is provided, the method comprising:

[0006] During the driving process of the vehicle, the driving environment data of the current vehicle is obtained; wherein the driving environment data includes the current driving trajectory;

[0007] Extracting features from the driving environment data to obtain a global feature vector of the environment;

[0008] Inputting the global feature vector of the environment into a pre-trained trajectory planning model, and outputting a reference driving trajectory;

[0009] Detecting a trajectory deviation degree between the current driving trajectory and the reference driving trajectory;

[0010] When the track deviation degree is greater than a preset track deviation degree, the driving record data of the current time period is obtained and fed back to the novice driver for the novice driver to learn.

[0011] According to another aspect of the present invention, a driving record data extraction device is provided, the device comprising:

[0012] A driving environment data acquisition module is used to acquire the current driving environment data of the vehicle during the driving process of the vehicle; wherein the driving environment data includes the current driving trajectory;

[0013] A global feature vector extraction module, used to extract features from the driving environment data to obtain a global feature vector of the environment;

[0014] A trajectory planning module, used for inputting the global feature vector of the environment into a pre-trained trajectory planning model and outputting a reference driving trajectory;

[0015] A trajectory deviation degree detection module, used to detect the trajectory deviation degree between the current driving trajectory and the reference driving trajectory;

[0016] The driving record data feedback module is used to obtain the driving record data of the current time period when the trajectory deviation degree is greater than the preset trajectory deviation degree, and feed back the driving record data to the novice driver for the novice driver to learn.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the driving record data extraction method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the driving record data extraction method described in any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the driving record data extraction method according to any embodiment of the present invention is implemented.

[0023] The technical solution of the embodiment of the present invention is to obtain driving environment data of the current vehicle during vehicle driving, extract features of the driving environment data, obtain a global feature vector of the environment, input the global feature vector of the environment into a pre-trained trajectory planning model, output a reference driving trajectory, detect the trajectory deviation degree between the current driving trajectory and the reference driving trajectory, and when the trajectory deviation degree is greater than the preset trajectory deviation degree, obtain the driving record data of the current time period and feed it back to the novice driver for the novice driver to learn, so that the novice driver can learn the reference driving trajectory. When the trajectory deviation degree between the current driving trajectory of the novice driver and the reference driving trajectory is too large, obtain the driving record data of the current time period that needs to be learned. By the novice driver learning and replaying the driving record data, the driving level of the novice driver can be quickly improved; thereby, the trial and error cost of the novice driver in the initial driving can be reduced, and the safety of the novice driver in the initial driving stage can be improved.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a driving record data extraction method provided according to Embodiment 1 of the present invention;

[0027] Figure 2 is a flow chart of a driving record data extraction method provided according to Embodiment 2 of the present invention;

[0028] Figure 3 is a schematic diagram of a process for extracting local vectors of environmental elements according to Embodiment 2 of the present invention;

[0029] Figure 4 is a schematic diagram of the structure of a driving record data extraction device provided according to Embodiment 3 of the present invention;

[0030] Figure 5 It is a structural schematic diagram of an electronic device for implementing the driving record data extraction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 A flowchart of a driving record data extraction method provided in Embodiment 1 of the present invention. The embodiment of the present invention can be applied to extracting driving record data that a novice driver needs to focus on learning during actual driving. The method can be executed by a driving record data extraction device, which can be implemented in the form of hardware and / or software. The driving record data extraction device can be configured in an electronic device that carries the driving record data extraction function, such as a vehicle-mounted terminal.

[0035] See also Figure 1 The driving record data extraction method shown includes:

[0036] S110. Acquire driving environment data of the current vehicle during driving of the vehicle.

[0037] The current vehicle may be the vehicle currently being driven by the novice driver. The driving environment data may be used to reflect the actual driving environment of the current vehicle. The driving environment data includes the current driving trajectory. Exemplarily, the driving environment data may include but is not limited to the location of traffic signs, obstacle locations, navigation routes, building locations, and the current driving trajectory. The current driving trajectory may be the actual driving trajectory of the current vehicle. The current driving trajectory may reflect the driving conditions of the novice driver.

[0038] In an optional embodiment of the present invention, the driving environment data includes traffic marking line positions, obstacle positions, navigation routes and current driving tracks.

[0039] The location of traffic signs is set according to the specific traffic environment, road conditions and traffic management needs. Traffic signs can serve as instructions, warnings and prohibitions on the road to ensure safe and smooth road traffic. Drivers should strictly follow the guidance of these signs during driving to ensure the traffic safety of themselves and others. For example, traffic signs may include lane dividing lines, intersection guide lines, pedestrian crossing lines, school area markings, parking and no-parking markings, speed reduction markings and special lane lines. For example, lane dividing lines may include white dashed lines, white solid lines, yellow solid lines and yellow dashed lines. Among them, white dashed lines are usually used to separate lanes traveling in the same direction, allowing vehicles to change lanes in these lanes. White solid lines are usually set in front of intersections or roadside parking areas, indicating that crossing the line or overtaking is prohibited. Yellow solid lines are used to distinguish lanes in different directions, indicating that driving on the line, overtaking or U-turning is prohibited. Yellow dashed lines are usually set between opposite lanes, allowing crossing the line and overtaking under safe conditions. For example, intersection guide lines may include stop lines and guide lines. Among them, the stop line is horizontally set in front of the zebra crossing at the intersection, and the vehicle must stop behind the line when the red light is on. The guide line is set at the intersection or ramp to guide the vehicle to drive along the specified route, and parking is not allowed in this area. The pedestrian crossing line can be set at the place where pedestrians cross the street to remind vehicles to slow down and give way. The school area marking line can be set on the surrounding roads of the school to remind vehicles to pay special attention to slowing down and avoiding. The parking and no-parking markings include parking space markings and yellow grid no-parking lines. Among them, the parking space markings are used to specify the specific location and direction of the vehicle parking. The yellow grid no-parking line is used to indicate that parking for any reason is strictly prohibited in this area to prevent traffic congestion. Exemplarily, the deceleration markings may include deceleration reminder lines and oscillating markings. Among them, the deceleration reminder lines can be set in places such as schools and intersections where deceleration is required to remind the driver to pay attention to deceleration. The oscillating marking line can adopt a concave-convex design to remind the driver to slow down by generating noise. Exemplarily, the dedicated lane line may include bus lanes and non-motorized vehicle lanes. Among them, the bus lane is a lane specially set up for buses, and other vehicles are not allowed to enter. Non-motorized vehicle lanes are specially designed for non-motorized vehicles to ensure their safe driving.

[0040] Obstacles encountered by vehicles during driving can include static obstacles and dynamic obstacles. Static obstacles refer to objects that are fixed in position and do not move actively, such as trees, buildings, traffic signs or roadblocks on the roadside. Dynamic obstacles refer to objects that are constantly changing in position, such as other vehicles, pedestrians or animals.

[0041] A navigation route is a recommended path from a starting point to a destination provided by the navigation system for the current vehicle in motion. The generation of a navigation route relies on advanced map data and algorithms, which can take into account factors such as traffic conditions, road closures, and speed limits in real time to provide the best driving plan for novice drivers. The navigation route can include signal light status and remaining signal light time, etc.

[0042] This solution concretizes the driving environment data into traffic sign line locations, obstacle locations, navigation routes and current driving trajectories, comprehensively considers the typical data in the driving environment during vehicle driving, takes into account the impact of the vehicle's own driving trajectory and driving environment, and further improves the efficiency and accuracy of driving record data extraction.

[0043] S120: Extract features from the driving environment data to obtain a global feature vector of the environment.

[0044] The global environmental feature vector can reflect the comprehensive situation of the current vehicle's driving environment in a vectorized manner. The global environmental feature vector can be a feature extraction result of the driving environment data.

[0045] Specifically, a pre-trained first feature extraction model can be used to extract features from the driving environment data to obtain a global feature vector of the environment. The training samples of the first feature extraction model may include driving environment data samples and global feature vector samples of the environment. Exemplarily, the feature extraction model may be Point Core (unsupervised point cloud anomaly detection model) or UMF (self-attention and cross-attention mechanism enhancement model), etc.

[0046] S130: Input the global feature vector of the environment into a pre-trained trajectory planning model, and output a reference driving trajectory.

[0047] The reference driving trajectory can be used to characterize the recommended driving trajectory of the current vehicle. The reference driving trajectory can be used to correct the current driving trajectory. The trajectory planning model can be used to plan the trajectory of the current vehicle according to the global feature vector of the environment. The input data of the trajectory planning model can be the global feature variable of the environment of the current vehicle. The output data of the trajectory planning model can be the reference driving trajectory. Exemplarily, the trajectory planning model can be an LSTM (Long Short Term Memory Network).

[0048] In an optional embodiment of the present invention, the training process of the trajectory planning model includes: obtaining a sample of a global feature vector of the environment and a real reference driving trajectory; inputting the sample of the global feature vector of the environment into the trajectory planning model and outputting a predicted reference driving trajectory; and adjusting the parameters of the trajectory planning model according to the difference between the predicted reference driving trajectory and the real reference driving trajectory.

[0049] The environmental global feature vector sample can be generated based on historical driving environment data or virtual driving environment data. The real reference driving trajectory is generated based on old drivers within a preset driving age range. The preset driving age range can be a pre-set driving age range for experienced drivers. Exemplarily, the preset driving age range can be more than 10 years. The predicted reference driving trajectory can be the prediction result of the trajectory planning model. In comparison, the real reference driving trajectory can be the real trajectory of the driving environment in which the current vehicle is located, and can also be understood as a recommended driving trajectory for the driving environment in which the current vehicle is located.

[0050] Specifically, a sample of the global feature vector of the environment and a real reference driving trajectory can be obtained. The sample of the global feature vector of the environment can be input into the trajectory planning model to output a predicted reference driving trajectory. The loss function can be calculated based on the difference between the predicted reference driving trajectory and the real reference driving trajectory. According to the loss function, the parameters in the trajectory planning model can be adjusted according to the gradient descent method until the trajectory planning model converges. Referring to the above example, the trajectory planning model can be an LSTM network. In the LSTM network, the input gate, the output gate, and the forget gate network all use a fully connected network, and the number of neurons is 512, 1024, 2048, and 120, respectively. The 120-dimensional vector finally outputted, in groups of 6 in the order of dimensions, can represent the reference driving trajectory points in the next 10 seconds. The loss function value of the reference driving trajectory output by the LSTM network and the real current driving trajectory of the novice driver in the scenario is calculated. According to the loss function, the gradient descent method is used to optimize the neural network.

[0051] Exemplarily, the following formula can be used to represent the loss function of the trajectory planning model:

[0052]

[0053] In the formula, loss is the loss function; is the horizontal coordinate of the point in the real reference driving trajectory; i is the horizontal coordinate of the point in the predicted reference driving trajectory; is the ordinate of the point in the real reference driving trajectory; i is the ordinate of the point in the real reference driving trajectory; is the lateral velocity of the point in the predicted reference driving trajectory; v xi is the lateral velocity of the point in the real reference driving trajectory; v is the longitudinal velocity of the point in the predicted reference driving trajectory; yi is the longitudinal velocity of the point in the real reference driving trajectory; is the lateral acceleration of the predicted point in the reference driving trajectory; a xi is the lateral acceleration of the point in the real reference driving trajectory; is the longitudinal acceleration of the point in the predicted reference driving trajectory; a yi is the longitudinal acceleration of the point in the real reference driving trajectory.

[0054] This scheme uses the real reference driving trajectory of experienced drivers as training samples, learns the driving trajectory of experienced drivers based on the trajectory planning model, and compares it with the current driving trajectory of novice drivers, thereby discovering and recording the shortcomings of novice drivers in the actual driving process, thereby realizing the teaching of novice drivers.

[0055] S140: Detecting a trajectory deviation degree between the current driving trajectory and the reference driving trajectory.

[0056] The trajectory deviation degree can be used to characterize the closeness between the current driving trajectory and the reference driving trajectory. It can be understood that the greater the trajectory deviation degree, the less close the current driving trajectory is to the reference driving trajectory; the smaller the trajectory deviation degree, the closer the current driving trajectory is to the reference driving trajectory.

[0057] Specifically, the position, speed and acceleration of each point in the current driving trajectory can be compared with the position, speed and acceleration of each point in the reference driving trajectory in sequence. The comparison results corresponding to each position, speed and acceleration can be weighted summed to obtain the trajectory deviation degree between the current driving trajectory and the reference driving trajectory.

[0058] For example, the following formula may be used to calculate the trajectory deviation degree between the current driving trajectory and the reference driving trajectory:

[0059]

[0060] Where Δ is the degree of trajectory deviation; x ti is the horizontal coordinate of the point in the current driving trajectory; si is the horizontal coordinate of the point in the reference driving trajectory; ti is the ordinate of the point in the current driving trajectory; si is the ordinate of the point in the reference driving trajectory; v txi is the lateral velocity of the point in the current driving trajectory; v tyi v is the longitudinal velocity of the point in the predicted reference driving trajectory; sxi is the lateral velocity of the point in the reference driving trajectory; v syi is the longitudinal velocity of the point in the reference driving trajectory; a txi is the lateral acceleration of the point in the current driving trajectory; a tyi is the longitudinal acceleration of the point in the reference driving trajectory; a sxi is the lateral acceleration of the point in the current driving trajectory; a syiis the longitudinal acceleration of the point in the reference driving trajectory.

[0061] S150: When the trajectory deviation degree is greater than a preset trajectory deviation degree, the driving record data of the current time period is obtained and fed back to the novice driver for the novice driver to learn from.

[0062] If the trajectory deviation degree is greater than the preset trajectory deviation degree, it can be understood that the current driving trajectory deviates too much from the reference driving trajectory, and it is necessary to review the driving conditions of the current driving trajectory near the position. The current time period may be a preset time period at the current moment. Exemplarily, the current time period may be within 30 seconds before and after the current moment. The driving record data may be used to record the driving conditions of the current vehicle. Exemplarily, the driving record data may include sensor data, driving environment data and a reference driving trajectory of the current vehicle. The driving record data may be used for review learning of novice drivers. Thus, by obtaining the driving record data of the preset time period and feeding it back to the novice driver for the novice driver to learn, the driving level of the novice driver in the early stage of driving can be quickly improved.

[0063] Specifically, when the trajectory deviation degree is greater than a preset trajectory deviation degree, the driving record data of the current time period can be obtained and fed back to the novice driver for the novice driver to learn from.

[0064] Optionally, in the actual driving scene of the vehicle, the active safety system (with emergency braking and emergency steering functions) and the 360° panoramic image are both turned on. When the active safety system of the current vehicle is triggered, the driving record data of the current time period can also be obtained and fed back to the novice driver for the novice driver to learn. By combining the active safety system of the car, the safety of the novice driver is maximized, and the driving behavior of the novice driver before and after the collision is triggered can be recorded for the novice driver to learn and improve.

[0065] The technical solution of the embodiment of the present invention is to obtain driving environment data of the current vehicle during vehicle driving, extract features of the driving environment data, obtain a global feature vector of the environment, input the global feature vector of the environment into a pre-trained trajectory planning model, output a reference driving trajectory, detect the trajectory deviation degree between the current driving trajectory and the reference driving trajectory, and when the trajectory deviation degree is greater than the preset trajectory deviation degree, obtain the driving record data of the current time period and feed it back to the novice driver for the novice driver to learn, so that the novice driver can learn the reference driving trajectory. When the trajectory deviation degree between the current driving trajectory of the novice driver and the reference driving trajectory is too large, obtain the driving record data of the current time period that needs to be learned. By the novice driver learning and replaying the driving record data, the driving level of the novice driver can be quickly improved; thereby, the trial and error cost of the novice driver in the initial driving can be reduced, and the safety of the novice driver in the initial driving stage can be improved.

[0066] Embodiment 2

[0067] Figure 2 A flow chart of a driving record data extraction method provided for the second embodiment of the present invention. Based on the above embodiments, the embodiment of the present invention concretizes "extracting features from driving environment data to obtain a global environmental feature vector" as "vectorizing and encoding each environmental element data contained in the driving environment data to obtain an original environmental element vector corresponding to each environmental element; extracting features from the original environmental element vector corresponding to each environmental element to obtain a local feature vector of each environmental element; synthesizing the local feature vectors of each environmental element by taking the maximum value of each dimension to obtain a global environmental feature vector", thereby improving the accuracy and typicality of the global environmental feature vector, and further improving the accuracy of the driving record data of the current time period determined based on the predicted reference driving trajectory and the current driving trajectory. It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the descriptions of other embodiments.

[0068] See also Figure 2 The driving record data extraction method shown includes:

[0069] S210: Acquire driving environment data of the current vehicle during driving of the vehicle.

[0070] Among them, the driving environment data includes the current driving trajectory.

[0071] S220 , vectorize and encode each environmental element data contained in the driving environment data to obtain an original environmental element vector corresponding to each environmental element.

[0072] The driving environment data may include environmental element data. The environmental element data may be used to characterize the elements in the driving environment in which the current vehicle is located. Exemplarily, the environmental elements may include, but are not limited to, traffic signs, obstacles, navigation routes, and buildings. The environmental element raw vectors may be used to characterize the environmental element data and facilitate subsequent processing of the environmental element data.

[0073] Specifically, a word vector calculation method can be used to vectorize and encode each environmental element data contained in the driving environment data to obtain each word vector corresponding to each environmental element data. The word vectors corresponding to each environmental element data can be combined according to different dimensions to obtain the original environmental element vector corresponding to each environmental element.

[0074] In an optional embodiment of the present invention, each environmental element data contained in the driving environment data is vectorized and encoded to obtain the original vector of the environmental element corresponding to each environmental element, including: according to a preset encoding dimension, each environmental element data contained in the driving environment data is vectorized and encoded to obtain the original vector of the environmental element corresponding to each environmental element.

[0075] The preset coding dimensions may be pre-set dimensions of vectorized coding. The preset coding dimensions may be set and adjusted by technicians based on experience.

[0076] Specifically, according to the preset coding dimension, each environmental element data can be mapped to each dimension included in the preset coding dimension, and each environmental element data included in the driving environment data can be vectorized and encoded to obtain the original environmental element vector corresponding to each environmental element.

[0077] Exemplarily, the preset encoding dimension may be 9. The following table may be used to represent the original environmental element vector corresponding to each environmental element.

[0078]

[0079] Referring to the above table, the environmental element data included in the driving environment data may include the location of traffic sign lines, obstacle locations, navigation routes, and current driving tracks. Specifically, traffic sign lines may include straight traffic sign lines and curved traffic sign lines. Among them, straight traffic sign lines may be traffic sign lines whose shape is fixed as a straight line. The location of straight traffic sign lines may be determined by the starting and ending points (x s ,y s ,x e ,y e ) in the form of s is the horizontal coordinate of the starting point; y s is the vertical coordinate of the starting point; x e is the horizontal coordinate of the end point; eis the ordinate of the end point. The curved traffic sign line can be a traffic sign line that may be curved. For example, a lane line. Optionally, after the traditional cubic spline fitting, sampling can be performed at intervals of 0.25m in the horizontal distance, discretized into a set of coordinate points, and finally, according to the indicator line type, the position of the curved traffic sign line is supplemented to the fifth dimension, that is, (x s ,y s ,x i ,y i ,x e ,y e , line_type) to identify the position of the curve traffic sign line; where x i is the horizontal coordinate of the middle point; i is the ordinate of the middle point; the possible values ​​of line_type include 1-dashed line, 2-single solid line, 3-double solid line, 4-left dashed and right solid line, and 5-right dashed and left solid line. Obstacles can be recorded in the form of center point and length and width. For example (x0, y0, l, w); where x0 is the horizontal coordinate of the center point of the obstacle; y0 is the horizontal coordinate of the center point of the obstacle; l is the length of the obstacle; w is the width of the obstacle. The navigation route can be recorded in the form of discrete coordinate points combined with the status of the traffic light and the remaining time of the traffic light, for example (x i ,y i ,state,time); where x i is the horizontal coordinate of the discrete coordinate point; i is the ordinate of the discrete coordinate point; state is the state of the traffic light, such as 1 red light, 2 yellow light and 3 green light; time is the remaining time of the traffic light. The current driving trajectory can be recorded in the form of discrete coordinate points. For example, each discrete point can include (x i ,y i ,v ix ,v iy ,a ix ,a iy );where v ix is the lateral velocity of the corresponding discrete point; v iy is the longitudinal velocity of the corresponding discrete point; a ix is the lateral acceleration of the corresponding discrete point; a iy is the longitudinal acceleration of the corresponding discrete point. For the above information, the discrete point recording time interval can be 0.5s respectively. In each environmental element data, except for the navigation route ID which is constant at 0, the IDs of other environmental element data are generated in random order. It is only necessary to ensure that the IDs of the environmental element data corresponding to the same environmental element in the environmental element data are the same.

[0080] This solution vectorizes the environmental element data contained in the driving environment data according to a preset coding dimension to obtain the original vector of the environmental element corresponding to each environmental element, thereby ensuring the format consistency of the original vector of the environmental element corresponding to the environmental element, and facilitating subsequent feature extraction and determination of driving record data.

[0081] S230 , extracting features from the original environmental element vectors corresponding to each environmental element to obtain local feature vectors of each environmental element.

[0082] Specifically, a pre-trained second feature extraction model can be used to extract features from the original environmental element vectors corresponding to each environmental element to obtain local feature vectors of each environmental element. The training samples of the second feature extraction model may include samples of the original environmental element vectors corresponding to each environmental element and samples of the local feature vectors of each environmental element. Exemplarily, the second feature extraction model may be Point Core (unsupervised point cloud anomaly detection model) or UMF (self-attention and cross-attention mechanism enhancement model), etc.

[0083] In an optional embodiment of the present invention, a single environmental element corresponds to at least one environmental element original vector; accordingly, feature extraction is performed on the environmental element original vectors corresponding to each environmental element to obtain a local feature vector of each environmental element, including: grouping the environmental element original vectors corresponding to each environmental element to obtain each environmental element group; for a single environmental element group, inputting the original vectors of each environmental element into the first fully connected layer, performing feature extraction on the original vectors of each environmental element, and obtaining a feature vector of each environmental element; for a single environmental element group, synthesizing the feature vectors of each environmental element in a manner of taking the maximum value of each dimension to obtain a feature vector of the environmental element group; for a single environmental element group, splicing the feature vector of the environmental element group with the original vectors of each environmental element to obtain a spliced ​​vector of each environmental element; for a single environmental element group, inputting the spliced ​​vectors of each environmental element into the second fully connected layer, the third fully connected layer and the fourth fully connected layer in sequence, performing feature extraction on the spliced ​​vectors of each environmental element, and obtaining a local vector of the environmental element corresponding to the environmental element group.

[0084] The environmental element group may be the grouping result of the environmental element data corresponding to the same environmental element. It can be understood that the driving environment data may include environmental element data at different times. Each environmental element data has multiple environmental element data for the same environmental element. Based on multiple environmental element data, the state of the environmental element can be more comprehensively reflected. The first fully connected layer can be used to extract features from the original vector of the environmental element. The environmental element feature vector may be the feature extraction result of the original vector of the environmental element by the first fully connected layer. The environmental element feature vector may be used to characterize the features of a single environmental element at multiple acquisition times. The environmental element group feature vector may be used to comprehensively reflect the features of the environmental element. The environmental element splicing vector may be used to comprehensively reflect the feature extraction state and original state of a single environmental element at multiple acquisition times. The second fully connected layer, the third fully connected layer, and the fourth fully connected layer may be used to extract local features from the spliced ​​splicing vectors of each environmental element to obtain the local vectors of the environmental elements corresponding to each environmental element.

[0085] Specifically, the original vectors of each environmental element corresponding to each environmental element can be grouped to obtain each environmental element group. For a single environmental element group, the original vectors of each environmental element can be input into the first fully connected layer, and the features of the original vectors of each environmental element can be extracted to obtain the feature vectors of each environmental element. For a single environmental element group, the feature vectors of each environmental element can be integrated in a manner of taking the maximum value of each dimension to obtain the feature vector of the environmental element group. For a single environmental element group, the feature vector of the environmental element group can be spliced ​​with the original vectors of each environmental element to obtain the spliced ​​vectors of each environmental element. For a single environmental element group, the spliced ​​vectors of each environmental element can be input into the second fully connected layer, the third fully connected layer and the fourth fully connected layer in sequence, and the features of the spliced ​​vectors of each environmental element can be extracted to obtain the local vectors of the environmental elements corresponding to the environmental element group.

[0086] For example, Figure 3 The following is a schematic diagram of the process of extracting local vectors of environmental elements. Figure 3As shown in FIG. 1 , the process of extracting local vectors of environmental elements is as follows: the original vectors of environmental elements with the same ID (i.e., the same environmental element) can be divided into the same group to obtain groups of environmental elements. For a group of vectors with the same ID, that is, for a single group of environmental elements, each of its original vectors of environmental elements passes through the fully connected layer 1 (i.e., the first fully connected layer) to extract a 9-dimensional environmental element feature vector. After extracting the 9-dimensional environmental element feature vector for each original vector of the environmental element in this group, the maximum value is taken for each dimension of the feature vector of each environmental element as the feature of the group of vectors to obtain the feature vector of the environmental element group. The feature vector of the environmental element group is concatenated with the original vectors of each environmental element into an 18-dimensional concatenated vector of environmental elements. Among them, the fully connected layer 1 has a total of 4 layers, and the number of neurons in each layer is 9, 72, 144, and 9, respectively. According to the above steps, until the local feature extraction of each original vector of the environmental element is completed, the concatenated concatenated vector of the environmental element is obtained. Then let these environmental element concatenation vectors pass through the fully connected layer Q (i.e. the second fully connected layer), the fully connected layer K (i.e. the third fully connected layer), and the fully connected layer V (i.e. the fourth fully connected layer) one by one, and finally obtain a local vector of environmental elements corresponding to an environmental element group. As the local feature corresponding to the environmental element group, the fully connected layer Q, the fully connected layer K and the fully connected layer V are all 4-layer fully connected networks, and the number of neurons in each layer is 18, 128, 1024, and 512 respectively. Therefore, the local vector of environmental elements corresponding to the environmental element group is a 512-dimensional vector.

[0087] For example, the following formula can be used to calculate the local vector of the environmental element corresponding to the environmental element group:

[0088] y = softmax(Q K)V;

[0089] Among them, y is the local vector of environmental elements corresponding to the environmental element group; the meaning of the softmax function is x is the concatenated vector of environmental elements; Q is the fully connected layer Q (i.e. the second fully connected layer); K is the fully connected layer K (i.e. the third fully connected layer); V is the fully connected layer V (i.e. the fourth fully connected layer).

[0090] This solution concretizes a single environmental element into a raw vector corresponding to at least one environmental element, groups the raw vectors of each environmental element corresponding to each environmental element, obtains each environmental element group, inputs the raw vectors of each environmental element into the first fully connected layer for each environmental element group, extracts features from the raw vectors of each environmental element, obtains feature vectors of each environmental element, synthesizes the feature vectors of each environmental element group by taking the maximum value of each dimension for each environmental element group, concatenates the feature vectors of the environmental element group with the raw vectors of each environmental element, obtains concatenated vectors of each environmental element, inputs the concatenated vectors of each environmental element into the second fully connected layer, the third fully connected layer and the fourth fully connected layer for the single environmental element group, extracts features from the concatenated vectors of each environmental element, and obtains the local vectors of the environmental elements corresponding to the environmental element group, thereby ensuring that the local vectors of the environmental elements corresponding to the environmental element group can take into account the feature extraction state and the original state of the single environmental element at multiple acquisition times, and can improve the accuracy of the determined local vectors of the environmental elements.

[0091] S240 , synthesizing the local feature vectors of each environmental element by taking the maximum value in each dimension to obtain a global feature vector of the environment.

[0092] Specifically, the global environmental feature vector can be obtained by taking the maximum value in each dimension for the local feature vector of each environmental element.

[0093] Referring to the above example, after extracting the local feature vectors of the environmental elements for all environmental element groups with the same ID, the maximum value of each dimension can be taken to obtain the environmental global feature vector of the same 512 dimensions.

[0094] S250: Input the global feature vector of the environment into a pre-trained trajectory planning model, and output a reference driving trajectory.

[0095] S260: Detect the degree of trajectory deviation between the current driving trajectory and the reference driving trajectory.

[0096] S270: When the trajectory deviation degree is greater than the preset trajectory deviation degree, the driving record data of the current time period is obtained and fed back to the novice driver for the novice driver to learn from.

[0097] The technical solution of the embodiment of the present invention performs vectorized encoding on the environmental element data contained in the driving environment data to obtain the original environmental element variables corresponding to each environmental element, so as to facilitate subsequent processing of the environmental element data; performs feature extraction on the original environmental element vectors corresponding to each environmental element to obtain the local feature vectors of each environmental element, and synthesizes the local feature vectors of each environmental element by taking the maximum value of each dimension to obtain the global environmental feature vector, thereby improving the accuracy and typicality of the global environmental feature vector, and further improving the accuracy of the driving record data of the current time period determined based on the predicted reference driving trajectory and the current driving trajectory.

[0098] Embodiment 3

[0099] Figure 4 A schematic diagram of the structure of a driving record data extraction device provided in Embodiment 3 of the present invention. The embodiment of the present invention can be applied to extract driving record data that a novice driver needs to focus on learning during actual driving. The device can execute a driving record data extraction method. The device can be implemented in the form of hardware and / or software. The device can be configured in an electronic device that carries the driving record data extraction function, such as a vehicle-mounted terminal.

[0100] See also Figure 4 The driving record data extraction device shown includes: a driving environment data acquisition module 410, which is used to obtain the driving environment data of the current vehicle during the driving process of the vehicle; wherein the driving environment data includes the current driving trajectory; a global feature vector extraction module 420, which is used to extract features from the driving environment data to obtain an environmental global feature vector; a trajectory planning module 430, which is used to input the environmental global feature vector into a pre-trained trajectory planning model and output a reference driving trajectory; a trajectory deviation degree detection module 440, which is used to detect the trajectory deviation degree between the current driving trajectory and the reference driving trajectory; a driving record data feedback module 450, which is used to obtain the driving record data of the current time period when the trajectory deviation degree is greater than a preset trajectory deviation degree, and feed it back to the novice driver for the novice driver to learn.

[0101] The technical solution of the embodiment of the present invention is to obtain driving environment data of the current vehicle during vehicle driving, extract features of the driving environment data, obtain a global feature vector of the environment, input the global feature vector of the environment into a pre-trained trajectory planning model, output a reference driving trajectory, detect the trajectory deviation degree between the current driving trajectory and the reference driving trajectory, and when the trajectory deviation degree is greater than the preset trajectory deviation degree, obtain the driving record data of the current time period and feed it back to the novice driver for the novice driver to learn, so that the novice driver can learn the reference driving trajectory. When the trajectory deviation degree between the current driving trajectory of the novice driver and the reference driving trajectory is too large, obtain the driving record data of the current time period that needs to be learned. By the novice driver learning and replaying the driving record data, the driving level of the novice driver can be quickly improved; thereby, the trial and error cost of the novice driver in the initial driving can be reduced, and the safety of the novice driver in the initial driving stage can be improved.

[0102] In an optional embodiment of the present invention, the global feature vector extraction module 420 includes: a vector encoding unit, which is used to perform vector encoding on each environmental element data contained in the driving environment data to obtain the original environmental element vector corresponding to each environmental element; a local feature extraction unit, which is used to perform feature extraction on the original environmental element vector corresponding to each environmental element to obtain the local feature vector of each environmental element; and a global feature extraction unit, which is used to integrate the local feature vectors of each environmental element by taking the maximum value of each dimension to obtain the global feature vector of the environment.

[0103] In an optional embodiment of the present invention, a single environmental element corresponds to at least one environmental element original vector; accordingly, the local feature extraction unit includes: an original vector grouping subunit, which is used to group the environmental element original vectors corresponding to each environmental element to obtain each environmental element group; a first feature extraction subunit, which is used to input the original vectors of each environmental element into the first fully connected layer for a single environmental element group, perform feature extraction on the original vectors of each environmental element, and obtain feature vectors of each environmental element; a feature vector synthesis subunit, which is used to synthesize the feature vectors of each environmental element in a manner of taking the maximum value of each dimension for a single environmental element group, and obtain a feature vector of the environmental element group; a feature vector splicing subunit, which is used to splice the feature vector of the environmental element group with the original vectors of each environmental element for a single environmental element group, and obtain a splicing vector of each environmental element; a local feature extraction subunit, which is used to input the splicing vectors of each environmental element into the second fully connected layer, the third fully connected layer and the fourth fully connected layer for a single environmental element group, perform feature extraction on the splicing vectors of each environmental element, and obtain a local vector of the environmental element corresponding to the environmental element group.

[0104] In an optional embodiment of the present invention, the vectorized coding unit includes: a vectorized coding sub-unit, which is used to vectorize the environmental element data contained in the driving environment data according to a preset coding dimension to obtain the original environmental element vector corresponding to each environmental element.

[0105] In an optional embodiment of the present invention, the device also includes: a training sample acquisition module, used to obtain environmental global feature vector samples and real reference driving trajectories; wherein the real reference driving trajectory is generated based on old drivers within a preset driving experience range; a predicted reference driving trajectory output module, used to input the environmental global feature vector samples into a trajectory planning model and output a predicted reference driving trajectory; a trajectory planning model parameter adjustment module, used to adjust the parameters of the trajectory planning model according to the difference between the predicted reference driving trajectory and the real reference driving trajectory.

[0106] In an optional embodiment of the present invention, the driving environment data includes traffic marking line positions, obstacle positions, navigation routes and current driving tracks.

[0107] The driving record data extraction device provided in the embodiment of the present invention can execute the driving record data extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0108] In the technical solution of the embodiment of the present invention, the acquisition, storage and application of the current vehicle's driving environment data, environmental global feature vector samples and real reference driving trajectory, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0109] Embodiment 4

[0110] Figure 5 A schematic diagram of the structure of an electronic device 500 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0111] like Figure 5As shown, the electronic device 500 includes at least one processor 501, and a memory connected to the at least one processor 501 in communication, such as a read-only memory (ROM) 502, a random access memory (RAM) 503, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 501 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 502 or the computer program loaded from the storage unit 508 to the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0112] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0113] The processor 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 501 performs the various methods and processes described above, such as the driving record data extraction method.

[0114] In some embodiments, the driving record data extraction method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the processor 501, one or more steps of the driving record data extraction method described above may be performed. Alternatively, in other embodiments, the processor 501 may be configured to perform the driving record data extraction method in any other appropriate manner (e.g., by means of firmware).

[0115] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0117] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0119] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0120] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS (Virtual Private Server) services.

[0121] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0122] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A driving record data extraction method, characterized in that: The method comprises: During the driving process of the vehicle, the driving environment data of the current vehicle is obtained; wherein the driving environment data includes the current driving trajectory; Extracting features from the driving environment data to obtain a global feature vector of the environment; Inputting the global feature vector of the environment into a pre-trained trajectory planning model, and outputting a reference driving trajectory; Detecting a trajectory deviation degree between the current driving trajectory and the reference driving trajectory; When the track deviation degree is greater than a preset track deviation degree, the driving record data of the current time period is obtained and fed back to the novice driver for the novice driver to learn.

2. The method according to claim 1, characterized in that The step of extracting features from the driving environment data to obtain a global feature vector of the environment includes: Performing vector coding on each environmental element data contained in the driving environment data to obtain an original environmental element vector corresponding to each environmental element; Extracting features from the original environmental element vectors corresponding to the environmental elements to obtain local feature vectors of the environmental elements; The local feature vectors of the environmental elements are integrated by taking the maximum value in each dimension to obtain the global feature vector of the environment.

3. The method according to claim 2, characterized in that A single environmental element corresponds to at least one environmental element original vector; Correspondingly, the feature extraction of the original environmental element vector corresponding to each environmental element to obtain the local feature vector of each environmental element includes: Grouping the original vectors of the environmental elements corresponding to the environmental elements to obtain environmental element groups; For a single group of environmental elements, inputting the original vector of each environmental element into the first fully connected layer, performing feature extraction on the original vector of each environmental element, and obtaining a feature vector of each environmental element; For a single environmental factor group, the characteristic vectors of the environmental factors are integrated by taking the maximum value in each dimension to obtain a characteristic vector of the environmental factor group; For a single environmental element group, concatenating the environmental element group feature vector with the original vectors of each environmental element to obtain a concatenated vector of each environmental element; For a single group of environmental elements, each concatenated vector of the environmental elements is sequentially input into the second fully connected layer, the third fully connected layer and the fourth fully connected layer, and features are extracted from each concatenated vector of the environmental elements to obtain a local vector of the environmental elements corresponding to the group of environmental elements.

4. The method according to claim 2, characterized in that: The vectorized encoding of each environmental element data contained in the driving environment data to obtain the original environmental element vector corresponding to each environmental element includes: According to a preset coding dimension, each environmental element data contained in the driving environment data is vectorized and encoded to obtain an environmental element original vector corresponding to each environmental element.

5. The method according to claim 1, characterized in that: The training process of the trajectory planning model includes: Acquire a sample of the global feature vector of the environment and a real reference driving trajectory; wherein the real reference driving trajectory is generated based on old drivers within a preset driving age range; Inputting the global feature vector sample of the environment into a trajectory planning model, and outputting a predicted reference driving trajectory; According to the difference between the predicted reference driving trajectory and the actual reference driving trajectory, the parameters of the trajectory planning model are adjusted.

6. The method according to claim 1, characterized in that The driving environment data includes traffic marking line positions, obstacle positions, navigation routes and current driving tracks.

7. A driving record data extraction device, characterized in that: The device comprises: A driving environment data acquisition module is used to acquire the current driving environment data of the vehicle during the driving process of the vehicle; wherein the driving environment data includes the current driving trajectory; A global feature vector extraction module, used to extract features from the driving environment data to obtain a global feature vector of the environment; A trajectory planning module, used for inputting the global feature vector of the environment into a pre-trained trajectory planning model and outputting a reference driving trajectory; A trajectory deviation degree detection module, used to detect the trajectory deviation degree between the current driving trajectory and the reference driving trajectory; The driving record data feedback module is used to obtain the driving record data of the current time period when the trajectory deviation degree is greater than the preset trajectory deviation degree, and feed back the driving record data to the novice driver for the novice driver to learn.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the driving record data extraction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the driving record data extraction method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method for extracting driving record data according to any one of claims 1 to 6 is implemented.

Images