Classification Method, Device, Electronic Device and Storage Medium for Driving Scene Data

Abstract

The present application discloses a classification method, device, electronic device and storage medium for driving scenario data, belonging to the technical field of data analysis. The method includes: obtaining the driving scenario data of a first vehicle within a specified time period, where the driving scenario data at least includes multiple frames of point cloud data collected by a radar on the first vehicle; analyzing the relative position relationship between the first vehicle and a second vehicle around the first vehicle based on each frame of point cloud data to obtain a relative position feature vector; determining a scene representation vector at the time of collecting this frame of point cloud data based on the relative position feature vector; analyzing the association relationship between the scene representation vectors at the time of collecting each frame of point cloud data to obtain a scene feature vector; and performing driving scenario classification based on the scene feature vector to obtain the driving scenario classification result of the first vehicle within the specified time period. In this way, a solution for classifying driving scenarios with the help of a radar is provided.

Description

Technical Field

[0001] This application relates to the technical field of data analysis, and in particular, to a method, apparatus, electronic device, and storage medium for classifying driving scenario data. Background Art

[0002] Currently, generally, an autonomous driving simulation system is used to test the performance of an autonomous driving algorithm. To comprehensively evaluate the autonomous driving algorithm, the test scenarios of the autonomous driving simulation system should cover various driving scenarios in actual driving, such as following a vehicle, overtaking, and lane changing.

[0003] Therefore, in the related art, driving scenario data will be collected and provided for use by the autonomous driving simulation system so that the autonomous driving simulation system can simulate various real driving scenarios. However, since driving scenario data is usually massive, in order to ensure that the provided driving scenario data can cover various actual driving scenarios, there is a need to classify the driving scenario data before providing it to the autonomous driving simulation system. Summary of the Invention

[0004] Embodiments of this application provide a method, apparatus, electronic device, and storage medium for classifying driving scenario data to provide a solution for classifying driving scenario data.

[0005] In a first aspect, embodiments of this application provide a method for classifying driving scenario data, including:

[0006] Obtain driving scenario data of a first vehicle within a specified time period, where the driving scenario data at least includes multiple frames of point cloud data collected by a radar on the first vehicle;

[0007] Based on each frame of point cloud data, analyze the relative position relationship between the first vehicle and a second vehicle around the first vehicle to obtain a relative position feature vector;

[0008] Based on the relative position feature vector, determine a scene representation vector when the point cloud data is collected;

[0009] Analyze the association relationship between the scene representation vectors when each frame of point cloud data is collected to obtain a scene feature vector;

[0010] Based on the scene feature vector, perform driving scenario classification to obtain a driving scenario classification result of the first vehicle within the specified time period.

[0011] In some embodiments, based on each frame of point cloud data, analyzing the relative position relationship between the first vehicle and a second vehicle around the first vehicle to obtain a relative position feature vector includes:

[0012] Analyze the point cloud data to determine the target point cloud data belonging to the second vehicle in the point cloud data;

[0013] Generate a position reference map from an aerial view perspective between the first vehicle and the second vehicle based on the target point cloud data;

[0014] Extract features from the position reference map to obtain the relative position feature vector.

[0015] In some embodiments, extracting features from the position reference map to obtain the relative position feature vector includes:

[0016] Extract features from the position reference map through a feature extraction model to obtain the relative position feature vector;

[0017] Among them, the feature extraction model is established according to the following steps:

[0018] Input the obtained position reference map sample into the encoding network for feature extraction;

[0019] Input the extraction result into the decoding network for feature parsing to obtain a predicted image;

[0020] Based on the predicted image and the position reference map sample, adjust the network parameters of the encoding network and / or the decoding network until it is determined that the difference between the predicted image and the position reference map sample is less than a preset difference, and then determine the current encoding network as the feature extraction model.

[0021] In some embodiments, the driving scenario data further includes control characterization data of the first vehicle, and the control characterization data is used to characterize the control state of the first vehicle, and further includes:

[0022] Select target control characterization data with a collection time matching that of the point cloud data from the control characterization data;

[0023] Analyze the target control characterization data to obtain the control feature vector of the first vehicle;

[0024] Based on the relative position feature vector, determine the scene characterization vector at the time of collecting the point cloud data, including:

[0025] Perform a fusion process on the relative position feature vector and the control feature vector to obtain the scene characterization vector at the time of collecting the point cloud data.

[0026] In some embodiments, analyzing the target control characterization data to obtain the control feature vector of the first vehicle includes:

[0027] Normalize each type of data in the target control characterization data to obtain the normalized value of this type of data;

[0028] Combine the normalized values of various types of data to obtain the control feature vector of the first vehicle.

[0029] In some embodiments, analyzing the correlation relationship between the scene characterization vectors when collecting each frame of point cloud data to obtain the scene feature vector, including:

[0030] Input each scene characterization vector into the feature analysis model for analysis according to the acquisition order of the point cloud data to obtain the scene feature vector;

[0031] Among them, the feature analysis model is established according to the following steps:

[0032] Obtain the scene characterization vector sequence of the vehicle sample;

[0033] Input the scene characterization vector sequence of the vehicle sample into the sequence analysis network for correlation analysis;

[0034] Input the analysis result into the sequence parsing network for driving scene parsing to obtain the predicted vector sequence;

[0035] Based on the predicted vector sequence and the scene characterization vector sequence, adjust the network parameters of the sequence analysis network and / or the sequence parsing network until it is determined that the difference between the predicted vector sequence and the scene characterization vector sequence is less than the set difference, and then determine the current sequence analysis network as the feature analysis model.

[0036] In a second aspect, an embodiment of the present application provides a classification device for driving scene data, including:

[0037] An acquisition module, configured to acquire driving scene data of a first vehicle within a specified time period, where the driving scene data at least includes multiple frames of point cloud data collected by a radar on the first vehicle;

[0038] A position analysis module, configured to analyze the relative position relationship between the first vehicle and a second vehicle around the first vehicle based on each frame of point cloud data to obtain a relative position feature vector;

[0039] A determination module, configured to determine the scene characterization vector when collecting the point cloud data based on the relative position feature vector;

[0040] A relationship analysis module, configured to analyze the correlation relationship between the scene characterization vectors when collecting each frame of point cloud data to obtain the scene feature vector;

[0041] A classification module, configured to classify driving scenarios based on the scenario feature vectors, so as to obtain the driving scenario classification result of the first vehicle within the specified time period.

[0042] In some embodiments, the position analysis module is specifically configured to:

[0043] Analyze the point cloud data to determine the target point cloud data belonging to the second vehicle in the point cloud data;

[0044] Based on the target point cloud data, generate a position reference map from a bird's-eye view perspective between the first vehicle and the second vehicle;

[0045] Extract features from the position reference map to obtain the relative position feature vector.

[0046] In some embodiments, the position analysis module is specifically configured to:

[0047] Extract features from the position reference map through a feature extraction model to obtain the relative position feature vector;

[0048] Wherein, the feature extraction model is established according to the following steps:

[0049] Input the obtained position reference map samples into an encoding network for feature extraction;

[0050] Input the extraction result into a decoding network for feature parsing to obtain a predicted image;

[0051] Based on the predicted image and the position reference map samples, adjust the network parameters of the encoding network and / or the decoding network until it is determined that the difference between the predicted image and the position reference map samples is less than a preset difference, and then determine the current encoding network as the feature extraction model.

[0052] In some embodiments, the driving scenario data further includes control characterization data of the first vehicle, and the control characterization data is used to characterize the control state of the first vehicle, and further includes:

[0053] A control analysis module, configured to select target control characterization data whose acquisition time matches the acquisition time of the point cloud data from the control characterization data; analyze the target control characterization data to obtain the control feature vector of the first vehicle;

[0054] The determination module is specifically configured to perform a fusion process on the relative position feature vector and the control feature vector to obtain a scenario characterization vector when the point cloud data is collected.

[0055] In some embodiments, the control analysis module is specifically configured to:

[0056] Normalize each type of data in the target control characterization data to obtain the normalized value of this type of data;

[0057] Combine the normalized values of various types of data to obtain the control feature vector of the first vehicle.

[0058] In some embodiments, the relationship analysis module is specifically configured to:

[0059] Input each scene characterization vector into the feature analysis model for analysis according to the acquisition order of the point cloud data to obtain the scene feature vector;

[0060] Among them, the feature analysis model is established according to the following steps:

[0061] Obtain the scene characterization vector sequence of the vehicle sample;

[0062] Input the scene characterization vector sequence of the vehicle sample into the sequence analysis network for correlation analysis;

[0063] Input the analysis result into the sequence parsing network for driving scene parsing to obtain the predicted vector sequence;

[0064] Based on the predicted vector sequence and the scene characterization vector sequence, adjust the network parameters of the sequence analysis network and / or the sequence parsing network until it is determined that the difference between the predicted vector sequence and the scene characterization vector sequence is less than the set difference, and then determine the current sequence analysis network as the feature analysis model.

[0065] In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor, and a memory communicatively connected to the at least one processor, wherein:

[0066] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor can execute the above-mentioned classification method for driving scene data.

[0067] In a fourth aspect, an embodiment of the present application provides a storage medium, when the computer program in the storage medium is executed by the processor of the electronic device, the electronic device can execute the above-mentioned classification method for driving scene data.

[0068] In the embodiments of the present application, driving scenario data of a first vehicle within a specified time period is obtained. The driving scenario data at least includes multiple frames of point cloud data collected by a radar on the first vehicle. Based on each frame of point cloud data, the relative position relationship between the first vehicle and a second vehicle around the first vehicle is analyzed to obtain a relative position feature vector. Based on the relative position feature vector, a scene representation vector at the time of collecting this frame of point cloud data is determined. The association relationship between the scene representation vectors at the time of collecting each frame of point cloud data is analyzed to obtain a scene feature vector. Based on the scene feature vector, driving scenario classification is performed to obtain the driving scenario classification result of the first vehicle within the specified time period. In this way, a solution for classifying driving scenarios by means of a radar is provided. Description of the Drawings

[0069] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0070] Figure 1 is a flowchart of a method for classifying driving scenario data provided by an embodiment of the present application;

[0071] Figure 2 is a flowchart of a method for determining a relative position feature vector provided by an embodiment of the present application;

[0072] Figure 3 is a flowchart of another method for classifying driving scenario data provided by an embodiment of the present application;

[0073] Figure 4 is a schematic diagram of a process for classifying driving scenario data provided by an embodiment of the present application;

[0074] Figure 5 is a position reference diagram provided by an embodiment of the present application;

[0075] Figure 6 is a schematic diagram of the network structure of a convolutional autoencoder provided by an embodiment of the present application;

[0076] Figure 7 is a schematic diagram of the structure of an LSTM provided by an embodiment of the present application;

[0077] Figure 8 is a schematic diagram of the structure of a classification device for driving scenario data provided by an embodiment of the present application;

[0078] Figure 9 is a schematic diagram of the hardware structure of an electronic device for implementing the method for classifying driving scenario data provided by an embodiment of the present application. Detailed Embodiments

[0079] In order to provide a solution for classifying driving scenario data, an embodiment of the present application provides a method, an apparatus, an electronic device, and a storage medium for classifying driving scenario data.

[0080] The following describes the preferred embodiments of the present application with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. And without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0081] Generally, a radar can be installed on a vehicle. The radar can collect data during the operation of the vehicle to obtain multiple frames of point cloud data. Among them, one frame of point cloud data refers to multiple laser points scanned by the radar within one scanning cycle.

[0082] Figure 1 It is a flowchart of a method for classifying driving scenario data provided by an embodiment of the present application. This process includes the following steps.

[0083] In step 11, obtain the driving scenario data of the first vehicle within a specified time period. The driving scenario data at least includes multiple frames of point cloud data collected by the radar on the first vehicle.

[0084] Among them, the first vehicle can be an autonomous vehicle or a manually driven vehicle, as long as a radar is installed on the first vehicle.

[0085] In step 12, based on each frame of point cloud data, analyze the relative position relationship between the first vehicle and the second vehicles around the first vehicle to obtain a relative position feature vector.

[0086] Among them, there may be one, two, three or more second vehicles. The relative position feature vector can be used to characterize the relative position relationship between the first vehicle and the surrounding second vehicles.

[0087] Specifically, when implementing, the relative position feature vector can be determined according to the Figure 2 shown process, and this process includes the following steps.

[0088] In step 121, analyze each frame of point cloud data to determine the target point cloud data belonging to the second vehicles around the first vehicle in this frame of point cloud data.

[0089] For example, use a 3D point cloud detection algorithm to analyze the laser points in each frame of point cloud data to determine which laser points belong to the second vehicles around the first vehicle. Among them, each laser point contains position information, so the relative position information between the second vehicle and the first vehicle can be known based on the laser points.

[0090] In step 122, based on the target point cloud data, a position reference map from an aerial view perspective between the first vehicle and the second vehicle is generated.

[0091] For example, an image with a pixel value of 0.5 for each pixel initially is generated. The specified area in this image is taken as the position area of the first vehicle, and the pixel values in this position area are set to 0. Based on the target point cloud data, the relative position information between the second vehicle and the first vehicle is determined. Based on the relative position information, the position area of the second vehicle is determined in this image, and the pixel values in the position area of the second vehicle in this image are set to 1, thereby obtaining a position reference map from an aerial view perspective between the first vehicle and the second vehicle.

[0092] In step 123, feature extraction is performed on the position reference map to obtain a relative position feature vector between the first vehicle and the second vehicle.

[0093] For example, feature extraction is performed on the position reference map through a feature extraction model to obtain a relative position feature vector between the first vehicle and the second vehicle.

[0094] In specific implementation, a feature extraction model can be established according to the following steps:

[0095] The obtained position reference map samples are input into an encoding network for feature extraction, and the extraction results are input into a decoding network for feature parsing to obtain a predicted image. Based on the predicted image and the position reference map samples, the network parameters of the encoding network and / or the decoding network are adjusted until it is determined that the difference between the predicted image and the position reference map samples is less than a preset difference, and the current encoding network is determined as the feature extraction model.

[0096] That is, the reference map samples are encoded by the encoding network, and the encoding results are decoded by the decoding network until it is determined that the predicted image obtained by decoding is similar enough to the reference map samples (i.e., the difference between the two is less than the preset difference), and the current encoding network is determined as the feature extraction model. In this way, a feature extraction model with relatively good extraction effect can be obtained without manually annotating the position reference map samples.

[0097] In step 13, based on the relative position feature vector, a scene representation vector at the time of collecting this frame of point cloud data is determined.

[0098] For example, the relative position feature vector is determined as the scene representation vector at the time of collecting this frame of point cloud data.

[0099] In step 14, the correlation relationship between the scene representation vectors at the time of collecting each frame of point cloud data is analyzed to obtain a scene feature vector.

[0100] For example, according to the acquisition order of the point cloud data, each scene representation vector is input into the feature analysis model for analysis to obtain the scene feature vector. That is, in the order of acquisition time from early to late, the scene representation vectors when collecting each frame of point cloud data are composed into a scene representation vector sequence, and then the scene representation vector sequence is input into the feature analysis model for analysis.

[0101] In specific implementation, the feature analysis model can be established according to the following steps:

[0102] Obtain the scene representation vector sequence of the vehicle sample, input the scene representation vector sequence of the vehicle sample into the sequence analysis network for correlation analysis, input the analysis result into the sequence parsing network for driving scene parsing to obtain the prediction vector sequence, and based on the prediction vector sequence and the scene representation vector sequence, adjust the network parameters of the sequence analysis network and / or the sequence parsing network until it is determined that the difference between the prediction vector sequence and the scene representation vector sequence is less than the set difference, and then determine the current sequence analysis network as the feature analysis model.

[0103] That is, use the sequence analysis network to perform correlation analysis on the scene representation vector sequence of the vehicle sample, and use the sequence parsing network to parse the analysis result until it is determined that the parsed prediction vector sequence and the scene representation vector sequence are similar enough (that is, the difference between the two is less than the set difference), and then determine the current sequence analysis network as the feature analysis model. In this way, a feature analysis model with relatively good analysis effect can be obtained without manually annotating the scene representation vector sequence of the vehicle sample.

[0104] In step 15, based on the scene feature vector, perform driving scene classification to obtain the driving scene classification result of the first vehicle within the specified time period.

[0105] For example, use the k-means++ clustering algorithm to perform driving scene classification on the scene feature vector to obtain the driving scene classification result of the first vehicle within the specified time period, where the classification of driving scenes includes many categories such as following a vehicle, overtaking, and lane changing.

[0106] Considering that control representation data such as vehicle speed, acceleration, and steering angle can be used to represent the control state of the first vehicle, and the control state of the first vehicle in different driving scenes will also be different, that is, the control state of the first vehicle can also reflect to a certain extent the driving scene in which the first vehicle is located. Therefore, the embodiment of the present application can also combine the point cloud data collected by the radar on the first vehicle and the control representation data of the first vehicle to perform driving scene classification.

[0107] Figure 3 It is a flowchart of another classification method for driving scene data provided by the embodiment of the present application, and this process includes the following steps.

[0108] In step 31, driving scenario data of the first vehicle within a specified time period is obtained. The driving scenario data includes multiple frames of point cloud data collected by the radar during the driving of the first vehicle and control characterization data of the first vehicle.

[0109] Among them, control characterization data such as vehicle speed, acceleration, steering angle, etc. can characterize the control state of the first vehicle.

[0110] In step 32, based on each frame of point cloud data, the relative position relationship between the first vehicle and the second vehicle around the first vehicle is analyzed to obtain a relative position feature vector.

[0111] For the implementation of this step, reference can be made to the implementation of step 12, which will not be elaborated here.

[0112] In step 33, target control characterization data whose acquisition time matches the acquisition time of this frame of point cloud data is selected from the control characterization data.

[0113] Among them, the acquisition time matching includes that the acquisition time difference is less than a specified value, the acquisition times are the same, etc.

[0114] In step 34, the target control characterization data is analyzed to obtain a control feature vector of the first vehicle.

[0115] For example, each type of data in the target control characterization data is normalized to obtain the normalized value of this type of data, and then, the normalized values of various types of data are combined to obtain a control feature vector of the first vehicle.

[0116] In step 35, the relative position feature vector and the control feature vector are fused to obtain a scene characterization vector when this frame of point cloud data is collected.

[0117] Among them, the fusion is, for example, splicing the relative position feature vector and the control feature vector together.

[0118] In step 36, the correlation relationship between the scene characterization vectors when each frame of point cloud data is collected is analyzed to obtain a scene feature vector.

[0119] For the implementation of this step, reference can be made to the implementation of step 14, which will not be elaborated here.

[0120] In step 37, based on the scene feature vector, driving scenario classification is performed to obtain a driving scenario classification result of the first vehicle within a specified time period.

[0121] In this way, by combining multiple frames of point cloud data of the first vehicle collected by the radar and the control characterization data of the first vehicle, the driving scenario of the first vehicle is classified. Considering more dimensions is conducive to improving the classification accuracy.

[0122] The solutions of the embodiments of the present application will be introduced below in conjunction with specific embodiments.

[0123] Figure 4 It is a schematic process diagram for classifying driving scenario data provided by an embodiment of the present application, mainly including the following parts: (1) Point cloud data encoding (2) Feature extraction (3) Feature fusion (4) Association relationship analysis (5) Clustering classification. The following will be introduced Figure 4 for each of these parts respectively.

[0124] (1) Encode the point cloud data to obtain a position reference map.

[0125] In specific implementation, a 3D point cloud detection algorithm can be used to detect other traffic participants (mainly referring to surrounding vehicles) around the vehicle from each frame of point cloud data, and a position reference map between the vehicle and the surrounding vehicles is generated from a bird's-eye view perspective.

[0126] For example, first generate an image with a pixel value of 0.5 for each pixel, and then, based on the relative position information between the vehicle and the surrounding vehicles, the position area of the vehicle and the position areas of the surrounding vehicles can be determined in the image respectively. Then, set the pixel values in the position area of the vehicle to 0, and set the pixel values in the position areas of the surrounding vehicles to 1, so as to obtain a position reference map from a bird's-eye view perspective between the vehicle and the surrounding vehicles. See Figure 5 .

[0127] (2) Feature extraction.

[0128] Extract features from each position reference map to obtain FV (i.e., relative position feature vector), thus completing the encoding from frame to vector (Frame2vector).

[0129] In some embodiments, feature extraction can be performed by a Convolutional Variational Autoencoder. Figure 6 It is a schematic network structure diagram of a Convolutional Variational Autoencoder provided by an embodiment of the present application. G is the position reference map. G is converted into FV through an Encoder (encoding) composed of a Convolutional Neural Networks (CNN), and FV generates (i.e., the predicted image) through a Decoder (decoding) composed of a CNN network.

[0130] The loss function formula for training the CNN network is as follows:

[0131] where G i is the i-th position reference map sample, is the i-th predicted image.

[0132] During the process of training the Encoder, by minimizing the loss function L CVA , L is decreased through the gradient descent algorithm. When it is less than the preset value, the trained Encoder, which is the feature extraction model, is completed. CVA In this way, the feature extraction model is trained by the self-supervised method without the need for manual sample annotation, which can reduce the model training cost.

[0133] It should be noted that

[0134] The figure shows the schematic diagram of the network structure in the training stage. In the model usage stage, the FV output from the Encoder is obtained, and there is no need to use the Decoder to process the FV anymore. Figure 6

[0135] (3) Feature fusion. (3) Feature fusion.

[0136] To distinguish the FV at different times, the time t is used as the subscript to distinguish the FV. For the FV corresponding to the point cloud data collected at time t t , the control representation data such as vehicle speed, acceleration, steering angle, etc. in the Controller Area Network (CAN) information at time t can be obtained. Each type of data is normalized, that is, the vehicle speed, acceleration, steering angle, etc. are normalized separately, and the normalized values of each type of data are combined to generate the CV t vector (i.e., the control feature vector). For example, the CV t after normalizing [vehicle speed, acceleration,...] is [0.1, 0.3,...].

[0137] After that, the FV t and the CV t are connected to obtain the V t vector (i.e., the scene representation vector of the vehicle at time t).

[0138] (4) Association relationship analysis.

[0139] The association relationship analysis is performed on the V t at multiple times to obtain the V clu (i.e., the scene feature vector).

[0140] In some embodiments, the sequence-to-sequence (Seq2seq) network composed of the Long Short-Term Memory (LSTM) network can be used for the association relationship analysis. Figure 7 This is the schematic diagram of the structure of an LSTM provided by the embodiment of the present application. The vehicle sample scene representation vector sequence V i= [V t-n , …, V t-1 , V t , V t+1 , …] is input into the previous Seq2seq network (i.e., the sequence analysis network) for correlation analysis to obtain V clu . V clu is input into the subsequent Seq2seq network (i.e., the sequence parsing network) for driving scenario parsing to obtain (i.e., the predicted vector sequence).

[0141] The loss function for training is defined as follows:

[0142] where V i represents the i-th scenario characterization vector sequence of the vehicle sample, represents the i-th output sequence.

[0143] Minimize the loss function L seq2seq . Through the gradient descent algorithm, make L seq2seq decrease. When it is less than the set value, the correlation analysis ability of the previous Seq2seq network for sequences can be trained. The trained previous Seq2seq network is the feature analysis model.

[0144] In this way, the feature analysis model is trained by the self-supervised method, without the need for manual sample annotation, which can reduce the model training cost.

[0145] Similarly, Figure 7 The schematic diagram of the network structure in the training stage is shown. In the model usage stage, what is obtained is V clu output from the previous Seq2seq network, and there is no need to use the subsequent Seq2seq network to process V clu anymore.

[0146] (5) Driving scenario classification.

[0147] The k-means++ clustering algorithm is used to classify the driving scenarios of V clu to obtain categories such as following a vehicle, overtaking, and lane changing.

[0148] In the embodiments of the present application, without manual sample annotation, driving scenarios such as overtaking and lane changing can be automatically classified, and the classification cost is relatively low. Moreover, the relative distances between other vehicles around the vehicle and other vehicles are encoded based on the point cloud data, and at the same time, information such as vehicle speed, acceleration, and direction angle in the vehicle CAN information of the vehicle is combined for driving scenario classification, and the classification accuracy is also relatively high.

[0149] Based on the same inventive concept, an embodiment of the present application further provides a classification device for driving scenario data. The principle of the classification device for driving scenario data to solve problems is similar to the above-mentioned classification method for driving scenario data. Therefore, the implementation of the classification device for driving scenario data can refer to the implementation of the classification method for driving scenario data, and the repeated parts will not be elaborated.

[0150] Figure 8 FIG. 4 is a schematic structural diagram of a classification device for driving scenario data provided by an embodiment of the present application, including an acquisition module 801, a position analysis module 802, a determination module 803, a relationship analysis module 804, and a classification module 805.

[0151] The acquisition module 801 is configured to acquire driving scenario data of a first vehicle within a specified time period, and the driving scenario data at least includes multiple frames of point cloud data collected by a radar on the first vehicle;

[0152] The position analysis module 802 is configured to analyze the relative position relationship between the first vehicle and a second vehicle around the first vehicle based on each frame of point cloud data to obtain a relative position feature vector;

[0153] The determination module 803 is configured to determine a scene representation vector at the time of collecting the point cloud data based on the relative position feature vector;

[0154] The relationship analysis module 804 is configured to analyze the association relationship between the scene representation vectors at the time of collecting each frame of point cloud data to obtain a scene feature vector;

[0155] The classification module 805 is configured to perform driving scenario classification based on the scene feature vector to obtain a driving scenario classification result of the first vehicle within the specified time period.

[0156] In some embodiments, the position analysis module 802 is specifically configured to:

[0157] Analyze the point cloud data to determine target point cloud data belonging to the second vehicle in the point cloud data;

[0158] Generate a position reference map from a bird's-eye view perspective between the first vehicle and the second vehicle based on the target point cloud data;

[0159] Extract features from the position reference map to obtain the relative position feature vector.

[0160] In some embodiments, the position analysis module 802 is specifically configured to:

[0161] Extract features from the position reference map through a feature extraction model to obtain the relative position feature vector;

[0162] Among them, the feature extraction model is established according to the following steps:

[0163] Input the obtained position reference map samples into the encoding network for feature extraction;

[0164] Input the extraction result into the decoding network for feature parsing to obtain a predicted image;

[0165] Based on the predicted image and the position reference map samples, adjust the network parameters of the encoding network and / or the decoding network until it is determined that the difference between the predicted image and the position reference map samples is less than a preset difference, and then determine the current encoding network as the feature extraction model.

[0166] In some embodiments, the driving scene data further includes control characterization data of the first vehicle, and the control characterization data is used to characterize the control state of the first vehicle, and further includes:

[0167] A control analysis module 806, configured to select target control characterization data whose acquisition time matches the acquisition time of the point cloud data from the control characterization data; analyze the target control characterization data to obtain a control feature vector of the first vehicle

[0168] The determining module 803 is specifically configured to perform a fusion process on the relative position feature vector and the control feature vector to obtain a scene characterization vector when the point cloud data is acquired.

[0169] In some embodiments, the control analysis module 806 is specifically configured to:

[0170] Perform a normalization process on each type of data in the target control characterization data to obtain a normalized value of this type of data;

[0171] Combine the normalized values of various types of data to obtain a control feature vector of the first vehicle.

[0172] In some embodiments, the relationship analysis module 804 is specifically configured to:

[0173] Input each scene characterization vector into a feature analysis model for analysis according to the acquisition order of the point cloud data to obtain a scene feature vector;

[0174] Among them, the feature analysis model is established according to the following steps:

[0175] Obtain a sequence of scene characterization vectors of vehicle samples;

[0176] Input the sequence of scene characterization vectors of the vehicle samples into a sequence analysis network for correlation analysis;

[0177] Input the analysis result into a sequence parsing network for driving scenario parsing to obtain a sequence of predicted vectors;

[0178] Based on the sequence of predicted vectors and the sequence of scenario representation vectors, adjust the network parameters of the sequence analysis network and / or the sequence parsing network until it is determined that the difference between the sequence of predicted vectors and the sequence of scenario representation vectors is less than a set difference. Then, determine the current sequence analysis network as the feature analysis model.

[0179] The division of modules in the embodiments of the present application is illustrative, merely a logical function division. In actual implementation, there may be other division methods. In addition, each functional module in the embodiments of the present application can be integrated in a processor, or exist separately physically, or two or more modules can be integrated in one module. The coupling between each module can be realized through some interfaces, and these interfaces are usually electrical communication interfaces, but it does not exclude the possibility of being mechanical interfaces or other forms of interfaces. Therefore, the modules described as separate components may or may not be physically separated, and can be located in one place, or distributed to different positions of the same or different devices. The above integrated modules can be implemented in the form of hardware or in the form of software function modules.

[0180] After introducing the classification method and device for driving scenario data in the exemplary embodiments of the present application, next, an electronic device according to another exemplary embodiment of the present application will be introduced.

[0181] Next, refer to Figure 9 to describe the electronic device 130 implemented according to this embodiment of the present application. Figure 9 The shown electronic device 130 is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present application.

[0182] As Figure 9 shown, the electronic device 130 is presented in the form of a general electronic device. The components of the electronic device 130 may include, but are not limited to: the above at least one processor 131, the above at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0183] The bus 133 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a processor, or a local bus using any bus structure in a variety of bus structures.

[0184] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0185] The memory 132 may also include a program / utilities 1325 having a set (at least one) of program modules 1324. Such program modules 1324 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data, and each or some combination of these examples may include an implementation of a network environment.

[0186] The electronic device 130 may also communicate with one or more external devices 134 (such as a keyboard, a pointing device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or may communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (such as a router, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 135. Further, the electronic device 130 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 136. As shown in the figure, the network adapter 136 communicates with other modules for the electronic device 130 through a bus 133. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0187] In an exemplary embodiment, a storage medium is also provided. When the computer program in the storage medium is executed by a processor of an electronic device, the electronic device can execute the above classification method for driving scenario data. Optionally, the storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, etc.

[0188] In an exemplary embodiment, the electronic device of the present application may at least include at least one processor and a memory communicatively connected to the at least one processor. Among them, the memory stores a computer program executable by the at least one processor. When the computer program is executed by the at least one processor, the at least one processor can execute the steps of any classification method for driving scenario data provided by the embodiments of the present application.

[0189] In an exemplary embodiment, a computer program product is further provided. When the computer program product is executed by an electronic device, the electronic device can implement any of the exemplary methods provided in this application.

[0190] Moreover, the computer program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0191] The program product for classifying driving scenario data in the embodiments of this application can adopt a CD-ROM and include program code, and can run on a computing device. However, the program product of this application is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0192] The readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0193] The program code contained on the readable medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber, radio frequency (RF), etc., or any suitable combination of the above.

[0194] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0195] It should be noted that although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-described units can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0196] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0197] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, as well as the combination of flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0199] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0200] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0201] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be interpreted to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0202] Obviously, those skilled in the art can make various changes and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A classification method for driving scenario data, characterized in that, Including: Obtain the driving scenario data of the first vehicle within a specified time period, where the driving scenario data at least includes multiple frames of point cloud data collected by a radar on the first vehicle; Based on each frame of point cloud data, analyze the relative position relationship between the first vehicle and a second vehicle around the first vehicle to obtain a relative position feature vector; Based on the relative position feature vector, determine a scene representation vector at the time when the point cloud data is collected, and the scene representation vector is the relative position feature vector; In the order of collection of the point cloud data, input each scene representation vector into a feature analysis model for analysis to obtain a scene feature vector; Based on the scene feature vector, perform driving scenario classification to obtain the driving scenario classification result of the first vehicle within the specified time period.

2. The method according to claim 1, characterized in that, Based on each frame of point cloud data, analyzing the relative position relationship between the first vehicle and a second vehicle around the first vehicle to obtain a relative position feature vector includes: Analyze the point cloud data to determine the target point cloud data belonging to the second vehicle in the point cloud data; Based on the target point cloud data, generate a position reference map from a bird's-eye view perspective between the first vehicle and the second vehicle; Extract features from the position reference map to obtain the relative position feature vector.

3. The method according to claim 2, wherein Extracting features from the position reference map to obtain the relative position feature vector includes: Extract features from the position reference map through a feature extraction model to obtain the relative position feature vector; Wherein, the feature extraction model is established according to the following steps: Input the obtained position reference map sample into an encoding network for feature extraction; Input the extraction result into a decoding network for feature parsing to obtain a predicted image; Based on the predicted image and the position reference map sample, adjust the network parameters of the encoding network and / or the decoding network until it is determined that the difference between the predicted image and the position reference map sample is less than a preset difference, and then determine the current encoding network as the feature extraction model.

4. The method according to claim 1, characterized in that If the driving scenario data further includes control representation data of the first vehicle, and the control representation data is used to represent the control state of the first vehicle, then it further includes: Select target control representation data from the control representation data whose collection time matches the collection time of the point cloud data; Analyze the target control representation data to obtain a control feature vector of the first vehicle; and The scene representation vector is obtained by performing a fusion process on the relative position feature vector and the control feature vector.

5. The method according to claim 4, characterized in that Analyzing the target control representation data to obtain a control feature vector of the first vehicle includes: Perform normalization processing on each type of data in the target control representation data to obtain the normalized value of this type of data; Combine the normalized values of various types of data to obtain the control feature vector of the first vehicle.

6. The method according to any one of claims 1 to 5, characterized in that The feature analysis model is established according to the following steps: Obtain a sequence of scene representation vectors of a vehicle sample; Input the sequence of scene representation vectors of the vehicle sample into a sequence analysis network for correlation analysis; Input the analysis result into a sequence parsing network for driving scenario parsing to obtain a sequence of predicted vectors; Based on the sequence of predicted vectors and the sequence of scenario representation vectors, adjust the network parameters of the sequence analysis network and / or the sequence parsing network until it is determined that the difference between the sequence of predicted vectors and the sequence of scenario representation vectors is less than a set difference. Then, determine the current sequence analysis network as the feature analysis model.

7. A classification device for driving scenario data, characterized in that, It includes: An acquisition module for acquiring driving scenario data of a first vehicle within a specified time period. The driving scenario data at least includes multiple frames of point cloud data collected by a radar on the first vehicle; A position analysis module for analyzing the relative position relationship between the first vehicle and a second vehicle around the first vehicle based on each frame of point cloud data to obtain a relative position feature vector; A determination module for determining a scenario representation vector at the time of collecting the point cloud data based on the relative position feature vector. The scenario representation vector is the relative position feature vector; A relationship analysis module for inputting each scenario representation vector into a feature analysis model for analysis in the acquisition order of the point cloud data to obtain a scenario feature vector; A classification module for classifying the driving scenario based on the scenario feature vector to obtain a driving scenario classification result of the first vehicle within the specified time period.

8. The device according to claim 7, characterized in that, If the driving scenario data further includes control representation data of the first vehicle, and the control representation data is used to represent the control state of the first vehicle, it further includes: A control analysis module for selecting target control representation data whose acquisition time matches the acquisition time of the point cloud data from the control representation data; analyzing the target control representation data to obtain a control feature vector of the first vehicle; and the scenario representation vector is obtained by fusing the relative position feature vector and the control feature vector.

9. An electronic device, characterized in that, It includes: At least one processor, and a memory communicatively connected to the at least one processor, wherein: The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-6.

10. A storage medium, characterized in that, When the computer program in the storage medium is executed by a processor of an electronic device, the electronic device can execute the method according to any one of claims 1-6.

Images