A cross-intention discrimination method, medium, device and vehicle

Abstract

The application discloses a crossing intention discrimination method, medium, device and vehicle, comprising: constructing a data set with crossing positive examples of vulnerable road users meeting first preset conditions and crossing negative examples of vulnerable road users meeting second preset conditions; wherein the first preset condition is that the vulnerable road user is in the front of the ego vehicle within the region of interest of the ego vehicle and has a crossing behavior within a preset future time; the second preset condition is that the vulnerable road user is in the rear of the ego vehicle within the region of interest of the ego vehicle and has a crossing behavior within a preset future time; training a preset binary classification model using the data set, the binary classification model being built based on a deep learning model; and deploying the trained binary classification model to an autonomous driving system to discriminate the crossing intention of the vulnerable road user. Through the above manner, the application can improve the accuracy of the crossing intention discrimination of the vulnerable road user, and further improve the safety and comfort of the passengers.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of autonomous driving, and in particular to a crossing intention determination method, a medium, an equipment and a vehicle. BACKGROUND

[0002] The traditional crossing intention determination method based on trajectory prediction needs to calculate the preliminary planning trajectory of the ego vehicle and the predicted trajectory of the object, which has strong dependence on the predicted trajectory and does not consider the interaction between the vulnerable road user and the ego vehicle, resulting in inaccurate determination of the crossing intention of the vulnerable road user and affecting the driving planning of the ego vehicle and the safety and comfort of the passengers. SUMMARY

[0003] The present application mainly provides a crossing intention determination method, a medium, an equipment and a vehicle to solve the problem of poor safety and comfort of passengers caused by inaccurate determination of the crossing intention of the vulnerable road user.

[0004] To solve the above technical problems, one technical solution adopted by the present application is to provide a crossing intention determination method based on a deep learning model, comprising: constructing a data set with crossing positive examples of vulnerable road users meeting a first preset condition and crossing negative examples of vulnerable road users meeting a second preset condition; wherein the first preset condition is that the vulnerable road user is in the region of interest of the ego vehicle and has a crossing behavior in front of the ego vehicle within a future preset time; the second preset condition is that the vulnerable road user is in the region of interest of the ego vehicle and has a crossing behavior behind the ego vehicle within a future preset time; training a preset binary classification model using the data set, the binary classification model being built based on a deep learning model; and deploying the trained binary classification model to an autonomous driving system to determine the crossing intention of the vulnerable road user.

[0005] By marking the crossing positive example label and the crossing negative example label of the vulnerable road user in the real driving data to construct the data set, and using the data set for training of the binary classification model, the accuracy of the binary classification model in determining the crossing intention of the vulnerable road user can be further improved, so as to reasonably plan the route and speed and improve the safety and comfort of the passengers.

[0006] In some embodiments, the construction of the data set with the crossing positive examples of the vulnerable road users meeting the first preset condition and the crossing negative examples of the vulnerable road users meeting the second preset condition comprises: filtering out closed road scene data in the driving data; traversing each of the vulnerable road users perceived by the ego vehicle in the driving data; marking a positive example label for the vulnerable road user meeting the first preset condition, and marking a negative example label for the vulnerable road user meeting the second preset condition.

[0007] By filtering out high-speed scene data in the driving data segment, the positive example label and the negative example label are re-labeled, which can avoid the data without reference value to reduce the speed of the label marking process and reduce the labeling time.

[0008] In some embodiments, the data set is constructed by the vulnerable road user crossing the positive example meeting the first preset condition and the vulnerable road user crossing the negative example meeting the second preset condition, and further includes: marking the vulnerable road user not meeting the first preset condition and the second preset condition as a negative example label according to a preset proportion.

[0009] By marking the part of the vulnerable road user not meeting the first preset condition and the second preset condition as a negative example label, the vulnerable road user on the road irrelevant to the self-vehicle can also be learned by the binary classification model, avoiding the irrelevant vulnerable road user being misjudged as a positive example, and further improving the accuracy of the binary classification model in judging the crossing intention of the vulnerable road user.

[0010] In some embodiments, the first preset condition further includes that an acute angle between an extension direction of the trajectory of the vulnerable road user and an extension direction of the trajectory of the self-vehicle is greater than a preset angle.

[0011] By specifically distinguishing the crossing behavior, the binary classification model can further learn the crossing behavior of the vulnerable road user in the training process, and consider the deceleration or stopping behavior of the vulnerable road user in the crossing process.

[0012] In some embodiments, the crossing positive example includes a first vulnerable road user and a positive example label, and the crossing negative example includes a second vulnerable road user and a negative example label; the data set is used to train a preset binary classification model, including: obtaining a feature vector of a target vulnerable road user; the target vulnerable road user includes the first vulnerable road user and / or the second vulnerable road user in the data set; the feature vector is interaction information between the self-vehicle and the target vulnerable road user; the feature vector of the target vulnerable road user, motion information of the vulnerable road user at a historical time, and navigation information of the self-vehicle are input into the binary classification model to obtain a probability of the target vulnerable road user crossing; a label corresponding to the target vulnerable road user is obtained, and an error between the probability of the vulnerable road user crossing and the label is calculated, wherein the label is a positive example label or a negative example label corresponding to the target vulnerable road user; the binary classification model is corrected based on the error until the error between the probability and the label is less than a preset error threshold.

[0013] The binary classification model is trained by data obtained from the driving data segment, and the binary classification model can learn the next action trajectory of the vulnerable road user according to the motion information and interaction information of the vulnerable road user, and accurately determine whether the vulnerable road user will cross.

[0014] In some embodiments, the inputting the feature vector of the target vulnerable road user, the motion information of the vulnerable road user at the historical time, and the navigation information of the ego vehicle into the binary classification model to obtain the probability of the target vulnerable road user crossing includes: inputting the feature vector of the target vulnerable road user, the motion information of the vulnerable road user at the historical time, and the navigation information of the ego vehicle into the convolution kernel network layer to learn and extract core features, to obtain the core features in the feature vector of the target vulnerable road user and the motion information at the historical time; and inputting the core features extracted by the convolution kernel network layer into the linear network layer to perform linear transformation, to obtain the probability of the target vulnerable road user crossing.

[0015] The learning and extraction of the core features by the convolution kernel network layer improves the efficiency and accuracy of feature extraction, is applicable to data with diversity, has high generalization, and enables the binary classification model to have good performance in different scenarios; the linear transformation process can be efficiently completed by the efficient calculation characteristics of the linear network layer, to meet the real-time calculation requirements of the vulnerable road user crossing intention determination.

[0016] In some embodiments, the deploying the trained binary classification model to the autonomous driving system includes: serializing the trained binary classification model into binary model data by an inference engine; and deploying the binary model data to the autonomous driving system by a programming language.

[0017] By serializing the binary classification model into binary model data and then deploying the binary model data, the machine code can be directly corresponded, and complex syntax and format analysis is not required during parsing, which helps to greatly improve the parsing speed and accelerate the inference speed of the model.

[0018] To solve the above technical problems, another technical solution adopted by the present application is to provide a storage medium having program data stored thereon, the program data being executed by a processor to implement the steps of the crossing intention determination method described above.

[0019] The beneficial effects of the storage medium are referred to the beneficial effects of the crossing intention determination method described above, and will not be repeated here.

[0020] The application further provides a vehicle-mounted device, comprising a processor and a memory connected to each other, and the memory stores a computer program, and the processor implements the steps of the crossing intention identification method as described above when executing the computer program.

[0021] The vehicle-mounted device is a computer type device, and the beneficial effects of the vehicle-mounted device refer to those of the crossing intention identification method as described above, which are not repeated here.

[0022] The application further provides a vehicle comprising the vehicle-mounted device as described above. The vehicle can complete the steps of identifying the crossing intention of the vulnerable road user during driving through the control of the vehicle-mounted device.

[0023] The beneficial effects of the application are that, different from the prior art, the application discloses a crossing intention identification method, medium, device and vehicle. By traversing driving data segments from real road driving data, marking a crossing positive example label for a vulnerable road user in the driving data segment that meets a first preset condition, marking a crossing negative example label for a vulnerable road user that meets a second preset condition, and using the labels for the training of a binary classification model, the accuracy of the binary classification model in identifying the crossing intention of the vulnerable road user can be further improved, so as to reasonably plan the route and speed and improve the safety and comfort of the passengers. The first preset condition is further set as being in the region of interest of the ego vehicle and having a crossing behavior in front of the ego vehicle in the future time, the user who has the crossing behavior can be determined according to the behavior mode of the vulnerable road user in a period of time; the second preset condition is further set as being in the region of interest of the ego vehicle and having a crossing behavior behind the ego vehicle in the future time, the user who does not have the crossing behavior can be determined, so that the binary classification model can accurately identify the behavior mode of the vulnerable road user in a period of time in different driving scenarios after being trained, and has flexibility. After the binary classification model is deployed to an automatic driving system, the automatic driving vehicle can accurately identify the crossing intention of the vulnerable road user, and decide whether to brake to show courtesy to ensure safety or to rush through to ensure the feeling. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only show some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0025] Figure 1 is a flowchart of an embodiment of the crossing intention identification method provided by the application;

[0026] Figure 2is a flowchart of an embodiment of the cross intention discrimination method based on a deep learning model step 10 as shown in Figure 1

[0027] Figure 3 is a flowchart of an embodiment of the cross intention discrimination method step 20 as shown in Figure 1

[0028] Figure 4 is a flowchart of an embodiment of the cross intention discrimination method step 22 as shown in Figure 3

[0029] Figure 5 is a flowchart of an embodiment of the cross intention discrimination method step 30 as shown in Figure 1

[0030] Figure 6 is a structural diagram of an embodiment of the storage medium provided by the present application;

[0031] Figure 7 is a structural diagram of an embodiment of the vehicle-mounted device provided by the present application;

[0032] Figure 8 is a structural diagram of an embodiment of the autonomous vehicle provided by the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0034] The terms “first”, “second”, “third” in the embodiments of the present application are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first”, “second”, “third” can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of “plurality” is at least two, for example, two, three, etc., unless otherwise specifically limited. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0035] ​​​​In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0036] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the traversal intent determination method based on a deep learning model provided in this application. The traversal intent determination method includes:

[0037] Step 10: Construct a dataset with positive examples of vulnerable road users crossing the road that meet the first preset condition and negative examples of vulnerable road users crossing the road that meet the second preset condition.

[0038] Based on the perceived vulnerable road users in the traffic scene, a first preset condition for identifying vulnerable road users who cross the road and a second preset condition for identifying vulnerable road users who do not cross the road are set. Vulnerable road users who meet the first preset condition are labeled as positive examples, and vulnerable road users who meet the second preset condition are labeled as negative examples. A dataset is constructed using the positive and negative examples of vulnerable road user data.

[0039] Vulnerable road users are participants who are relatively vulnerable to injury in road traffic, including but not limited to pedestrians, cyclists, e-scooter riders, and people pushing strollers.

[0040] The first precondition is that a vulnerable road user is crossing in front of the vehicle within the vehicle's area of ​​interest and within a pre-defined timeframe. If a vulnerable road user crosses the road in front of the vehicle within a few seconds, the autonomous vehicle should immediately take safety measures, such as slowing down, stopping, or yielding, to ensure the safety of both the pedestrian and the vehicle.

[0041] The forward crossing behavior includes situations where the trajectory of the vehicle and the vulnerable road user intersect within a predetermined time in the future, and the vulnerable road user passes the intersection point before the vehicle, and situations where the trajectory of the vehicle and the vulnerable road user do not intersect within a predetermined time in the future, but the vulnerable road user passes in front of the vehicle.

[0042] Specifically, the length of the preset time should be flexibly set according to actual conditions such as road type, vehicle speed, and traffic flow to ensure that it is neither too sensitive to cause frequent misjudgments nor too sluggish to increase the risk of collisions, such as 2s, 3s, 5s, 8s, 10s, 15s, or 17s.

[0043] The region of interest (ROI) is the area that the autonomous vehicle sensor focuses on monitoring, which is usually dynamically adjusted according to the speed, direction, road environment, and other factors of the vehicle. The region of interest can include the area within a certain distance and angle range in front of the vehicle to ensure that potential traffic participants can be discovered and identified in time.

[0044] Optionally, the first preset condition further includes that an acute angle formed between the trajectory extension direction of the vulnerable road user and the trajectory extension direction of the ego vehicle is greater than a preset angle.

[0045] When the acute angle formed between the trajectory extension direction of the vulnerable road user and the trajectory extension direction of the ego vehicle is greater than the preset angle, in combination with the condition that the vulnerable road user has a crossing behavior in front of the ego vehicle within the ego vehicle's region of interest and within a future preset time, it can be considered that there is a high collision risk between the vulnerable road user and the ego vehicle. At this time, the autonomous vehicle should immediately start the corresponding safety response mechanism, such as deceleration, parking, avoidance, etc., to ensure the safety of the vulnerable road user and the vehicle itself.

[0046] Specifically, the preset angle can be 7°, 10°, 15°, 19°, 20°, 23°, or 30°, etc.

[0047] By identifying the crossing behavior through specific angles, the binary classification model can further learn the crossing behavior of the vulnerable road user during the training process, while considering the possible deceleration or stopping behavior of the vulnerable road user during the crossing process.

[0048] The second preset condition is that the vulnerable road user is within the region of interest of the ego vehicle and has a crossing behavior behind the ego vehicle within a future preset time. If the trajectory of the vulnerable road user within the future preset time intersects with the ego vehicle, and the ego vehicle passes through the intersection point before the vulnerable road user, it means that the vulnerable road user is crossing from behind the ego vehicle and will not affect the driving of the ego vehicle.

[0049] The vulnerable road user meeting the first preset condition is taken as a positive example, and the vulnerable road user meeting the second preset condition is taken as a negative example, to construct a data set for judging the crossing intention of the vulnerable road user, and to train an effective model for judging the crossing intention of the vulnerable road user, which is beneficial to improve the accuracy of the autonomous vehicle in judging the crossing intention of the vulnerable road user.

[0050] Further, referring to Figure 2 The data set is constructed with the vulnerable road user crossing positive examples meeting the first preset condition and the vulnerable road user crossing negative examples meeting the second preset condition, including:

[0051] Step 11: Filter out the closed road scene data in the driving data.

[0052] The closed road is often located in a non-public area and can be identified by comparing the geographic location information (such as latitude and longitude) with a public road database. After identifying the closed road scene in the driving data, the part of the data is removed from the driving data. Since it is not easy for vulnerable road users to cross in a closed road, filtering out the closed road scene data in the driving data can shorten the time for subsequent label marking, thereby improving the construction speed of the data set.

[0053] Step 12: Traverse each vulnerable road user perceived by the ego vehicle in the driving data.

[0054] The driving data is traversed in chronological order, and the data at each time point is checked. For each time point, all related sensor data is checked to determine whether there is a vulnerable road user. For each perceived vulnerable road user, relevant information such as location (such as latitude and longitude or coordinates in the vehicle coordinate system), speed, direction, type (such as pedestrian or bicycle), and size is recorded.

[0055] Step 13: Mark the positive example label for the vulnerable road user meeting the first preset condition, and mark the negative example label for the vulnerable road user meeting the second preset condition.

[0056] Among the obtained vulnerable road user information, the positive example label is marked as a training label for the vulnerable road user with crossing intention through the first preset condition, and the negative example label is marked as a training label for the vulnerable road user without crossing intention through the second preset condition.

[0057] The labeled labels are used for binary classification model training, which helps the autonomous vehicle to more accurately identify potential traffic risks and take appropriate measures to ensure safe driving.

[0058] Optionally, step 13 further comprises: marking the vulnerable road user not meeting the first preset condition and the second preset condition as a negative example label at a preset ratio.

[0059] Among the vulnerable road user data in the driving data that is not marked as a crossing positive example or a crossing negative example because it does not meet the first preset condition and the second preset condition, a portion of the data is selected at a ratio to be marked as a crossing negative example.

[0060] By marking part of the vulnerable road users not meeting the first preset condition and the second preset condition as a negative example label, the vulnerable road users on the road who are irrelevant to the ego vehicle can also be learned by the binary classification model, avoiding the irrelevant vulnerable road users being misjudged as positive examples, and further improving the accuracy of the binary classification model in determining the crossing intention of the vulnerable road user.

[0061] Step 20: Train the preset binary classification model using the dataset.

[0062] The preset binary classification model is trained using the dataset as input data and validation data to improve the accuracy of the binary classification model in identifying the crossing intention of vulnerable road users.

[0063] The binary classification model is built using a deep learning framework, including designing the architecture of the binary classification model such as input layer, hidden layer, and output layer, and specifying the loss function and evaluation metrics to improve the performance of the binary classification model. The binary classification model is built based on a deep learning model to classify the input data samples into one of two categories.

[0064] Specifically, the types of binary classification models can include: logistic regression, which maps the output of a linear model to the (0, 1) interval using a function to obtain the probability of belonging to a certain category; decision tree, which classifies data into one of two categories by learning a series of rules; neural network, which can learn complex nonlinear relationships by stacking multiple layers, and uses a function in the output layer to obtain the probability of belonging to a certain category, etc.

[0065] The binary classification model built based on the deep learning framework is trained using the dataset, enabling the binary classification model to learn accurate and detailed recognition methods and improve its performance. Meanwhile, a large amount of training data can reduce the model's over-reliance on training data, reduce the risk of overfitting, and adapt to new data, enhancing the generalization ability of the binary classification model.

[0066] Wherein, the crossing positive example includes the first vulnerable road user and the positive example label, and the crossing negative example includes the second vulnerable road user and the negative example label.

[0067] Further, referring to Figure 3 The preset binary classification model is trained using the dataset, including:

[0068] Step 21: Obtain the feature vector of the target vulnerable road user.

[0069] According to the problem requirements and data characteristics, select appropriate features, extract the features of the target vulnerable road user from the dataset through image processing algorithms or data analysis methods, convert the extracted features into numerical form, and form a feature vector to represent various attributes and behavior patterns related to the vulnerable road user.

[0070] Specifically, the target vulnerable road user includes the first vulnerable road user and / or the second vulnerable road user in the dataset.

[0071] Further, the feature vector is the interaction information between the ego vehicle and the target vulnerable road user, including: basic position and motion characteristics of the target vulnerable road user such as position, speed, acceleration, motion direction, etc.; environmental interaction characteristics such as distance from the vehicle, distance from road elements, path selection, etc.; appearance and behavior characteristics such as size and shape, posture, behavior pattern, etc.; time characteristics such as timestamp or time period; map interaction information, etc. Through the feature vector, the binary classification model can distinguish different types of vulnerable road users and preliminarily judge their possible motion patterns. For example, the gait characteristics of a pedestrian may indicate that it is approaching or moving away from the road, while the speed characteristics of a cyclist may reflect whether it intends to cross the road.

[0072] Step 22: input the feature vector of the target vulnerable road user, the motion information of the vulnerable road user at the historical time, and the ego vehicle navigation information into the binary classification model to obtain the probability of the target vulnerable road user crossing.

[0073] The label of each sample is whether the target vulnerable road user has performed a crossing behavior, which is a binary classification problem. A suitable binary classification model is selected for model architecture and parameter setting. The feature vector of the target vulnerable road user, the motion information of the vulnerable road user at the historical time, and the ego vehicle navigation information in the data set are input into the binary classification model for training. During the training process, the model learns the mapping relationship between the input data and the label, and judges which training label the target vulnerable road user is closer to according to the characteristics of the input data.

[0074] Specifically, the motion information of the vulnerable road user at the historical time includes trajectory data, speed information, acceleration information, motion direction, and behavior pattern of the vulnerable road user such as pedestrians and cyclists. The historical motion information provides detailed records about its past behavior, including position, speed, acceleration, motion direction, etc. By analyzing the trajectory changes, speed fluctuations, and acceleration patterns of the vulnerable road user, the binary classification model can identify the motion trend and intention of the vulnerable road user. For example, if the vulnerable road user is gradually approaching the edge of the road and decelerating, it is likely to indicate that it has the intention to cross the road.

[0075] The ego vehicle navigation information includes the state and position information of the vehicle itself, as well as the driving plan of the vehicle. The ego vehicle navigation information provides the state and position information of the vehicle itself, as well as the driving plan and the perception of the surrounding traffic environment of the vehicle. By comparing the position of the vulnerable road user and the driving route of the vehicle, it is evaluated whether the vulnerable road user is likely to enter the driving path of the vehicle. At the same time, the vehicle can also combine the traffic signal light state, road signs and markings, etc. information to further judge whether the crossing intention of the vulnerable road user is legal and safe.

[0076] By integrating the appearance features, motion features and the ego vehicle navigation information of the target vulnerable road user, the binary classification model can capture the key factors affecting the target behavior and make accurate predictions accordingly.

[0077] Specifically, the binary classification model includes a convolution kernel network layer and a linear network layer.

[0078] The convolution kernel network layer slides over the input data through multiple convolution kernels and performs multiplication and addition operations at each position, thereby extracting core features of the input data.

[0079] The linear network layer is located after the convolution layer and is used for further processing and classification of the features extracted by the convolution layer.

[0080] Referring to Figure 4 The feature vector of the target vulnerable road user, the motion information of the vulnerable road user at historical time and the ego vehicle navigation information are input into the binary classification model to obtain the probability of the target vulnerable road user crossing, including:

[0081] Step 221: input the feature vector of the target vulnerable road user, the motion information of the vulnerable road user at historical time and the ego vehicle navigation information into the convolution kernel network layer for learning and core feature extraction, to obtain the core features of the feature vector of the target vulnerable road user and the motion information at historical time.

[0082] The feature vector of the target vulnerable road user, the motion information of the vulnerable road user at historical time and the ego vehicle navigation information are input into the convolution kernel network layer, which improves the attention of the binary classification model to the target vulnerable road user within the recognition range through the implicit features in the feature vector of the target vulnerable road user, the motion information of the vulnerable road user at historical time and the ego vehicle navigation information, saves the operation cost of the model and improves the data processing efficiency of the binary classification model.

[0083] The neural convolution network automatically learns and extracts local features from the data set for short-term motion pattern or trend prediction of the target vulnerable road user for subsequent analysis, reduces the complexity and subjectivity of feature engineering, and improves the generalization ability and accuracy of the model.

[0084] Step 222: input the core features extracted by the convolution kernel network layer into the linear network layer for linear transformation to obtain the probability of the target vulnerable road user crossing.

[0085] All feature maps outputted by the convolutional layer are flattened into a one-dimensional vector, which contains all the core features extracted from the input data. The flattened one-dimensional vector is inputted into the linear network layer. Each neuron of the linear network layer is connected to each element in the input vector and performs a linear transformation. The result of the linear transformation is a new vector whose length depends on the number of neurons in the linear network layer. An activation function such as a sigmoid activation function is added after the linear network layer to map any real number value to the interval (0, 1), generating a floating-point number between 0 and 1, which represents the probability of the target vulnerable road user to cross.

[0086] The linear transformation process can be efficiently completed through the efficient computing characteristics of the linear network layer, meeting the real-time computing requirements of the vulnerable road user crossing intention discrimination.

[0087] Step 23: Obtain the label corresponding to the target vulnerable road user and calculate the error between the probability of the vulnerable road user to cross and the label, wherein the label is the positive example label or the negative example label corresponding to the target vulnerable road user.

[0088] The positive example label or the negative example label corresponding to the target vulnerable road user whose current calculated probability value is obtained, and the error between the probability and the label is calculated, and the error is used to correct the binary classification model.

[0089] For example, the output of the binary classification model corresponding to vulnerable road user A is 0.32, which means that the probability of vulnerable road user A having a crossing intention is 0.32. In the data set, the positive example label is assigned a value of 1 and the negative example label is assigned a value of 0. Vulnerable road user A is labeled as a negative example in the data set, so the difference between 0.32 and 0 is calculated and the difference is returned to update the data of the binary classification model.

[0090] For another example, the output of the binary classification model corresponding to vulnerable road user B is 0.89, which means that the probability of vulnerable road user B having a crossing intention is 0.89. Vulnerable road user B is labeled as a positive example in the data set, so the difference between 0.89 and 1 is calculated and the difference is returned to update the data of the binary classification model.

[0091] Step 24: Correct the binary classification model based on the error until the error between the probability and the label is less than the preset error threshold.

[0092] The error threshold is defined as the standard for evaluating the prediction ability of the binary classification model. When the error between the probability and the label is less than the preset error threshold, the binary classification model meets the training requirements. The parameters of the binary classification model are corrected based on the error until the error between the probability and the label is less than the preset error threshold.

[0093] For example, the output of the binary classification model corresponding to vehicle C is 0.92, and the difference between the corresponding positive example label value 1 is 0.06. Based on the difference value of 0.08, the parameters of the binary classification model are updated, and the output of the binary classification model corresponding to vehicle A is 0.95, which is less than the preset threshold 0.01. At this time, the binary classification model meets the training requirements.

[0094] By setting the preset threshold, the binary classification model can accurately fit the training data and generalize to new data, which is beneficial to improve the performance of the binary classification model.

[0095] Step 30: Deploy the trained binary classification model to the autonomous driving system to determine the crossing intention of the vulnerable road user.

[0096] Deploy the binary classification model in the autonomous driving system and obtain road information such as image type information or sensor type information through the autonomous driving vehicle. The binary classification model analyzes the abnormal stillness of the target vehicle within the recognition range on the current road based on the obtained road information to determine the probability of crossing of each target vulnerable road user.

[0097] Specifically, the content of the road information includes the driving speed, relative speed, acceleration, direction of the autonomous driving vehicle and the target vulnerable road user, the relative position between the autonomous driving vehicle and the target vulnerable road user, and the current signal light state, traffic sign recognition result, and possible behavior prediction.

[0098] By successfully deploying the trained binary classification model into the autonomous driving system, the crossing intention of the vulnerable road user is determined to perform automatic driving operations such as avoidance or stopping, thereby improving the safety and reliability of the autonomous driving system.

[0099] Further, referring to Figure 5 Deploy the trained binary classification model to the autonomous driving system, including:

[0100] Step 31: Serialize the trained binary classification model to binary model data through the inference engine.

[0101] Export the binary model from the training framework of the binary classification model, and convert the trained binary classification model into a format more suitable for deployment in the autonomous driving system through the inference engine, forming a binary model data with smaller size, faster loading speed and higher running efficiency.

[0102] The inference engine is responsible for performing the inference task of the model, that is, processing the real-time road information in the autonomous driving system through the loaded binary classification model and outputting the corresponding classification result. The inference engine is a bridge connecting the binary classification model and the actual application scenario. It enables the trained binary classification model to perform efficient inference on different hardware and software platforms.

[0103] The binary model data is binary classification model data saved in binary format, which is more compact and efficient than text or source code format. It can be directly read by a machine and only stores the weights, structure and other necessary information of the model without any human-readable tags or annotations.

[0104] Serializing the binary classification model into binary model data removes all unnecessary formatting and tags, saving resource space on the device where the autonomous driving system is located. Binary model data can be quickly processed by computer memory and processors, so loading binary model data is usually much faster than parsing text or source code format, which helps to reduce the delay during model inference. Binary model data can maintain a consistent format across different operating systems and hardware architectures, making it easy to deploy and run in multiple environments without additional conversion or modification, and is more cross-platform compatible.

[0105] Step 32: Deploy the binary model data to the autonomous driving system through the programming language.

[0106] After loading the binary model file using the C++ API provided by the inference engine, the road information data collected by the autonomous driving system, such as camera images, is preprocessed into the format expected by the binary model, and then the preprocessed data is passed to the inference engine for inference and output result acquisition, completing the deployment of binary model data on the autonomous driving system.

[0107] The autonomous driving vehicle can distinguish the crossing intention of vulnerable road users in the road through the binary model data in the autonomous driving system and respond in time to ensure the safety of the autonomous driving vehicle in the traffic environment.

[0108] Referring to Figure 6 , Figure 6 is a structural schematic diagram of an embodiment of the storage medium provided by the present application.

[0109] The storage medium 300 stores program data 301, which, when executed by a processor, implements the steps of the crossing intention discrimination method as Figure 1 described.

[0110] The program data 301 is stored in a storage medium 300, including a plurality of instructions for causing a network device (such as a router, a personal computer, a server, etc.) or a processor to execute all or part of the steps of the method described in various embodiments of the present application.

[0111] Optionally, the storage medium 300 can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0112] Referring to Figure 7 , Figure 7 is a structural schematic diagram of an embodiment of the vehicle-mounted device provided by the present application.

[0113] The vehicle-mounted device 400 is installed in a vehicle and is a computer-type device, which is linked with various sensing modules and control modules in the vehicle to realize visual perception, sensor data processing and image display process. The vehicle-mounted device 400 includes a processor 420 and a memory 410 connected with each other. The memory 410 stores a computer program, and the processor 420 implements the steps of the above-mentioned crossing intention discrimination method when executing the computer program.

[0114] Referring to Figure 8 , Figure 8 A vehicle 500 is provided, which includes the vehicle-mounted device 400 as described above. The vehicle 500 can complete the process of discriminating the crossing intention of the vulnerable road user to assist automatic driving through the control of the vehicle-mounted device 400 during driving.

[0115] Different from the prior art, the present application provides a crossing intention discrimination method, medium, device and vehicle. By screening the scenes with vulnerable road users from the driving data and marking positive example labels and negative example labels for binary classification model training, the trained binary classification model can judge whether the vulnerable road user has a crossing intention according to the action of the vulnerable road user, and slow down or avoid by the downstream automatic driving system. The intention discrimination of the vulnerable road user based on deep learning can accurately judge the crossing intention of the vulnerable road user and improve the accuracy. A large amount of learning data enables the binary classification model to discriminate the crossing intention of the vulnerable road user in different scenes; at the same time, learning new data can also help the binary classification model to be applied in scenes with different driving habits and traffic rules, which has generality.

[0116] The various embodiments in the specification are described in progressive manner, and the same or similar parts between the embodiments can be mutually referred to. Each embodiment focuses on the difference from other embodiments. In particular, the storage medium embodiments and the computer device embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0117] The present application can be used in many general or special-purpose vehicle-mounted computing system environments or configurations. For example: personal computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, distributed computing environments that include any of the above systems or devices, and the like.

[0118] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed.

[0119] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0120] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0121] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for crossing intention discrimination based on a deep learning model, characterized in that, The method comprises: constructing a data set comprising positive examples of vulnerable road users crossing in accordance with a first preset condition and negative examples of vulnerable road users crossing in accordance with a second preset condition; wherein the first preset condition is that the vulnerable road user has a crossing behavior in front of the ego vehicle within a region of interest of the ego vehicle and within a preset future time; and the second preset condition is that the vulnerable road user has a crossing behavior behind the ego vehicle within the region of interest of the ego vehicle and within a preset future time; training a preset binary classification model based on the data set, the binary classification model being built based on a deep learning model; deploying the trained binary classification model to an autonomous driving system to determine the crossing intention of the vulnerable road user.

2. The cross intention discrimination method according to claim 1, characterized by, The method of constructing a data set comprising positive examples of vulnerable road users crossing in accordance with a first preset condition and negative examples of vulnerable road users crossing in accordance with a second preset condition comprises: filtering out closed road scene data in driving data; traversing each vulnerable road user perceived by the ego vehicle in the driving data; labeling the positive examples of the vulnerable road users in accordance with the first preset condition and labeling the negative examples of the vulnerable road users in accordance with the second preset condition.

3. The cross intention discrimination method according to claim 2, characterized by, The method of constructing a data set comprising positive examples of vulnerable road users crossing in accordance with a first preset condition and negative examples of vulnerable road users crossing in accordance with a second preset condition further comprises: labeling the vulnerable road users not in accordance with the first preset condition and the second preset condition as negative examples in accordance with a preset proportion.

4. The cross intention discrimination method according to any one of claims 1 to 3, characterized by, The first preset condition further comprises that an acute angle between an extension direction of the trajectory of the vulnerable road user and an extension direction of the trajectory of the ego vehicle is greater than a preset angle.

5. The cross intention discrimination method according to claim 1, wherein The positive example comprises a first vulnerable road user and a positive example label, and the negative example comprises a second vulnerable road user and a negative example label. The method of training a preset binary classification model based on the data set comprises: obtaining a feature vector of a target vulnerable road user; the target vulnerable road user comprises the first vulnerable road user and / or the second vulnerable road user in the data set; and the feature vector is interaction information between the ego vehicle and the target vulnerable road user; inputting the feature vector of the target vulnerable road user, motion information of the target vulnerable road user at a historical time, and ego vehicle navigation information into the binary classification model to obtain a probability of the target vulnerable road user crossing; obtaining a label corresponding to the target vulnerable road user, and calculating an error between the probability of the target vulnerable road user crossing and the label, wherein the label is a positive example label or a negative example label corresponding to the target vulnerable road user; based on the error, modifying the binary classification model until the error between the probability and the label is less than a preset error threshold.

6. The cross intention discrimination method according to claim 5, wherein The binary classification model comprises a convolution kernel network layer and a linear network layer. The feature vector of the target vulnerable road user, the motion information of the vulnerable road user at the historical time, and the navigation information of the ego vehicle are input into the binary classification model to obtain a probability of the target vulnerable road user crossing. The feature vector of the target vulnerable road user, the motion information of the vulnerable road user at the historical time, and the navigation information of the ego vehicle are input into the convolution kernel network layer for learning and core feature extraction to obtain core features in the feature vector of the target vulnerable road user and the motion information at the historical time. The core features extracted by the convolution kernel network layer are input into the linear network layer for linear transformation to obtain the probability of the target vulnerable road user crossing.

7. The cross intention discrimination method of claim 1, wherein The trained binary classification model is deployed to an autonomous driving system, including: The trained binary classification model is serialized into binary model data by an inference engine. The binary model data is deployed to the autonomous driving system by a programming language.

8. A storage medium having stored thereon program data, characterized in that The program data, when executed by a processor, implements the steps of the crossing intention discrimination method according to any one of claims 1-7.

9. An in-vehicle device characterized by comprising: A processor and a memory are connected to each other, and the memory stores a computer program, and the processor executes the computer program to implement the steps of the crossing intention discrimination method according to any one of claims 1-7.

10. A vehicle characterized by comprising: The vehicle-mounted device according to claim 9.

Images