Transverse intention discrimination method, medium, equipment and vehicle

Abstract

The invention discloses a crossing intention judgment method, a medium, equipment and a vehicle. The crossing intention judgment method comprises the following steps: constructing a data set by using a vulnerable road user crossing positive example meeting a first preset condition and a vulnerable road user crossing negative example meeting a second preset condition; wherein the first preset condition is that the vulnerable road user has a passing behavior in front of the vehicle in the region of interest of the vehicle in the future preset time; the second preset condition is that the vulnerable road user is in the region of interest of the vehicle and has a passing behavior behind the vehicle in the future preset time; training a preset dichotomy model by adopting the data set, wherein the dichotomy model is built based on a deep learning model; and deploying the trained dichotomy model to the automatic driving system so as to judge the crossing intention of the vulnerable road user. Through the above mode, the judgment accuracy of the crossing intention of the vulnerable road user can be improved, and then the safety and comfort of passengers are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to a method, medium, device and vehicle for determining crossing intention. Background Art

[0002] The traditional crossing intention identification method based on trajectory prediction requires a calculation based on the current preliminary planned trajectory of the vehicle and the predicted trajectory of the object. It is highly dependent on the predicted trajectory and does not consider the interaction between vulnerable road users and the vehicle, resulting in inaccurate results for the crossing intention identification of vulnerable road users, thus affecting the driving planning of the vehicle and the safety and comfort of the occupants. Summary of the invention

[0003] The present application mainly provides a crossing intention judgment method, medium, equipment and vehicle to solve the problem of poor occupant safety and comfort caused by inaccurate crossing intention judgment of vulnerable road users.

[0004] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a method for judging crossing intention based on a deep learning model, including: constructing a data set with positive examples of crossing by vulnerable road users that meet a first preset condition, and negative examples of crossing by vulnerable road users that meet a second preset condition; wherein the first preset condition is that the vulnerable road user is in the area of ​​interest of the vehicle and has a crossing behavior in front of the vehicle within a preset time in the future; the second preset condition is that the vulnerable road user is in the area of ​​interest of the vehicle and has a crossing behavior behind the vehicle within a preset time in the future; using the data set to train a preset binary classification model, the binary classification model is built based on a deep learning model; deploying the trained binary classification model to the autonomous driving system to judge the crossing intention of the vulnerable road user.

[0005] By marking the positive and negative crossing labels of vulnerable road users in real driving data to construct a dataset and using the dataset for training a binary classification model, the accuracy of the binary classification model in identifying the crossing intentions of vulnerable road users can be further improved, thereby making reasonable route and speed planning and improving the safety and comfort of passengers.

[0006] In some embodiments, constructing a data set with positive examples of vulnerable road users crossing the road that meet a first preset condition and negative examples of vulnerable road users crossing the road that meet a second preset condition includes: filtering out closed road scene data in driving data; traversing each vulnerable road user perceived by the vehicle in the driving data; marking positive example labels for the vulnerable road users that meet the first preset condition, and marking negative example labels for the vulnerable road users that meet the second preset condition.

[0007] By filtering out high-speed scene data in the driving data segment and then marking the positive and negative labels, we can avoid data with no reference value from slowing down the labeling process and reducing the labeling time.

[0008] In some embodiments, constructing a data set using positive examples of vulnerable road users crossing the road that meet a first preset condition and negative examples of vulnerable road users crossing the road that meet a second preset condition further includes: marking the vulnerable road users that do not meet the first preset condition and the second preset condition as negative example labels according to a preset ratio.

[0009] By marking some vulnerable road users who do not meet the first preset condition and the second preset condition with negative labels, vulnerable road users on the road that are not related to the vehicle's movement can also be learned by the binary classification model, avoiding irrelevant vulnerable road users from being misjudged as positive examples, and further improving the accuracy of the binary classification model in distinguishing the crossing intention of vulnerable road users.

[0010] In some embodiments, the first preset condition also includes that an acute angle formed between an extension direction of the trajectory of the vulnerable road user and an extension direction of the trajectory of the vehicle is greater than a preset angle.

[0011] By identifying crossing behaviors from specific angles, the binary classification model can further learn the crossing behaviors of vulnerable road users during the training process, taking into account the deceleration or stopping behaviors that may occur during the crossing process of vulnerable road users.

[0012] In some embodiments, the crossing positive example includes a first vulnerable road user and a positive example label, and the crossing negative example includes a second vulnerable road user and a negative example label; the data set is used to train a preset binary classification model, including: obtaining a feature vector of a target vulnerable road user; the target vulnerable road user includes the first vulnerable road user and / or the second vulnerable road user in the data set; the feature vector is interaction information between the vehicle and the target vulnerable road user; the feature vector of the target vulnerable road user, the motion information of the vulnerable road user in historical time, and the navigation information of the vehicle are input into the binary classification model to obtain the probability of the target vulnerable road user crossing; the label corresponding to the target vulnerable road user is obtained, and the error between the probability of the vulnerable road user crossing and the label is calculated, wherein the label is a positive example label or a negative example label corresponding to the target vulnerable road user; the binary classification model is corrected based on the error until the error between the probability and the label is less than a preset error threshold.

[0013] The binary classification model is trained by obtaining data from driving data segments. The binary classification model can learn the next action trajectory of the vulnerable road user based on the motion information and interaction information of the vulnerable road user, and accurately determine whether the vulnerable road user will cross the road.

[0014] In some embodiments, the step of inputting the feature vector of the target vulnerable road user, the motion information of the vulnerable road user in historical time, and the self-vehicle navigation information into the binary classification model to obtain the probability of the target vulnerable road user crossing includes: inputting the feature vector of the target vulnerable road user, the motion information of the vulnerable road user in historical time, and the self-vehicle navigation information into the convolution kernel network layer for learning and core feature extraction to obtain the feature vector of the target vulnerable road user and the core features in the motion information in historical time; and inputting the core features extracted by the convolution kernel network layer into the linear network layer for linear transformation to obtain the probability of the target vulnerable road user crossing.

[0015] By learning and extracting core features through the convolution kernel network layer, the efficiency and accuracy of feature extraction are improved. It can be applied to diverse data and has high generalization, so that the binary classification model can have good performance in different scenarios. Through the efficient computing characteristics of the linear network layer, the linear transformation process can be completed efficiently, meeting the real-time computing needs of judging the crossing intention of vulnerable road users on the road.

[0016] In some embodiments, deploying the trained binary classification model to the autonomous driving system includes: serializing the trained binary classification model into binary model data through an inference engine; and deploying the binary model data to the autonomous driving system through a programming language.

[0017] By serializing the binary classification model into binary model data and then deploying it, it can directly correspond to machine code. When parsing, there is no need for complex syntax and format parsing like text format, which helps to greatly improve the parsing speed and speed up the model's reasoning speed.

[0018] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a storage medium on which program data is stored, and when the program data is executed by a processor, the steps of the crossing intention determination method as described above are implemented.

[0019] The beneficial effects of the storage medium refer to the beneficial effects of the above-mentioned crossing intention determination method, which will not be repeated here.

[0020] The present application also provides a vehicle-mounted device, including a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the crossing intention determination method as described above are implemented.

[0021] The vehicle-mounted device is a computer-type device. The beneficial effects of the vehicle-mounted device refer to the beneficial effects of the above-mentioned crossing intention judgment method, which will not be repeated here.

[0022] The present application also provides a vehicle, including the above-mentioned vehicle-mounted device. During the driving process of the vehicle, the vehicle can complete the above-mentioned step of determining the crossing intention of the vulnerable road user through the control of the vehicle-mounted device.

[0023] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses a method, medium, device and vehicle for judging the intention of crossing. By traversing driving data segments from real road driving data, marking crossing positive labels for vulnerable road users that meet the first preset condition in the driving data segments, and marking crossing negative labels for vulnerable road users that meet the second preset condition, and using them for training the binary classification model, the accuracy of the binary classification model in judging the intention of crossing of vulnerable road users can be further improved, thereby making reasonable route and speed planning and improving the safety and comfort of passengers. Further setting the first preset condition as being within the area of ​​interest of the vehicle and having crossing behavior in front of the vehicle in the future, the user who has crossed can be determined according to the behavior pattern of vulnerable road users within a period of time; further setting the second preset condition as being within the area of ​​interest of the vehicle and having crossing behavior behind the vehicle in the future, the user who has not crossed can be determined, so that after the binary classification model is trained, it can also accurately judge according to the behavior pattern of vulnerable road users within a period of time in different driving scenarios, and it has flexibility. After deploying the binary classification model to the autonomous driving system, the autonomous driving vehicle can accurately determine the crossing intention of vulnerable road users and decide whether to brake and give way to ensure safety, or rush through to ensure physical comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0025] Figure 1 It is a flowchart of an embodiment of a method for determining a crossing intention provided by the present application;

[0026] Figure 2Yes Figure 1 The flowchart of step 10 of an embodiment of a method for determining crossing intention based on a deep learning model is shown;

[0027] Figure 3 Yes Figure 1 The flowchart of step 20 of the method for determining the crossing intention is shown as an embodiment;

[0028] Figure 4 Yes Figure 3 The flowchart of step 22 of the crossing intention determination method is shown as an embodiment;

[0029] Figure 5 Yes Figure 1 The flowchart of step 30 of the method for determining the crossing intention is shown in FIG.

[0030] Figure 6 It is a structural schematic diagram of an embodiment of a storage medium provided by the present application;

[0031] Figure 7 It is a structural schematic diagram of an embodiment of the vehicle-mounted device provided by the present application;

[0032] Figure 8 It is a structural schematic diagram of an embodiment of an autonomous driving vehicle provided by the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0035] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] See also Figure 1 , Figure 1 : is a flow chart of an embodiment of a method for distinguishing crossing intention based on a deep learning model provided by the present application, and the method for distinguishing crossing intention includes:

[0037] Step 10: Construct a data set using positive examples of vulnerable road users crossing the road that meet the first preset condition and negative examples of vulnerable road users crossing the road that meet the second preset condition.

[0038] According to the vulnerable road users in the perceived traffic scenes, a first preset condition for identifying vulnerable road users with crossing behavior and a second preset condition for identifying vulnerable road users without crossing behavior are set, and vulnerable road users that meet the first preset condition are marked as positive examples, and vulnerable road users that meet the second preset condition are marked as negative examples. A data set is constructed with the positive vulnerable road user data and the negative vulnerable road user data.

[0039] Vulnerable road users are participants who are relatively vulnerable to harm in road traffic, including but not limited to pedestrians, cyclists, electric scooter riders, people pushing strollers, etc.

[0040] The first preset condition is that the vulnerable road user is in the area of ​​interest of the vehicle and crosses the road in front of the vehicle within a preset time in the future. If the vulnerable road user crosses the road in front of the vehicle within a few seconds in the future, the autonomous vehicle should immediately take safety measures, such as slowing down, stopping, and avoiding, to ensure the safety of pedestrians and vehicles.

[0041] Front crossing behavior includes the following situations: the trajectories of the ego vehicle and the vulnerable road user intersect within a preset time in the future, and the vulnerable road user passes the intersection before the ego vehicle; and the trajectories of the ego vehicle and the vulnerable road user do not intersect within a preset time in the future, but the vulnerable road user will pass in front of the ego vehicle.

[0042] Specifically, the length of the preset time in the future should be flexibly set according to actual conditions such as road type, vehicle speed, traffic flow, etc., to ensure that it is neither too sensitive to cause frequent misjudgments nor too slow to increase the risk of collision, such as: 2s, 3s, 5s, 8s, 10s, 15s or 17s, etc.

[0043] The Region of Interest (ROI) is the area that the autonomous vehicle sensor focuses on monitoring, and is usually adjusted dynamically based on factors such as the vehicle's speed, direction, and road environment. The ROI can include areas within a certain distance and angle range in front of the vehicle to ensure that potential traffic participants can be discovered and identified in a timely manner.

[0044] Optionally, the first preset condition also includes that an acute angle formed between an extension direction of the trajectory of the disadvantaged road user and an extension direction of the trajectory of the vehicle is greater than a preset angle.

[0045] When the acute angle between the extension direction of the vulnerable road user's trajectory and the extension direction of the ego vehicle's trajectory is greater than the preset angle, combined with the condition that the vulnerable road user is in the ego vehicle's area of ​​interest and has the intention to cross in front of the ego vehicle within the preset time in the future, it can be considered that there is a high risk of collision between the vulnerable road user and the ego vehicle. At this time, the autonomous vehicle should immediately initiate the corresponding safety response mechanism, such as deceleration, parking, avoidance, etc., to ensure the safety of the vulnerable road user and the vehicle itself.

[0046] Specifically, the preset angle may be 7°, 10°, 15°, 19°, 20°, 23° or 30°, etc.

[0047] By identifying crossing behaviors from specific angles, the binary classification model can further learn the crossing behaviors of vulnerable road users during the training process, while taking into account the possible deceleration or stopping behaviors of vulnerable road users during the crossing process.

[0048] The second preset condition is that the vulnerable road user is in the area of ​​interest of the ego vehicle and has the behavior of crossing behind the ego vehicle within the preset time in the future. If the trajectory of the vulnerable road user intersects with the ego vehicle within the preset time in the future, and the ego vehicle passes the intersection before the vulnerable road user, it means that the vulnerable road user is crossing from behind the ego vehicle and will not affect the driving of the ego vehicle.

[0049] Vulnerable road users who meet the first preset condition are taken as positive examples, and vulnerable road users who meet the second preset condition are taken as negative examples. A data set for identifying the crossing intention of vulnerable road users is constructed, and an effective model is trained to identify the crossing intention of vulnerable road users, which is conducive to improving the accuracy of autonomous driving vehicles in identifying the crossing intention of vulnerable road users.

[0050] Further, see Figure 2 , a dataset is constructed with positive examples of vulnerable road users crossing the road that meet the first preset condition and negative examples of vulnerable road users crossing the road that meet the second preset condition, including:

[0051] Step 11: Filter out closed road scene data from the driving data.

[0052] Closed roads are often located in non-public areas and can be identified by comparing geographic location information (such as longitude and latitude) with the public road database. After identifying the closed road scenes in the driving data, this part of the data is removed from the driving data. Since vulnerable road users are less likely to cross on closed roads, filtering out closed road scene data in driving data can shorten the subsequent labeling time, thereby increasing the speed of building the data set.

[0053] Step 12: Traverse each vulnerable road user perceived by the vehicle in the driving data.

[0054] Traverse the driving data in chronological order and check the data at each time point. For each time point, check all relevant sensor data to determine if there is a vulnerable road user. For each perceived vulnerable road user, record its relevant information, such as location (latitude and longitude or coordinates in the vehicle coordinate system), speed, direction, type (pedestrian or bicycle), and size.

[0055] Step 13: Marking a positive label for vulnerable road users that meet the first preset condition, and marking a negative label for vulnerable road users that meet the second preset condition.

[0056] In the traversed vulnerable road user information, a positive example label is marked as a training label for vulnerable road users with crossing intentions according to a first preset condition; and a negative example label is marked as a training label for vulnerable road users without crossing intentions according to a second preset condition.

[0057] By marking labels for binary classification model training, autonomous driving vehicles can more accurately identify potential traffic risks and take corresponding measures to ensure driving safety.

[0058] Optionally, step 13 further includes: marking vulnerable road users who do not meet the first preset condition and the second preset condition as negative example labels according to a preset ratio.

[0059] In the driving data, among the vulnerable road user data that is not marked as a crossing positive example or a crossing negative example because it does not meet the first preset condition and the second preset condition, a portion of the data is selected in proportion and marked as a crossing negative example.

[0060] By marking some vulnerable road users who do not meet the first preset condition and the second preset condition with negative labels, vulnerable road users on the road that are not related to the vehicle's movement can also be learned by the binary classification model, avoiding irrelevant vulnerable road users from being misjudged as positive examples, and further improving the accuracy of the binary classification model in distinguishing the crossing intention of vulnerable road users.

[0061] Step 20: Use the data set to train the preset binary classification model, which is built based on the deep learning model.

[0062] The dataset is used as input data and verification data to train the preset binary classification model to improve the accuracy of the binary classification model in distinguishing the crossing intention of vulnerable road users.

[0063] Use the deep learning framework to build a binary classification model, including designing the architecture of the binary classification model such as the input layer, hidden layer, and output layer, and specifying the loss function and evaluation indicators to improve the performance of the binary classification model. The binary classification model is built based on the deep learning model and is used to divide the input data samples into one of two categories.

[0064] Specifically, the types of binary classification models may include: logistic regression, which maps the output of a linear model to the interval (0, 1) through a function to obtain the probability of belonging to a certain category; decision trees, which classify data into one of two categories by learning a series of rules; neural networks, which can learn complex nonlinear relationships by stacking multiple layers, and use functions in the output layer to obtain the probability of belonging to a certain category, etc.

[0065] By training the binary classification model built based on the deep learning framework through the data set, the binary classification model can learn accurate and detailed recognition methods, thereby improving the performance of the binary classification model. At the same time, a large amount of training data can reduce the model's excessive dependence on training data, reduce the risk of overfitting, and be able to adapt to new data, thereby enhancing the generalization ability of the binary classification model.

[0066] The crossing positive example includes the first vulnerable road user and the positive example label, and the crossing negative example includes the second vulnerable road user and the negative example label.

[0067] Further, see Figure 3 , use the data set to train the preset binary classification model, including:

[0068] Step 21: Obtain the feature vector of the target vulnerable road user.

[0069] Select appropriate features based on problem requirements and data characteristics, extract features of target vulnerable road users from the data set through image processing algorithms or data analysis methods, convert the extracted features into numerical form to form feature vectors, which are used to represent various attributes and behavior patterns related to vulnerable road users.

[0070] Specifically, the target vulnerable road user includes the first vulnerable road user and / or the second vulnerable road user in the data set.

[0071] Furthermore, the feature vector is the interaction information between the vehicle and the target vulnerable road user, including: the basic position and motion characteristics of the target vulnerable road user such as position, speed, acceleration, direction of motion, etc.; environmental interaction characteristics such as distance from the vehicle, distance from road elements, path selection, etc.; appearance and behavior characteristics such as size and shape, posture, behavior pattern, etc.; time characteristics such as timestamp or time period; map interaction information, etc. Through the feature vector, the binary classification model can distinguish different types of vulnerable road users and preliminarily judge their possible movement patterns. For example, the gait characteristics of a pedestrian may indicate that he is approaching or moving away from the road, while the speed characteristics of a cyclist may reflect whether he intends to cross the road.

[0072] Step 22: Input the feature vector of the target vulnerable road user, the historical movement information of the vulnerable road user, and the self-vehicle navigation information into the binary classification model to obtain the probability of the target vulnerable road user crossing.

[0073] The label of each sample is whether the target vulnerable road user has crossed the road. This is a binary classification problem. A suitable binary classification model is selected for model architecture and parameter setting. The feature vector of the target vulnerable road user in the data set, the motion information of the vulnerable road user in historical time, and the self-vehicle navigation information are input into the binary classification model for training. During the training process, the model learns the mapping relationship between the input data and the label, and determines which training label the target vulnerable road user is closer to based on the characteristics of the input data.

[0074] Specifically, the historical movement information of vulnerable road users includes trajectory data, speed information, acceleration information, movement direction and behavior patterns of vulnerable road users such as pedestrians and cyclists. The historical movement information provides a detailed record of their past behavior, including position, speed, acceleration, movement direction, etc. By analyzing the trajectory changes, speed fluctuations and acceleration patterns of vulnerable road users, the binary classification model can identify the movement trends and intentions of vulnerable road users. For example, if a vulnerable road user is gradually approaching the edge of the road and slowing down, it is likely that he or she intends to cross the road.

[0075] The vehicle's navigation information includes the vehicle's own status and location information, as well as the vehicle's driving plan. The vehicle's navigation information provides the vehicle's own status and location information, as well as the vehicle's driving plan and perception of the surrounding traffic environment. By comparing the vulnerable road user's location with the vehicle's own driving route, the vehicle assesses whether the vulnerable road user is likely to enter the vehicle's driving path. At the same time, the vehicle can also combine information such as traffic light status, road signs and signs to further determine whether the vulnerable road user's crossing intention is legal and safe.

[0076] By integrating the appearance features, motion features and ego-vehicle navigation information of the target vulnerable road user, the binary classification model can capture the key factors affecting the target behavior and make accurate predictions based on them.

[0077] Specifically, the binary classification model includes a convolutional kernel network layer and a linear network layer.

[0078] The convolution kernel network layer slides over the input data through multiple convolution kernels and performs multiplication and addition operations at each position to extract the core features of the input data.

[0079] The linear network layer is located after the convolutional layer and is used to further process and classify the features extracted by the convolutional layer.

[0080] See also Figure 4 , the feature vector of the target vulnerable road user, the historical movement information of the vulnerable road user and the navigation information of the vehicle are input into the binary classification model to obtain the probability of the target vulnerable road user crossing, including:

[0081] Step 221: Input the feature vector of the target vulnerable road user, the motion information of the vulnerable road user in historical time, and the navigation information of the vehicle into the convolution kernel network layer for learning and core feature extraction, and obtain the feature vector of the target vulnerable road user and the core features in the motion information in historical time.

[0082] The feature vector of the target vulnerable road user, the historical motion information of the vulnerable road user and the navigation information of the vehicle are input into the convolution kernel network layer. The implicit features in the feature vector of the target vulnerable road user, the historical motion information of the vulnerable road user and the navigation information of the vehicle are used to improve the attention of the binary classification model to the target vulnerable road users within the identification range, save the computational cost of the model, and improve the data processing efficiency of the binary classification model.

[0083] The neural convolutional network automatically learns and extracts local features from the dataset to predict the short-term movement patterns or trends of target vulnerable road users for subsequent analysis, reducing the complexity and subjectivity of feature engineering and improving the generalization ability and accuracy of the model.

[0084] Step 222: The core features extracted by the convolution kernel network layer are input into the linear network layer for linear transformation to obtain the probability of the target vulnerable road user crossing.

[0085] All feature maps output by the convolutional layer are flattened into a one-dimensional vector, which contains all the core features extracted from the input data. The flattened one-dimensional vector is input to the linear network layer. Each neuron in the linear network layer is connected to each element in the input vector to perform a linear transformation. The result of the linear transformation is a new vector whose length depends on the number of neurons in the linear network layer. An activation function such as the sigmoid activation function is added after the linear network layer to map any real value to the interval (0, 1) and generate a floating point number between 0 and 1, which represents the probability of the target vulnerable road user crossing.

[0086] Through the efficient computing characteristics of the linear network layer, the linear transformation process can be completed efficiently, meeting the real-time computing needs of identifying the crossing intention of vulnerable road users.

[0087] Step 23: Obtain a label corresponding to the target vulnerable road user, and calculate the error between the probability of the vulnerable road user crossing and the label, where the label is a positive label or a negative label corresponding to the target vulnerable road user.

[0088] The positive or negative label of the dataset corresponding to the target vulnerable road user whose probability value is currently calculated is obtained, and the error between the probability and the label is calculated. The error is used to correct the binary classification model.

[0089] For example, the output of the binary classification model for vulnerable road user A is 0.32, which means that the probability that vulnerable road user A has the intention to cross is 0.32. In the data set, the positive label is assigned 1 and the negative label is assigned 0. Vulnerable road user A is marked as a negative label in the data set, so the difference between 0.32 and 0 is calculated and returned to update the data of the binary classification model.

[0090] For another example, the output of the binary classification model for vulnerable road user B is 0.89, which means that the probability that vulnerable road user B has the intention to cross is 0.89. Vulnerable road user B is marked as a positive example label in the data set, and the difference between 0.89 and 1 is calculated and returned to update the data of the binary classification model.

[0091] Step 24: Modify the binary classification model based on the error until the error between the probability and the label is less than the preset error threshold.

[0092] The error threshold is defined as the standard for evaluating the prediction ability of the binary classification model. When the error between the probability and the label is less than the preset error threshold, the binary classification model meets the training requirements. Based on the error, the parameters of the binary classification model are modified until the error between the probability and the label is less than the preset error threshold.

[0093] For example, the output of the binary classification model corresponding to vehicle C is 0.92, and the difference between the corresponding positive example label value 1 is 0.06. The parameters of the binary classification model are updated based on the difference of 0.08 until the output of the binary classification model corresponding to vehicle A is 0.95, which is less than the preset threshold value of 0.01. At this time, the binary classification model meets the training requirements.

[0094] By setting a preset threshold, the binary classification model can not only accurately fit the training data, but also generalize to new data, which is beneficial to improving the performance of the binary classification model.

[0095] Step 30: Deploy the trained binary classification model to the autonomous driving system to determine the crossing intention of the vulnerable road user.

[0096] A binary classification model is deployed in the autonomous driving system, and road information such as image type information or sensor type information is obtained through the autonomous driving vehicle. The binary classification model uses the obtained road information to perform abnormal stationary analysis on target vehicles within the recognition range on the current road to determine the probability of each target vulnerable road user crossing.

[0097] Specifically, the content of road information includes the driving speed, relative speed, acceleration, direction, relative position between the autonomous driving vehicle and the target vulnerable road user, as well as the current traffic light status, traffic sign recognition results, and possible behavior predictions.

[0098] By successfully deploying the trained binary classification model into the autonomous driving system, the crossing intention of vulnerable road users can be used to perform autonomous driving operations such as evasion or stopping, thereby improving the safety and reliability of the autonomous driving system.

[0099] Further, see Figure 5 , deploy the trained binary classification model into the autonomous driving system, including:

[0100] Step 31: Serialize the trained binary classification model into binary model data through the inference engine.

[0101] The binary model is exported from the training framework of the binary classification model, and the trained binary classification model is converted into a format more suitable for deployment in the autonomous driving system through the inference engine, forming binary model data with smaller size, faster loading speed and higher operating efficiency.

[0102] The inference engine is responsible for executing the model's inference task, that is, processing the real-time road information in the autonomous driving system through the loaded binary classification model and outputting the corresponding classification results. The inference engine is a bridge between the binary classification model and the actual application scenario, which enables the trained binary classification model to perform efficient inference on different hardware and software platforms.

[0103] Binary model data is binary classification model data saved in binary format, which is more compact and efficient than text or source code format. It can be directly read by machines and only stores the weights, structure, and other necessary information of the model without including any human-readable tags or annotations.

[0104] Serializing the binary classification model into binary model data removes all unnecessary formatting and markup, saving resource space on the device where the autonomous driving system is located. Binary model data can be directly processed by the computer's memory and processor, so loading binary model data is usually much faster than parsing text or source code formats, helping to reduce latency during model inference. Binary model data can maintain a consistent format on different operating systems and hardware architectures, and can be easily deployed and run in a variety of environments without additional conversion or modification, making it more cross-platform compatible.

[0105] Step 32: Deploy the binary model data to the autonomous driving system via a programming language.

[0106] After using the C++ API provided by the inference engine to load the binary model file, the road information data collected by the autonomous driving system, such as camera images, is preprocessed into the format expected by the binary model. The preprocessed data is then passed to the inference engine to perform inference and obtain the output results, completing the deployment of the binary model data on the autonomous driving system.

[0107] Autonomous driving vehicles can use the binary model data in the autonomous driving system to identify the crossing intentions of vulnerable road users on the road and respond in a timely manner to ensure the safety of autonomous driving vehicles in traffic environments.

[0108] See also Figure 6 , Figure 6 It is a structural diagram of an embodiment of the storage medium provided by the present application.

[0109] The storage medium 300 stores program data 301. When the program data 301 is executed by the processor, the following is achieved: Figure 1 The steps of the described crossing intention determination method.

[0110] The program data 301 is stored in a storage medium 300, and includes a number of instructions for enabling a network device (such as a router, a personal computer, a server, or other network device) or a processor to execute all or part of the steps of the method described in each embodiment of the present application.

[0111] Optionally, the storage medium 300 may be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or other medium that can store program data.

[0112] See also Figure 7 , Figure 7 It is a structural schematic diagram of an embodiment of the vehicle-mounted device provided by the present application.

[0113] The vehicle-mounted device 400 is installed in the vehicle and is a computer-type device. It works in conjunction with various sensor modules and control modules in the vehicle to achieve visual perception, sensor data processing, and image display. The vehicle-mounted device 400 includes a processor 420 and a memory 410 that are interconnected. The memory 410 stores a computer program. When the processor 420 executes the computer program, the steps of the above-mentioned crossing intention judgment method are implemented.

[0114] See also Figure 8 , Figure 8 A vehicle 500 is provided, comprising the above-mentioned vehicle-mounted device 400. During driving, the vehicle 500 can complete the process of determining the crossing intention of vulnerable road users through the control of the vehicle-mounted device 400 to assist in automatic driving.

[0115] Different from the prior art, the present application provides a method, medium, device and vehicle for judging the intention to cross. By screening out scenes with vulnerable road users in the driving data, and marking positive and negative labels for binary classification model training, the trained binary classification model can judge whether the vulnerable road users have the intention to cross based on their actions, and the downstream autonomous driving system will slow down or avoid them. Vulnerable road user intention judgment based on deep learning can accurately judge the crossing intention of vulnerable road users and improve accuracy. A large amount of learning data enables the binary classification model to judge the crossing intention of vulnerable road users in different scenarios; at the same time, learning new data can also help the binary classification model be applied in scenarios with different driving habits and traffic rules, and it is versatile.

[0116] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium embodiment and the computer device embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0117] The present application can be used in many general or special vehicle-mounted computing system environments or configurations, such as personal computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, and distributed computing environments including any of the above systems or devices.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not performed.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for identifying crossing intention based on a deep learning model, characterized in that: include: A data set is constructed using positive examples of vulnerable road users crossing the road that meet the first preset condition and negative examples of vulnerable road users crossing the road that meet the second preset condition; The first preset condition is that the vulnerable road user is in the area of ​​interest of the vehicle and crosses in front of the vehicle within a preset time in the future; the second preset condition is that the vulnerable road user is in the area of ​​interest of the vehicle and crosses behind the vehicle within a preset time in the future; The data set is used to train a preset binary classification model, where the binary classification model is built based on a deep learning model; The trained binary classification model is deployed to the autonomous driving system to determine the crossing intention of the vulnerable road user.

2. The method for determining the crossing intention according to claim 1, characterized in that: The constructing of a data set using positive examples of vulnerable road users crossing the road that meet the first preset condition and negative examples of vulnerable road users crossing the road that meet the second preset condition includes: Filter out closed road scene data from driving data; Traversing each of the vulnerable road users perceived by the vehicle in the driving data; The vulnerable road users meeting the first preset condition are marked with positive labels, and the vulnerable road users meeting the second preset condition are marked with negative labels.

3. The method for determining the crossing intention according to claim 2, characterized in that: The constructing of a data set using positive examples of vulnerable road users crossing the road that meet the first preset condition and negative examples of vulnerable road users crossing the road that meet the second preset condition also includes: The vulnerable road users who do not meet the first preset condition and the second preset condition are marked as negative example labels according to a preset ratio.

4. The method for determining the crossing intention according to any one of claims 1 to 3, characterized in that: The first preset condition also includes that an acute angle formed between an extension direction of the trajectory of the vulnerable road user and an extension direction of the trajectory of the vehicle is greater than a preset angle.

5. The method for determining the crossing intention according to claim 1, characterized in that: The crossing positive example includes a first vulnerable road user and a positive example label, and the crossing negative example includes a second vulnerable road user and a negative example label; The preset binary classification model is trained using the data set, including: Acquire a feature vector of a target vulnerable road user; the target vulnerable road user includes the first vulnerable road user and / or the second vulnerable road user in the data set; the feature vector is interaction information between the vehicle and the target vulnerable road user; Inputting the feature vector of the target vulnerable road user, the historical movement information of the vulnerable road user and the self-vehicle navigation information into the binary classification model to obtain the probability of the target vulnerable road user crossing; Acquire a label corresponding to the target vulnerable road user, and calculate an error between the probability of the vulnerable road user crossing and the label, wherein the label is a positive label or a negative label corresponding to the target vulnerable road user; The binary classification model is modified based on the error until the error between the probability and the label is less than a preset error threshold.

6. The method for determining the crossing intention according to claim 5, characterized in that: The binary classification model includes a convolutional kernel network layer and a linear network layer; The step of inputting the feature vector of the target vulnerable road user, the historical movement information of the vulnerable road user and the self-vehicle navigation information into the binary classification model to obtain the probability of the target vulnerable road user crossing the road comprises: Inputting the feature vector of the target vulnerable road user, the motion information of the vulnerable road user in historical time, and the navigation information of the vehicle into the convolution kernel network layer for learning and core feature extraction, and obtaining the feature vector of the target vulnerable road user and the core features in the motion information in historical time; The core features extracted by the convolution kernel network layer are input into the linear network layer for linear transformation to obtain the probability of the target vulnerable road user crossing.

7. The method for determining the crossing intention according to claim 1, characterized in that: The step of deploying the trained binary classification model to the autonomous driving system includes: Serializing the trained binary classification model into binary model data through an inference engine; Deploy the binary model data to the autonomous driving system via a programming language.

8. A storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the steps of the abnormal stationary vehicle identification method according to any one of claims 1 to 7 are implemented.

9. A vehicle-mounted device, characterized in that: The method comprises a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the abnormal stationary vehicle identification method as described in any one of claims 1 to 8 are implemented.

10. A vehicle, characterized in that: Comprising the vehicle-mounted device as claimed in claim 9.

Images