Vehicle starting intention prediction method and device, terminal equipment and storage medium

A deep learning model enhances vehicle starting intention prediction by analyzing motion data fluctuations before and after starting, addressing the inaccuracy of current models and improving autonomous driving performance.

CN119058673APending Publication Date: 2024-12-03HAOMO TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310637124.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Current models are unable to accurately predict the starting intention of vehicles due to small fluctuations in motion data before starting and low feature differentiation, which hinders accurate prediction in autonomous driving scenarios.

Method used

A method utilizing a deep learning model that considers the distinct fluctuations in motion data of a target vehicle before and after starting, incorporating historical motion data of multiple vehicles to enhance prediction accuracy by distinguishing between stationary and starting states.

Benefits of technology

Improves the accuracy of predicting vehicle starting intention by leveraging the deep learning model to differentiate between stationary and starting states, enabling better anticipation of vehicle movements in autonomous driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119058673A_ABST
    Figure CN119058673A_ABST
Patent Text Reader

Abstract

The invention is suitable for the field of automobile automatic driving, and provides a target vehicle starting intention prediction method and device, terminal equipment and a storage medium, and the prediction method comprises the steps: obtaining first motion data of an unstarted target vehicle at each moment within a set duration from a current moment; inputting the first motion data into a preset starting intention prediction model, and outputting a starting prediction result of the target vehicle by the starting intention prediction model; the starting intention prediction model is formed by training historical motion data of a plurality of vehicles within a set duration before starting. Through the scheme, prediction of the starting intention of the target vehicle is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of automobile autonomous driving, and in particular to a method, apparatus, terminal device and storage medium for predicting vehicle starting intention. Background Art

[0002] Accurate start intention prediction is of great significance to the development of the field of autonomous driving. It can further solve the problems of lagging reaction of following vehicles in autonomous driving, especially the start intention prediction of the target vehicle in the traffic light scene, which can obtain the start intention of the vehicle in front of the autonomous driving in advance, and help the autonomous driving vehicle to follow the vehicle in time and improve comfort. Figure 1 Generally refers to the intention of a stationary vehicle to start moving away from its current position. Currently, due to the small fluctuation of the vehicle's motion data before starting and the low feature differentiation, it is currently impossible to use the model to accurately predict the starting intention, which has many shortcomings. Summary of the invention

[0003] The purpose of this application is to provide a method, device, terminal equipment and storage medium for predicting vehicle starting intention, aiming to solve the problem that the current model cannot be used to accurately predict starting intention.

[0004] A first aspect of an embodiment of the present application provides a method for predicting a target vehicle's starting intention, the method comprising:

[0005] Obtain the first motion data of the target vehicle that has not started at each moment within a set time from the current moment;

[0006] The first motion data is input into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle; the starting intention prediction model is formed by training historical motion data of multiple vehicles within a set time period before starting.

[0007] In an optional embodiment, the prediction method further includes:

[0008] Acquire the second motion data of at least one of the vehicles closest to and farthest from the target vehicle at each moment within a set time period from the current moment within the user vehicle detection range;

[0009] The second motion data is input into the starting intention prediction model, and the starting intention prediction model inputs the first motion data and the second motion data, and outputs a starting prediction result of the target vehicle.

[0010] In an optional embodiment, before obtaining the motion data of the target vehicle that has not started at each moment within a set time period from the current moment, the prediction method further includes:

[0011] Acquire historical movement data of multiple vehicles at each moment in a preset historical time period; the preset historical time period is before the vehicles complete starting;

[0012] Split the preset historical time period into multiple sub-time periods, and the length of each sub-time period is the set length;

[0013] According to the manually input start marking results of each vehicle in each sub-time period, the historical movement data of each vehicle in each sub-time period are marked;

[0014] The manually input start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are composed of a training data, which is input into a deep learning model to obtain the start intention prediction model; wherein the start labeling results of each vehicle in each sub-time period are used as the output of the deep learning model, and the corresponding historical motion data are used as the input of the deep learning model.

[0015] In an optional embodiment, before obtaining the motion data of the target vehicle that has not started at each moment within a set time period from the current moment, the prediction method further includes:

[0016] Acquire historical movement data of multiple vehicles at each moment in a preset historical time period; the preset historical time period is before the vehicles complete starting;

[0017] Split the preset historical time period into multiple sub-time periods, and the length of each sub-time period is the set length;

[0018] According to the historical movement data of each vehicle in each sub-time period, combined with the preset start time interval, determine the start marking result corresponding to the historical movement data of each vehicle in each sub-time period;

[0019] The start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are composed of a training data, which is input into a deep learning model to obtain the start intention prediction model; wherein the start labeling results of each vehicle in each sub-time period are used as the output of the deep learning model, and the corresponding historical motion data are used as the input of the deep learning model.

[0020] In an optional embodiment, determining the start marking result corresponding to the historical motion data of each vehicle in each sub-time period according to the historical motion data of each vehicle in each sub-time period in combination with the preset start time interval includes:

[0021] According to the preset first time window and the second time window, in combination with the motion data, the preset first frame number and the second frame number, starting from the first time frame in each of the sub-time periods, determining whether each frame is a start frame frame by frame;

[0022] Determine the start time interval of each vehicle according to the start frame and the first time frame in each sub-time period;

[0023] For each vehicle, if the corresponding start time interval is longer than the preset start time interval, the corresponding sub-time interval of the vehicle is marked as start, and the other sub-time intervals of the vehicle are marked as not start.

[0024] In an optional embodiment, the motion data includes speed data and displacement data, and the step of determining whether each frame is a startup frame frame by frame starting from the first time frame in each of the sub-time periods according to the preset first time window and the second time window, in combination with the motion data, the preset first frame number and the second frame number, includes:

[0025] Starting from the first time frame in each sub-time period, determine frame by frame whether the speed of all frames in the first time window is higher than a first set speed value, whether the speed of a set number of frames in the second time window is higher than a second set speed value, and whether the displacement in the second time window is higher than a set displacement value;

[0026] If both are yes, then the currently judged time frame is determined to be the startup frame.

[0027] In an optional embodiment, before acquiring the motion data of the target vehicle that has not started at each moment within a set time period from the current moment, the method includes:

[0028] For all vehicles within the user vehicle detection range, if the displacement data of the vehicle is within a first preset threshold and the speed data is within a second preset threshold, the vehicle is determined as a target vehicle that has not started.

[0029] A second aspect of an embodiment of the present application provides a device for predicting a target vehicle's starting intention, the device comprising:

[0030] A data acquisition module, used to acquire the first motion data of the target vehicle that has not started at each moment within a set time from the current moment;

[0031] A starting intention prediction module is used to input the first motion data into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle; the starting intention prediction model is formed by training historical motion data of multiple vehicles within a set time period before starting.

[0032] A third aspect of an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0033] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0034] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the method described in the first aspect.

[0035] Beneficial effects of this application

[0036] A method for predicting vehicle starting intention provided by the present application is adopted, firstly, first motion data of a target vehicle that has not started at each moment within a set time length from the current moment is obtained, and then the first motion data is input into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle, thereby making use of the fact that the fluctuation change of its motion data within the set time length before the target vehicle starts is different from the fluctuation change of the motion data in the "stationary state" without starting intention, and the motion data in the "stationary state" and from the start to the end of starting are both used as inputs of a deep learning model, so that the deep learning model takes into account the difference characteristics between the two inputs when predicting, thereby improving the accuracy of the prediction model, and further forming a solution for predicting starting intention using a deep learning model, which solves the problem that the current model cannot accurately predict starting intention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 One of the flowcharts of a method for predicting the starting intention of a target vehicle provided in an embodiment of the present application;

[0039] Figure 2 A second flowchart of a method for predicting a target vehicle's starting intention provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of the structure of a device for predicting the starting intention of a target vehicle provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0043] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0044] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0045] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0046] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0047] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0048] It should be understood that the size of the serial numbers of the steps in this embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0049] Starting intention Figure 1 Generally refers to the intention that a vehicle in a stationary state may generate a starting action, thereby driving away from the current position. At present, due to the small fluctuation of the vehicle's motion data before starting and the low feature differentiation, it is currently impossible to use a model to accurately predict the starting intention. Based on the above, the inventor of the present application has found that within the set time before the target vehicle starts, the fluctuation of its motion data is different from the fluctuation of the motion data of the "stationary state" without the intention of starting. The motion data in the "stationary state" and "start to end of start" are both used as inputs of the deep learning model, so that the deep learning model can consider the difference between the two inputs when predicting, thereby improving the accuracy of the prediction model. In view of this, the present application proposes a method for predicting the starting intention of the target vehicle. According to the motion data of the target vehicle at each moment within the set time from the current moment, combined with the starting intention prediction model obtained by training based on multiple vehicle historical motion data, the vehicle's starting intention can be effectively predicted, solving the problem that the current model cannot accurately predict the starting intention.

[0050] The scenario of the present application includes a target vehicle, a user vehicle, and in a preferred embodiment, other vehicles within the detection range of the user vehicle.

[0051] In this scenario, the target vehicle can be the vehicle closest to the user vehicle in the direction of travel of the user vehicle, or it can be the vehicle that hinders the travel of the user vehicle in the direction of travel of the user vehicle. Since the sensing direction of the sensor of a general vehicle is the front and rear direction of the vehicle, the user vehicle is generally the vehicle behind the target vehicle, or the vehicle in front of the target vehicle. This application does not elaborate on this. It can be understood that for the sensor whose sensing direction is the left and right direction of the vehicle, or a global sensor, such as an image sensor, the user vehicle is not limited to whether it is the front vehicle or the rear vehicle of the target vehicle, and this application does not impose any restrictions on this.

[0052] For example, the present application can be used in lane scenarios, where the target vehicle, the user vehicle and other vehicles are all in the same lane. The user vehicle determines whether the front vehicle or the rear vehicle has the intention to start, so that predictive operations such as early starting or braking can be performed. The present application can also be used in lane scenarios that are not in the same lane. For example, by determining whether vehicles in different lanes have the intention to start, it can assist in lane changing. Or for non-lane scenarios, the present application is also applicable. For example, in parking scenarios, it can be determined whether the target vehicle at the detected position has the intention to start, so as to facilitate occupying the parking space of the target vehicle after the target vehicle leaves. For example, the other vehicles are the first vehicle or the last vehicle.

[0053] In the scenario of the present application, the motion data of the target vehicle is detected by the detection sensor of the user's vehicle, so that the motion data of the target vehicle can be input into a deep learning model based on the motion data of the target vehicle, and the result of whether the target vehicle has started can be obtained through the model.

[0054] Specifically, the user vehicle of the present application obtains the motion data of the target vehicle at each moment within a set time period from the current moment through a detection sensor. Since the starting intention prediction model can distinguish the characteristic differences between the changes in motion data obtained by the sensor when the target vehicle is stationary and the changes in motion data obtained by the sensor when the target vehicle starts, it can accurately predict whether the target vehicle has the intention to start.

[0055] In the present application, the motion data may be speed data and distance data, or speed data and coordinate data, wherein the sensor may be a lidar, which first detects the speed and distance of the target vehicle, and then calculates the speed and displacement of the target vehicle through the vehicle's processor, such as an MCU. In other preferred embodiments, trajectory data may also be generated through the lidar's detection data to facilitate user visualization, etc. This application does not impose any restrictions on this.

[0056] The specific implementation methods of the present application are described in detail below.

[0057] Reference Figure 1 , shows one of the flow charts of a method for predicting the starting intention of a target vehicle provided in an embodiment of the present application.

[0058] Step 101, obtaining first motion data of a target vehicle that has not started at each moment within a set time period from the current moment.

[0059] Among them, the first motion data includes the speed data and displacement data of the vehicle. Correspondingly, the first motion data can be acquired and processed by sensors installed on the user's vehicle, such as: laser radar, camera, millimeter wave radar, etc.

[0060] Exemplarily, a laser radar mounted on a user's vehicle transmits a detection signal (laser beam) to a target vehicle, and the received signal reflected from the target vehicle is compared with the transmitted signal. After appropriate processing, the speed data and displacement data of the target vehicle can be obtained.

[0061] Among them, the set duration can be a set duration before the current moment. In the embodiment of the present application, a set duration of 2s is taken as an example for detailed explanation, that is, 2s before the current moment.

[0062] Among them, the target vehicle that has not started can be determined as follows: for all vehicles within the detection range of the user's vehicle, if the displacement data of the vehicle is within a first preset threshold and the speed data is within a second preset threshold, the vehicle is determined as a target vehicle that has not started.

[0063] For example, in this application, vehicles that meet the following conditions are considered to be vehicles that have not started: within 2 seconds before the current moment, 2 seconds corresponds to 20 frames, that is, 2 seconds is represented by the [-19,0] interval, the speed set in the [-4,0] interval is less than 1m / s, and the displacement in the [-19,0] interval is less than 3m.

[0064] Exemplarily, the user vehicle obtains the speed data and displacement data of the target vehicle that meets the non-starting condition at each moment within 2 seconds before the current moment through the laser radar carried by the user vehicle.

[0065] In this embodiment of the present application, by acquiring the motion data of the target vehicle at each moment within a set time period from the current moment, the specific characteristics of the motion data and the difference between the motion data of the vehicle that has not started can be obtained, so as to facilitate the subsequent input into the starting intention prediction model for prediction.

[0066] Step 102, inputting the first motion data into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle; the starting intention prediction model is formed by training the historical motion data of multiple vehicles within a set time period before starting.

[0067] Exemplarily, the speed data and displacement data of the target vehicle at each moment within 2 seconds before the current moment are input into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle.

[0068] In some preferred embodiments of the present application, other non-target vehicles within the detection range of the user's vehicle can be combined to assist in predicting the start intention of the target vehicle, thereby utilizing the surrounding environment to improve the prediction accuracy of the start intention. In these embodiments, the prediction method of the present application may further include:

[0069] S101: within the detection range of the user's vehicle, obtaining second motion data of at least one of the vehicles closest to and farthest from the target vehicle at each moment within a set time period from the current moment;

[0070] S102: Inputting the second motion data into the starting intention prediction model, the starting intention prediction model inputs the first motion data and the second motion data, and outputs a starting prediction result of the target vehicle.

[0071] Among them, the second motion data may include speed data and displacement data, which may be the speed data and displacement data of the vehicle closest to the target vehicle, the speed data and displacement data of the vehicle farthest from the target vehicle, or the speed data and displacement data of the vehicle closest to and farthest from the target vehicle.

[0072] Exemplarily, within the detection range of the user's vehicle, the speed data and displacement data of the vehicle closest to the target vehicle are obtained through a laser radar, and the speed data and displacement data of the vehicle closest to the target vehicle and the speed data and displacement data of the target vehicle are input into the starting intention prediction model to obtain the starting prediction result of the target vehicle. The above-mentioned acquisition of the motion data of the vehicle closest to the target vehicle can assist in predicting the starting intention of the target vehicle. If the vehicle closest to the target vehicle is in a state of not starting, then the target vehicle is more likely to remain in a state of not starting, and vice versa. For example, in the scene of a traffic light, when the red light switches to the green light, if the vehicle in front of the target vehicle and closest to the target vehicle has the intention to start, then the possibility of the target vehicle having the intention to start will be greatly increased.

[0073] Exemplarily, within the detection range of the user's vehicle, the speed data and displacement data of the vehicle farthest from the target vehicle are obtained through the laser radar, and the speed data and displacement data of the vehicle farthest from the target vehicle and the speed data and displacement data of the target vehicle are input into the starting intention prediction model to obtain the starting prediction result of the target vehicle. The above-mentioned acquisition of the motion data of the vehicle farthest from the target vehicle within the detection range can also assist in predicting the starting intention of the target vehicle. If the vehicle farthest from the target vehicle is in a state of not starting, then the target vehicle is also likely to remain in a state of not starting, and vice versa. For example, in the scene of a traffic light, when the red light switches to the green light, if the vehicle in front of the target vehicle in the same lane and the vehicle farthest from the target vehicle within the detection range remains in a state of not starting, then the possibility of the target vehicle behind having the intention to start will be reduced.

[0074] Exemplarily, within the detection range of the user's vehicle, the speed data and displacement data of the vehicles closest to and farthest from the target vehicle are obtained by a laser radar, and the speed data and displacement data of the vehicles closest to and farthest from the target vehicle, as well as the speed data and displacement data of the target vehicle, are input into a starting intention prediction model to obtain a starting prediction result of the target vehicle.

[0075] In the implementation method that can be realized in the present application, before obtaining the motion data of the target vehicle, it is necessary to construct a number of training samples to train the deep learning model to obtain a start intention prediction model. In the process of constructing the training samples, it is necessary to input the start annotation results corresponding to the historical motion data of the vehicle. The input operation can be performed manually or automatically by computer. The steps corresponding to the manual input method include:

[0076] S201: Acquire historical movement data of multiple vehicles at each moment in a preset historical time period, where the preset historical time period is before the vehicles complete starting;

[0077] S202: Split the preset historical time period into multiple sub-time periods, and the length of each sub-time period is the set length;

[0078] S203: marking the historical movement data of each vehicle in each sub-time period according to the manually input start marking results of each vehicle in each sub-time period;

[0079] S204: The manually input start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are combined into a training data, which is input into a deep learning model to obtain a start intention prediction model.

[0080] In this implementation, the start labeling results of each vehicle in each sub-time period are used as the output of the deep learning model, and the corresponding historical motion data are used as the input of the deep learning model.

[0081] Among them, the multiple vehicles may include an obstacle vehicle, a vehicle in front of the obstacle vehicle and closest to the obstacle vehicle, and a vehicle that is farthest from the obstacle vehicle within the user's detection range and belongs to the same lane as the obstacle vehicle. The above-mentioned obstacle vehicles are not limited to vehicles that actually constitute an obstacle to the user's vehicle. Only the historical motion data of the obstacle vehicle is used as training data for training the deep learning model to train a starting intention prediction model.

[0082] Among them, the historical motion data of each moment in the preset historical time period, for example, a current moment is set as ts in the historical time, and correspondingly, the preset historical time period includes: αs before the current moment ts, βs after the current moment ts and the current moment ts, (0≤αs≤5s, 0≤βs≤10s). Exemplarily, with a time interval of 0.1s as a moment, αs is set to 2s, βs is set to 4s, that is, the historical motion data of each moment in the preset historical time period is the historical motion data corresponding to every 0.1s in the preset historical time period of 6s.

[0083] Among them, the time length of each sub-time period is the set time length. For example, the set time length is determined to be 2s. Correspondingly, the preset historical time period of 6s is divided into 3 sub-time periods, represented as [-20,0], [0,20] and [20,40] (0 represents the current time).

[0084] Exemplarily, historical motion data corresponding to each 0.1s of a preset historical time period of 6s is obtained for an obstacle vehicle, a vehicle in front of the obstacle vehicle and closest to the obstacle vehicle, and a vehicle within a user detection range that is farthest from the obstacle vehicle and belongs to the same lane as the obstacle vehicle, the preset historical time period of 6s is determined to be 2s, and the preset historical time period of 6s is divided into three sub-time periods, represented as [-20,0], [0,20] and [20,40] (0 represents the current moment); the historical motion data of each vehicle in each sub-time period is annotated according to the manually input start labeling result of each vehicle in each sub-time period; the manually input start labeling result of each vehicle in each sub-time period and the corresponding historical motion data are combined into a training data, which is input into a deep learning model to obtain a start intention prediction model; wherein the start labeling result of each vehicle in each sub-time period is used as the output of the deep learning model, and the corresponding historical motion data is used as the input of the deep learning model.

[0085] In another implementation that can be implemented in the present application, the steps corresponding to the computer automatic input method include:

[0086] S301: Acquire historical movement data of multiple vehicles at each moment in a preset historical time period; the preset historical time period is before the vehicles complete starting;

[0087] S302: Split the preset historical time period into multiple sub-time periods, and the length of each sub-time period is the set length;

[0088] S303: Determine the start marking result corresponding to the historical movement data of each vehicle in each sub-time period according to the historical movement data of each vehicle in each sub-time period and the preset start time interval;

[0089] S304: The start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are combined into a training data, and input into the deep learning model to obtain the start intention prediction model;

[0090] In this implementation, the start labeling result of each vehicle in each sub-time period is used as the output of the deep learning model, and the corresponding historical motion data is used as the input of the deep learning model.

[0091] The starting labeling results of the historical motion data of each vehicle may be determined by the following steps:

[0092] S401: According to the preset first time window and the second time window, in combination with the motion data, the preset first frame number and the second frame number, starting from the first time frame in each of the sub-time periods, determining whether each frame is a start frame frame by frame;

[0093] S402: Determine the start time interval of each vehicle according to the start frame and the first time frame in each sub-time interval; for each vehicle, if the corresponding start time interval is longer than the preset start time interval, mark the corresponding sub-time interval of the vehicle as start, and mark the other sub-time intervals of the vehicle as not started.

[0094] Exemplarily, the first time window is set to [t, t+2], the second time window is set to [t, t+29], the first frame number is 2, the second frame number is 15, and the preset startup interval is 2s. Taking the sub-time period [0, 20] as an example, starting from the first time frame 0 in the sub-time period, each frame is judged frame by frame whether it is a startup frame.

[0095] The following steps may be used to determine the startup frame:

[0096] S501: Starting from the first time frame in each sub-time period, determine frame by frame whether the speed of all frames in the first time window is higher than a first set speed value, whether the speed of a set number of frames in the second time window is higher than a second set speed value, and whether the displacement in the second time window is higher than a set displacement value;

[0097] S502: If both are yes, determine that the currently determined time frame is the startup frame.

[0098] Exemplarily, the first set speed value is 1m / s, the second set speed value is 2m / s, and the set displacement value is 5m. Correspondingly, taking the sub-time period [0,20] as an example, starting from the first time frame 0 in the sub-time period, frame by frame judgment, if the speed of all frames in the first time window [t,t+2] is greater than the first set speed value of 1m / s, and in the second time window [t,t+29], more than 15 frames have speeds higher than the second set speed value of 2m / s, and the displacement in the second time window [t,t+29] is higher than the set displacement value of 5m, then time t can be determined as the start time, and each frame in [t,t+29] is a start frame.

[0099] Exemplarily, the vehicle start time distance is calculated as: start time distance = t-ts, where ts represents the current time and t represents the start time. For each vehicle, if the corresponding start time distance is longer than the preset start time distance, the corresponding sub-time period of the vehicle is marked as 1 (1 represents start), and the other sub-time periods of the vehicle are marked as 0 (0 represents not start).

[0100] Exemplarily, the manually input start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are combined into a training data, which is input into a deep learning model to train and obtain a start intention prediction model.

[0101] Among them, the deep learning model can be an XGBoost traditional machine learning model.

[0102] For example, taking the sub-time period [0,20] as an example, the start-up annotation results of the obstacle vehicle, the vehicle in front of the obstacle vehicle and closest to the obstacle vehicle, and the vehicle farthest from the obstacle vehicle within the user detection range and in the same lane as the obstacle vehicle in the sub-time period [0,20] and the corresponding historical motion data are composed of a training data, which is input into the XGBoost deep learning model to train and obtain the start intention prediction model. Among them, the start-up annotation results of each vehicle in each sub-time period are used as the output of the deep learning model, and the corresponding historical motion data are used as the input of the deep learning model. The input historical motion data is as follows:

[0103] [agent_speed_0,...,agent_speed_-9,agent_pos_0,...,agent_pos_-9,

[0104] min_speed_0,...,min_speed_-9,min_pos_0,...,min_pos_-9,

[0105] max_speed_0,...,max_speed_-9,max_pos_0,...,max_pos_-9]

[0106] Among them, the suffix 0 represents the current moment, -9 represents the historical moment 2s, agent represents the obstacle vehicle, speed represents the speed data, pos represents the displacement data, min represents the vehicle in front of the obstacle vehicle and closest to the obstacle vehicle, and max represents the vehicle farthest from the obstacle vehicle within the user's detection range and in the same lane as the obstacle vehicle. The construction of the above training data and the starting intention prediction model obtained by training can achieve the purpose of predicting the vehicle's starting intention by utilizing the small fluctuations in the speed data and displacement data in the time period before the vehicle completes the start.

[0107] In another implementation that can be implemented in the present application, the construction of the training sample can also be carried out through the following steps:

[0108] S601: Obtaining a manual calibration result of the prediction result, and if the manual calibration result is inconsistent with the prediction result, marking the starting intention state of the target vehicle as the manual calibration result;

[0109] S602: Generate a training sample based on the motion data of the target vehicle at each moment within a set time from the current moment and the manual calibration result, and train the start intention prediction model.

[0110] In this implementation, using manual calibration results and motion data to generate training samples to train the start intention prediction model can gradually improve the results of model prediction and improve the accuracy of model prediction.

[0111] In other embodiments that can be implemented in this application, refer to Figure 2 , showing the second flowchart of a method for predicting the starting intention of a target vehicle provided in an embodiment of the present application.

[0112] Exemplarily, in the offline training of the starting intention prediction model, firstly, a training database is constructed, and the user vehicle collects the historical motion data of all vehicles within the detection range through the sensor equipment and saves it locally, wherein the historical motion data includes the speed data and trajectory data of the vehicle, and the trajectory data of each frame of the vehicle is traversed to obtain the trajectory coordinate (x, y) sequence and speed sequence (the sum of the speeds in the x and y directions) of the obstacle vehicle, the vehicle in front of the obstacle vehicle and closest to the obstacle vehicle, and the vehicle farthest from the obstacle vehicle within the user's detection range and belonging to the same lane as the obstacle vehicle. The coordinate sequence and speed sequence of the three obstacles are sampled respectively, and the sampling step is 0.2s. The trajectory coordinate sequence is converted into displacement data of two adjacent frames, and the historical motion data of the vehicle in the historical time period is annotated by determining the start time. Secondly, the starting intention prediction model is trained, and the historical motion data of the three obstacle vehicles are sequentially spliced ​​into a one-dimensional vector in the following manner and saved in data_features to form a training sample, and the model training (model_train) is performed to obtain the starting prediction result (model_predict), and the prediction result is evaluated.

[0113] [agent_speed_0,...,agent_speed_-9,agent_pos_0,...,agent_pos_-9,

[0114] min_speed_0,...,min_speed_-9,min_pos_0,...,min_pos_-9,

[0115] max_speed_0,...,max_speed_-9,max_pos_0,...,max_pos_-9]

[0116] Among them, the suffix 0 represents the current time, -9 represents the historical time 2s, agent represents the obstacle vehicle, speed represents the speed data, pos represents the displacement data, min represents the vehicle in front of the obstacle vehicle and closest to the obstacle vehicle, and max represents the vehicle farthest from the obstacle vehicle within the user's detection range and in the same lane as the obstacle vehicle.

[0117] For example, when training the model, multiple training samples are processed in random order, 20% of the training samples are extracted for testing the model, and the remaining 80% of the training samples are used for training the model. During the training process, a grid parameter adjustment method is used to finally determine a set of parameters with the best effect, and the model parameters are as follows:

[0118] n_estimators: 220

[0119] max_depth: 10

[0120] min_child_weight: 3

[0121] gamma: 0.3

[0122] learning_rate: 0.05

[0123] For example, the prediction results are evaluated and the evaluation indicator is recall. + , recall - , accuracy + , accuracy - First, define TP, TN, FP, FN and positive and negative samples. Set the data marked as obstacle vehicles that have not started as negative samples, and the obstacle vehicles marked as started as positive samples. Correspondingly, TP: True Positive, correctly classified positive samples; TN: True Negative, correctly classified negative samples; FP: False Positive, incorrectly classified positive samples; FN: False Negative, incorrectly classified negative samples.

[0124] further,

[0125] recall + =TP / (TP+FN)

[0126] accuracy + =TP / (TP+FP)

[0127] recall - =TN / (TN+FP)

[0128] accuracy - =TN / (TN+FN)

[0129] The model training results are evaluated according to the above evaluation indicators, and the results are shown in Table 1:

[0130] recall accuracy stationary 0.90 0.85 launch 0.82 0.87

[0131] Table 1

[0132] The above table shows the performance of the start intention prediction model under the condition of a preset start time domain of 1.5s. For the obstacle vehicle that starts (launch), the prediction accuracy of the model is + ) reached 87%, and the recall rate +) reached 82%; for the obstacle vehicle that has not started (stationary), the prediction accuracy of the model (accuracy - ) reached 85%, and the recall rate - ) reaches 90%. If the start-up time interval threshold is shortened, the start-up intention prediction model can achieve a higher prediction index. The larger the start-up time interval, the earlier the model can predict the start-up intention of the obstacle, which can enable the user's vehicle to have better driving performance in intersection start scenarios and congestion scenarios.

[0133] For example, refer to Figure 2 In the online real-time prediction of the starting intention prediction model, the user vehicle collects the speed data and trajectory data of the target vehicle, the vehicle in front of the target vehicle and closest to the target vehicle, and the vehicle farthest from the target vehicle and in the same lane as the target vehicle in the user detection range in real time, performs data processing, samples the trajectory data to obtain displacement data, splices the motion data of the three vehicles into a one-dimensional vector and saves it in the sampling features, inputs it into the starting intention prediction model, and obtains the starting intention prediction result of the target vehicle. By adding the motion data of the vehicle in front of the target vehicle and closest to the target vehicle and the vehicle farthest from the target vehicle and in the same lane as the target vehicle in the user detection range to the input data, the starting intention prediction model is assisted in predicting the starting intention of the target vehicle, thereby improving the accuracy of the model prediction results.

[0134] Figure 3 A schematic diagram of the structure of a device for predicting the starting intention of a target vehicle provided in an embodiment of the present application. For the sake of convenience of explanation, only the part related to the embodiment of the present application is shown.

[0135] The target vehicle starting intention prediction device may specifically include the following modules:

[0136] The data acquisition module 301 is used to acquire the first motion data of the target vehicle that has not started at each moment within a set time period from the current moment;

[0137] The starting intention prediction module 302 is used to input the first motion data into a preset starting intention prediction model, and the starting intention prediction model outputs the starting prediction result of the target vehicle; the starting intention prediction model is formed by training the historical motion data of multiple vehicles within a set time period before starting.

[0138] Figure 4 The terminal device 400 includes: at least one processor 401 ( Figure 4Only one is shown in the figure) a processor, a memory 402, and a computer program 403 stored in the memory 402 and executable on the at least one processor 401, and when the processor 401 executes the computer program 403, the steps in the above-mentioned embodiment of the auxiliary processing method for meeting the standards for rainfall water quality in a river are implemented.

[0139] The terminal device 400 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 It is only an example of the terminal device 400 and does not constitute a limitation on the terminal device 400. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0140] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0141] In some embodiments, the memory 402 may be an internal storage unit of the terminal device 400, such as a hard disk or memory of the terminal device 400. In other embodiments, the memory 402 may also be an external storage device of the terminal device 400, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 400. Further, the memory 402 may also include both an internal storage unit of the terminal device 400 and an external storage device. The memory 402 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 402 may also be used to temporarily store data that has been output or is to be output.

[0142] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. If a rated part is not described or recorded in detail in a certain embodiment, reference may be made to the relevant description of other embodiments.

[0144] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0145] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0149] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for predicting a target vehicle's starting intention, characterized in that: The prediction method comprises: Obtaining the first motion data of the target vehicle that has not started at each moment within a set time period from the current moment; The first motion data is input into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle; the starting intention prediction model is formed by training historical motion data of multiple vehicles within a set time period before starting.

2. The method according to claim 1, characterized in that The prediction method further comprises: Acquire second motion data of at least one of the vehicles closest to and farthest from the target vehicle within the user vehicle detection range at each moment within a set time period from the current moment; The second motion data is input into the starting intention prediction model, and the starting intention prediction model inputs the first motion data and the second motion data, and outputs a starting prediction result of the target vehicle.

3. The method according to claim 1, characterized in that Before obtaining the motion data of the target vehicle that has not started at each moment within a set time period from the current moment, the prediction method further includes: Obtaining historical movement data of a plurality of vehicles at each moment within a preset historical time period, wherein the preset historical time period is before the vehicles complete starting; Split the preset historical time period into multiple sub-time periods, with each sub-time period having the set duration. Based on the manually input start marking results of each vehicle in each sub-time period, the historical movement data of each vehicle in each sub-time period is marked; The manually input start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are combined into training data and input into a deep learning model to obtain the start intention prediction model; wherein, the start labeling results of each vehicle in each sub-time period are used as the output of the deep learning model, and the corresponding historical motion data are used as the input of the deep learning model.

4. The method according to claim 1, wherein Before obtaining the motion data of the target vehicle that has not started at each moment within a set time period from the current moment, the prediction method further includes: Obtaining historical movement data of a plurality of vehicles at each moment within a preset historical time period; the preset historical time period is before the vehicles complete starting; Split the preset historical time period into multiple sub-time periods, with each sub-time period having the set duration. Determine the start marking result corresponding to the historical movement data of each vehicle in each sub-time period based on the historical movement data of each vehicle in each sub-time period and the preset start time interval; The start labeling results of each vehicle in each sub-time period and the corresponding historical motion data are combined into training data and input into a deep learning model to obtain the start intention prediction model; wherein the start labeling results of each vehicle in each sub-time period are used as the output of the deep learning model, and the corresponding historical motion data are used as the input of the deep learning model.

5. The method according to claim 4, characterized in that The method of determining the start marking result corresponding to the historical movement data of each vehicle in each sub-time period based on the historical movement data of each vehicle in each sub-time period and the preset start time interval includes: According to the preset first time window and the second time window, in combination with the motion data, the preset first frame number and the preset second frame number, starting from the first time frame in each of the sub-time periods, determining whether each frame is a startup frame frame by frame; Determining a start time interval of each vehicle according to the start frame and the first time frame in each sub-time period; For each vehicle, if the corresponding start time interval is longer than the preset start time interval, the corresponding sub-time interval of the vehicle is marked as start, and the other sub-time intervals of the vehicle are marked as not started.

6. The method according to claim 5, characterized in that The motion data includes speed data and displacement data, and the determining, based on the preset first time window and the second time window, in combination with the motion data, the preset first frame number and the preset second frame number, starting from the first time frame in each of the sub-time periods, whether each frame is a startup frame, frame by frame, includes: Starting from the first time frame in each sub-time period, determine frame by frame whether the speed of all frames in the first time window is higher than a first set speed value, whether the speed of a set number of frames in the second time window is higher than a second set speed value, and whether the displacement in the second time window is higher than a set displacement value; If both are yes, then the currently judged time frame is determined to be the startup frame.

7. The method according to claim 1, characterized in that Before obtaining the motion data of the target vehicle that has not started at each moment within a set time period from the current moment, the method includes: For all vehicles within the user vehicle detection range, if the vehicle's displacement data is within a first preset threshold and the speed data is within a second preset threshold, the vehicle is determined to be a target vehicle that has not started.

8. A device for predicting the starting intention of a target vehicle, characterized in that: The prediction device comprises: A data acquisition module is used to obtain the first motion data of the target vehicle that has not started at each moment within a set time period from the current moment; A starting intention prediction module is used to input the first motion data into a preset starting intention prediction model, and the starting intention prediction model outputs a starting prediction result of the target vehicle; the starting intention prediction model is formed by training historical motion data of multiple vehicles within a set time period before starting.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.