Obstacle intrusion intention recognition method and vehicle control method

Abstract

The application provides an obstacle intrusion intention recognition method and a vehicle control method, wherein the obstacle intrusion intention recognition method comprises the following steps: determining a target obstacle, a plurality of position information of the target obstacle relative to a target lane line corresponding to a lane where a target vehicle is located, and a driving scene type of the target vehicle according to a plurality of vehicle environment data; determining motion trajectory information of the target obstacle and a spatial change relationship between the target obstacle and the target lane line according to the plurality of position information; determining a corresponding intrusion judgment condition according to the driving scene type; and determining an intrusion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located according to the motion trajectory information, the spatial change relationship and the intrusion judgment condition. In this way, the time change information and the spatial change information of the target obstacle are fully considered, and the intrusion judgment condition corresponding to the driving scene type of the vehicle is fully considered, so that the intrusion intention recognition can be more accurately realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle driving scene obstacle recognition, and in particular to an obstacle intrusion intention recognition method and a vehicle control method. BACKGROUND

[0002] In the process of vehicle driving, the vehicle usually needs to recognize the intention of the obstacles such as other vehicles, pedestrians and the like around the ego vehicle to intrude into the lane currently occupied by the ego vehicle, so as to determine whether there is a collision risk. And in the case of determining that there is a collision risk, the risk is avoided in advance to ensure the safety of driving. Therefore, how to recognize the information such as whether the obstacles around the vehicle have the intention to intrude into the lane currently occupied by the ego vehicle and obtain more accurate intrusion intention recognition results is crucial to the safety of driving. SUMMARY

[0003] The embodiments of the present application provide an obstacle intrusion intention recognition method and a vehicle control method, which can more accurately realize obstacle intrusion intention recognition, obtain more accurate obstacle intrusion intention recognition results, and further enable the vehicle to make corresponding processing such as early warning more timely and accurately, so as to improve the safety of driving.

[0004] To solve the above technical problems, in a first aspect, the embodiments of the present application provide an obstacle intrusion intention recognition method, which comprises: acquiring a plurality of vehicle environment data corresponding to a target vehicle, determining a target obstacle, a plurality of position information of the target obstacle relative to a target lane line corresponding to a lane currently occupied by the target vehicle, and a driving scene type of the target vehicle according to the plurality of vehicle environment data, each vehicle environment data corresponding to a different data collection time, and the vehicle environment data comprising visual image data and radar point cloud data; determining motion trajectory information of the target obstacle and a spatial change relationship between the target obstacle and the target lane line according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the lane currently occupied by the target vehicle; and determining a corresponding intrusion judgment condition according to the driving scene type; and determining an intrusion intention recognition result of the target obstacle corresponding to the lane currently occupied by the target vehicle according to the motion trajectory information, the spatial change relationship and the intrusion judgment condition.

[0005] By combining the vehicle visual image data and the radar point cloud data, more accurate position information of the target obstacle relative to the target lane line can be obtained. Furthermore, according to the plurality of position information of the target obstacle relative to the target lane line, the motion trajectory information of the target obstacle in time variation and the spatial variation relationship between the target obstacle and the target lane line in spatial variation can be obtained. Then, the driving scene type of the vehicle is fully considered, and according to the intrusion judgment condition corresponding to the driving scene type, the motion trajectory information of the target obstacle in time variation and the spatial variation relationship in spatial variation are combined to determine whether the target obstacle has the intention to intrude into the lane where the target vehicle is located. In this way, the time variation information and the spatial variation information of the target obstacle are fully considered, and the intrusion judgment condition corresponding to the driving scene type of the vehicle is fully considered, so that the intrusion intention recognition can be more accurately realized, a more accurate obstacle intrusion intention recognition result can be obtained, and the vehicle can make a warning or other corresponding processing more timely and accurately, so as to improve the driving safety.

[0006] In a possible implementation of the first aspect, according to the motion trajectory information, the spatial variation relationship, and the intrusion judgment condition, the intrusion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located is determined, including:

[0007] According to the motion trajectory information, the motion trajectory trend of the target obstacle and the corresponding intrusion risk level are predicted. According to the motion trajectory trend of the target obstacle and the spatial variation relationship, the intrusion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is determined. According to the intrusion judgment condition, the intrusion score of the target obstacle corresponding to the lane where the target vehicle is located is determined. According to the intrusion intention probability, the intrusion risk level, and the intrusion score, whether the target obstacle has the intention to intrude into the lane where the target vehicle is located is determined as the intrusion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located.

[0008] According to the motion trajectory trend of the target obstacle, the time variation information of the target obstacle is obtained, and the corresponding intrusion risk level is predicted. According to the time variation information of the target obstacle and the spatial variation relationship, the intrusion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is obtained. According to the intrusion judgment condition, the intrusion score of the target obstacle corresponding to the lane where the target vehicle is located is determined. In this way, the intrusion intention probability, the intrusion risk level, and the intrusion score are comprehensively considered to identify whether the target obstacle has the intention to intrude into the lane where the target vehicle is located, and a more accurate intrusion intention recognition result is obtained.

[0009] In a possible implementation of the first aspect, the invasion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is determined according to the motion trajectory trend of the target obstacle and the spatial variation relationship, including: the invasion intention probability is obtained by the following manner:

[0010]

[0011] wherein, is the invasion intention probability, σ is a Sigmoid activation function, W and b are learnable parameters, is a function of the motion trajectory trend, is a function of the spatial variation relationship.

[0012] By adopting the above technical solution, the invasion intention probability of the target obstacle is determined in combination with the motion trajectory information of the target obstacle in time variation and the spatial variation relationship in spatial variation, the invasion trend of the target obstacle can be considered in all aspects, and the invasion intention can be more accurately judged.

[0013] In a possible implementation of the first aspect, in the case that there are multiple invasion judgment conditions, the invasion score of the target obstacle invading the lane where the target vehicle is located is determined according to the invasion judgment conditions, including: determining the weight coefficient corresponding to each invasion judgment condition; determining the similarity information corresponding to each invasion judgment condition according to the motion trajectory information and the spatial variation relationship; and determining the invasion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type according to the weight coefficient and the similarity information corresponding to each invasion judgment condition.

[0014] By adopting the above technical solution, the similarity information corresponding to each invasion judgment condition is determined according to the motion trajectory information and the spatial variation relationship, and the invasion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type is determined according to the weight coefficient and the similarity information corresponding to each invasion judgment condition. In this way, the similarity between the invasion judgment condition and the current motion information of the target obstacle under the driving scene type is considered, the invasion score representing the invasion intention of the target obstacle is obtained, and the invasion intention identification of the target obstacle can be more comprehensive.

[0015] In a possible implementation of the first aspect, the invasion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type is determined according to the weight coefficient and the similarity information corresponding to each invasion judgment condition, including: the invasion score is obtained by the following manner:

[0016]

[0017] wherein, is the invasion score, is the weight coefficient corresponding to the i th invasion judgment condition, The similarity information corresponding to the i-th invasion judgment condition, and n is the number of the invasion judgment conditions.

[0018] In a possible implementation of the first aspect, the method further includes determining the weight coefficients corresponding to the invasion judgment conditions, including: determining environment perception information of an environment in which the target vehicle is located according to the visual image data; and determining the weight coefficients corresponding to the invasion judgment conditions according to the environment perception information.

[0019] With the technical solution described above, the weight coefficients are dynamically adjusted based on the environment perception information of the environment in which the vehicle is located, so that the invasion score can better reflect the invasion intention of the target obstacle under the current environment.

[0020] In a possible implementation of the first aspect, the multiple vehicle environment data corresponding to the target vehicle are acquired, and the multiple position information of the target obstacle and the target lane line corresponding to the target obstacle relative to the target vehicle are determined according to the multiple vehicle environment data, including: acquiring visual image data collected by a camera of the target vehicle and radar point cloud data collected by a lidar of the target vehicle at different data collection times, and performing time alignment processing on the visual image data and the radar point cloud data with the same data collection time to obtain multiple vehicle environment data corresponding to different data collection times; determining the target obstacle in the visual image data in an image coordinate system and distance information of the target obstacle and the target lane line in the image coordinate system at different data collection times; converting the visual image data corresponding to different data collection times in the image coordinate system to a bird's-eye view coordinate system to obtain visual image data corresponding to different data collection times in the bird's-eye view coordinate system, and obtaining three-dimensional position information of the target obstacle and three-dimensional position information of the target lane line corresponding to different data collection times according to the visual image data at different data collection times in the bird's-eye view coordinate system and the radar point cloud data with the same data collection time as the visual image data; and obtaining distance information of the target obstacle and the target lane line corresponding to different data collection times in the bird's-eye view coordinate system as the multiple position information of the target obstacle relative to the target lane line according to the three-dimensional position information of the target obstacle and the three-dimensional position information of the target lane line corresponding to different data collection times.

[0021] With the technical solution described above, the position of the target obstacle relative to the target lane line is determined based on multi-source perception information of the camera and the lidar, and more accurate position information of the target obstacle relative to the target lane line can be obtained.

[0022] In a possible implementation of the first aspect, the method further includes: determining the spatial variation relationship between the target obstacle and the target lane line according to a plurality of position information of the target obstacle relative to the target lane line corresponding to a lane in which the target vehicle is located, including: encoding the topological structure of the target lane line and the position information of the target obstacle to obtain an encoding vector; obtaining a spatial feature map corresponding to an aerial view coordinate system according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the lane in which the target vehicle is located in the aerial view coordinate; and modeling the spatial variation relationship between the target obstacle and the lane line based on a graph convolution network model according to the encoding vector and the spatial feature map.

[0023] According to the above technical solution, the spatial variation relationship between the target obstacle and the target lane line is determined, and the intention of the target obstacle to invade the target lane line can be more intuitively obtained.

[0024] In a possible implementation of the first aspect, the method further includes: determining the corresponding invasion judgment condition according to the driving scene type, including: determining the invasion judgment condition corresponding to the driving scene type from a preset scene template library according to the driving scene type.

[0025] According to the above technical solution, the invasion judgment condition is obtained based on the preset scene template library, so that different invasion judgment conditions are obtained under different driving scene types, and the invasion intention can be more flexibly and accurately judged.

[0026] In a second aspect, the implementation of the present application further discloses a vehicle control method, including: controlling the target vehicle to perform corresponding processing according to the invasion intention recognition result, the invasion intention recognition result being obtained according to the obstacle invasion intention recognition method provided in any one of the implementations of the first aspect.

[0027] In a third aspect, the implementation of the present application further discloses a vehicle, which is configured to execute the obstacle invasion intention recognition method provided in any one of the implementations of the first aspect and / or execute the vehicle control method provided in the second aspect.

[0028] In a fourth aspect, the implementation of the present application further discloses a computer readable storage medium, which stores a computer program, and the computer program can be executed by an electronic device to implement the obstacle invasion intention recognition method provided in any one of the implementations of the first aspect.

[0029] In a fifth aspect, the implementation of the present application further discloses a computer program product, including a computer program, and the computer program is executed by an electronic device to implement the obstacle invasion intention recognition method provided in any one of the implementations of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the embodiments description will be briefly introduced as follows.

[0031] Figure 1 A flowchart of an obstacle intrusion intention recognition method provided by an embodiment of the present application;

[0032] Figure 2 A flowchart of determining position information of a target obstacle relative to a target lane line provided by an embodiment of the present application;

[0033] Figure 3 A flowchart of determining a spatial variation relationship between a target obstacle and a target lane line provided by an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a scene template library provided by an embodiment of the present application;

[0035] Figure 5 A flowchart of determining an intrusion intention recognition result provided by an embodiment of the present application;

[0036] Figure 6 A flowchart of determining an intrusion score provided by an embodiment of the present application;

[0037] Figure 7 Another flowchart of an obstacle intrusion intention recognition method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] In an intelligent driving system, accurately recognizing the intrusion intention of surrounding target obstacles (such as vehicles, pedestrians, etc.) is crucial for driving safety. Current obstacle intrusion intention recognition mainly relies on a single sensor (such as a camera) to obtain target obstacles, which is difficult to stably obtain the spatial relationship between the target obstacle and the lane line in complex scenes (such as tunnels, scenes with unclear light at night, etc.). Moreover, the current intention recognition only considers the current motion state of the target obstacle, ignoring the motion trend of the target obstacle in the time dimension, resulting in delayed warning or false positives. And the current intrusion intention recognition method only recognizes the position of the target obstacle to simply judge the intrusion intention of the target obstacle, but the intrusion behavior characteristics are quite different for different traffic scenes (such as highways or urban intersections), and the existing recognition method cannot achieve accurate intention recognition, which poses a great safety hazard.

[0039] Based on this, the application provides an obstacle intrusion intention recognition method, which combines vehicle visual image data and radar point cloud data to obtain more accurate position information of a target obstacle relative to a target lane line, fully considers the motion trajectory information of the target obstacle in time variation, the spatial variation relationship between the target obstacle and the target lane line in spatial variation, and fully considers the driving scene type of the vehicle, so as to realize more accurate intrusion intention judgment and recognition of the target obstacle, and obtain more accurate intrusion intention recognition results.

[0040] As shown in Figure 1 The obstacle intrusion intention recognition method provided by the implementation manner of the application includes the following steps.

[0041] S100, a plurality of vehicle environment data corresponding to a target vehicle are obtained, a plurality of position information of a target obstacle relative to a target lane line corresponding to a lane where the target vehicle is located are determined according to the plurality of vehicle environment data, and a driving scene type of the target vehicle is determined, each vehicle environment data corresponds to a different data collection time, and the vehicle environment data includes visual image data and radar point cloud data.

[0042] S200, motion trajectory information of the target obstacle is determined according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the lane where the target vehicle is located, and a spatial variation relationship between the target obstacle and the target lane line is determined.

[0043] S300, a corresponding intrusion judgment condition is determined according to the driving scene type.

[0044] S400, an intrusion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located is determined according to the motion trajectory information, the spatial variation relationship and the intrusion judgment condition.

[0045] Among them, about steps S200 and S300, S200 can be executed first, and then S300 can be executed, S300 can be executed first, and then S200 can be executed, of course, it can also be executed at the same time.

[0046] The obstacle intrusion intention recognition method provided by the implementation manner of the present application can obtain more accurate position information of the target obstacle relative to the target lane line in combination with vehicle visual image data and radar point cloud data. Further, the movement trajectory information of the target obstacle in time variation and the spatial variation relationship between the target obstacle and the target lane line in spatial variation are obtained according to the plurality of position information of the target obstacle relative to the target lane line. Then, the driving scene type of the vehicle is fully considered, and whether the target obstacle has the intention to intrude into the lane where the target vehicle is located is determined according to the intrusion judgment condition corresponding to the driving scene type, in combination with the movement trajectory information of the target obstacle in time variation and the spatial variation relationship in spatial variation. In this way, the time variation information and the spatial variation information of the target obstacle are fully considered, and the intrusion judgment condition corresponding to the driving scene type of the vehicle is fully considered, so that more accurate intrusion intention judgment and recognition can be realized.

[0047] Firstly, step S100 is performed, in which, as shown in Figure 2 In the implementation manner of the present application, a plurality of vehicle environment data corresponding to the target vehicle is obtained, and a plurality of position information of a target obstacle relative to a target lane line corresponding to a lane where the target vehicle is located is determined according to the plurality of vehicle environment data, including the following steps.

[0048] In S110, visual image data collected by a camera of the target vehicle and radar point cloud data collected by a laser radar of the target vehicle at different data collection times are obtained, and the visual image data and the radar point cloud data are time-aligned to obtain a plurality of vehicle environment data corresponding to different data collection times.

[0049] For example, a fusion perception model based on visual and radar sensors is constructed, a plurality of frames of visual image data of a target obstacle and a lane line photographed by a surround-view camera and a binocular camera at different data collection times are obtained, and a plurality of frames of radar point cloud data perceived by a laser radar are obtained. The plurality of frames of visual image data and the plurality of frames of radar point cloud data corresponding to different data collection times are input into the fusion perception model, and the fusion perception model performs time alignment processing on each frame of visual image data and each frame of radar point cloud data to obtain visual image data and radar point cloud data corresponding to a plurality of data collection times. The visual image data and the radar point cloud data with the same data collection time can be visual image data and radar point cloud data with exactly the same time, or visual image data and radar point cloud data within a preset time difference.

[0050] For example, 4-way surround-view camera (1920x1080@30fps) visual image data and 1 set of front-view binocular camera visual image data are collected simultaneously, and radar point cloud data of a front-mounted 1024-line laser radar are obtained to cover a 360° field of view of the vehicle.

[0051] S120, determine the target obstacle in the visual image data in the image coordinate system at different data collection times, and the distance information of the target obstacle and the target lane line in the image coordinate system.

[0052] For example, spatial relationship modeling is performed, and based on the recognition processing of the target obstacle and the distance information of the target obstacle and the target lane line in the image coordinate system by the fusion perception model, the target obstacle is detected by using, for example, YOLOv7 technology, the lane line is segmented by using, for example, LaneNet technology, the pixel-level distance between the bounding box of the target obstacle and the lane line is calculated, and the distance information of the target obstacle and the target lane line in the image coordinate system is obtained.

[0053] S130, based on an inverse perspective transformation method, the visual image data corresponding to different data collection times in the image coordinate system is converted to the bird's eye view coordinate system to obtain the visual image data corresponding to different data collection times in the bird's eye view coordinate system, and the three-dimensional position information of the target obstacle and the three-dimensional position information of the target lane line corresponding to different data collection times are obtained according to the visual image data corresponding to different data collection times in the bird's eye view coordinate system and the radar point cloud data with the same data collection time as the visual image data.

[0054] For example, the bird's eye view (BEV) coordinate system conversion is performed, and the visual image data at different data collection times in the image coordinate system is converted to the bird's eye view coordinate system by using the inverse perspective transformation method (IPM), so that the three-dimensional coordinates (as an example of three-dimensional position information) of the bounding box of the target obstacle and the three-dimensional coordinates of the target lane line at different data collection times in the bird's eye view coordinate system are obtained according to the visual image data corresponding to different data collection times in the bird's eye view coordinate system and the radar point cloud data with the same data collection time as the visual image data at different data collection times.

[0055] S140, according to the three-dimensional position information of the target obstacle and the three-dimensional position information of the target lane line corresponding to different data collection times, the distance information of the target obstacle and the target lane line corresponding to different data collection times in the bird's eye view coordinate system is obtained as a plurality of position information of the target obstacle relative to the target lane line.

[0056] For example, the transverse / longitudinal distance between the bounding box of the target obstacle and the lane line corresponding to different data collection times in the bird's eye view coordinate system is calculated based on the fusion perception model. The transverse distance is the shortest distance between the target obstacle and the lane line, and the longitudinal distance is the distance between the target obstacle and the position of the lane line where the target vehicle is located.

[0057] Further, the driving scene type of the target vehicle is determined.

[0058] In the implementation of the present application, the fusion perception model also obtains environmental perception information of the environment in which the vehicle is located based on visual image information, including traffic signs, roadside facilities, weather, visibility, and the like.

[0059] Further, a driving scene recognition and pattern matching model is constructed, and the environmental perception information is input into the driving scene recognition and pattern matching model to identify the current driving scene type according to the traffic signs and roadside facilities.

[0060] The driving scene recognition and pattern matching model can be a visual language model (VLM) in particular. The driving scene type includes an urban road, an expressway, an intersection, a school intersection, and the like.

[0061] Next, step S200 is performed to determine the motion trajectory information of the target obstacle according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the lane in which the target vehicle is located.

[0062] For example, a transformer-based spatio-temporal coupling intention recognition model is constructed, and the spatio-temporal coupling intention recognition model is first used for time dimension processing.

[0063] Specifically, the position information of the target obstacle relative to the target lane line corresponding to a plurality of continuous frames (i.e., a plurality of data collection times) (about 0.33 seconds) is input, and the spatio-temporal coupling intention recognition model obtains the motion trajectory information of the target obstacle according to the position changes of the target obstacle relative to the target lane line in the continuous frames. The motion trajectory information of the target obstacle includes position, speed, acceleration, and the like.

[0064] Further, as shown in Figure 3 In the implementation of the present application, the spatial change relationship between the target obstacle and the target lane line is determined according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the lane in which the target vehicle is located, including the following steps.

[0065] S210, the position information of the target obstacle and the topological structure of the target lane line are encoded to obtain an encoded vector.

[0066] For example, the spatio-temporal coupling intention recognition model is used for spatial dimension processing.

[0067] Specifically, the spatio-temporal coupling intention recognition model uses a VectorNet encoding method to encode the target lane line into a plurality of line segments to obtain the topological structure of the target lane line. The spatio-temporal coupling intention recognition model is used to encode the topological structure of the target lane line and the coordinates of the target obstacle in the bird's eye view coordinate system to obtain the encoded vector of the target obstacle and the target lane line in the bird's eye view coordinate system.

[0068] S220, obtain the spatial feature map corresponding to the bird's eye coordinate system according to the plurality of position information of the target lane line corresponding to the target obstacle relative to the lane in which the target vehicle is located in the bird's eye coordinate system.

[0069] Illustratively, the spatial feature map in the bird's eye coordinate system is displayed / implicitly constructed according to the plurality of position information of the target lane line corresponding to the target obstacle relative to the lane in which the target vehicle is located in the bird's eye coordinate system based on the spatio-temporal coupling intention recognition model.

[0070] S230, modeling the spatial variation relationship between the target obstacle and the lane line based on the graph convolutional network according to the encoding vector and the spatial feature map.

[0071] Illustratively, the spatial variation relationship between the target obstacle and the lane line is modeled according to the encoding vector and the spatial feature map by the graph convolutional network (GCN) of the spatio-temporal coupling intention recognition model. For example, the angle and distance of the target obstacle deviating from the target lane line are obtained.

[0072] Next, step S300 is performed, and in the implementation manner of the present application, corresponding intrusion judgment conditions are determined according to the driving scene type, including: determining the intrusion judgment conditions corresponding to the driving scene type from the preset scene template library according to the driving scene type.

[0073] Illustratively, the scene template library is constructed in advance, as shown in Figure 4 The scene template library includes the scene type, the typical intrusion behavior, and the feature template example (i.e., the intrusion judgment condition).

[0074] The typical intrusion behavior is the possible intrusion behavior of the target obstacle in the corresponding driving scene. The feature template example is the intrusion judgment condition used to judge the intrusion of the target obstacle into the target lane line in the corresponding driving scene type.

[0075] For example, if it is determined that the current driving scene type is an urban road, the intrusion judgment condition is that the trajectory acceleration of the target obstacle > 2 m / s² and the lateral displacement suddenly changes; if it is determined that the current driving scene type is a highway, the intrusion judgment condition is that the target obstacle crosses the target lane line more than 3 times per 5 seconds; if it is determined that the current driving scene type is an intersection, the intrusion judgment condition is that the angle of the target obstacle relative to the ego vehicle (i.e., the target vehicle) is greater than 30° and the distance between the target obstacle and the target lane line in which the target vehicle is located is less than 5 m.

[0076] Next, step S400 is performed, and in the implementation manner of the present application, as shown in Figure 5 ​As shown, according to the motion trajectory information, the spatial change relationship, and the invasion judgment condition, the invasion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located is determined, including the following steps.

[0077] S410, according to the motion trajectory information, the motion trajectory trend of the target obstacle and the corresponding invasion danger level are predicted.

[0078] For example, the Transformer encoder based on the spatio-temporal coupling intention recognition model extracts the time sequence features based on the motion trajectory information of the target obstacle, predicts the motion trajectory trend of the target obstacle, and determines the invasion danger level according to the motion trajectory trend. The invasion danger level is a continuous value between 0 and 1.

[0079] Specifically, according to the motion trajectory trend, the probability of the target obstacle invading the lane where the target vehicle is located is determined, and the invasion danger level is determined according to the position of the target vehicle when the target obstacle invades the lane where the target vehicle is located.

[0080] S420, according to the motion trajectory trend of the target obstacle and the spatial change relationship, the invasion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is determined.

[0081] For example, the feature fusion and decision are performed based on the spatio-temporal coupling intention recognition model.

[0082] Specifically, the motion trajectory trend and the spatial change relationship and other feature information are spliced and input into the full connection layer of the spatio-temporal coupling intention recognition model, and the invasion intention probability is output.

[0083] In the implementation manner of the present application, according to the motion trajectory trend of the target obstacle and the spatial change relationship, the invasion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is determined, including obtaining the invasion intention probability by the following way:

[0084]

[0085] wherein, is the invasion intention probability, σ is the Sigmoid activation function, W and b are learnable parameters, is a function of the motion trajectory trend, is a function of the spatial change relationship.

[0086] S430, according to the invasion judgment condition, the invasion score of the target obstacle corresponding to the lane where the target vehicle is located is determined.

[0087] For example, Figure 6As shown, in the case where there are multiple intrusion judgment conditions, according to the intrusion judgment conditions, the intrusion score of the target obstacle corresponding to the lane where the target vehicle is located is determined, including the following steps.

[0088] S431, determine the weight coefficient corresponding to each intrusion judgment condition.

[0089] For example, based on the preset scene library, the initial weight value of the intrusion judgment condition corresponding to the current driving scene type is obtained.

[0090] Further, in the implementation manner of the present application, determining the weight coefficient corresponding to each intrusion judgment condition includes: determining the environmental information of the environment where the vehicle is located according to the visual image data, and determining the weight coefficient corresponding to each intrusion judgment condition according to the environmental information.

[0091] For example, according to the environmental perception information such as weather and visibility, the weight coefficient corresponding to each intrusion judgment condition is dynamically adjusted.

[0092] S432, according to the motion trajectory information and the spatial change relationship, determine the similarity information corresponding to each intrusion judgment condition.

[0093] For example, the feature information corresponding to each intrusion judgment condition is obtained from the motion trajectory information and the spatial change relationship, the spatial vectors of each intrusion judgment condition and the corresponding feature information are respectively determined, and the similarity information corresponding to each intrusion judgment condition is calculated according to the spatial vector of each intrusion judgment condition and the spatial vector of the corresponding feature information. , wherein, is the i th intrusion judgment condition, is the feature information corresponding to the i th intrusion judgment condition.

[0094] Specifically, taking the urban road as an example of the driving scene type, the trajectory acceleration feature information of the target obstacle is obtained according to the motion trajectory information, and the lateral displacement feature information of the target obstacle is obtained according to the spatial change relationship.

[0095] The similarity information of the intrusion judgment condition and the trajectory acceleration feature information of the trajectory acceleration>2m / s² is calculated, and the similarity information of the lateral displacement mutation and the lateral displacement feature information is determined.

[0096] S433, according to the weight coefficient and the similarity information corresponding to each intrusion judgment condition, determine the intrusion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type.

[0097] For example, according to the weight coefficient and the similarity information corresponding to each intrusion judgment condition, the intrusion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type is determined, including obtaining the intrusion score by the following way:

[0098]

[0099] wherein, is an invasion score, is a weight coefficient corresponding to the i-th invasion judgment condition, is similarity information corresponding to the i-th invasion judgment condition, and n is the number of invasion judgment conditions.

[0100] S440, according to the invasion intention probability, the invasion danger level and the invasion score, determining whether the target obstacle has the intention to invade the lane where the target vehicle is located as the invasion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located.

[0101] For example, if at least one of the invasion intention probability, the invasion danger level and the invasion score meets a target condition, it is determined that the target obstacle has the intention to invade the lane where the target vehicle is located.

[0102] wherein, the target condition is that the invasion intention probability is greater than or equal to a preset probability threshold, the invasion danger level is greater than or equal to a preset level threshold, and the invasion score is greater than or equal to a preset score threshold. The preset probability threshold can be 65%, 70%, 80%, 90%, etc., the preset level threshold can be 0.6, 0.7, 0.8, 0.85, etc., and the preset score threshold can be 65, 70, 80, 85, 90, etc.

[0103] In another implementation, the invasion intention probability, the invasion danger level and the invasion score can also be weighted and summed to obtain a target invasion value, and if the target invasion value is greater than or equal to a preset invasion threshold, it is determined that the target obstacle has the intention to invade the lane where the target vehicle is located.

[0104] For example, the invasion intention probability is a percentage number, the invasion danger level and the invasion score can be converted into percentage numbers, and according to a preset weight value, a target invasion value is obtained.

[0105] wherein, the invasion danger level is a value between 0 and 1, and the invasion danger level / 1 can obtain the corresponding invasion danger percentage number, the invasion score is a value between 0 and 100, and the invasion score / 100 can obtain the corresponding invasion score percentage.

[0106] It should be noted that the preset probability threshold, the preset level threshold, the preset score threshold and the preset invasion threshold can be dynamically adjusted according to the environmental information of the environment where the vehicle is located. For example, the threshold is lowered by 20% in rainy and foggy weather.

[0107] The obstacle intrusion intention recognition method provided by the implementation manner of the application is actually a kind of real-time multi-target intrusion intention recognition method for a vehicle, which can realize the recognition of the intrusion intention of a target such as a vehicle and a pedestrian in a complex traffic scene. The problem that a single sensor cannot accurately obtain vehicle environment data due to performance degradation caused by environmental influence is solved based on multi-source perception fusion. The early warning lag caused by the dependence on static spatial relationship is overcome based on spatio-temporal coupling modeling. The complex scene robustness is improved based on dynamic scene matching through a template library and a weight adjustment mechanism. In this way, the target obstacle intrusion intention recognition based on multi-source perception fusion, spatio-temporal coupling modeling and dynamic pattern matching can predict the risk of the target obstacle intruding into the lane where the ego vehicle is located in real time and accurately.

[0108] As shown in the another implementation manner of the application, the obstacle intrusion intention recognition method of the application further includes the following steps. Figure 7

[0109] S10, a plurality of vehicle environment data corresponding to a target vehicle are obtained, a plurality of position information of a target obstacle and a target lane line corresponding to a lane where the target obstacle is located relative to the target vehicle are determined according to the plurality of vehicle environment data, each vehicle environment data corresponds to a different data acquisition time, and the vehicle environment data includes visual image data and radar point cloud data.

[0110] Exemplarily, the multi-frame visual image data and the radar point cloud data are input into a fusion perception model, the visual image data and the radar point cloud data are subjected to time alignment processing by the fusion perception model, and a plurality of position information of the target obstacle and the target lane line corresponding to the lane where the target obstacle is located relative to the target vehicle are obtained according to the processed visual image data and the radar point cloud data in an image coordinate system and a bird's eye view coordinate system.

[0111] Further, the fusion perception model further obtains environmental perception information of the vehicle based on the visual image data, including traffic signs, road side facilities, weather, visibility and other information.

[0112] S20, motion trajectory information of the target obstacle is determined according to the plurality of position information of the target obstacle and the target lane line corresponding to the lane where the target obstacle is located relative to the target vehicle, and a spatial change relationship between the target obstacle and the target lane line is determined.

[0113] S30, a motion trajectory trend and a corresponding intrusion risk level of the target obstacle are predicted according to the motion trajectory information.

[0114] S40, an intrusion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is determined according to the motion trajectory trend and the spatial change relationship.

[0115] ​S50, if it is determined that the invasion intention probability is greater than or equal to a preset probability threshold and the invasion danger level is greater than or equal to a preset level threshold, it is determined that the target obstacle has the intention to invade the lane in which the target vehicle is located.

[0116] Exemplarily, the target obstacle in the bird's eye view coordinate system and a plurality of position information of the target obstacle relative to the target lane line corresponding to the lane in which the target vehicle is located are input into the spatio-temporal coupling intention recognition model, the spatio-temporal coupling intention recognition model obtains the motion trajectory trend and the spatial change information of the target obstacle according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the lane in which the target vehicle is located, and then obtains the invasion intention probability and the invasion danger level according to the motion trajectory trend and the spatial change information, so as to output the judgment result of whether the target obstacle has the intention to invade the lane in which the target vehicle is located according to the invasion intention probability and the invasion danger level.

[0117] S60, if it is determined that the invasion intention probability is less than the preset probability threshold or the invasion danger level is less than the preset level threshold, the driving scene type of the target vehicle is determined, and the corresponding invasion judgment condition is determined according to the driving scene type.

[0118] S70, according to the invasion judgment condition, the invasion score of the target obstacle corresponding to the lane in which the target vehicle is located is determined.

[0119] S80, if the invasion score is greater than or equal to a preset score threshold, it is determined that the target obstacle has the intention to invade the lane in which the target vehicle is located.

[0120] S90, if the invasion score is less than the preset score threshold, it is determined that the target obstacle does not have the intention to invade the lane in which the target vehicle is located.

[0121] Exemplarily, the environmental perception information, the motion trajectory trend and the spatial change information are input into the driving scene recognition and pattern matching model, the driving scene recognition and pattern matching model identifies the current driving scene type according to the traffic signs and the roadside facilities, determines the invasion judgment condition corresponding to the driving scene type from the preset scene template library according to the driving scene type, dynamically adjusts the weight coefficient corresponding to each invasion judgment condition according to the environmental perception information such as weather and visibility, matches the feature information in the motion trajectory trend and the spatial change information according to the invasion judgment condition, determines the invasion score of the target obstacle corresponding to the lane in which the target vehicle is located according to the similarity of the invasion judgment condition and the corresponding feature information and the weight coefficient corresponding to each invasion judgment condition, and then outputs the judgment result of whether the target obstacle has the intention to invade the lane in which the target vehicle is located according to the invasion score.

[0122] In the implementation of the present application, the position information of the target obstacle relative to the target lane line corresponding to the lane where the target vehicle is located can be obtained more accurately by combining visual image data and radar point cloud data, the motion trajectory information of the target obstacle is determined according to the position information, the spatial change relationship between the target obstacle and the target lane line is determined, the motion trajectory trend of the target obstacle and the corresponding invasion risk level are predicted according to the motion trajectory information, and the invasion intention probability of the target obstacle corresponding to the lane where the target vehicle is located is determined according to the motion trajectory trend and the spatial change relationship. In this way, the invasion intention probability is more accurate considering the motion trajectory trend of the target obstacle in the time change and the spatial change relationship in the spatial change, and the target obstacle can be more accurately judged and identified whether it has the intention to invade the lane where the target vehicle is located according to the invasion intention probability and the invasion risk level. Further, if the invasion intention probability is less than a preset probability threshold or the invasion risk level is less than a preset level threshold, the driving scene type of the target vehicle is also determined, the corresponding invasion judgment condition is determined according to the driving scene type, the invasion score of the target obstacle corresponding to the lane where the target vehicle is located is determined according to the invasion judgment condition, and whether the target obstacle has the intention to invade the lane where the target vehicle is located is determined according to the invasion score. In this way, even if a misjudgment occurs according to the invasion intention probability and the invasion risk level, the target obstacle can also be secondarily judged and identified whether it has the intention to invade the lane where the target vehicle is located according to the current driving scene type and the corresponding invasion judgment condition, the intention of the target obstacle to invade the lane where the target vehicle is located can be better judged and identified, and the driving safety is further improved.

[0123] The present application also provides a vehicle control method, comprising: controlling the target vehicle to perform corresponding processing according to the invasion intention identification result, the invasion intention identification result being obtained according to the aforementioned obstacle invasion intention identification method.

[0124] For example, according to the invasion intention identification result, if it is determined that the target obstacle has the intention to invade the lane where the target vehicle is located, or if it is determined that at least one of the invasion intention probability, the invasion risk level, and the invasion score meets the target condition, or if it is determined that the target invasion value is greater than or equal to a preset invasion threshold, the ego vehicle is controlled to slow down, change lanes, or stop.

[0125] For example, taking the detection of a pedestrian (as an example of a target obstacle) on an urban road as an example, if it is determined that the lateral distance of the pedestrian from the target lane line in the bird's eye view coordinate system is 3.5 meters, the lateral speed of the pedestrian increases from 0.2 m / s to 1.5 m / s in the motion trajectory of the pedestrian in the next 5 seconds, and the current driving scene type is identified as a school section, the weight coefficient of the pedestrian trajectory sudden change invasion judgment condition is increased, the invasion score is calculated, and if the invasion score is greater than a preset score threshold, a warning is issued 1.8 seconds before the pedestrian steps into the lane.

[0126] The application also provides a vehicle for executing the aforementioned obstacle intrusion intention recognition method and / or executing the aforementioned vehicle control method.

[0127] The application also provides a chip for executing the aforementioned obstacle intrusion intention recognition method and / or vehicle control method in the technical solutions of the embodiments.

[0128] The application also provides a computer readable storage medium, which stores computer instructions, and when the computer instructions are executed on a processor of a vehicle, the processor of the vehicle executes the technical solutions of the aforementioned obstacle intrusion intention recognition method and / or vehicle control method in the embodiments.

[0129] In some possible implementation manners, various aspects of the method provided by the application can also be implemented in the form of a program product, which includes program codes for causing a processor of a vehicle to execute the steps in the method according to various exemplary implementation manners of the application described in the specification, for example, the vehicle can execute the obstacle intrusion intention recognition method and / or vehicle control method described in the embodiments of the application.

[0130] The program product can adopt any combination of one or more readable media. The readable medium can be a readable data medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CDROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] The implementation manners of the application also provide a computer program product, which includes a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and the at least one processor executes the computer program to implement the technical solutions of the aforementioned obstacle intrusion intention recognition method and / or vehicle control method in the embodiments.

[0132] It is to be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. Although the application has been described with reference to the preferred embodiments, persons skilled in the art will recognize that changes can be made in form and detail without departing from the spirit and the scope of the application. Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments described herein. It is the following claims, including any amendments thereto, that define the scope of the application.

[0133] It is to be understood that the same may be employed in the same or other embodiments without departing from the scope of the application. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0134] It should be noted that the terms "first", "second", and so on do not necessarily indicate any relative importance.

[0135] It should be noted that in the drawings, some structural or methodical features can be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order can not be required. Rather, in some embodiments, these features can be arranged in a different manner and / or order than shown in the illustrative drawings. Additionally, inclusion of structural or methodical features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, these features can not be included or can be combined with other features.

[0136] While the application has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, it being understood that only the preferred embodiments have been shown and described and that all changes and modifications that come within the spirit of the application are desired to be protected. There are many alternative ways of implementing the application. The disclosed embodiments are illustrative of the application and not restrictive.

Claims

1. A method of obstacle intrusion intention recognition, characterized by, The method comprises: obtaining a plurality of vehicle environment data corresponding to a target vehicle, determining a target obstacle, a plurality of position information of a target lane line corresponding to the target obstacle relative to a lane where the target vehicle is located, and a driving scene type of the target vehicle according to the plurality of vehicle environment data, each of the vehicle environment data corresponding to a different data collection time, and the vehicle environment data comprising visual image data and radar point cloud data; determining motion trajectory information of the target obstacle and a spatial change relationship between the target obstacle and the target lane line according to the plurality of position information of the target lane line corresponding to the target obstacle relative to the lane where the target vehicle is located; determining a corresponding invasion judgment condition according to the driving scene type; predicting a motion trajectory trend of the target obstacle and a corresponding invasion risk level according to the motion trajectory information; determining an invasion intention probability of the target obstacle corresponding to the lane where the target vehicle is located according to the motion trajectory trend of the target obstacle and the spatial change relationship, the invasion intention probability being obtained in the following manner: wherein, is the intrusion intent probability, σ is a Sigmoid activation function, W and b are learnable parameters, is a function of the motion trajectory trend, is a function of the spatial variation relationship; determining an invasion score of the target obstacle corresponding to the lane where the target vehicle is located according to the invasion judgment condition; determining whether the target obstacle has an intention to invade the lane where the target vehicle is located as an invasion intention recognition result of the target obstacle corresponding to the lane where the target vehicle is located according to the invasion intention probability, the invasion risk level and the invasion score.

2. The obstacle intrusion intention recognition method according to claim 1, characterized by, In the case that there are a plurality of invasion judgment conditions, determining an invasion score of the target obstacle corresponding to the lane where the target vehicle is located according to the invasion judgment condition comprises: determining a weight coefficient corresponding to each of the invasion judgment conditions; determining similarity information corresponding to each of the invasion judgment conditions according to the motion trajectory information and the spatial change relationship; determining an invasion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type according to the weight coefficient and the similarity information corresponding to each of the invasion judgment conditions.

3. The obstacle intrusion intention recognition method according to claim 2, characterized by, Determining an invasion score of the target obstacle corresponding to the lane where the target vehicle is located under the driving scene type according to the weight coefficient and the similarity information corresponding to each of the invasion judgment conditions comprises obtaining the invasion score in the following manner: wherein, is the invasion score, is the weight coefficient corresponding to the i-th invasion judgment condition, is the similarity information corresponding to the i-th invasion judgment condition, and n is the number of invasion judgment conditions.

4. The obstacle intrusion intention recognition method according to claim 3, characterized by, determining a weight coefficient corresponding to each of the invasion judgment conditions comprises: determining environment perception information of an environment where the target vehicle is located according to the visual image data; determining the weight coefficient corresponding to each of the invasion judgment conditions according to the environment perception information.

5. The obstacle intrusion intention recognition method according to claim 4, characterized by, Obtaining a plurality of vehicle environment data corresponding to a target vehicle, determining a target obstacle, a plurality of position information of a target lane line corresponding to the target obstacle relative to a lane where the target vehicle is located, comprises: acquire the visual image data collected by the camera of the target vehicle and the radar point cloud data collected by the laser radar of the target vehicle at different data acquisition times, and perform time alignment processing on the visual image data and the radar point cloud data to obtain a plurality of vehicle environment data corresponding to different data acquisition times; determine the target obstacle in the visual image data in the image coordinate system at different data acquisition times, and distance information of the target obstacle and the target lane line in the image coordinate system; convert the visual image data corresponding to different data acquisition times in the image coordinate system to the bird's eye view coordinate system based on an inverse perspective transformation method to obtain the visual image data corresponding to different data acquisition times in the bird's eye view coordinate system, and obtain three-dimensional position information of the target obstacle and three-dimensional position information of the target lane line corresponding to different data acquisition times according to the visual image data corresponding to different data acquisition times in the bird's eye view coordinate system and the radar point cloud data collected at the same data acquisition time as the visual image data; obtain distance information of the target obstacle and the target lane line corresponding to different data acquisition times in the bird's eye view coordinate system as a plurality of position information of the target obstacle relative to the target lane line according to the three-dimensional position information of the target obstacle and the three-dimensional position information of the target lane line corresponding to different data acquisition times.

6. The obstacle intrusion intent recognition method according to claim 5, characterized by, determine the spatial variation relationship between the target obstacle and the target lane line according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the target vehicle lane, including: perform encoding processing on the topological structure of the target lane line and the position information of the target obstacle to obtain an encoding vector; obtain a spatial feature map corresponding to the bird's eye view coordinate system according to the plurality of position information of the target obstacle relative to the target lane line corresponding to the target vehicle lane in the bird's eye view coordinate system; model the spatial variation relationship between the target obstacle and the lane line based on a graph convolution network model according to the encoding vector and the spatial feature map.

7. The obstacle intrusion intent recognition method according to claim 6, characterized by, determine the corresponding invasion judgment condition according to the driving scene type, including: determine the invasion judgment condition corresponding to the driving scene type from a preset scene template library according to the driving scene type.

8. A vehicle control method characterized by, The method comprises: control the target vehicle to perform corresponding processing according to the invasion intention recognition result, wherein the invasion intention recognition result is obtained according to the obstacle invasion intention recognition method of any one of claims 1-7.

Images