Intersection collision early warning method, device, equipment, medium and product based on roadside perception

Abstract

The invention discloses an intersection collision early warning method, device, equipment, medium and product based on roadside perception, and the method comprises the steps: obtaining an intersection region image collected by a roadside camera in a port scene, and carrying out the target detection and tracking, and obtaining the type of a traffic participant and a detection frame; traversing all traffic participants, and screening vehicles to be pre-warned; fitting historical track points of the vehicle to be pre-warned according to the pixel coordinates of the detection frame, and determining the driving state of the vehicle to be pre-warned; and judging whether the vehicle to be pre-warned is in a preset intersection collision pre-warning area according to the driving state of the vehicle to be pre-warned, and if so, reporting intersection collision pre-warning. According to the method, corresponding equipment does not need to be installed at the vehicle end, intersection collision early warning information of all vehicles in the intersection range is judged in real time through roadside sensing, the cost is effectively reduced, the driving state of the vehicle is determined through fitting of the historical track points of the vehicle through image pixel coordinates, and dependence on model sensing precision is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to an intersection collision warning method, device, equipment, medium and product based on roadside perception. Background Art

[0002] At present, there are two mainstream methods for intersection collision warning. One is to use sensors, cameras, radars and other devices installed on vehicles to sense the surrounding environment and obstacles such as other vehicles and pedestrians, and judge intersection collision warning events at the vehicle end. This method does not rely on roadside equipment, has strong flexibility, but has a limited sensing range, high cost, and requires corresponding equipment to be installed at the vehicle end. The other method is to use roadside intelligent sensors to sense traffic participants on the road, judge intersection warning events, and report the results to vehicles or platforms. However, this method is highly dependent on the accuracy of the perceived target vehicle speed and heading angle. Moreover, the accuracy of vehicle speed and heading angle requires a very high accuracy for the sensing model. Currently, mainstream object detection models cannot guarantee very high accuracy in complex scenarios. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an intersection collision warning method, device, equipment, medium and product based on roadside perception, which does not require corresponding equipment to be installed at the vehicle end, can judge the intersection collision warning information of all vehicles within the intersection range in real time through roadside perception, effectively reduce costs, and determine the driving state of the vehicle by fitting the historical trajectory points of the vehicle with image pixel coordinates, effectively reducing the dependence on the accuracy of the model perception.

[0004] To achieve the above object, an embodiment of the present invention provides an intersection collision warning method based on roadside perception, including:

[0005] Obtain an intersection area image collected by a roadside camera in a port scene;

[0006] Detect and track traffic participants in the intersection area image according to an object detection algorithm, and obtain the types and detection frames of the traffic participants;

[0007] Traverse all the traffic participants, and screen the vehicles to be warned according to the types;

[0008] Fit the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame, and determine the driving state of the vehicle to be warned;

[0009] Judge whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned. If so, report an intersection collision warning.

[0010] As an improvement to the above solution, fitting the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame to determine the driving state of the vehicle to be warned includes:

[0011] Calculating the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame;

[0012] Calculating the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to determine whether the vehicle to be warned is in a stationary state;

[0013] Fitting the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determining the driving state of the vehicle to be warned according to the slope of the driving trajectory equation.

[0014] As an improvement to the above solution, the pixel coordinates of the detection frame include the coordinates of the upper left corner point and the lower right corner point. Then, calculating the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame includes:

[0015] Calculating the first center point coordinates of the first detection frame according to the pixel coordinates of the first detection frame;

[0016] Calculating the second center point coordinates of the second detection frame according to the pixel coordinates of the second detection frame;

[0017] Calculating the difference between the first center point coordinates and the second center point coordinates, and judging whether the driving direction of the vehicle to be warned is upward or downward according to the difference on the y-axis.

[0018] As an improvement to the above solution, calculating the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to determine whether the vehicle to be warned is in a stationary state includes:

[0019] Calculating the center point coordinates of the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame;

[0020] Calculating the fluctuation variances on the x-axis and y-axis respectively according to the center point coordinates of the historical trajectory points;

[0021] If the fluctuation variances on the x-axis and y-axis are both less than the first preset threshold, it is determined that the vehicle to be warned is in a stationary state.

[0022] As an improvement to the above solution, fitting the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determining the driving state of the vehicle to be warned according to the slope of the driving trajectory equation includes:

[0023] Calculate the center point coordinates of the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection box;

[0024] Fit the driving trajectory equation of the vehicle to be warned according to the center point coordinates of the historical trajectory points;

[0025] Calculate the difference between the slope of the driving trajectory equation and the slope of the lane equation;

[0026] If the difference is less than the second preset threshold, it is determined that the vehicle to be warned is driving along the lane.

[0027] As an improvement to the above solution, the preset intersection collision warning area includes warning start lines and warning end lines in different driving directions, and both the warning start line and the warning end line adopt a segmented form.

[0028] As an improvement to the above solution, determining whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned includes:

[0029] Calculate the distances between the upper left corner point coordinates and the lower right corner point coordinates of the detection box and the warning start line and the warning end line respectively;

[0030] Judge whether the minimum distance value between the upper left corner point coordinates of the detection box and the upward warning start line is greater than the vehicle type preset threshold of the vehicle to be warned; if so, it is determined that the front of the vehicle to be warned has entered the upward warning start line, otherwise it has not entered;

[0031] Judge whether the minimum distance value between the lower right corner point coordinates of the detection box and the downward warning start line is greater than the vehicle type preset threshold of the vehicle to be warned; if so, it is determined that the front of the vehicle to be warned has entered the downward warning start line, otherwise it has not entered.

[0032] An embodiment of the present invention further provides an intersection collision warning device based on roadside perception, including:

[0033] An image acquisition module, configured to acquire an intersection area image collected by a roadside camera in a port scene;

[0034] A target detection module, configured to detect and track traffic participants in the intersection area image according to a target detection algorithm, and obtain the types of the traffic participants and detection boxes;

[0035] A vehicle screening module, configured to traverse all the traffic participants and screen the vehicle to be warned according to the type;

[0036] A trajectory fitting module, configured to fit historical trajectory points of the vehicle to be warned according to pixel coordinates of the detection frame, and determine a driving state of the vehicle to be warned;

[0037] A collision warning module, configured to determine whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned, and if so, report an intersection collision warning.

[0038] An embodiment of the present invention further provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the method for intersection collision warning based on roadside perception described in any one of the above is implemented.

[0039] An embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program. When the computer program runs, the device where the computer-readable storage medium is located is controlled to execute the method for intersection collision warning based on roadside perception described in any one of the above.

[0040] An embodiment of the present invention further provides a computer program product. The computer program product includes a computer program or computer instructions. When the computer program or the computer instructions are executed by a processor, the method for intersection collision warning based on roadside perception described in any one of the above is implemented.

[0041] Compared with the prior art, the beneficial effects of a method, device, equipment, medium, and product for intersection collision warning based on roadside perception provided by an embodiment of the present invention are as follows: By acquiring an intersection area image collected by a roadside camera in a port scenario; detecting and tracking traffic participants in the intersection area image according to a target detection algorithm to obtain types of the traffic participants and detection frames; traversing all the traffic participants and screening the vehicle to be warned according to the types; fitting historical trajectory points of the vehicle to be warned according to pixel coordinates of the detection frame to determine a driving state of the vehicle to be warned; and determining whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned, and if so, reporting an intersection collision warning. In the embodiment of the present invention, there is no need to install corresponding equipment at the vehicle end, and intersection collision warning information of all vehicles within the intersection range is judged in real time through roadside perception, effectively reducing costs, and the driving state of the vehicle is determined by fitting the vehicle historical trajectory points with image pixel coordinates, effectively reducing the dependence on the model perception accuracy. Description of the Drawings

[0042] Figure 1 It is a schematic flowchart of a preferred embodiment of a method for intersection collision warning based on roadside perception provided by the present invention;

[0043] Figure 2 It is a schematic structural diagram of a preferred embodiment of an intersection collision warning device based on roadside perception provided by the present invention;

[0044] Figure 3 It is a schematic structural diagram of a preferred embodiment of a terminal device provided by the present invention. Detailed implementation manners

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] Please refer to Figure 1 , Figure 1 It is a schematic flowchart of a preferred embodiment of an intersection collision warning method based on roadside perception provided by the present invention. The intersection collision warning method based on roadside perception includes:

[0047] S1, obtaining an intersection area image collected by a roadside camera in a port scene;

[0048] S2, detecting and tracking traffic participants in the intersection area image according to an object detection algorithm to obtain the types and detection frames of the traffic participants;

[0049] S3, traversing all the traffic participants and screening the vehicles to be warned according to the types;

[0050] S4, fitting the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame to determine the driving state of the vehicle to be warned;

[0051] S5, judging whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned. If so, report an intersection collision warning.

[0052] Specifically, the embodiment of the present invention provides a low-cost intersection collision warning method based on roadside perception for the port scenario. It should be noted that the road surface in the port scenario is flat, without obstacles such as buildings and greenery, and there are relatively high roadside facilities for installing equipment. Compared with the ordinary intersection scenario, the vision is relatively open (a wide-angle camera can be installed on the roadside at a higher position without relying on vehicle-side cameras and lidar). The operating speed of the vehicles operating in it is relatively slow compared to ordinary intersections, and the actual requirements for warning accuracy and latency are not very high. Therefore, a low-cost algorithm with acceptable accuracy can be adopted. Since there are many operating vehicles, and most of them are manned and unmanned container trucks, the cost of adding additional vehicle-end cameras, radars and other equipment is relatively high. Therefore, using hardware such as roadside wide-angle cameras can save more costs and meet the usage requirements of the port scenario. However, the accuracy of the camera is difficult to compare with that of lidar. Firstly, the accuracy of the current target detection model is difficult to reach 100%. Secondly, when performing target positioning, the internal and external parameters of the camera are required, and the calibration accuracy of the internal and external parameters and external factors such as weather will have a very large impact on this, and the positioning results are often not so accurate. For this reason, the embodiment of the present invention uses pixel coordinates to replace longitude and latitude coordinates and directly calculates the intersection collision warning on the image because the position of the vehicle in the picture is determined and is not affected by the internal and external parameters of the camera positioning. Secondly, whether it is a camera or lidar, there are certain accuracy problems with the detected speed and heading angle, and these are very important variable values in the intersection collision warning algorithm. Therefore, the embodiment of the present invention determines the driving state of the vehicle by fitting the historical trajectory points of the vehicle through image pixel coordinates, effectively avoiding the problems caused by speed and heading angle errors.

[0053] Furthermore, in the embodiment of the present invention, first, an image of the intersection area collected by a roadside camera in the port scenario is obtained. Due to cost and environmental limitations, only one wide-angle camera is installed at each container yard intersection to monitor vehicles such as driverless and manned container trucks traveling on the main road, and one wide-angle camera is installed on the road in the container yard to monitor container yard operation vehicles, etc. Then, according to the target detection algorithm, for example, the YOLOV6+SORT algorithm, traffic participants in the intersection area image are detected and tracked to obtain the IDs, types, and detection frames of the traffic participants. All traffic participants are traversed, and the vehicles to be warned are screened according to their types. The historical trajectory points of the vehicles to be warned are fitted according to the pixel coordinates of the detection frames to determine the driving states of the vehicles to be warned. Finally, it is judged whether the vehicles to be warned are in the preset intersection collision warning area according to the driving states of the vehicles to be warned. If so, the intersection collision warning is reported to the platform, and the platform conducts traffic analysis and issues it to the corresponding vehicles. Exemplarily, when there is a collision risk between a vehicle traveling on the main road and a vehicle driving out of the container yard, an intersection collision warning event needs to be reported to the platform. Among them, the standard for evaluating the reporting accuracy is that if a warning is reported between the warning start time (the time when the vehicle enters the main road area of the monitored container yard and there is a collision risk) and the warning end time (when the vehicle drives out of the intersection and there is no collision risk), it is determined that the warning event is reported successfully. The above solution has been verified by the project. One camera can cover one port intersection, and the reporting accuracy of intersection collision warning events reaches more than 95%.

[0054] In the embodiment of the present invention, there is no need to install corresponding equipment on the vehicle end. The intersection collision warning information of all vehicles within the intersection range is judged in real time through roadside perception, effectively reducing costs. Roadside perception has strong universality, makes up for the perception deficiencies of single-vehicle intelligence, and can also provide relevant services for non-intelligent vehicles, effectively expanding the safety range of vehicle travel. Secondly, roadside perception can share the hardware cost, can be deployed on a large scale, reduce the high computing power demand on the vehicle end, and reduce the delay impact of cloud computing and transmission, which plays a crucial role in realizing future full autonomous driving. The embodiment of the present invention can realize the warning reminder for all vehicles in the intersection warning area at a low cost. Its warning information can be reported to the platform for traffic situation analysis and can also be issued by the platform to the vehicle end to provide more effective safety information for drivers. Moreover, for the impact brought by the perception accuracy and positioning accuracy, the driving state of the vehicle is determined by fitting the historical trajectory points of the vehicle through the image pixel coordinates, effectively reducing the dependence on the model perception accuracy and positioning accuracy.

[0055] In another preferred embodiment, step S4, fitting the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame to determine the driving state of the vehicle to be warned, includes:

[0056] S401. Calculate the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame;

[0057] S402. Calculate the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to determine whether the vehicle to be warned is in a stationary state;

[0058] S403. Fit to obtain the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determine the driving state of the vehicle to be warned according to the slope of the driving trajectory equation.

[0059] Specifically, in view of the problem of poor long-distance positioning accuracy in the embodiments of the present invention, since the position of the vehicle in the picture is determined and not affected by internal and external camera positioning parameters, pixel coordinates are used instead of longitude and latitude coordinates for position judgment. In view of the problem of unstable vehicle heading angle, it is difficult for 2D object detection algorithms to determine the vehicle head, and due to the influence of surrounding objects, the calculated heading angle fluctuates greatly, and it is difficult to determine the accurate driving direction of the vehicle through the heading angle. Therefore, a method of fitting the motion trajectory is adopted to judge the driving direction of the vehicle. Calculate the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame. Fit to obtain the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determine the driving state of the vehicle to be warned according to the slope of the driving trajectory equation. For long-distance slow-moving vehicles, a single object detection algorithm calculates their speeds inaccurately. In practice, there are often problems such as the vehicle speed of stationary vehicles not being 0 or the vehicle speed of slow-moving vehicles being 0. Embodiments of the present invention propose a new judgment method for this. Calculate the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to determine whether the vehicle to be warned is in a stationary state.

[0060] In another preferred embodiment, the pixel coordinates of the detection frame include the upper left corner point coordinates and the lower right corner point coordinates. Then, in S401, calculating the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame includes:

[0061] S4011. Calculate the first center point coordinates of the first detection frame according to the pixel coordinates of the first detection frame;

[0062] S4012. Calculate the second center point coordinates of the second detection frame according to the pixel coordinates of the second detection frame;

[0063] S4013. Calculate the difference between the first center point coordinates and the second center point coordinates, and judge whether the driving direction of the vehicle to be warned is upward or downward according to the difference on the y-axis.

[0064] Specifically, in the embodiments of the present invention, the pixel coordinates of the detection frame include the coordinates of the upper left corner point and the coordinates of the lower right corner point. Exemplarily, externally define a dictionary variable to save the 20 (set according to empirical values such as the perception message frequency) historical trajectory points of the vehicle closest to the vehicle. The trajectory data is the 2D detection frame data (coordinates of the upper left corner point, coordinates of the lower right corner point) of the vehicle. For example, if t0 = [lefttopx, lefttopy, rightbottomx, rightbottomy], the information in the dictionary is as follows: {id1: [t0, t1,... t20], id2: [t0, t1,... t20],...}. The driving direction of the vehicle can be calculated through the historical trajectory points. Calculate the center point coordinates of the detection frames of t20 and t1, and judge whether the driving direction of the vehicle is upward or downward through the difference in the y-axis.

[0065] In yet another preferred embodiment, in step S402, according to the pixel coordinates of the detection frame, calculate the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and the y-axis to determine whether the vehicle to be warned is in a stationary state, including:

[0066] S4021, according to the pixel coordinates of the detection frame, calculate the center point coordinates of the historical trajectory points of the vehicle to be warned;

[0067] S4022, according to the center point coordinates of the historical trajectory points, calculate the fluctuation variances on the x-axis and the y-axis respectively;

[0068] S4023, if the fluctuation variances on both the x-axis and the y-axis are less than a first preset threshold, it is determined that the vehicle to be warned is in a stationary state.

[0069] Specifically, according to the characteristics of the port scene, vehicles travel slowly during operation, there are parked vehicles on the road, and 2D target detection obtains the vehicle speed by measuring the position (longitude and latitude points) change of the vehicle in two frames of images and the time interval between the two frames of images. However, due to positioning accuracy problems and the influence of surrounding changing targets, there are often situations where the vehicle speed of a slowly moving vehicle is 0, or the vehicle speed of a stationary vehicle is not 0. Therefore, the embodiments of the present invention propose a new method for judging whether a vehicle is stationary, which calculates the fluctuation conditions of the current vehicle's historical trajectory on the x-axis and the y-axis. By calculating the 20 historical trajectory center point coordinates x1, x2,... x20 and y1, y2,... y20, the variances s_x and s_y are respectively solved. When the variances are respectively less than a certain threshold, it can be determined that the vehicle is stationary. Among them, the selection of the threshold is related to the scene and is obtained through testing multiple stationary vehicles and slow-moving vehicles in actual projects, which belongs to empirical values.

[0070] In yet another preferred embodiment, in S403, according to the pixel coordinates of the detection frame, the driving trajectory equation of the vehicle to be warned is fitted, and the driving state of the vehicle to be warned is determined according to the slope of the driving trajectory equation, including:

[0071] S4031, according to the pixel coordinates of the detection frame, calculate the center point coordinates of the historical trajectory points of the vehicle to be warned;

[0072] S4032, according to the center point coordinates of the historical trajectory points, fit the driving trajectory equation of the vehicle to be warned;

[0073] S4033, calculate the difference between the slope of the driving trajectory equation and the slope of the lane equation;

[0074] S4034, if the difference is less than the second preset threshold, it is determined that the vehicle to be warned is driving along the lane.

[0075] Specifically, in the 2D object detection algorithm, the heading angle is obtained by calculating the angle between the vehicle speed direction and the due north direction. However, in actual projects, there are problems such as large fluctuations in the heading angle left and right. Therefore, the embodiment of the present invention proposes a new method for judgment. First, according to the pixel coordinates of the detection frame, the center point coordinates of the vehicle historical trajectory points points = [(x1, y1), (x2, y2),... (x20, y20)] are calculated. Then, the random sample consensus (RANSAC) algorithm is used for trajectory fitting to find the outliers (points far from the others) in the processed data. After fitting, a straight line equation y = kx + b can be obtained, where k is the slope of the straight line. When judging the vehicle trajectory direction by the slope, first, the lane point on the collected image is obtained to get the slope k0 of the lane straight line equation. If the difference between the vehicle trajectory slope k and the lane slope k0 is within a certain threshold, it can be determined that the vehicle is driving along the lane. Among them, the threshold setting is affected by the scene. In actual projects, different slope values are obtained by testing vehicles driving along the lane, vehicles driving out of the compartment area, and vehicles making U-turns, etc., so as to obtain the threshold range.

[0076] In yet another preferred embodiment, the preset intersection collision warning area includes warning start lines and warning end lines in different driving directions, and both the warning start line and the warning end line adopt a segmented form.

[0077] Specifically, in the embodiment of the present invention, according to the range to be monitored at the intersection, the warning start line and the warning end line for vehicles traveling in different directions are predefined. Taking the upward direction as an example, when the vehicle head crosses the upward warning start line, it indicates the start of the warning, and when the vehicle tail exits the upward warning end line, it indicates the end of the warning. Due to the positioning accuracy problem, when defining the area in the embodiment of the present invention, pixel coordinate points are used. For example, the upward warning start line is composed of a set of points [(x1, y1), (x2, y2),...], and it is segmented. Every 3 points form a segment. Through 3 points, the Bézier curve of this line segment can be obtained. It is a smooth parametric curve defined by a series of control points and can be expressed as [B1, B2,...].

[0078] In yet another preferred embodiment, determining whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned includes:

[0079] Calculating the distances between the coordinates of the upper left corner point and the lower right corner point of the detection frame and the warning start line and the warning end line respectively;

[0080] Judging whether the minimum distance value between the coordinate of the upper left corner point of the detection frame and the upward warning start line is greater than the preset vehicle type threshold of the vehicle to be warned; if so, it is determined that the vehicle head of the vehicle to be warned has entered the upward warning start line, otherwise it has not entered;

[0081] Judging whether the minimum distance value between the coordinate of the lower right corner point of the detection frame and the downward warning start line is greater than the preset vehicle type threshold of the vehicle to be warned; if so, it is determined that the vehicle head of the vehicle to be warned has entered the downward warning start line, otherwise it has not entered.

[0082] Specifically, in the embodiments of the present invention, it is determined whether the vehicle is in the area between the warning start line and the warning end line according to the driving state. According to scenarios such as not entering, just entering, fully entering, just leaving, and fully leaving, it is judged through the 2D detection box information and the warning line. The specific judgment is as follows: Calculate the distances between the coordinates of the upper left corner point and the lower right corner point of the detection box and the warning start line and the warning end line respectively. Taking the vehicle moving upward as an example, calculate the distances between the upper left corner point and the lower right corner point of the vehicle target box and the upward warning start section and the upward warning end section respectively, and find the nearest line segment index and the distance value, which can be expressed as [lefttop_nearest_start_idx, lefttop_nearest_start_dist], [lefttop_nearest_end_idx, lefttop_nearest_end_dist], [rightbottom_nearest_start_idx, rightbottom_nearest_start_dist], [rightbottom_nearest_end_idx, rightbottom_nearest_end_dist] for easy understanding. When the vehicle has not entered, its lefttopy is less than the minimum y value of the nearest upward warning start section; when the vehicle is just entering, due to 2D detection problems, the highest point of the target box is not the vehicle head. When lefttopy exceeds the upward warning start line, in fact, the vehicle head has not entered the warning area. This problem is also an unavoidable problem in 2D detection. In the embodiments of the present invention, to improve the impact brought by this problem, different crossing thresholds are set for judgment at different positions (warning line segmentation) for different vehicle types. Specifically, it is judged whether the distance value of the upper left corner point to the nearest upward section is greater than the preset threshold of its vehicle type. If it is greater, it means that the vehicle head has entered the upward warning start line, otherwise it has not entered. Among them, the threshold is selected through tests of different vehicle types. The detection model can output the vehicle type. In the port scenario, since the operation vehicle types are limited, it can be obtained through preset traversal tests. When the vehicle types are rich and the traversal test is difficult, considering that the camera field of view is fixed and the warning line position is fixed, different vehicle type detection boxes can be compared and distinguished at this position, so the threshold can be selected through the size of the detection box (since this scenario requires intersection collision warning judgment between the vehicle entering and leaving the warning area, the requirement for time delay is not very high, and it is not necessary to strictly control the threshold size, and it can be within a certain range). When the vehicle has fully entered, its lefttopy is less than the minimum y value of the nearest warning end section, and rightbottomy is greater than the maximum y value of the nearest warning start line. When the vehicle is just leaving or has fully left, due to the perspective relationship, it can be directly judged by the distance between rightbottomy and the nearest warning end section.

[0083] In view of the characteristics of flat, unobstructed road surface and low vehicle speed with low precision requirements in the port scenario, the embodiments of the present invention propose a low-cost roadside solution for full-port intersection collision warning, avoiding the influence brought by perception accuracy and positioning accuracy, and being more cost-saving compared with other solutions. By using a single roadside wide-angle camera to cover the intersection area for monitoring intersection collision warning events, in order to avoid the influence of camera extrinsic parameter errors and positioning accuracy caused by long distances, when judging whether a vehicle is in the intersection collision warning area and during state judgment, pixel coordinates are used instead of longitude and latitude coordinates for calculation. At the same time, considering that there are certain errors in the vehicle speed and heading angle obtained by 2D detection under low speed and occlusion conditions, the embodiments of the present invention use the fluctuation variances of historical trajectory points on the x-axis and y-axis to determine whether the vehicle is in a parked state, and use the slope fitted by historical trajectory points to judge the driving state of the vehicle, greatly reducing the false alarm rate of intersection collision warning for stationary vehicles and vehicles making U-turns or driving out of the box area.

[0084] Correspondingly, the present invention also provides an intersection collision warning device based on roadside perception, which can implement all processes of the intersection collision warning method based on roadside perception in the above embodiments.

[0085] Please refer to Figure 2 , Figure 2 FIG. is a schematic structural diagram of a preferred embodiment of an intersection collision warning device based on roadside perception provided by the present invention. The intersection collision warning device based on roadside perception includes:

[0086] An image acquisition module 201, configured to acquire an image of the intersection area collected by a roadside camera in the port scenario;

[0087] A target detection module 202, configured to detect and track traffic participants in the intersection area image according to a target detection algorithm, and obtain the types and detection frames of the traffic participants;

[0088] A vehicle screening module 203, configured to traverse all the traffic participants and screen the vehicles to be warned according to the types;

[0089] A trajectory fitting module 204, configured to fit the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame, and determine the driving state of the vehicle to be warned;

[0090] A collision warning module 205, configured to judge whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned, and if so, report an intersection collision warning.

[0091] Preferably, the trajectory fitting module 204 is specifically configured to:

[0092] Calculate the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame;

[0093] Calculate the variance of fluctuations of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to determine whether the vehicle to be warned is in a stationary state;

[0094] Fit the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determine the driving state of the vehicle to be warned according to the slope of the driving trajectory equation.

[0095] Preferably, the pixel coordinates of the detection frame include the upper left corner point coordinates and the lower right corner point coordinates. Then, calculating the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame includes:

[0096] Calculate the first center point coordinates of the first detection frame according to the pixel coordinates of the first detection frame;

[0097] Calculate the second center point coordinates of the second detection frame according to the pixel coordinates of the second detection frame;

[0098] Calculate the difference between the first center point coordinates and the second center point coordinates, and judge the driving direction of the vehicle to be warned as going up or down according to the difference on the y-axis.

[0099] Preferably, calculating the variance of fluctuations of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to determine whether the vehicle to be warned is in a stationary state includes:

[0100] Calculate the center point coordinates of the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame;

[0101] Calculate the variance of fluctuations on the x-axis and y-axis respectively according to the center point coordinates of the historical trajectory points;

[0102] If the variances of fluctuations on the x-axis and y-axis are both less than the first preset threshold, it is determined that the vehicle to be warned is in a stationary state.

[0103] Preferably, fitting the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determining the driving state of the vehicle to be warned according to the slope of the driving trajectory equation includes:

[0104] Calculate the center point coordinates of the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame;

[0105] Fit the driving trajectory equation of the vehicle to be warned according to the center point coordinates of the historical trajectory points;

[0106] Calculate the difference between the slope of the calculated driving trajectory equation and the slope of the lane equation;

[0107] If the difference is less than a second preset threshold, it is determined that the vehicle to be warned is driving along the lane.

[0108] Preferably, the preset intersection collision warning area includes warning start lines and warning end lines in different driving directions, and both the warning start line and the warning end line are in a segmented form.

[0109] Preferably, determining whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned includes:

[0110] Calculate the distances between the coordinates of the upper left corner point and the lower right corner point of the detection frame and the warning start line and the warning end line respectively;

[0111] Judge whether the minimum distance value between the coordinates of the upper left corner point of the detection frame and the upward warning start line is greater than the vehicle type preset threshold of the vehicle to be warned; if so, it is determined that the front of the vehicle to be warned has entered the upward warning start line, otherwise it has not entered;

[0112] Judge whether the minimum distance value between the coordinates of the lower right corner point of the detection frame and the downward warning start line is greater than the vehicle type preset threshold of the vehicle to be warned; if so, it is determined that the front of the vehicle to be warned has entered the downward warning start line, otherwise it has not entered.

[0113] In specific implementation, the working principle, control process and achieved technical effects of the intersection collision warning device based on roadside perception provided by the embodiments of the present invention are the same as those of the intersection collision warning method based on roadside perception in the above embodiments, and will not be elaborated here.

[0114] Please refer to Figure 3 , Figure 3 is a schematic structural diagram of a preferred embodiment of a terminal device provided by the present invention. The terminal device includes a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301. When the processor 301 executes the computer program, it implements the intersection collision warning method based on roadside perception described in any of the above embodiments.

[0115] Preferably, the computer program may be divided into one or more modules / units (such as computer program 1, computer program 2, ……), and the one or more modules / units are stored in the memory 302 and executed by the processor 301 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the terminal device.

[0116] The processor 301 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 301 may also be any conventional processor. The processor 301 is the control center of the terminal device and connects various parts of the terminal device through various interfaces and lines.

[0117] The memory 302 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc., and the data storage area can store relevant data, etc. In addition, the memory 302 may be a high-speed random access memory, or may also be a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., or the memory 302 may also be other volatile solid-state storage devices.

[0118] It should be noted that the above terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 3 the structural schematic diagram is only an example of the above terminal device and does not constitute a limitation on the above terminal device. It may include more or fewer components than shown in the figure, or combine certain components, or different components.

[0119] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the intersection collision warning method based on roadside perception described in any one of the above embodiments.

[0120] An embodiment of the present invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or the computer instructions are executed by a processor, they implement the intersection collision warning method based on roadside perception described in any one of the above embodiments.

[0121] An embodiment of the present invention provides an intersection collision warning method, device, equipment, medium and product based on roadside perception. By obtaining the intersection area image collected by the roadside camera in the port scene; detecting and tracking traffic participants in the intersection area image according to the target detection algorithm to obtain the types and detection frames of the traffic participants; traversing all the traffic participants, screening the vehicles to be warned according to the types; fitting the historical trajectory points of the vehicles to be warned according to the pixel coordinates of the detection frames to determine the driving states of the vehicles to be warned; judging whether the vehicles to be warned are in the preset intersection collision warning area according to the driving states of the vehicles to be warned. If so, report the intersection collision warning. In the embodiment of the present invention, there is no need to install corresponding equipment on the vehicle end, and the intersection collision warning information of all vehicles within the intersection range is judged in real time through roadside perception, effectively reducing the cost. And the driving state of the vehicle is determined by fitting the historical trajectory points of the vehicle through the image pixel coordinates, effectively reducing the dependence on the model perception accuracy.

[0122] It should be noted that the system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0123] The above is the preferred embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches are also regarded as the protection scope of the present invention.

Claims

1. An intersection collision warning method based on roadside perception, characterized in that, Including: Obtain the intersection area image collected by the roadside camera in the port scene; Detect and track traffic participants in the intersection area image according to the target detection algorithm to obtain the types of the traffic participants and detection frames; Traverse all the traffic participants and screen the vehicles to be warned according to the types; Fit the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame to determine the driving state of the vehicle to be warned; Judge whether the vehicle to be warned is in the preset intersection collision warning area according to the driving state of the vehicle to be warned. If so, report the intersection collision warning.

2. The intersection collision warning method based on roadside perception according to claim 1, characterized in that, The step of fitting the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame to determine the driving state of the vehicle to be warned includes: Calculate the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame; Calculate the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to judge whether the vehicle to be warned is in a stationary state; Fit to obtain the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame, and determine the driving state of the vehicle to be warned according to the slope of the driving trajectory equation.

3. The intersection collision warning method based on roadside perception according to claim 2, wherein If the pixel coordinates of the detection frame include the upper left corner point coordinates and the lower right corner point coordinates, then the step of calculating the driving direction of the vehicle to be warned according to the pixel coordinates of the detection frame includes: Calculate the first center point coordinates of the first detection frame according to the pixel coordinates of the first detection frame; Calculate the second center point coordinates of the second detection frame according to the pixel coordinates of the second detection frame; Calculate the difference between the first center point coordinates and the second center point coordinates, and judge the driving direction of the vehicle to be warned as going up or down according to the difference on the y-axis.

4. The intersection collision warning method based on roadside perception according to claim 3, characterized in that The step of calculating the fluctuation variances of the historical trajectory points of the vehicle to be warned on the x-axis and y-axis according to the pixel coordinates of the detection frame to judge whether the vehicle to be warned is in a stationary state includes: Calculate the center point coordinates of the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame; Calculate the fluctuation variances on the x-axis and y-axis respectively according to the center point coordinates of the historical trajectory points; If the fluctuation variances on the x-axis and y-axis are both less than the first preset threshold, it is determined that the vehicle to be warned is in a stationary state.

5. The intersection collision warning method based on roadside perception according to claim 4, characterized in that, The step of fitting to obtain the driving trajectory equation of the vehicle to be warned according to the pixel coordinates of the detection frame and determining the driving state of the vehicle to be warned according to the slope of the driving trajectory equation includes: Calculate the center point coordinates of the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame; Fit to obtain the driving trajectory equation of the vehicle to be warned according to the center point coordinates of the historical trajectory points; Calculate the difference between the slope of the driving trajectory equation and the slope of the lane equation; If the difference is less than the second preset threshold, it is determined that the vehicle to be warned is driving along the lane.

6. The intersection collision warning method based on roadside perception according to claim 1, wherein The preset intersection collision warning area includes warning start lines and warning end lines in different driving directions, and both the warning start line and the warning end line are in a segmented form.

7. The intersection collision warning method based on roadside perception according to claim 6, characterized in that, Determining whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned includes: Calculating the distances between the coordinates of the upper left corner point and the lower right corner point of the detection frame and the warning start line and the warning end line respectively; Judging whether the minimum distance value between the coordinate of the upper left corner point of the detection frame and the upper warning start line is greater than the vehicle type preset threshold of the vehicle to be warned; if so, it is determined that the front of the vehicle to be warned has entered the upper warning start line, otherwise it has not entered; Judging whether the minimum distance value between the coordinate of the lower right corner point of the detection frame and the lower warning start line is greater than the vehicle type preset threshold of the vehicle to be warned; if so, it is determined that the front of the vehicle to be warned has entered the lower warning start line, otherwise it has not entered.

8. An intersection collision warning device based on roadside perception, characterized in that, It includes: An image acquisition module for acquiring an intersection area image collected by a roadside camera in a port scene; A target detection module for detecting and tracking traffic participants in the intersection area image according to a target detection algorithm to obtain the types of the traffic participants and detection frames; A vehicle screening module for traversing all the traffic participants and screening the vehicle to be warned according to the types; A trajectory fitting module for fitting the historical trajectory points of the vehicle to be warned according to the pixel coordinates of the detection frame to determine the driving state of the vehicle to be warned; A collision warning module for judging whether the vehicle to be warned is in a preset intersection collision warning area according to the driving state of the vehicle to be warned, and if so, reporting an intersection collision warning.

9. A terminal device, characterized in that, It includes a processor and a memory, and a computer program is stored in the memory, and the computer program is configured to be executed by the processor. When the processor executes the computer program, the method for intersection collision warning based on roadside perception according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program. When the device where the computer-readable storage medium is located executes the computer program, the method for intersection collision warning based on roadside perception according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions. When the computer program or the computer instructions are executed by a processor, the method for intersection collision warning based on roadside perception according to any one of claims 1 to 7 is implemented.

Images