A pedestrian crossing collision risk warning method and system based on holographic perception

Abstract

The present invention discloses a pedestrian crossing collision risk warning method and system based on holographic perception. The method includes the following steps: collecting data of pedestrians and vehicles in a non-signal-controlled section based on holographic perception devices; constructing a prediction model of speed using the LSTM algorithm to predict the trajectories of pedestrians and vehicles, and determining the risk analysis range. The risk analysis range is divided into N sub-regions, and the upper and lower limits of the speed of each sub-region are calculated using the empirical cumulative distribution function to determine the corresponding potential risk region for each sub-region; the shape of the potential risk region is assumed to be a rectangle with a certain length and width; determining the time difference TDTC when the potential risk regions of pedestrians and vehicles coincide with the potential conflict point; determining the threshold value of TDTC by plotting the cumulative frequency curve of TDTC; dividing the risk types according to the comparison between the detected value and the threshold, and performing real-time warning. The present invention improves the accuracy of the pedestrian crossing warning timing and the accuracy of the warning.

Description

Technical Field

[0001] The present invention belongs to the field of road traffic safety, and particularly relates to a pedestrian crossing collision risk warning method and system based on holographic perception. Background Art

[0002] According to the "2020 Annual Report on Road Traffic Accident Statistics", the number of accidents at crosswalks is about 4,395, and the cumulative number of deaths is 960. The potential safety hazards for pedestrians, who are in a vulnerable position among road users, when passing through sections without signal control, especially crosswalks, cannot be ignored.

[0003] Current pedestrian crossing warning devices are mainly divided into traditional pedestrian crossing risk warning devices and vehicle networking pedestrian crossing risk warning devices. Traditional pedestrian crossing risk warning devices adopt a trigger-based response warning method based on cross-section vehicle and pedestrian traffic detection data, that is, when it is detected that a pedestrian is about to enter the crosswalk and a detector set in front of the crosswalk detects a vehicle passing by, the response mechanism is immediately triggered to send a stop-and-yield warning to the vehicle. This method does not consider the spatio-temporal dynamic changes of traffic elements such as vehicle and pedestrian positions and speeds, the warning timing is inaccurate, and the warning accuracy is not high. The vehicle networking pedestrian crossing risk warning devices only focus on the dynamic changes of vehicle traffic status, and the detection of pedestrian status only stays at the level of "whether there are pedestrians crossing the street", without considering the dynamic changes of traffic elements such as pedestrian position and speed, and without considering the avoidance measures of pedestrians when there is a collision risk, and cannot accurately judge the interaction status between vehicles and pedestrians, and the warning accuracy is also not high. Therefore, both types of devices have certain limitations.

[0004] The holographic perception device integrates a camera, a millimeter-wave radar and a high-performance processor, performs fusion tracking processing on video targets and radar targets, so as to realize real-time vectorization and tracking of global targets. It can obtain continuous trajectory data and information such as the positions, speeds and moving directions of vehicles and pedestrians, and has strong real-time performance.

[0005] In addition, in terms of risk warning methods, traffic conflict technology is usually used to judge collision risks and risk prediction is carried out according to trajectory data information. Most of the current such prediction methods analyze based on the conflict critical situations between the predicted trajectory points, and there are certain errors in the positions of the next moment trajectory points predicted by the trajectory prediction algorithm. If the positions of the trajectory points of the actual trajectory are predicted, it is very likely that the accumulated errors will become larger and larger, thus affecting the prediction effect. Summary of the Invention

[0006] The object of the present invention is to provide a pedestrian crossing collision risk warning method and system based on holographic perception, so as to solve the problems of inaccurate warning timing, low warning accuracy and error in trajectory prediction of existing pedestrian crossing warning devices.

[0007] The technical solution of the present invention is as follows:

[0008] A pedestrian crossing collision risk warning method based on holographic perception, comprising the following steps:

[0009] Using holographic perception technology based on holographic perception devices to collect the position, movement direction and speed data of pedestrians and vehicles in a section without signal control;

[0010] According to the collected data, use the LSTM algorithm to construct a prediction model of speed to predict the trajectories of pedestrians and vehicles, and determine the risk analysis range. Divide the risk analysis range into N sub-regions, and use the empirical cumulative distribution function to calculate the upper and lower limits of the speed of each sub-region. Determine the corresponding potential risk area of each sub-region according to the upper and lower limits of the speed of each sub-region; the shape of the potential risk area is assumed to be a rectangle with a certain length and width;

[0011] Combining the potential risk areas of pedestrians and vehicles, determine the time difference when the potential risk areas of the two coincide with the potential conflict points, that is, the conflict time difference TDTC, as the safety evaluation index for the conflict between pedestrians and vehicles; among them, the potential conflict point is the intersection of the trajectories of pedestrians and vehicles; Determine the threshold value of TDTC by drawing the cumulative frequency curve of TDTC through the cumulative frequency curve method;

[0012] According to the comparison between the detected conflict time difference TDTC value and the threshold, divide the risk types and give warnings.

[0013] According to the above solution, the holographic perception device adopts a radar-vision integrated machine. The radar-vision fusion simultaneously accesses the original video stream and radar data stream into the embedded processor in the integrated machine through the MIPI and SPI interfaces. Directly perform AI target extraction on the original video stream in the built-in embedded processor, then project the video target into the radar coordinate system through the built-in coordinate mapping system, and finally perform fusion tracking processing on the video target and radar target to realize real-time vectorization and tracking of global targets.

[0014] According to the above solution, the trajectory is expressed as:

[0015] T v / p ={L 1 ,L 2 ,L 3 ,…,L n}

[0016] In the formula, T v represents the trajectory of the vehicle; Tp Represents the trajectory of a pedestrian; L i =(x i , y i ) represents the position of a pedestrian / vehicle at the i-th moment;

[0017] The speed prediction model for predicting the trajectories of pedestrians and vehicles is as follows:

[0018] L i+1 =(x i+1 , y i+1 )=(x i +Δx, y i +Δy)

[0019] Δx = v x ×Δt, Δy = v y ×Δt

[0020] In the formula, x i , y i respectively represent the horizontal and vertical coordinates of a pedestrian / vehicle at the i-th moment, x i+1 , y i+1 respectively represent the horizontal and vertical coordinates of a pedestrian / vehicle at the (i + 1)-th moment, Δx and Δy respectively represent the corresponding coordinate increments, v x , v y respectively represent the instantaneous speed values in the horizontal and vertical directions, and Δt represents the time step.

[0021] According to the above scheme, Δt takes a value of 2 s.

[0022] According to the above scheme, the risk analysis range is a section of a specific length in front of the crosswalk.

[0023] According to the above scheme, calculating the upper and lower limits of the speed of each sub-region using the empirical cumulative distribution function includes:

[0024] For the speed data x 1 , x 2 , …, x n of each sub-region, construct its distribution function:

[0025]

[0026] E(F n ′ (x)) = F(x)

[0027] In the formula, is the frequency of speeds less than x, and n is the total number of speed data; the empirical cumulative distribution function ECDF is a non-decreasing function between 0 and 1. When n approaches infinity, for each x value, F n ′ (x) converges to F(x);

[0028] Define the confidence intervals for all speed data in this sub-region as follows: The confidence intervals for all speed data in this sub-region are as follows:

[0029]

[0030] where B(x) is the function of the confidence interval;

[0031] By the Dvoretzky-Kiefer-Wolfowitz inequality, the two-sided confidence interval is obtained, and the formula is as follows:

[0032]

[0033] L(x) = max{F n ′ (x) - ε, 0}

[0034] U(x) = min{F n ′ (x) + ε, 1}

[0035]

[0036] where sup is the upper bound function; L(x) is the lower bound, and U(x) is the upper bound, that is, the upper and lower limits of the speed.

[0037] According to the above scheme, the TDTC values corresponding to the 85% cumulative frequency and the 15% cumulative frequency are used as the critical values for potential risk and general risk, and general risk and severe risk, respectively.

[0038] According to the above scheme, real-time warning is carried out by setting variable message signs on the roadside.

[0039] According to the above scheme, when the risk is potential risk, the content displayed on the variable message sign is set to "Yield to pedestrians"; when the risk is general risk, the content displayed on the variable message sign is set to "License plate information, your current speed is XX, please slow down and avoid pedestrians"; when the risk is severe risk, the content displayed on the variable message sign is set to "License plate information, your current speed is XX, please stop and avoid pedestrians"; if the vehicle does not stop and give way when the warning corresponding to the severe risk is carried out, it will be captured, and the continuous trajectory data will be stored to provide relevant evidence for subsequent handling.

[0040] A holographic perception-based pedestrian crossing collision risk warning system for implementing the holographic perception-based pedestrian crossing collision risk warning method described in any one of the above, comprising:

[0041] A holographic perception device for collecting the position, movement direction and speed data of pedestrians and vehicles in a non-signal-controlled section;

[0042] Variable message sign, used for real-time warning.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] The present invention is no longer a warning system based on sectional data and trigger response mechanism. Instead, it obtains continuous trajectory data of vehicles and pedestrians when pedestrians cross the street at signal-free control sections through holographic sensing devices, taking into account the spatio-temporal change dynamics of basic traffic elements of vehicles and pedestrians. The dynamic changes in information such as the speeds of vehicles and pedestrians can reflect the avoidance measures taken by vehicles and pedestrians. It can judge the interaction state between vehicles and pedestrians in real time, divide the collision risk levels, and issue different warnings corresponding to their levels. In this way, it improves the accuracy of the warning timing and the accuracy of the warning, greatly enhances the efficiency of yielding to pedestrians, and also solves the defects of drivers in obtaining information and the problem of poor sight distance, improving the safety of pedestrians crossing the street at sections. At the same time, it can also collect evidence for behaviors violating the regulation of yielding to pedestrians based on the entire continuous trajectory information and can provide relevant trajectory data support.

[0045] The present invention obtains the dynamic information of the traffic states of vehicles and pedestrians in real time through holographic sensing devices, discriminates the interaction state between vehicles and pedestrians, and at the same time takes into account the avoidance measures of vehicles and pedestrians, solving the problems that both traditional pedestrian crossing risk warning devices and vehicle-to-internet pedestrian crossing risk warning devices do not simultaneously consider the spatio-temporal change dynamics of the positions of vehicles and pedestrians, with inaccurate warning timing and low warning accuracy. By obtaining this information, it can also solve problems such as poor sight distance, incomplete observation, and even sight distance blind spots when drivers are driving, reducing the collision risk of pedestrians crossing the street.

[0046] The present invention determines the size of the corresponding potential risk area by calculating the upper and lower limits of the speeds of each sub-region. This method takes into account the speed distribution when passing through different sub-regions within the analysis range, enabling the determined size range of the potential risk area to basically cover any position where vehicles / pedestrians may appear at the next moment, reasonably expanding the prediction range and improving the accuracy of the warning.

[0047] When conducting trajectory prediction, the present invention updates the calculation method of the safety evaluation index according to the concept of the potential risk area, considers any position where the vehicle / pedestrian may appear at the next moment, and replaces the overlap of the "point" with the potential conflict point with the overlap of the "plane", which reduces the influence brought by the cumulative error during trajectory prediction to a certain extent. That is, in terms of trajectory prediction, the prediction method is updated, the prediction range is expanded according to any position where the vehicle / pedestrian at the current position may appear at the next moment, and an attempt is made to replace the "point" with the "plane". Applying this method to the calculation of the safety evaluation index and updating the calculation method of the index by replacing the overlap of the "point" with the overlap of the "plane" can improve the accuracy of trajectory prediction and the accuracy of collision risk warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flowchart of the pedestrian crossing collision risk warning method based on holographic perception of the present invention;

[0049] Figure 2 is a specification diagram for determining the shape and size of the potential risk area of the present invention;

[0050] Figure 3 is a layout diagram of the pedestrian crossing collision risk warning system based on holographic perception of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0052] Aiming at the deficiencies in the existing research on the detection of pedestrian crossing collision risks on roads, such as inaccurate warning timing, low warning accuracy, and errors in trajectory prediction caused by not considering the spatio-temporal variation dynamics of traffic elements such as the positions and speeds of vehicles and pedestrians at the same time, the present invention provides a short-term position prediction method for traffic elements based on holographic perception trajectory data and the fusion of the LSTM algorithm, establishes a collision risk prediction model for human-vehicle cooperation based on spatio-temporal data of traffic elements, and designs a pedestrian crossing collision risk warning system with the above functions for vehicle travel.

[0053] The pedestrian crossing collision risk warning method based on holographic perception of the present invention includes the following steps:

[0054] Step 1: Use holographic perception technology based on holographic perception devices to collect basic data such as the positions, speeds, movement directions, and trajectories of pedestrians and vehicles in a non-signal-controlled section, and determine the risk analysis range;

[0055] Step 2: According to the data collected in Step 1 and the trajectory file, use the LSTM algorithm to predict the trajectories of pedestrians and vehicles, and determine potential risk areas;

[0056] Step 3: Use the conflict detection algorithm, combined with the potential risk areas, to calculate the safety evaluation index value; determine the threshold value of the safety evaluation index through the cumulative frequency curve method, and divide the corresponding different risk types according to the comparison between the detected value and the threshold;

[0057] Step 4: For different risk types and vehicle-related information, set variable message signs on the roadside to give real-time warnings to vehicles.

[0058] Among them, the specific content of the holographic perception technology in Step 1 is as follows:

[0059] The holographic perception device can adopt a radar-vision integrated machine, which is a traffic sensor that combines a camera, a millimeter-wave radar, and a high-performance processor. The radar-vision fusion simultaneously accesses the original video stream and radar data stream into the embedded processor in the integrated machine through the MIPI and SPI interfaces. In the built-in embedded processor, directly perform AI target extraction on the original video stream, then project the video target into the radar coordinate system through the built-in coordinate mapping system, and finally perform fusion tracking processing on the video target and the radar target to achieve real-time vectorization and tracking of the global target. It can obtain continuous trajectory data and information such as the positions, speeds, and movement directions of vehicles and pedestrians, with strong real-time performance.

[0060] The specific method of using the LSTM algorithm for trajectory prediction in Step 2 is as follows:

[0061] Assume that the trajectories extracted within continuous time stamps are also continuous. Then, according to the positions of the vehicles / pedestrians collected, their trajectories can be expressed as:

[0062] T v / p ={L 1 ,L 2 ,L 3 ,…,L n}

[0063] In the formula, T v represents the trajectory of the vehicle;

[0064] T p represents the trajectory of the pedestrian;

[0065] L i =(x i ,y i ) represents the position information of the vehicle / pedestrian at the i-th moment.

[0066] To specifically perform trajectory prediction, the trajectory prediction model constructed in the present invention is a speed prediction model. Considering the reaction time of the driver, the predicted time step is taken as 2 s, and the trajectory of the vehicle / pedestrian can be expressed as:

[0067] L i+1 =(x i+1 , y i+1 )=(x i +Δx, y i +Δy)

[0068] Δx = v x ×Δt, Δy = v y ×Δt

[0069] In the formula, x i , y i respectively represent the horizontal and vertical coordinates of the vehicle and pedestrian at the i-th moment;

[0070] x i+1 , y i+1 respectively represent the horizontal and vertical coordinates of the vehicle and pedestrian at the (i + 1)-th moment;

[0071] Δx and Δy respectively represent the corresponding coordinate increments;

[0072] v x , v y respectively represent the instantaneous speed values in the horizontal and vertical directions;

[0073] Δt represents the time step, which is taken as 2 s here.

[0074] The specific method for determining the potential risk area in step 2 is as follows:

[0075] Referring to the relevant regulations in the "Code for Design of Urban Road Engineering" CJJ37 - 2012, the maximum design speed of urban roads is 60 km / h, and the corresponding stopping sight distance is 70 m. Therefore, the analysis range is limited to the section 70 m before the crosswalk. Move the trajectory origin of all vehicles in the analysis range to the same point to observe the 1 - s trajectory of the vehicles. According to the trajectory characteristics of the vehicles on the section, assume the shape of the potential risk area as a rectangle with a certain length and width. Considering the different speed distributions in different areas of the section, in order to determine the size of the potential risk area in different areas, divide the analysis range into N sub - areas, and use the empirical cumulative distribution function to calculate the upper and lower limits of the speed of each sub - area to determine the size of the corresponding potential risk area in each sub - area.

[0076] The empirical cumulative distribution function (ECDF) provides a simple estimate of the probability distribution of F(x) by "best guessing" the correct form of the unknown distribution F(x). Given a dataset of independent and identically distributed random observations (historical speed data of a certain sub - area) x1 , x 2 , …, x n , the ECDF can obtain the following distribution:

[0077]

[0078] wherein, is the frequency of speeds less than x, and n is the total number of speed data.

[0079] The ECDF is a non - decreasing function between 0 and 1. When n approaches infinity, it is almost certain that for each x value, F′ n (x) converges to F(x). Meanwhile, the confidence interval of the ECDF is defined as a function of a fixed x value and a point - wise confidence interval for x, as follows: The function of the point - wise confidence interval is as follows:

[0080] E(F′ n (x)) = F(x)

[0081]

[0082] wherein, C(x) is the region given by the point - wise confidence interval. Although each x value adheres to the confidence interval (since the point - wise confidence interval is calculated for each x value), it cannot be guaranteed that C(x) simultaneously adheres to the confidence region for each individual x value. Instead, we can define the confidence interval for all x as follows: the confidence interval for all x is as follows:

[0083]

[0084] wherein, B(x) is the function of the confidence interval. Then, through the Dvoretzky - Kiefer - Wolfowitz (DKW) inequality, the formula for the two - sided confidence interval (lower bound L(x), upper bound U(x)) can be obtained as follows:

[0085]

[0086] L(x) = max{F′ n (x) - ε, 0}

[0087] U(x) = min{F′ n (x) + ε, 1}

[0088] wherein, sup is the ceiling function.

[0089] The size of the potential risk area within the corresponding sub-region is determined by calculating the ECDF of each sub-region and the upper and lower confidence interval boundaries L(x) and U(x). Different sub-regions determine the upper and lower speed limits based on historical data to determine the size of the potential risk area for each vehicle passing through the sub-region.

[0090] The specific content of the safety evaluation index in Step 3 is as follows:

[0091] To analyze the process of the mutual influence of vehicle-pedestrian conflicts, the present invention introduces the conflict time difference (TDTC) as a safety evaluation index for vehicle-pedestrian conflicts. Combining with the potential risk area, the definition of the conflict time difference (TDTC) is updated as: if the speeds of the vehicle and the pedestrian remain unchanged, then the time difference when their potential risk areas coincide with the potential conflict point. The potential conflict point is the intersection of the vehicle and pedestrian trajectories. TDTC can be expressed as:

[0092] TDTC p,v (t) = T p (t) - T v (t)

[0093] In the formula, T p (t) represents the time when the pedestrian's potential risk area coincides with the potential conflict point; T v (t) represents the time when the vehicle's potential risk area coincides with the potential conflict point.

[0094] To achieve the safety evaluation of vehicle-pedestrian conflicts, the present invention uses the cumulative frequency curve method to determine the threshold value of TDTC. The cumulative frequency curve method is a commonly used method in traffic engineering. For example, to determine the road speed limit, the road speed is usually generated according to the speed distribution of the measured cumulative frequency curve, and the 85% cumulative frequency corresponding to the speed limit is selected as the basis (that is, under normal driving conditions, 85% of the vehicles do not exceed the speed). Similar analysis can be introduced to determine the severity of the risk. By plotting the cumulative frequency curve of TDTC, the TDTC values corresponding to the 85% percentile cumulative frequency and the 15% percentile cumulative frequency are used as the critical values for potential risk and general risk, and general risk and severe risk, respectively.

[0095] The specific content of the real-time warning to the vehicle in Step 4 is as follows:

[0096] According to the information such as the risk type, license plate number, and vehicle speed corresponding to each vehicle, the real-time warning information is transmitted to the driver through the content displayed on the variable message sign.

[0097] When the detected risk is a potential risk, the content displayed on the variable message sign is set to "Yield to Pedestrians"; when the detected risk is a general risk, the content displayed on the variable message sign is set to "License plate information. Your current speed is XX. Please slow down and give way to pedestrians"; when the detected risk is a serious risk, the content displayed on the variable message sign is set to "License plate information. Your current speed is XX. Please stop and give way to pedestrians". If the vehicle does not stop and give way when giving a warning corresponding to a serious risk, it will be captured, and the continuous trajectory data will be stored to provide relevant evidence for subsequent handling.

[0098] Embodiment:

[0099] Figure 1 This is the overall flowchart of the present invention. The method for warning pedestrian crossing collision risks based on holographic perception in this embodiment includes the following steps:

[0100] Step 1: Real-time collect the basic traffic information of pedestrians and vehicles in the non-signal-controlled section through holographic perception devices, including the positions of pedestrians and vehicles, the moving directions of pedestrians and vehicles, and the speeds of pedestrians and vehicles. The holographic perception device can adopt a radar-vision integrated machine. The radar-vision fusion simultaneously accesses the original video stream and radar data stream into the embedded processor in the integrated machine through the MIPI and SPI interfaces. In the built-in embedded processor, directly perform AI target extraction on the original video stream, then project the video target into the radar coordinate system through the built-in coordinate mapping system, and finally perform fusion tracking processing on the video target and radar target to realize the real-time vectorization and tracking of the global target. It can obtain continuous trajectory data and information such as the positions, speeds, and moving directions of vehicles and pedestrians.

[0101] Step 2: According to the collected data and trajectory files, use the LSTM algorithm to construct a prediction model of speed to predict the trajectories of pedestrians and vehicles, and use the empirical cumulative distribution function to determine the potential risk area, as Figure 2 shown.

[0102] The empirical cumulative distribution function (ECDF) provides a simple estimate of the probability distribution of F(x) by "best guessing" the correct form of the unknown distribution F(x). Given a dataset of independent and identically distributed random observations (historical speed data in a certain sub-region) x 1 , x 2 , …, x n , the ECDF can obtain the following distribution:

[0103]

[0104] In the formula, is the frequency of speeds less than x, and n is the total number of speed data.

[0105] The ECDF is a non - decreasing function between 0 and 1. When n approaches infinity, it is almost certain that for each value of x, F n ′ (x) converges to F(x). Meanwhile, the confidence interval of the ECDF is defined as a function of a fixed x - value and a point - wise confidence interval for x, as follows: As follows:

[0106] E(F n ′ (x)) = F(x)

[0107]

[0108] where C(x) is the region given by the point - wise confidence interval. Although each value of x adheres to the confidence interval (since the point - wise confidence interval is calculated for each value of x), it is not guaranteed that C(x) simultaneously adheres to the confidence region for each individual x - value. Instead, we can define the confidence interval for all x as follows: as follows: As follows:

[0109]

[0110] where B(x) is the function of the confidence interval. Then, through the Dvoretzky - Kiefer - Wolfowitz (DKW) inequality, the formula for the two - sided confidence interval (lower bound L(x), upper bound U(x)) is as follows:

[0111]

[0112] L(x) = max{F n ′ (x)-ε, 0}

[0113] U(x) = min{F n ′ (x)+ε, 1}

[0114] where sup is the ceiling function.

[0115] The size of the potential risk area within the corresponding sub - area is determined by calculating the ECDF and the upper and lower confidence interval boundaries L(x) and U(x) for each sub - area.

[0116] Step 3: Apply the conflict detection algorithm to calculate the value of the safety evaluation index TDTC, where the potential conflict points are Figure 3 the white explosion - shaped area in

[0117] Combined with the potential risk areas, the definition of the Time Difference of Traffic Conflict (TDTC) is updated as follows: If the speeds of the vehicle and the pedestrian remain constant, then it is the time difference when the potential risk areas of the two coincide with the potential conflict point. The potential conflict point is the intersection of the trajectories of the vehicle and the pedestrian. TDTC can be expressed as:

[0118] TDTC p,v (t) = T p (t) - T v (t)

[0119] In the formula, T p (t) represents the time when the potential risk area of the pedestrian coincides with the potential conflict point; T v (t) represents the time when the potential risk area of the vehicle coincides with the potential conflict point.

[0120] By plotting the cumulative frequency curve of TDTC, the TDTC values corresponding to the 85% cumulative frequency and the 15% cumulative frequency are used as the critical values for potential risk and general risk, and general risk and severe risk respectively. Based on this, the detected values are classified into risk categories.

[0121] Step 4: For different risk types and vehicle-related information, set up variable message signs on the roadside to give real-time warnings to vehicles. The equipment layout is as Figure 3 shown. According to the risk type, license plate number, vehicle speed and other information corresponding to each vehicle, the content displayed on the variable message sign is used to transmit real-time warning information to the driver. According to the different driving directions of the vehicle on the road, the layout of the holographic perception equipment and the variable message sign is symmetrically distributed in the form of a cantilever. Figure 3 Only the specific layout of one driving direction is shown in detail.

[0122] When the detected risk is a potential risk, the content displayed on the variable message sign is set to "Yield to Pedestrians"; when the detected risk is a general risk, the content displayed on the variable message sign is set to "License plate information, your current speed is XX, please slow down and avoid pedestrians"; when the detected risk is a severe risk, the content displayed on the variable message sign is set to "License plate information, your current speed is XX, please stop and avoid pedestrians". If the vehicle does not stop and give way when the warning corresponding to the severe risk is given, it will be captured and the continuous trajectory data will be stored to provide relevant evidence for subsequent handling.

[0123] In summary, the present invention provides a pedestrian crossing collision risk warning method and system based on holographic perception. By using holographic perception devices to collect dynamic traffic element information such as the positions and speeds of pedestrians and vehicles in signal-free control sections, deep learning algorithms are utilized to predict the movement trajectories of both. According to the concept of potential risk areas, the calculation method of safety evaluation indicators is updated, replacing the overlap of a "point" with that of a "surface" and potential conflict points, thereby improving the risk warning accuracy. At the same time, the thresholds of safety evaluation indicators are determined, and the risk levels are classified into potential risks, general risks, and serious risks to achieve the identification of the pedestrian crossing collision risk status on the road section. Finally, real-time warnings are sent to the moving vehicles through roadside devices such as variable message signs.

[0124] It should be noted that according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, or two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0125] It is easy for those skilled in the art to understand that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A pedestrian crossing collision risk warning method based on holographic perception, characterized in that, it includes the following steps: Collect the position, movement direction and speed data of pedestrians and vehicles in a section without signal control by using holographic perception technology based on holographic perception devices; According to the collected data, use the LSTM algorithm to construct a speed prediction model to predict the trajectories of pedestrians and vehicles, determine the risk analysis range, divide the risk analysis range into N sub-regions, use the empirical cumulative distribution function to calculate the upper and lower limits of the speed of each sub-region, and determine the corresponding potential risk regions of each sub-region according to the upper and lower limits of the speed of each sub-region; The shape of the potential risk region is assumed to be a rectangle with a certain length and width; Combining the potential risk regions of pedestrians and vehicles, determine the time difference when the potential risk regions of the two coincide with the potential conflict points, that is, the conflict time difference TDTC, as the safety evaluation index for the conflict between pedestrians and vehicles; among them, the potential conflict point is the intersection of the trajectories of pedestrians and vehicles; determine the threshold value of TDTC by drawing the cumulative frequency curve of TDTC through the cumulative frequency curve method; Divide the risk types according to the comparison between the detected conflict time difference TDTC value and the threshold value, and give a warning.

2. The pedestrian crossing collision risk warning method based on holographic perception according to claim 1, characterized in that, The holographic perception device adopts a radar-vision integrated machine. The radar-vision fusion simultaneously accesses the original video stream and radar data stream into the embedded processor in the integrated machine through the MIPI and SPI interfaces. Directly perform AI target extraction on the original video stream in the built-in embedded processor, then project the video target into the radar coordinate system through the built-in coordinate mapping system, and finally perform fusion tracking processing on the video target and radar target to realize the real-time vectorization and tracking of global targets.

3. The pedestrian crossing collision risk warning method based on holographic perception according to claim 1, characterized in that, The trajectory is expressed as: T v / p = {L 1 , L 2 , L 3 , …, L n} where T v represents the trajectory of the vehicle; T p represents the trajectory of a pedestrian; L i = (x i , y i ) represents the position of a pedestrian / vehicle at the i-th moment; The speed prediction model for predicting the trajectories of pedestrians and vehicles is as follows: L i+t = (x i+1 , y i+1 ) = (x i + Δx, y i + Δy) Δx = v x ×Δt, Δy = v y ×Δt where x i , y i represent the horizontal and vertical coordinates of the pedestrian / vehicle at the i-th moment respectively, x i+1 , y i+1 represent the horizontal and vertical coordinates of the pedestrian / vehicle at the (i + 1)-th moment respectively, Δx and Δy represent the corresponding coordinate increments, v x , v y represent the instantaneous velocity values in the horizontal and vertical directions respectively, and Δt represents the time step.

4. The pedestrian crossing collision risk warning method based on holographic perception according to claim 3, characterized in that, The value of Δt is 2s.

5. The pedestrian crossing collision risk warning method based on holographic perception according to claim 1, characterized in that, The risk analysis range is a section with a specific length in front of the crosswalk.

6. The pedestrian crossing collision risk warning method based on holographic perception according to claim 1, characterized in that, Calculating the upper and lower limits of the speed of each sub-region by using the empirical cumulative distribution function includes: For the speed data x of each sub-region 1 , x 2 , …, x n , construct its distribution function: E(F n ′ (x)) = F(x) Wherein, is the frequency with a speed less than x, and n is the total number of speed data; the empirical cumulative distribution function ECDF is a non-decreasing function between 0 and 1. When n approaches infinity, for each x value, F n ′ (x) converges to F(x); Define the confidence interval for all velocity data in this sub-region as follows: The confidence interval is as follows: In the formula, B(x) is the function of the confidence interval; Through the Dvoretzky-Kiefer-Wolfowitz inequality, obtain the two-sided confidence interval, and the formula is as follows: L(x) = max{F n ′ (x) - ε, 0} U(x) = min{F n ′ (x) + ε, 1} In the formula, sup is the upper limit function; L(x) is the lower bound, and U(x) is the upper bound, that is, the upper and lower limits of the speed.

7. The pedestrian crossing collision risk warning method based on holographic perception according to claim 1, characterized in that, Use the TDTC values corresponding to the 85% percentile cumulative frequency and 15% percentile cumulative frequency as the critical values for potential risk and general risk, and general risk and serious risk respectively.

8. The pedestrian crossing collision risk warning method based on holographic perception according to claim 7, characterized in that, real-time warning is carried out by setting a variable message sign on the roadside.

9. The pedestrian crossing collision risk warning method based on holographic perception according to claim 8, characterized in that, when the risk is a potential risk, the content displayed on the variable message sign is set to "Yield to pedestrians"; when the risk is a general risk, the content displayed on the variable message sign is set to "License plate information, your current speed is XX, please slow down and avoid pedestrians"; when the risk is a serious risk, the content displayed on the variable message sign is set to "License plate information, your current speed is XX, please stop and avoid pedestrians"; if the vehicle does not stop and give way when the warning corresponding to the serious risk is carried out, it will be captured, and the continuous trajectory data will be stored to provide relevant evidence for subsequent disposal.

10. A pedestrian crossing collision risk warning system based on holographic perception for implementing the pedestrian crossing collision risk warning method based on holographic perception according to any one of claims 1 to 9, characterized in that, comprising: a holographic perception device for collecting the position, movement direction and speed data of pedestrians and vehicles in a section without signal control; a variable message sign for carrying out real-time warning.

Images