A method for sensing and warning blind spots in underground parking lots beyond visual range

Abstract

The present invention proposes a method for over-the-horizon perception and early warning of blind spots in underground parking lots, including collecting video images of the entire parking lot; obtaining vehicle posture information with monitoring as the origin; building a virtual scene simulation platform for the entire parking lot; deploying a virtual scene display screen device to provide a virtual picture from the first driver's perspective without the obstruction of wall pillars on the display screen to achieve blind spot perspective; predicting the change in vehicle posture data over a period of time in the future, measuring the relationship between the current perspective and the predicted posture data, and designing the optimal perspective change; and evaluating the risk of vehicle collision and providing early warning prompts. The present invention realizes the detection and early warning of vehicles and other targets in the blind spots, has the characteristics of low latency and low cost, and can effectively solve the safety problem of blind spots; relying on environmental perception and virtual space to realize over-the-horizon perception and early warning of blind spots in underground parking lots, provides important tools and means for solving the frequent accidents of blind spot conflict points.

Description

Technical Field

[0001] The present invention relates to the field of traffic technology, and in particular to a blind spot beyond-visual-range perception and early warning method for an underground parking lot. Background Art

[0002] In order to maintain the space required for urban functions and traffic, underground parking lots continue to emerge in cities. However, the internal walls and columns are dense, there are a large number of sharp turns and steep slopes, and the light is dim, which causes a large number of potential operational conflicts and safety risks, and a high rate of blind spot accidents. In order to expand the driver's field of vision and reduce the impact of blind spots, the traditional method is to install road wide-angle mirrors on key wall columns. However, they are easily affected by factors such as light, viewing angle, and placement angle, and have low reliability; wide-angle mirrors cause image distortion and are not intuitive enough. Drivers usually focus on finding parking spaces and are not easy to notice; wide-angle mirrors have a limited field of vision, giving drivers less reaction time. Therefore, the actual warning effect of road wide-angle mirrors is often greatly reduced, and their role in avoiding accidents is extremely limited.

[0003] At present, the digital perspective A-pillar system has been implemented in relevant enterprises. It realizes the perspective effect of eliminating A-pillar occlusion by displaying the transformed blind spot image on the A-pillar screen, which effectively reduces the occurrence of accidents caused by A-pillar occlusion. If the idea of ​​eliminating blind spots is directly transferred to the blind spot environment of the parking lot, because the camera is fixed and the position of the driver is constantly changing, it is impossible to display the picture with the driver as the first perspective in real time, and there is still no way to improve the problem that the wide-angle lens picture of the road is not intuitive enough. Therefore, it is urgent to use environmental perception and virtual platform to run visualization synchronously, and transmit the picture without the occlusion of the wall column in the virtual platform to the LED screen installed on the wall column, and provide warning marks for dangerous vehicles at the same time, finally realizing the blind spot perception and early warning function of the parking lot beyond the visual range, and providing new ideas and new means to solve the "chronic disease" of frequent accidents at the blind spot conflict point of underground parking lots. Summary of the invention

[0004] The purpose of the present invention is to provide a method with beyond-visual-range perception and early warning functions in a parking lot, which has solved the above-mentioned technical problems.

[0005] To achieve the above object, the present invention proposes a method for sensing and warning a blind spot in an underground parking lot beyond visual range, comprising the following steps:

[0006] S1: Collect video images of the entire parking lot;

[0007] S2: Perform depth estimation on the image data based on the DORN algorithm to obtain the vehicle posture information with the monitoring point as the origin;

[0008] S3: Build a virtual scene simulation platform for the entire parking lot;

[0009] S4: a virtual scene display screen device is arranged in the underground parking lot, and the display screen device provides a virtual picture from the first driver's perspective without the obstruction of the wall pillar on the display screen based on the vehicle posture information and the virtual scene simulation platform, so as to realize blind spot perspective;

[0010] S5: Based on the vehicle posture data, predict the changes in the vehicle posture data in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change;

[0011] S51. Sense user needs through trajectory analysis and prediction algorithms, and extract the position data of the user's vehicle Where x(t), y(t), z(t) are the position coordinates of the vehicle in three-dimensional space. is the vehicle's attitude angle (rotation angle around the x, y, z axis), and t is time. Prediction function through trajectory analysis Predict the pose data P'(t n+1 );

[0012] S52, through the objective function G(V(t),P'(t n+1 )) measures the relationship between the current viewing angle and the predicted pose data. The current viewing angle V(t) = (v x (t),v y (t),v z (t)), where v x (t),v y (t),v z (t) represents the current viewing angle in the direction of the x, y, and z axes respectively. The larger the objective function value is, the more the current viewing angle can meet the driver's needs;

[0013] S53, setting the optimal viewing angle change ΔV(t) so that the objective function G(V(t),P'(t n+1 )) value is the largest, that is And through V(t n+1 )=V(t)+ΔV(t) to update the position and posture of the camera in the virtual platform, so as to switch the viewing angle of the display screen according to user needs;

[0014] S6: Based on the time series information of the vehicle position data, the vehicle collision risk is evaluated, and the annotation data is generated to implement collision warning prompts between vehicles.

[0015] Furthermore, in S1, the video image is collected through monitoring devices installed in the parking lot; S1 also includes frame cutting and preprocessing of the collected video image.

[0016] Furthermore, the preprocessing includes enhancing the contrast, brightness and sharpness of the video image.

[0017] Further, the S2 comprises the following steps:

[0018] S21. Use the DORN algorithm to estimate the depth of the image sequence to obtain a depth image. The algorithm uses a deep regression neural network to estimate a single image and uses inter-frame information to obtain depth information of a continuous image sequence.

[0019] S22, input the depth image and the original RGB image into the D4LCN model, and use the depth-guided convolution to extract image features; the model adopts a multi-branch architecture, performs multi-scale and multi-channel convolution on the input depth image and RGB image, extracts image features, and then realizes the positioning and recognition of vehicle position information;

[0020] S23. Based on the image feature extraction, the non-maximum suppression target detection module of D4LCN is used to identify and locate the vehicle. The algorithm obtains the position, size and direction information of the target by classifying and regressing the targets in the image. At the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and located in continuous frames.

[0021] Further, the S3 comprises the following steps:

[0022] S31. Obtain basic spatial information of the corresponding underground parking lot, including the number and position of wall columns, the number and position of parking spaces, and the position and angle of camera layout, use the Vue framework and the three.is engine to render the scene, provide a user interaction interface, and display the basic information of the parking lot;

[0023] S32. Use the Flask framework to build a program to extract the vehicle posture data perceived by the camera, retrieve the vehicle posture information of the same ID, and sort it by time to form a trajectory. Through perspective transformation, a trajectory dataset P = {P1, P2, ···, P j ,···,P m},in is the trajectory of vehicle j, x j ,y j ,z j is the three-dimensional coordinate of vehicle j at time t, is the attitude angle of vehicle j at time t;

[0024] S33. The front and back ends interact with each other through RESTful API to realize the panoramic operation of the virtual platform.

[0025] Further, the S4 comprises the following steps:

[0026] S41. According to the parking lot layout and vehicle movement, a virtual scene display screen device shall be arranged on the wall pillars. Its height, size, direction and brightness shall be in principle convenient for the driver to observe;

[0027] S42. Track the driver's perspective based on the virtual space platform, and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective.

[0028] Further, the S6 comprises the following steps:

[0029] S61, extracting the corresponding motion trajectory and speed time series information based on the vehicle's posture data, and f =v i +at, s=(v i +v f )t / 2 to get the speed and acceleration of the vehicle, where v i and v f represents the initial and final speeds of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents the distance traveled by the vehicle, which can be obtained from the position data of the adjacent frames of the vehicle;

[0030] S62, assessing the risk of vehicle collision, and calculating the time required for two vehicles to collide at their current speeds TTC = L / (v1-v2), where L is the distance between vehicle 1 and vehicle 2, and v1 and v2 represent the speeds of vehicle 1 and vehicle 2, respectively;

[0031] S63. According to the operation of vehicles in the parking lot, a distance threshold between two vehicles is set. When L is lower than a certain threshold, different risk levels are marked according to the TTC value to complete the collision warning prompt.

[0032] Compared with the prior art, the advantages of the present invention are:

[0033] 1. The present invention realizes the detection and early warning of vehicles and other targets in blind spots, has the characteristics of low latency and low cost, and can effectively solve the safety problem of blind spots. It relies on environmental perception and virtual space to realize beyond-visual-range perception and early warning of blind spots in underground parking lots, providing important tools and means for solving the frequent accidents of blind spot conflict points.

[0034] 2. The present invention can accurately locate indoor vehicles, and the display screen device provides an accurate viewing angle according to vehicle information, thereby providing an effective guarantee for the safety of vehicles traveling in the basement. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The present invention is a flowchart of a method for sensing and warning a blind spot in an underground parking lot beyond visual range in an embodiment of the present invention.

[0036] Figure 2 Schematic diagram of image preprocessing in the blind spot beyond visual range perception and early warning method for underground parking lots in an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of target detection in a blind spot beyond visual range perception and early warning method for an underground parking lot in an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the actual application of the blind spot beyond-visual-range perception and early warning method for underground parking lots in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be further described below.

[0040] like Figure 1 As shown, the present invention is a method for sensing and warning blind spots in underground parking lots beyond visual range, comprising the following steps:

[0041] S1. Read in surveillance video data from the entire parking lot and pre-process the video images.

[0042] S11, the parking lot monitoring video stream data is transmitted to the environment perception module in real time through the interface;

[0043] S12, split the video frame by frame into images. Use the video processing library (OpenCV) to split the video frame by frame to obtain the video sequence images;

[0044] S13, image enhancement preprocessing. Figure 2 As shown in the figure, the video images are processed using the image enhancement tools in the Python pillow library. The enhancement algorithms include contrast, brightness, and sharpness. Image enhancement preprocessing can improve image quality and reduce the loss of feature details during depth extraction.

[0045] Furthermore, the specific algorithm of step S13 is:

[0046] Contrast Enhancement:

[0047]

[0048] Among them, Oi,j represents the enhanced pixel value, Ii,j represents the original pixel value, μ represents the average grayscale value of the image, σ represents the standard deviation of the image, and k represents the contrast gain.

[0049] Brightness Enhancement:

[0050] Oi,j=Ii,j×k+b

[0051] Where b represents the brightness offset, and the other variables are the same as above.

[0052] Sharpness Enhancement:

[0053] Oi,j=Ii,j+Ii,j×k-Li,j×k

[0054] Where Li,j represents the pixel value after Laplace filtering, k represents the sharpness gain, and the other variables are the same as above.

[0055] S2. Perform depth estimation on the image data based on the DORN algorithm, perform three-dimensional target detection on the obtained depth image and the original RGB image based on the D4LCN model, and output the vehicle posture information with the monitoring as the coordinate origin.

[0056] S21. Use the DORN algorithm to estimate the depth of the image sequence and obtain a depth image. Assume that the input RGB image is Where h and w represent the height and width of the image, respectively, and 3 represents the number of channels of the image. The output of DORN is a depth image The value of each pixel represents the depth of that point. The depth is calculated as follows:

[0057]

[0058] where p k (x, y) represents the probability that the pixel (x, y) belongs to the kth depth interval. The obtained depth is normalized to convert it into a grayscale depth image.

[0059] The algorithm uses a deep regression neural network to estimate a single image and uses inter-frame information to obtain depth information of a continuous image sequence to improve the accuracy and robustness of depth estimation.

[0060] S22. Input the depth image and the original RGB image into the D4LCN model and extract image features using depth-guided convolution. The depth information in the depth image is used as the information for guiding the convolution module for calculation:

[0061]

[0062] Among them, I' is the output image feature information, k is the convolution kernel scale, g i ,g j are the image unit translation scales, I is the input image matrix, and D is the depth image matrix.

[0063] The model adopts a multi-branch architecture to perform multi-scale, multi-channel convolution on the input depth image and RGB image to extract image features, thereby realizing the positioning and identification of vehicle position information.

[0064] S23. Based on the image feature extraction, the non-maximum suppression (NMS) target detection module of D4LCN is used to identify and locate the vehicle. The obtained vehicle location information is [x f,i y f,i z f,i ] 3D , represents the three-dimensional space coordinates of the vehicle in the monitoring probe's viewing angle coordinate system, where f is the sequence number of the preprocessed video segmentation image sequence, i is the target signal contained in a single image, and the collection information of the same vehicle bounding box is [h f,i w f,i l f,i ] 3D , where h, w, and l represent the height (vertical direction), length (parallel to the target moving direction), and width of the bounding box, respectively. The algorithm obtains the position, size, and direction information of the target by classifying and regressing the target in the image. At the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and located in consecutive frames.

[0065] S3, such as Figure 3 As shown in the figure, the basic information of the parking lot space layout is obtained, the posture data matching the vehicle is filtered and processed from the camera perception data set, and a simulation platform is built to realize the panoramic operation of the virtual platform.

[0066] S31. Obtain basic spatial information of the corresponding underground parking lot, including but not limited to the number and location of wall columns, the number and location of parking spaces, and the location and angle of camera layout, etc., use the Vue framework and three.is engine to render the scene, provide a user interaction interface, and display the basic information of the parking lot;

[0067] S32. Use the Flask framework to build a program to extract the vehicle posture data perceived by the camera, retrieve the vehicle posture information of the same ID, and sort it by time to form a trajectory. Through perspective transformation, a trajectory dataset P = {P1, P2, ···, P j ,···,P m},in is the trajectory of vehicle j, x j ,y j ,z j is the three-dimensional coordinate of vehicle j at time t, is the attitude angle of vehicle j at time t;

[0068] S33. The front and back ends interact with each other through RESTful API to realize the panoramic operation of the virtual platform.

[0069] S4. Deploy a virtual scene display screen device to track the user's perspective in the car on the virtual platform. By removing the wall and column blocking model in the virtual platform, a wide field of view is provided for the driver to eliminate blind spot hazards.

[0070] S41. According to the parking lot layout and vehicle movement, a virtual scene display screen device shall be arranged on the wall pillars. Its height, size, direction and brightness shall be in principle convenient for the driver to observe;

[0071] S42. Track the driver's perspective based on the virtual space platform, and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective.

[0072] S5. Based on the vehicle posture data, predict the changes in the vehicle's posture data in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change.

[0073] S51. Sense user needs through trajectory analysis and prediction algorithms and extract the position data of the user's vehicle Where x(t), y(t), z(t) are the position coordinates of the vehicle in three-dimensional space. is the vehicle's attitude angle (rotation angle around the x, y, z axis), and t is time. Prediction function through trajectory analysis Predict the pose data P'(t n+1 );

[0074] S52, through the objective function G(V(t),P'(t n+1 )) measures the relationship between the current viewing angle and the predicted pose data. The current viewing angle V(t) = (v x (t),v y (t),v z (t)), where v x (t),v y (t),v z (t) represents the current viewing angle in the direction of the x, y, and z axes respectively. The larger the objective function value is, the more the current viewing angle can meet the driver's needs;

[0075] S53, setting the optimal viewing angle change ΔV(t) so that the objective function G(V(t),P'(t n+1 )) value is the largest, that is And through V(t n+1 )=V(t)+ΔV(t) updates the position and posture of the camera in the virtual platform, thereby switching the display screen viewing angle according to user needs.

[0076] S6. Based on the time series information of the vehicle position data, the vehicle collision risk is evaluated, and annotation data is generated to implement collision warning prompts between vehicles.

[0077] S61, extracting the corresponding motion trajectory and speed time series information based on the vehicle's posture data, and f =v i +at, s=(v i +v f )t / 2 to get the speed and acceleration of the vehicle, where v i and v f represents the initial and final speeds of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents the distance traveled by the vehicle, which can be obtained from the position data of the adjacent frames of the vehicle;

[0078] S62, assessing the risk of vehicle collision, and calculating the time required for two vehicles to collide at their current speeds TTC = L / (v1-v2), where L is the distance between vehicle 1 and vehicle 2, and v1 and v2 represent the speeds of vehicle 1 and vehicle 2, respectively;

[0079] S63. According to the operation of vehicles in the parking lot, a distance threshold between two vehicles is set. When L is lower than a certain threshold, different risk levels are marked according to the TTC value to complete the collision warning prompt.

[0080] This example is based on a case study scenario in an underground parking lot in Pudong. There are ramps, sharp bends, and dense wall columns in the parking lot, with many blind spots and safety hazards. Multiple cameras are deployed to achieve full-area monitoring and environmental perception detection, process the acquired vehicle posture data, render the vehicle's motion status in the virtual space, and further perceive user needs to achieve personalized perspective switching and collision warning annotation.

[0081] This embodiment applies the above technical solution, such as Figure 4 As shown, the main process is:

[0082] Step 1: Read in surveillance video data from the entire parking lot and pre-process the video images.

[0083] Step 1.1, the parking lot monitoring video stream data is transmitted to the information processing module in real time through the interface;

[0084] Step 1.2: Split the video into images frame by frame. Use the video processing library (OpenCV) to split the video into frames and obtain the video sequence images;

[0085] Step 1.3: Image enhancement preprocessing. Use the image enhancement tool in the Python pillow library to process the video. The enhancement algorithm includes contrast, brightness, and sharpness. Image enhancement preprocessing can improve image quality and reduce the loss of feature details during depth extraction.

[0086] Furthermore, the specific algorithm of step 1.3 is:

[0087] Contrast Enhancement:

[0088]

[0089] Among them, Oi,j represents the enhanced pixel value, Ii,j represents the original pixel value, μ represents the average grayscale value of the image, σ represents the standard deviation of the image, and k represents the contrast gain.

[0090] Brightness Enhancement:

[0091] Oi,j=Ii,j×k+b

[0092] Where b represents the brightness offset, and the other variables are the same as above.

[0093] Sharpness Enhancement:

[0094] Oi,j=Ii,j+Ii,j×k-Li,j×k

[0095] Where Li,j represents the pixel value after Laplace filtering, k represents the sharpness gain, and the other variables are the same as above.

[0096] Step 2: Perform depth estimation on the image data based on the DORN algorithm, perform three-dimensional target detection on the obtained depth image and original RGB image based on the D4LCN model, and output the vehicle posture information with the monitoring as the coordinate origin.

[0097] Step 2.1: Use the DORN algorithm to estimate the depth of the image sequence and obtain a depth image. Assume that the input RGB image is Where h and w represent the height and width of the image, respectively, and 3 represents the number of channels of the image. The output of DORN is a depth image The value of each pixel represents the depth of that point. The depth is calculated as follows:

[0098]

[0099] where p k (x, y) represents the probability that the pixel (x, y) belongs to the kth depth interval. The obtained depth is normalized to convert it into a grayscale depth image.

[0100] The algorithm uses a deep regression neural network to estimate a single image and uses inter-frame information to obtain depth information of a continuous image sequence to improve the accuracy and robustness of depth estimation.

[0101] Step 2.2: Input the depth image and the original RGB image into the D4LCN model and use depth-guided convolution to extract image features. The depth information in the depth image is used as the information for guiding the convolution module for calculation:

[0102]

[0103] Among them, I' is the output image feature information, k is the convolution kernel scale, g i ,g j are the image unit translation scales, I is the input image matrix, and D is the depth image matrix.

[0104] The model adopts a multi-branch architecture to perform multi-scale, multi-channel convolution on the input depth image and RGB image to extract image features, thereby realizing the positioning and identification of vehicle position information.

[0105] Step 2.3: Based on the image feature extraction, the non-maximum suppression (NMS) target detection module of D4LCN is used to identify and locate the vehicle. The obtained vehicle location information is [x f,i y f,i z f,i ] 3D , represents the three-dimensional space coordinates of the vehicle in the monitoring probe's viewing angle coordinate system, where f is the sequence number of the preprocessed video segmentation image sequence, i is the target signal contained in a single image, and the collection information of the same vehicle bounding box is [h f,i w f,i l f,i ] 3d , where h, w, and l represent the height (vertical direction), length (parallel to the target moving direction), and width of the bounding box, respectively. The algorithm obtains the position, size, and direction information of the target by classifying and regressing the target in the image. At the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and located in consecutive frames.

[0106] Step 3: Obtain basic information on the parking lot space layout, filter and process the posture data that matches the vehicle from the camera perception data set, combine the two, build a simulation platform, and realize panoramic operation of the virtual platform.

[0107] Step 3.1, obtain the basic spatial information of the underground parking lot, including but not limited to the number and location of wall columns, the number and location of parking spaces, and the location and angle of camera layout, etc., use the Vue framework and three.is engine to render the scene, provide a user interaction interface, and display the basic information of the parking lot;

[0108] Step 3.2: Use the Flask framework to build a program to extract the vehicle posture data perceived by the camera, retrieve the vehicle posture information of the same ID, and sort it by time to form a trajectory. Through perspective transformation, the trajectory dataset P = {P1, P2, ···, P j ,···,P m},in is the trajectory of vehicle j, x j ,y j ,z j is the three-dimensional coordinate of vehicle j at time t, is the attitude angle of vehicle j at time t;

[0109] Step 3.3: The front-end and back-end interact with each other through RESTful API to realize the panoramic operation of the virtual platform.

[0110] Step 4: Set up a virtual scene display screen device, track the user's perspective in the car on the virtual platform, and provide the driver with a wide field of view and eliminate blind spot hazards by removing the wall column occlusion model in the virtual platform.

[0111] Step 4.1. According to the parking lot layout and vehicle movement, a virtual scene display screen device is installed on the wall column. Its height, size, direction and brightness should be convenient for the driver to observe.

[0112] Step 4.2: Track the driver's perspective based on the virtual space platform, and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective.

[0113] Step 5: Based on the vehicle posture data, predict the changes in the vehicle's posture data in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change.

[0114] Step 5.1: Sense user needs through trajectory analysis and prediction algorithms and extract the position data of the user's vehicle Where x(t), y(t), z(t) are the position coordinates of the vehicle in three-dimensional space. is the vehicle's attitude angle (rotation angle around the x, y, z axis), and t is time. Prediction function through trajectory analysis Predict the pose data P'(tn+1 );

[0115] Step 5.2: Through the objective function G(V(t),P'(t n+1 )) measures the relationship between the current viewing angle and the predicted pose data. The current viewing angle V(t) = (v x (t),v y (t),v z (t)), where v x (t),v y (t),v z (t) represents the current viewing angle in the direction of the x, y, and z axes respectively. The larger the objective function value is, the more the current viewing angle can meet the driver's needs;

[0116] Step 5.3: Set the optimal viewing angle change ΔV(t) so that the objective function G(V(t),P'(t n+1 )) value is the largest, that is And through V(t n+1 )=V(t)+ΔV(t) updates the position and posture of the camera in the virtual platform, thereby switching the display screen viewing angle according to user needs.

[0117] Step 6: Based on the time series information of the vehicle position data, evaluate the vehicle collision risk, generate annotation data, and implement collision warning prompts between vehicles.

[0118] Step 6.1: Extract the corresponding motion trajectory and speed time series information based on the vehicle's posture data, and use v f =v i +at, s=(v i +v f )t / 2 to get the speed and acceleration of the vehicle, where v i and v f represents the initial and final speeds of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents the distance traveled by the vehicle, which can be obtained from the position data of the adjacent frames of the vehicle;

[0119] Step 6.2, assess the risk of vehicle collision, and calculate the time required for the two vehicles to collide at their current speeds TTC = L / (v1-v2), where L is the distance between vehicle 1 and vehicle 2, and v1 and v2 represent the speeds of vehicle 1 and vehicle 2, respectively;

[0120] Step 6.3: According to the operation of vehicles in the parking lot, the distance threshold between the two vehicles is set to 10m. When L is lower than this threshold, different risk levels are marked according to the TTC value, where TTC≥3s is marked as low risk; 1.5s<TTC<3s is marked as medium risk; TTC≤1.5s is marked as high risk. Generate annotation data based on this judgment mechanism to complete the collision warning prompt.

[0121] In summary, the present invention aims at the frequent accidents in blind spots of underground parking lots, and proposes an over-the-horizon perception and warning device for underground parking lots. The device combines the vehicle posture data collected and perceived by the camera with the virtual space simulation to realize the virtualization of the real scene, and provides a wide field of view by removing the wall column occlusion in the virtual space to eliminate the hidden dangers of blind spots; analyzes the running state of the vehicle trajectory, and takes the posture change closest to the prediction as the constraint condition to realize the automatic switching of the personalized perspective; based on the time series information of the vehicle position change, sets the corresponding threshold for its distance and collision time, evaluates the collision risk, generates risk annotation, and completes the early warning prompt. The present invention builds a virtual space platform for the blind spots of underground parking lots, combines hardware equipment such as cameras and display screens, detects the movement traces of vehicles and pedestrians in the blind spots, and displays virtual pictures that run synchronously with the real scene, providing users with "intelligent, precise, and personalized" prompts and warnings. The present invention realizes the detection and early warning of vehicles and other targets in the blind spots, has the characteristics of low latency and low cost, and can effectively solve the safety problems of blind spots; relying on environmental perception and virtual space, it realizes over-the-horizon perception and early warning of blind spots in underground parking lots, and provides important tools and means for solving the frequent accidents of blind spot conflict points.

[0122] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A method for sensing and warning blind spots in underground parking lots beyond visual range, characterized in that: The following steps are involved: S1: Collect video images of the entire parking lot; S2: Perform depth estimation on the image data based on the DORN algorithm to obtain the vehicle posture information with the monitoring point as the origin; S3: Build a virtual scene simulation platform for the entire parking lot; S4: a virtual scene display screen device is arranged in the underground parking lot, and the display screen device provides a virtual picture from the first driver's perspective without the obstruction of the wall pillar on the display screen based on the vehicle posture information and the virtual scene simulation platform, so as to realize blind spot perspective; S5: Based on the vehicle posture data, predict the changes in the vehicle posture data in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change; S51. Sense user needs through trajectory analysis and prediction algorithms, and extract the position data of the user's vehicle ,in is the position coordinate of the vehicle in three-dimensional space, is the attitude angle of the vehicle, which is the rotation angle around the x, y, and z axes respectively, t is time, based on the posture data over a period of time , the prediction function through trajectory analysis , predict the pose data for a period of time in the future ; S52, through the objective function Measures the relationship between the current perspective and the predicted pose data. The current perspective ,in Respectively represent the current viewing angle x , y , z The direction of the axis, the larger the objective function value is, the more the current viewing angle can meet the driver's needs; S53, Setting the best viewing angle change , so that the objective function The maximum value, that is and through Update the position and posture of the camera in the virtual platform to switch the display screen perspective according to user needs; S6: Based on the time series information of the vehicle position data, the vehicle collision risk is evaluated, and the annotation data is generated to implement collision warning prompts between vehicles.

2. The method for sensing and warning the blind spot of an underground parking lot beyond visual range according to claim 1 is characterized in that: In S1, the video image is collected through monitoring devices installed in the parking lot; S1 also includes frame cutting and preprocessing of the collected video image.

3. The method for sensing and warning the blind spot of an underground parking lot beyond visual range according to claim 2 is characterized in that: The preprocessing includes enhancing the contrast, brightness and sharpness of the video image.

4. The method for sensing and warning the blind spot of an underground parking lot beyond visual range according to claim 1 is characterized in that: The S2 comprises the following steps: S21. Use the DORN algorithm to estimate the depth of the image sequence to obtain a depth image. The DORN algorithm uses a deep regression neural network to estimate a single image and uses inter-frame information to obtain depth information of a continuous image sequence. S22, inputting the depth image and the original RGB image into the D4LCN model, and extracting image features using depth-guided convolution; the D4LCN model adopts a multi-branch architecture, performs multi-scale and multi-channel convolution on the input depth image and RGB image, extracts image features, and then realizes positioning and recognition of vehicle position information; S23. Based on the image feature extraction, the non-maximum suppression target detection module of D4LCN is used to identify and locate the vehicle. The non-maximum suppression target detection module of D4LCN obtains the position, size and direction information of the target by classifying and regressing the target in the image. At the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and located in continuous frames.

5. The method for sensing and warning the blind spot of an underground parking lot beyond visual range according to claim 1 is characterized in that: The S3 comprises the following steps: S31. Obtain basic spatial information of the corresponding underground parking lot, including the number and position of wall columns, the number and position of parking spaces, and the position and angle of camera layout, use the Vue framework and the three.js engine to render the scene, provide a user interaction interface, and display the basic information of the parking lot; S32. Use the Flask framework to build a program to extract the vehicle posture data perceived by the camera, retrieve the vehicle posture information of the same ID, and sort it by time to form a trajectory. Through perspective transformation, it forms a trajectory dataset of the vehicle in the virtual panoramic view. ,in For vehicles j The trajectory of for t Time Vehicle j The three-dimensional coordinates of for t Time Vehicle j The attitude angle; S33. The front and back ends interact with each other through RESTful API to realize the panoramic operation of the virtual platform.

6. The method for sensing and warning blind spots in underground parking lots beyond visual range according to claim 1 is characterized in that: The S4 comprises the following steps: S41. According to the parking lot layout and vehicle movement, a virtual scene display screen device shall be arranged on the wall pillars. Its height, size, direction and brightness shall be in principle convenient for the driver to observe; S42. Track the driver's perspective based on the virtual space platform, and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective.

7. The method for sensing and warning blind spots in underground parking lots beyond visual range according to claim 1 is characterized in that: The S6 comprises the following steps: S61, extracting the corresponding motion trajectory and speed time series information based on the vehicle's posture data, , , Get the speed and acceleration of the vehicle, where v i and v f represents the initial and final velocities of the vehicle, a represents the vehicle's acceleration, t Indicates the time interval between adjacent frames. s Indicates the vehicle's travel distance, which can be obtained from the vehicle's position data in adjacent frames; S62: Assess the risk of vehicle collision and calculate the time required for two vehicles to collide at their current speeds. ,in L is the distance between vehicle 1 and vehicle 2, v 1 and v 2 Represent the speeds of vehicle 1 and vehicle 2 respectively; S63: According to the operation of the vehicles in the parking lot, a distance threshold between the two vehicles is set. L When it is below a certain threshold, different risk levels are marked according to the TTC value to provide a collision warning prompt.

Images