Stereo garage vehicle storage risk assessment method and system based on multi-source sensor data fusion

Abstract

A kind of stereo garage vehicle warehouse entry risk assessment method based on multi-source sensor data fusion, comprising the following steps: step 1, multi-source data synchronous acquisition;Step 2, the two-dimensional image data obtained is processed;Step 3, the vehicle 2D pixel level boundary box and comprehensive confidence obtained are processed;Step 4, the vehicle state characteristic quantity for risk assessment is extracted;Step 5, determine the risk assessment index of vehicle warehouse entry and carry out risk assessment;The implementation system of the above method includes sensor network module, data processing and fusion module, dynamic weight calculation module, comprehensive risk assessment module;The present application can accurately identify potential collision risks such as body inclination, overspeed, overheight, too close distance between rear wall and vehicle, etc., and can significantly reduce the probability of collision accidents under complex conditions.

Description

Technical Field

[0001] This invention belongs to the field of behavioral feature recognition technology, specifically involving a method and system for risk assessment of vehicle entry into a three-dimensional parking garage based on multi-source sensor data fusion. Background Technology

[0002] Currently, the number of vehicles owned by residents is rising rapidly, while land resources in urban core areas are extremely scarce. The area available for building new parking lots is very limited, and traditional surface parking lots have drawbacks such as large footprint and low space utilization, which has created significant parking problems in cities. Multi-level parking garages, as a technological means to improve land space utilization and alleviate parking problems, have gained widespread application in commercial complexes, residential communities, transportation hubs, and other scenarios urgently needing to alleviate parking pressure, thanks to their core advantage of efficiently utilizing vertical space, which can increase parking capacity per unit area by 3 to 5 times.

[0003] The vehicle entry process in automated parking garages requires extremely high precision in positional sensing. It necessitates real-time and accurate acquisition of the relative positions of various structures within the garage, as well as the vehicle's own movement status, to prevent collisions and accidents. However, existing automated parking garages generally use single sensors for data sensing. These individual sensors, due to their inherent limitations, are significantly affected by interference in complex weather conditions such as rain, snow, fog, or low-light environments at night. This results in insufficient accuracy in sensing various aspects of vehicle entry data and inadequate environmental adaptability, making it difficult to meet safety requirements.

[0004] The vehicle entry process involves interaction between the vehicle and multiple entities such as the garage door, the rear wall of the garage, and the lifting platform. It is necessary to assess the priority of different risk factors in real time. This requires the automated parking system to have a very high risk prediction capability. Existing risk assessment methods have problems such as fixed weight allocation and failure to consider changes in environmental factors. They lack dynamic adaptability. When risks such as "speeding + lane departure" occur, they cannot provide timely and appropriate warnings. This makes it impossible for the automated parking system to meet the requirement of accurately predicting entry risks. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method and system for risk assessment of vehicle entry into a three-dimensional parking garage based on multi-source sensor data fusion, so as to solve the defects in the background art.

[0006] The technical solution adopted to solve the above technical problems is: a method for risk assessment of vehicle entry into a three-dimensional parking garage based on multi-source sensor data fusion, characterized by the following steps:

[0007] Step 1: Multi-source data synchronous acquisition: Through the deployment of high-definition cameras, lidar and ultrasonic arrays, two-dimensional image data, three-dimensional point cloud data and near-field obstacle information are collected in real time during the vehicle entry process, and the data collected by the ultrasonic array is cleaned.

[0008] Step 2: Process the acquired 2D image data: Use the YOLOv5 detection model to process the acquired 2D image data, and use the bounding box regression algorithm to obtain the 2D pixel-level bounding boxes and overall confidence scores of the vehicles entering the warehouse. ;

[0009] Step 3: Process the acquired 2D pixel-level bounding box and overall confidence score of the vehicle: Obtain the preset calibration parameters of the high-definition camera and the LiDAR, and back-project the 2D pixel-level bounding box onto the 3D point cloud coordinate system of the LiDAR to obtain the 3D region of interest for subsequent point cloud processing.

[0010] Step 4: Extract vehicle state features for risk assessment: Extract features from the data in Step 3 to obtain vehicle state features for risk assessment. The vehicle state features include the height, attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the side pillars of the garage, and distance to the rear wall of the garage for the target vehicle entering the garage. If the height is greater than the threshold, the vehicle is directly prohibited from entering the garage. If the height is less than the threshold, the data is formatted, coordinate system is unified, time window averaging is performed, multivariate data time series alignment is performed, and mean fusion is performed.

[0011] Step 5: Determine vehicle entry risk assessment indicators and conduct risk assessment: The vehicle entry risk assessment indicators include comprehensive risk factors. Single-factor composite risk value .

[0012] The process of cleaning the data collected by the ultrasonic array in step 1 of this invention is as follows: First, scalar Kalman filtering is performed on each independent sensor in the ultrasonic array. Then, within the same interrupt cycle, the system clock counter value τ(t) and the current ultrasonic sensor confidence level are compared. splicing the data to the estimated values, and generating fixed-length data pairs using the following formula. :

[0013]

[0014]

[0015] Among them, the system clock counter value τ(t) has a resolution ≤ 1ms. The distance data for each ultrasonic array submodule after cleaning is shown. The current confidence level of the ultrasonic sensor; These are the weighting coefficients. To comprehensively evaluate signal quality, For environmental adaptability, For the sake of historical consistency, For geometric rationality.

[0016] Weighting coefficients of this invention The value range is (0, 1], and the weighting coefficient satisfies .

[0017] In step 2 of this invention, the overall confidence level is... We obtain it from the following formula:

[0018]

[0019] in, The confidence score of the original output of the YOLOv5 detection model. This represents the stability factor calculated based on the variance of bounding box positions across multiple consecutive frames. It is an image quality factor that combines image contrast, sharpness, and noise level. Considering the environmental adaptability coefficient under illumination conditions, The preset weighting coefficients have a value range of (0, 1].

[0020] Stability factor calculated based on the position variance of bounding boxes across multiple consecutive frames We obtain it from the following formula:

[0021]

[0022] in, The attenuation coefficient is a finite number of positive integers. The variance of the bounding box center point coordinates over N consecutive frames;

[0023] Image quality factor that integrates image contrast, sharpness, and noise level We obtain it from the following formula:

[0024]

[0025] in, The contrast of an image is represented by calculating the standard deviation of a local region of the image. Image sharpness is indicated by calculating the variance of the Laplacian operator response of the image; The noise level of the image is estimated by analyzing the high-frequency components of the image. , , The preset weighting coefficients satisfy... + =1, , , The value range is (0, 1];

[0026] Environmental adaptive coefficient considering illumination conditions From the following formula, we get

[0027]

[0028] in, This is the normalized light intensity value, ranging from 0 to 1. This represents an offset less than 1, used to preserve the baseline confidence level under optimal lighting conditions, indicating the system's robustness in extreme situations.

[0029] The process of obtaining the three-dimensional region of interest in step 3 of this invention is as follows: The obtained 2D pixel-level bounding box of the vehicle and the overall confidence level are... The Zhang Zhengyou calibration method is used to obtain preset calibration parameters for the high-definition camera and LiDAR. The 2D pixel-level bounding box of the vehicle is back-projected onto the 3D point cloud coordinate system of the LiDAR through perspective transformation, resulting in a 3D region of interest for subsequent point cloud data processing. After fusion, the system clock counter value is... and fusion confidence The data is spliced ​​into the fusion result, generating fixed-length data pairs as follows:

[0030]

[0031] in: This represents the coordinate data of the three-dimensional region of interest. This represents the fusion confidence score of the image and point cloud data. Record parameters for the time point when the image and point cloud data fusion is completed, with a resolution of ≤1ms, to mark the specific time when the fusion operation is completed;

[0032] Fusion confidence of image and point cloud data fusion Determined by the following formula:

[0033]

[0034] in This represents the number of valid point clouds within the ROI region. The threshold for the number of point clouds. Let Variance be the point cloud density distribution. The point cloud coverage ratio of the ROI region. This represents the maximum point cloud density distribution value within the ROI region. The weighting coefficient has a value range of (0, 1] and satisfies the following conditions: .

[0035] In step 4 of this invention, the data format is organized by arranging the multi-source sensor data into a unified data matrix.

[0036]

[0037] in, This represents the raw measurement value of the i-th sensor for the j-th risk factor, where i is the sensor number (from 1 to I), I is the number of sensors (a finite positive integer), and j is the number of risk factors (an integer from 1 to 5), corresponding to the five risk factors: the target vehicle's attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the garage's side pillars, and distance to the garage's rear wall. The coordinates in the sensor's local coordinate system are: ;

[0038] The coordinate system is one that transforms the measured values ​​in the local coordinate systems of each sensor to the world coordinate system using the following formula. :

[0039]

[0040] in, , , They are respectively around The rotation matrix of the axis is determined by the sensor. Installation pitch angle Yaw angle Roll angle Sure, The coordinates of the measurement point in the sensor's local coordinate system. The translation vector from the sensor's local coordinate system to the world coordinate system; the updated coordinate block data matrix. It will be used for time window averaging.

[0041] The time window averaging process involves applying a sliding window average to multiple data streams from different sensors that characterize the same physical quantity. For data from coordinate block data matrices... data stream Smooth output at time t for:

[0042]

[0043] in Where W is the system sampling time interval, and W is the system-level smoothing window width. The smoothed data matrix is ​​formed by smoothing all data streams using time indices within the sliding window. Perform time-series alignment of multivariate data;

[0044] The multi-data time-series alignment is based on a unified time base, whereby each asynchronous data stream is uniformly resampled to a fixed-frequency time grid using a linear interpolation method, for the target synchronization time. interpolation results for:

[0045]

[0046] in, The target synchronization time in the original data The corresponding previous nearest time The data stream values ​​below, The target synchronization time in the original data The corresponding next nearest time The data stream values ​​below;

[0047] Smoothing data matrix After interpolation, the measurement value of the j-th risk factor by the i-th sensor is obtained. , used for subsequent calculations.

[0048] The comprehensive risk factor R of this invention is determined by the following formula:

[0049]

[0050] The single-factor composite risk value of the j-th risk factor is determined by the following formula.

[0051]

[0052] The dynamic adjustment coefficient for the j-th risk factor is determined by the following formula.

[0053]

[0054] in, Let the risk sensitivity be the j-th risk factor. Let the risk threshold be the j-th risk factor. The weighted fusion result of the j-th risk factor measured by multiple sensors is determined by the following formula.

[0055]

[0056] in, The confidence level when the i-th sensor measures the j-th risk factor. Let be the measurement value of the j-th risk factor by the i-th sensor. When the i-th sensor cannot measure the j-th risk factor, =0, =0;

[0057] The corrected weight matrix The weight elements of the j-th risk factor, and the corrected weight matrix. Determined by the following formula:

[0058]

[0059] in, The basic weight matrix is ​​5×1. The matrix is ​​a 5×1 identity matrix, where γ is the weight adjustment coefficient (taking a finite positive integer), and N is a 5×1 dynamic adjustment matrix for risk factors, consisting of the dynamic adjustment coefficients corresponding to each risk factor. constitute.

[0060] The aforementioned multi-source sensor data fusion-based vehicle entry risk assessment system for automated parking garages includes a sensor network module, a data processing and fusion module, a dynamic weight calculation module, and a comprehensive risk assessment module.

[0061] The sensor network module is used to acquire various attitude data during vehicle parking, which is then used for subsequent fusion calculation of risk scores.

[0062] The data processing and fusion module is used to standardize and calibrate the raw data collected by the sensor network module, providing a high-quality, unified data foundation for subsequent fusion calculations;

[0063] The dynamic weight calculation module is used to dynamically calculate and allocate the weight of each risk factor in the final fusion score based on the different stages of vehicle entry into the warehouse and the real-time confidence level of each sensor data.

[0064] The comprehensive risk assessment module is used to calculate the comprehensive risk factors at different stages of vehicle entry into the warehouse, and to determine whether there is any risk and the magnitude of the risk based on the comprehensive risk factors.

[0065] The sensor network module of this invention includes an ultrasonic array, a high-definition camera, and a lidar.

[0066] The data processing and fusion module includes a data cleaning submodule, an image and point cloud data fusion submodule, a data format sorting submodule, a coordinate system submodule, a time window averaging unit, a multivariate data time series alignment submodule, and a mean fusion unit.

[0067] The dynamic weight calculation module includes a risk factor calculation submodule and a weight calculation submodule;

[0068] The comprehensive risk assessment module includes a risk scoring submodule and a risk evaluation submodule.

[0069] The risk factor calculation submodule of this invention includes a door-to-vehicle distance risk calculation unit, a height risk calculation unit, an attitude angle risk calculation unit, an entry speed risk calculation unit, and a distance risk calculation unit between the rear wall of the warehouse and the vehicle.

[0070] Compared with the prior art, the present invention has the following advantages:

[0071] 1. Traditional automated parking systems mostly rely on single sensors. Due to inherent limitations of these sensors, their perception accuracy drops significantly in complex environments. For example, lidar experiences beam scattering in rain and fog, resulting in distance errors exceeding ±15cm; ultrasonic sensors have a false negative rate exceeding 60% when detecting low obstacles; and high-definition cameras have poor distance recognition capabilities in low-light conditions at night, with errors exceeding ±8cm. Compared to traditional sensor perception, this invention combines data fusion from multiple sensors, including ultrasonic arrays, high-definition cameras, and lidar. By integrating the advantages of different sensor types and compensating for the inherent limitations of single sensors, the system's perception accuracy is significantly improved in complex environments such as rain, fog, nighttime, and strong light, and its environmental adaptability is significantly enhanced.

[0072] 2. Traditional risk assessments typically employ fixed weights and a single-dimensional approach, which can easily lead to delayed warnings or misjudgments. For example, when a vehicle is traveling slowly but is too close to the rear wall of the parking garage, the fixed weights may cause the vehicle to be mistakenly assessed as risk-free due to its slow speed, resulting in a collision. In contrast to traditional risk assessments, this invention combines sensor confidence levels and data to perform dynamic weight allocation and multi-dimensional risk assessment. The dynamic weight calculation module comprehensively covers six core risk dimensions: vehicle height, attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the side pillars of the garage, and distance to the rear wall of the garage. This allows for accurate identification of potential collision risks such as vehicle tilt, speeding, exceeding height limits, and excessively close proximity to the rear wall of the garage, significantly reducing the probability of collisions in complex situations.

[0073] 3. Traditional automated parking garages often do not have strict requirements on the placement of sensors, which can lead to blind spots and limited coverage, resulting in incomplete measurement of key vehicle data. Compared to traditional sensor placement, the first and second ultrasonic arrays are vertically installed on the front and right pillars of the garage entrance passage, respectively. The ultrasonic sensors on each side are vertically distributed to cover the bottom to the middle of the vehicle, avoiding misjudgments of vehicle tilt caused by the height limitation of a single sensor. The third ultrasonic array is horizontally installed on the rear wall of the garage parking space, with its height matching the common height restrictions of multi-level parking garages. The fourth ultrasonic array adopts a matrix layout, which can simultaneously collect distance data at different heights and widths of the rear of the vehicle, avoiding the measurement deviation of the rear distance caused by traditional single-point installation. The high-definition camera submodule is installed on the top of the front frame of the garage, which can capture the contour changes of the vehicle throughout the entire process of entering the garage from a full-view perspective, and monitor the relative position of the vehicle and the garage door in real time. The high-definition camera submodule is installed at a specific position on the rear wall of the garage, which avoids being blocked by the ultrasonic sensors and can focus on the rear posture of the vehicle. The lidar submodule is installed in an unobstructed area on the side or front of the garage entrance, which scans and covers the entire process of the vehicle entering the garage from all angles, eliminating point cloud blind spots. Attached Figure Description

[0074] Figure 1 This is a flowchart of the method of the present invention.

[0075] Figure 2 This is a schematic diagram of the system of the present invention.

[0076] Figure 3 This is a layout diagram of the sensor network module of the present invention in a garage.

[0077] Figure 4 This is a schematic diagram of the data processing and fusion module of the present invention.

[0078] Figure 5 This is a schematic diagram of the dynamic weight calculation module of the present invention.

[0079] Figure 6 This is a schematic diagram of the risk assessment module of the present invention. Detailed Implementation

[0080] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to these embodiments.

[0081] Example 1

[0082] exist Figure 1 The present invention relates to a method for risk assessment of vehicle entry into a multi-level parking garage based on multi-source sensor data fusion, comprising the following steps:

[0083] Step 1: Multi-source data synchronous acquisition: Using deployed high-definition cameras, LiDAR, and ultrasonic arrays, real-time acquisition of 2D image data, 3D point cloud data, and near-field obstacle information during vehicle entry is performed. The data collected by the ultrasonic array is then cleaned. Specifically, the cleaning process involves: first, performing scalar Kalman filtering on each independent sensor in the ultrasonic array; then, within the same interrupt cycle, combining the system clock counter value τ(t) with the current ultrasonic sensor confidence level. splicing the data to the estimated values, and generating fixed-length data pairs using the following formula. :

[0084]

[0085]

[0086] Among them, the system clock counter value τ(t) has a resolution ≤ 1ms. The distance data for each ultrasonic array submodule after cleaning is shown. The current confidence level of the ultrasonic sensor; Let be the weighting coefficients, and let the weighting coefficients satisfy . ; To comprehensively evaluate signal quality, For environmental adaptability, For the sake of historical consistency, For geometric rationality, the preferred weight allocation in this embodiment is as follows: 0.4, 0.2, 0.3, 0.1.

[0087] Step 2: Process the acquired 2D image data: The acquired 2D image data is processed using the YOLOv5 detection model, and the bounding box regression algorithm is used to obtain the 2D pixel-level bounding box of the vehicle entering the parking garage and the overall confidence score. In order to improve the stability and reliability of vehicle detection in the multi-level parking garage environment, the YOLOv5 detection model is trained on a dataset consisting of images of the scene in front of the multi-level parking garage. The dataset contains image samples under various lighting conditions and vehicle parking postures, so that the model has higher detection accuracy and robustness for the target scene.

[0088] Overall confidence level We obtain it from the following formula:

[0089]

[0090] in, The confidence score of the original output of the YOLOv5 detection model. This represents the stability factor calculated based on the variance of bounding box positions across multiple consecutive frames. It is an image quality factor that combines image contrast, sharpness, and noise level. Considering the environmental adaptability coefficient under illumination conditions, The preset weighting coefficients have a value range of (0, 1], and in this embodiment, the values ​​are 0.3 and 0.2 respectively.

[0091] Stability factor calculated based on the position variance of bounding boxes across multiple consecutive frames We obtain it from the following formula:

[0092]

[0093] in, The attenuation coefficient is a finite number of positive integers; in this embodiment, it is set to 1. This represents the variance of the bounding box center point coordinates over N consecutive frames.

[0094] Image quality factor that integrates image contrast, sharpness, and noise level We obtain it from the following formula:

[0095]

[0096] in, The contrast of an image is represented by calculating the standard deviation of a local region of the image. Image sharpness is indicated by calculating the variance of the Laplacian operator response of the image; The noise level of the image is estimated by analyzing the high-frequency components of the image. , , The preset weighting coefficients satisfy... + =1, and its value range is (0, 1]. In this embodiment... 0.3 =0.4、 =0.3.

[0097] Environmental adaptability coefficient considering illumination conditions From the following formula, we get

[0098]

[0099] in, The normalized light intensity value ranges from 0 to 1, and is 0.8 in this embodiment. This represents an offset less than 1, used to preserve the baseline confidence level under optimal lighting conditions, indicating the system's robustness in extreme cases. In this embodiment... 0.2.

[0100] Step 3: Process the acquired 2D pixel-level bounding box and overall confidence score of the vehicle: Obtain the preset calibration parameters of the high-definition camera and LiDAR, and back-project the 2D pixel-level bounding box onto the 3D point cloud coordinate system of the LiDAR to obtain the 3D region of interest for subsequent point cloud processing; the process of obtaining the 3D region of interest is as follows: the obtained 2D pixel-level bounding box and overall confidence score of the vehicle are processed... The Zhang Zhengyou calibration method is used to obtain preset calibration parameters for the high-definition camera and LiDAR. The 2D pixel-level bounding box of the vehicle is back-projected onto the 3D point cloud coordinate system of the LiDAR through perspective transformation, resulting in a 3D region of interest for subsequent point cloud data processing. After fusion, the system clock counter value is... and fusion confidence The data is then spliced ​​into the fusion result, generating fixed-length data pairs in the following format:

[0101]

[0102] in: This represents the coordinate data of the three-dimensional region of interest. This represents the fusion confidence score of the image and point cloud data. Record parameters for the time point when the image and point cloud data fusion is completed, with a resolution of ≤1ms, to mark the specific time when the fusion operation is completed;

[0103] Fusion confidence of image and point cloud data fusion Determined by the following formula:

[0104]

[0105] in This represents the number of valid point clouds within the ROI region. The threshold for the number of point clouds. Let Variance be the point cloud density distribution. The point cloud coverage ratio of the ROI region. This represents the maximum point cloud density distribution value within the ROI region. Let be the weighting coefficient, satisfying The value range is (0, 1). In this embodiment, the preferred values ​​are respectively... 0.3, 0.4, 0.3. Three-dimensional region of interest and overall confidence level Related, when When the value is below the threshold, the system automatically expands the range of the 3D region of interest obtained by backprojection of the 2D bounding box.

[0106] Step 4: Extract vehicle state features for risk assessment: Perform feature extraction on the above data to obtain vehicle state features for risk assessment. These features include the height, attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the side pillars of the garage, and distance to the rear wall of the garage for the target vehicle entering the garage. If the height is greater than the threshold, the vehicle is directly prohibited from entering the garage. If the height is less than the threshold, the data is formatted, coordinate system unified, averaged over a time window, aligned with multivariate data time series, and the mean is fused. All the above features need to be mapped to a dimensionless numerical range to eliminate the influence of different physical dimensions on the risk assessment model.

[0107] The above data format is used to organize multi-source sensor data into a unified data matrix.

[0108]

[0109] in, The measurement value of the i-th sensor for the j-th risk factor is given, where i is the sensor number (from 1 to I), I is the number of sensors (a finite positive integer), and j is the number of risk factors (an integer from 1 to 5), corresponding to five risk factors: the target vehicle's attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the garage's side pillars, and distance to the garage's rear wall. The coordinates in the sensor's local coordinate system are: ;

[0110] The aforementioned coordinate system one transforms the measured values ​​in the local coordinate systems of each sensor to the world coordinate system using the following formula. :

[0111]

[0112] in, , , They are respectively around The rotation matrix of the axis is determined by the sensor. Installation pitch angle Yaw angle Roll angle Sure, The coordinates of the measurement point in the sensor's local coordinate system. The translation vector from the sensor's local coordinate system to the world coordinate system; the updated coordinate block data matrix. It will be used for time window averaging.

[0113] The aforementioned time window averaging involves applying a sliding window average to multiple data streams from different sensors that represent the same physical quantity. This unit performs cross-sensor temporal smoothing at the system level on multiple data streams from different sensors that represent the same physical quantity (e.g., the lateral distance between a vehicle and a pillar on the side of a garage), suppressing instantaneous fluctuations in single-sensor data and providing a more stable input for subsequent mean fusion. For data from coordinate block data matrices... data stream Smooth output at time t for:

[0114]

[0115] in Where W is the system sampling time interval, and W is the system-level smoothing window width. The time index within the sliding window; the smoothed data matrix formed after smoothing all data streams. Perform multi-dimensional data time-series alignment. This embodiment... The value is 0.01, and W is 5.

[0116] The aforementioned multi-data timing alignment is based on a unified time reference. It resamples each asynchronous data stream to a fixed frequency time grid using a linear interpolation method. In this embodiment, the fixed frequency is 100Hz. This module is based on a unified time reference established by the Precision Clock Protocol (IEEE 1588) to accurately compensate for the small time deviations between different sensor data streams, ensuring that all data are strictly synchronized in time.

[0117] For the target synchronization time interpolation results for:

[0118]

[0119] in, The target synchronization time in the original data The corresponding previous nearest time The data stream values ​​below, The target synchronization time in the original data The corresponding next nearest time The data stream values ​​below.

[0120] Smoothing data matrix After interpolation, the measurement value of the j-th risk factor by the i-th sensor is obtained. , used for subsequent calculations.

[0121] Step 5: Determine vehicle entry risk assessment indicators and conduct risk assessment: Vehicle entry risk assessment indicators include dynamic weights and comprehensive risk factors;

[0122] The aforementioned comprehensive risk factor R is determined by the following formula.

[0123]

[0124] The single-factor composite risk value of the j-th risk factor is determined by the following formula.

[0125]

[0126] The dynamic adjustment coefficient for the j-th risk factor is determined by the following formula.

[0127]

[0128] in, Let the risk sensitivity be the j-th risk factor. For the risk threshold of the j-th risk factor, in this embodiment... =3, 3, =3, =3, =3, Let be the danger threshold for the j-th risk factor, with values ​​of 75°, 2m / s, 5m, 5m, and 2m respectively. The weighted fusion result of the j-th risk factor measured by multiple sensors is determined by the following formula.

[0129]

[0130] in, The confidence level when the i-th sensor measures the j-th risk factor, and the confidence level when the i-th sensor cannot measure the j-th risk factor. =0, =0;

[0131] The corrected weight matrix The weight elements of the j-th risk factor, and the corrected weight matrix. Determined by the following formula

[0132]

[0133] in, The basic weight matrix is ​​5×1. The matrix is ​​a 5×1 identity matrix, γ is the weight adjustment coefficient, and N is a 5×1 dynamic adjustment matrix of risk factors, consisting of the dynamic adjustment coefficients corresponding to each risk factor. constitute.

[0134] Based on comprehensive risk factors The magnitude of risk is determined by the value of R. When R exceeds certain thresholds, the risk level can be classified as low, medium, or high. The specific threshold values ​​need to be determined based on the actual needs of different scenarios, and will vary depending on the level of safety requirements. Additionally, a single-factor comprehensive risk value is also considered. The ranking is based on the single-factor composite risk value. The ranking determines the primary sources of risk, with the largest... The corresponding single factor is the current main risk.

[0135] like Figure 2 As shown, the above-mentioned multi-source sensor data fusion-based vehicle entry risk assessment system for a parking garage includes a sensor network module 100, a data processing and fusion module 200, a dynamic weight calculation module 300, and a comprehensive risk assessment module 400. The modules interact with each other through a system bus or a dedicated communication link. The core data flow is: sensor network module 100 → data processing and fusion module 200 → dynamic weight calculation module 300 → comprehensive risk assessment module 400.

[0136] The aforementioned sensor network module 100 is used to acquire various attitude data during vehicle parking, for subsequent fusion calculation of risk scores; such as Figure 3As shown, the sensor network module 100 includes an ultrasonic array 101, a high-definition camera 102, and a lidar 103. The ultrasonic array module 101 is assembled from multiple independent ultrasonic sensors, specifically: a first ultrasonic array 101a located on the front pillar, which consists of four ultrasonic sensors vertically and evenly spaced at intervals of 0.5m on the front pillar of the garage entrance passage, with the lowest ultrasonic sensor 0.3m above the ground and the highest ultrasonic sensor 1.8m above the ground, used to measure the distance between the vehicle and the front pillar; and a second ultrasonic array 101b located on the rear pillar, which consists of sensors of the same specifications symmetrically installed on the rear pillar. The system includes several ultrasonic sensors: a third ultrasonic array 101c, located on the upper part of the rear wall, with three ultrasonic sensors horizontally installed at a height of 2.2m above the rear wall of the parking space, spaced 0.8m apart and centered on the center line of the parking space; used to detect if a vehicle exceeds the height limit and transmit distance data; and a fourth ultrasonic array 101d, located on the lower part of the rear wall, with six ultrasonic sensors installed in a 2x3 matrix below the height limit line of the rear wall, with a row spacing of 0.6m and a column spacing of 0.8m, covering an area 0.4-1.6m high and 2.4m wide, used to measure the distance between the rear of the vehicle and the rear wall of the parking garage. High-definition camera submodules 102a and 102b are installed at the top of the front frame of the parking garage and at a height of 1.9m above the rear wall, respectively, located at the center of the gap between the third ultrasonic array 101c and the fourth ultrasonic array 101d, used to capture images of the rear of the vehicle from the front. A lidar module 103 is installed on the side or front of the parking garage entrance to detect vehicle point cloud data and obstacle distance information.

[0137] The data processing and fusion module 200 is used to standardize and calibrate the raw data collected by the sensor network module, providing a high-quality, uniformly formatted data foundation for subsequent fusion calculations; such as Figure 4 As shown, the data processing and fusion module 200 includes a data cleaning submodule 201, an image and point cloud data fusion submodule 202, a data format preparation submodule 203, a coordinate system submodule 204, a time window averaging unit 205, a multivariate data time series alignment submodule 206, and a mean fusion unit 207. The data processing and fusion module 200 receives raw multi-source heterogeneous data from the sensor network module 100 and completes standardization processing through the data cleaning submodule 201, the image and point cloud data fusion submodule 202, the data format preparation submodule 203, the coordinate system submodule 204, the time window averaging unit 205, the multivariate data time series alignment submodule 206, and the mean fusion unit 207. The data cleaning submodule includes an ultrasonic data cleaning unit 201a and an image data cleaning unit 201b.

[0138] The dynamic weight calculation module 300 is used to dynamically calculate and allocate the weights of each risk factor in the final fusion score based on different stages of vehicle entry and the real-time confidence levels of each sensor data; such as Figure 5 As shown, the dynamic weight calculation module 300 includes a risk factor calculation submodule 301 and a weight calculation submodule 302. The risk factor calculation submodule 301 includes a door-to-vehicle distance risk calculation unit 301a, a height risk calculation unit 301b, a posture angle risk calculation unit 301c, an entry speed risk calculation unit 301d, and a distance risk calculation unit 301e between the rear wall of the garage and the vehicle. The weight calculation submodule 302 dynamically adjusts the weights of each risk factor according to the entry stage, including posture angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the side pillars of the garage, and distance to the rear wall of the garage.

[0139] The comprehensive risk assessment module 400 is used to calculate the comprehensive risk factors at different stages of vehicle entry into the warehouse, and to determine whether there is any risk and the magnitude of the risk based on the comprehensive risk factors; for example... Figure 6 As shown, the comprehensive risk assessment module 400 includes a risk scoring submodule 401 and a risk evaluation submodule 402. The comprehensive risk assessment module 401 receives risk factors from the risk factor calculation submodule 301, calculates the comprehensive risk factor R through a weighted fusion algorithm, and this score can comprehensively reflect the overall risk level during the vehicle entry process and output it to the human-machine interface or vehicle control system.

Claims

1. A method for risk assessment of vehicle entry into a multi-level parking garage based on multi-source sensor data fusion, characterized in that... Includes the following steps: Step 1: Multi-source data synchronous acquisition: Through the deployment of high-definition cameras, lidar and ultrasonic arrays, two-dimensional image data, three-dimensional point cloud data and near-field obstacle information are collected in real time during the vehicle entry process, and the data collected by the ultrasonic array is cleaned. Step 2: Process the acquired 2D image data: Use the YOLOv5 detection model to process the acquired 2D image data, and use the bounding box regression algorithm to obtain the 2D pixel-level bounding boxes and overall confidence scores of the vehicles entering the warehouse. ; The overall confidence level We obtain it from the following formula: in, The confidence score of the original output of the YOLOv5 detection model. This represents the stability factor calculated based on the variance of bounding box positions across multiple consecutive frames. It is an image quality factor that combines image contrast, sharpness, and noise level. Considering the environmental adaptability coefficient under illumination conditions, The preset weighting coefficients have a value range of (0, 1]. Stability factor calculated based on the position variance of bounding boxes across multiple consecutive frames We obtain it from the following formula: in, The attenuation coefficient is a finite number of positive integers. The variance of the bounding box center point coordinates over N consecutive frames; Image quality factor that integrates image contrast, sharpness, and noise level We obtain it from the following formula: in, The contrast of an image is represented by calculating the standard deviation of a local region of the image. Image sharpness is indicated by calculating the variance of the Laplacian operator response of the image; The noise level of the image is estimated by analyzing the high-frequency components of the image. , , The preset weighting coefficients satisfy... + =1, , , The value range is (0, 1]; Environmental adaptability coefficient considering illumination conditions From the following formula, we get in, This is the normalized light intensity value, ranging from 0 to 1. This represents an offset less than 1, used to preserve the baseline confidence level under optimal lighting conditions, indicating the system's robustness in extreme situations. Step 3: Process the acquired 2D pixel-level bounding box and overall confidence score of the vehicle: Obtain the preset calibration parameters of the high-definition camera and the LiDAR, and back-project the 2D pixel-level bounding box onto the 3D point cloud coordinate system of the LiDAR to obtain the 3D region of interest for subsequent point cloud processing. Step 4: Extract vehicle state features for risk assessment: Extract features from the data in Step 3 to obtain vehicle state features for risk assessment. The vehicle state features include the height, attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the side pillars of the garage, and distance to the rear wall of the garage for the target vehicle entering the garage. If the height is greater than the threshold, the vehicle is directly prohibited from entering the garage. If the height is less than the threshold, the data is formatted, coordinate system is unified, time window averaging is performed, multivariate data time series alignment is performed, and mean fusion is performed. Step 5: Determine vehicle entry risk assessment indicators and conduct risk assessment: The vehicle entry risk assessment indicators include comprehensive risk factors. Single-factor composite risk value ; The comprehensive risk factor R is determined by the following formula: The single-factor composite risk value of the j-th risk factor is determined by the following formula. The dynamic correction coefficient for the j-th risk factor is determined by the following formula. in, Let j be the risk sensitivity of the j-th risk factor. Let the risk threshold be the j-th risk factor. The weighted fusion result of the j-th risk factor measured by multiple sensors is determined by the following formula. in, The confidence level when the i-th sensor measures the j-th risk factor. Let be the measurement value of the j-th risk factor by the i-th sensor. When the i-th sensor cannot measure the j-th risk factor, =0, =0; The corrected weight matrix The weight elements of the j-th risk factor, and the corrected weight matrix. Determined by the following formula: in, The basic weight matrix is ​​5×1. The matrix is ​​a 5×1 identity matrix, where γ is the weight adjustment coefficient (taking a finite positive integer), and N is a 5×1 dynamic adjustment matrix for risk factors, consisting of the dynamic adjustment coefficients corresponding to each risk factor. constitute.

2. The method for risk assessment of vehicle entry into a multi-level parking garage based on multi-source sensor data fusion as described in claim 1, characterized in that... The process of cleaning the data collected by the ultrasonic array in step 1 is as follows: First, a scalar Kalman filter is performed on each independent sensor in the ultrasonic array. Then, within the same interrupt cycle, the system clock counter value τ(t) and the current ultrasonic sensor confidence level are compared. splicing the data to the estimated values, and generating fixed-length data pairs using the following formula. : Among them, the system clock counter value τ(t) has a resolution ≤ 1ms. The distance data for each ultrasonic array submodule after cleaning is shown. The current confidence level of the ultrasonic sensor; These are the weighting coefficients. To comprehensively evaluate signal quality, For environmental adaptability, For the sake of historical consistency, For geometric rationality.

3. The method for risk assessment of vehicle entry into a multi-level parking garage based on multi-source sensor data fusion as described in claim 2, characterized in that... The weighting coefficient The value range is (0, 1], and the weighting coefficient satisfies .

4. The method for risk assessment of vehicle entry into a multi-level parking garage based on multi-source sensor data fusion as described in claim 1, characterized in that... The process of obtaining the three-dimensional region of interest in step 3 is as follows: The obtained 2D pixel-level bounding box of the vehicle and the overall confidence level are... The Zhang Zhengyou calibration method is used to obtain preset calibration parameters for the high-definition camera and LiDAR. The 2D pixel-level bounding box of the vehicle is back-projected onto the 3D point cloud coordinate system of the LiDAR through perspective transformation, resulting in a 3D region of interest for subsequent point cloud data processing. After fusion, the system clock counter value is... and fusion confidence The data is spliced ​​into the fusion result, generating fixed-length data pairs as follows: in: This represents the coordinate data of the three-dimensional region of interest. This represents the fusion confidence score of the image and point cloud data. Record parameters for the time point when the image and point cloud data fusion is completed, with a resolution of ≤1ms, to mark the specific time when the fusion operation is completed; Fusion confidence of image and point cloud data fusion Determined by the following formula: in This represents the number of valid point clouds within the ROI region. The threshold for the number of point clouds. Let Variance be the point cloud density distribution. The point cloud coverage ratio of the ROI region. This represents the maximum point cloud density distribution value within the ROI region. The weighting coefficient has a value range of (0, 1] and satisfies the following conditions: .

5. The method for risk assessment of vehicle entry into a multi-level parking garage based on multi-source sensor data fusion as described in claim 1, characterized in that... In step 4, the data format is organized by arranging the multi-source sensor data into a unified data matrix. in, This represents the raw measurement value of the i-th sensor for the j-th risk factor, where i is the sensor number (from 1 to I), I is the number of sensors (a finite positive integer), and j is the number of risk factors (an integer from 1 to 5), corresponding to the five risk factors: the target vehicle's attitude angle, instantaneous speed, longitudinal distance to the garage door frame, lateral distance to the garage's side pillars, and distance to the garage's rear wall. The coordinates in the sensor's local coordinate system are: ; The coordinate system is one that transforms the measured values ​​in the local coordinate systems of each sensor to the world coordinate system using the following formula. : in, , , They are respectively around The rotation matrix of the axis is determined by the sensor. Installation pitch angle Yaw angle Roll angle Sure, The coordinates of the measurement point in the sensor's local coordinate system. The translation vector from the sensor's local coordinate system to the world coordinate system; the updated coordinate block data matrix. It will be used for time window averaging; The time window averaging process involves applying a sliding window average to multiple data streams from different sensors that characterize the same physical quantity. For data from coordinate block data matrices... data stream Smooth output at time t for: in Where W is the system sampling time interval, and W is the system-level smoothing window width. The smoothed data matrix is ​​formed by smoothing all data streams using time indices within the sliding window. Perform time-series alignment of multi-source data; The multi-data time-series alignment is based on a unified time base, whereby each asynchronous data stream is uniformly resampled to a fixed-frequency time grid using a linear interpolation method, for the target synchronization time. interpolation results for: in, The target synchronization time in the original data The corresponding previous nearest time The data stream values ​​below, The target synchronization time in the original data The corresponding next nearest time The data stream values ​​below; Smoothing data matrix After interpolation, the measurement value of the j-th risk factor by the i-th sensor is obtained. , used for subsequent calculations.

6. An assessment system for a multi-source sensor data fusion-based vehicle entry risk assessment method for automated parking garages according to claim 1, characterized in that: It includes a sensor network module (100), a data processing and fusion module (200), a dynamic weight calculation module (300), and a comprehensive risk assessment module (400). The sensor network module (100) is used to acquire various attitude data during the vehicle parking process, which is then used for subsequent fusion calculation of risk scores; The data processing and fusion module (200) is used to standardize and calibrate the raw data collected by the sensor network module, providing a high-quality, unified data foundation for subsequent fusion calculations; The dynamic weight calculation module (300) is used to dynamically calculate and allocate the weight of each risk factor in the final fusion score based on the different stages of vehicle entry into the warehouse and the real-time confidence level of each sensor data. The comprehensive risk assessment module (400) is used to calculate the comprehensive risk factors at different stages of vehicle entry into the warehouse, and to determine whether there will be any risk and the magnitude of the risk based on the comprehensive risk factors.

7. The evaluation system according to claim 6, characterized in that: The sensor network module (100) includes an ultrasonic array (101), a high-definition camera (102), and a lidar (103). The data processing and fusion module (200) includes a data cleaning submodule (201), an image and point cloud data fusion submodule (202), a data format sorting submodule (203), a coordinate system submodule (204), a time window averaging unit (205), a multivariate data time series alignment submodule (206), and a mean fusion unit (207). The dynamic weight calculation module (300) includes a risk factor calculation submodule (301) and a weight calculation submodule (302). The comprehensive risk assessment module (400) includes a risk scoring submodule (401) and a risk evaluation submodule (402).

8. The evaluation system according to claim 7, characterized in that: The risk factor calculation submodule (301) includes a door-to-vehicle distance risk calculation unit (301a), a height risk calculation unit (301b), an attitude angle risk calculation unit (301c), an entry speed risk calculation unit (301d), and a distance risk calculation unit between the rear wall of the warehouse and the vehicle (301e).

Images