A laser radar-based stereo garage safety protection system and method

Abstract

The application discloses a kind of stereoscopic garage safety protection system and method based on laser radar, belong to positioning measurement technical field, it includes the laser radar point cloud data and visual image data in stereoscopic garage, laser radar point cloud data and visual image data are time-space synchronous calibration, generate synchronous multi-source perception data;Based on synchronous multi-source perception data, construct the real-time dynamic environment model containing target geometric information and semantic information;Dynamic target and static target are identified in real-time dynamic environment model, and the risk potential field of each target is calculated, and the global composite risk field atlas representing comprehensive risk distribution is generated.The laser radar point cloud and visual image fusion perception technology of time-space synchronization are used, and the global composite risk field atlas representing comprehensive risk distribution is constructed, the high-precision identification and positioning of dynamic and static target in garage can be realized.

Description

Technical Field

[0001] This invention relates to the field of positioning and measurement technology, and in particular to a safety protection system and method for a three-dimensional parking garage based on lidar. Background Technology

[0002] Automated parking garages, as a highly efficient parking solution for utilizing urban space, contain various automated mechanical equipment such as elevators and trolleys, operating in a complex and dynamically changing environment. To ensure the safety of personnel, vehicles, and equipment, automated parking garages typically deploy safety protection systems. These systems generally rely on sensor technology for environmental monitoring, with the use of optical methods such as lidar for distance measurement and target localization being one of the key technologies for obstacle detection.

[0003] Existing safety technologies for automated parking garages typically employ a single type of sensor solution. For example, infrared or ultrasonic sensors are installed in key passageways to form detection barriers, triggering an emergency stop when an object enters. Some solutions also use lidar for area scanning to detect obstacles within designated protected areas. Alternatively, video surveillance systems are employed, relying on manual monitoring or simple image algorithms for oversight.

[0004] However, infrared or ultrasonic sensing methods have limited detection range and resolution, and cannot provide detailed target information. While lidar alone can acquire precise location and contour information, it cannot distinguish the attributes of obstacles, affecting operational efficiency. Video surveillance alone lacks accurate three-dimensional spatial positioning capabilities, making accurate distance judgment difficult. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a three-dimensional parking garage safety protection system and method based on lidar. It employs a spatiotemporally synchronized lidar point cloud and visual image fusion perception technology, and constructs a global composite risk field map characterizing the comprehensive risk distribution, enabling high-precision identification and positioning of dynamic and static targets within the parking garage.

[0006] The above objectives can be achieved through the following approach:

[0007] A safety protection method for a three-dimensional parking garage based on lidar includes acquiring lidar point cloud data and visual image data within the parking garage; performing spatiotemporal synchronization calibration on the lidar point cloud data and the visual image data to generate synchronous multi-source perception data; constructing a real-time dynamic environment model containing target geometric and semantic information based on the synchronous multi-source perception data; identifying dynamic and static targets in the real-time dynamic environment model, calculating the vectorized risk potential field of each target, and generating a global composite risk field map representing the comprehensive risk distribution.

[0008] Optionally, the generation of synchronous multi-source sensing data includes: acquiring lidar point cloud data and visual image data within the automated parking garage; receiving a unified hardware synchronization pulse signal, and aligning the lidar point cloud data and the visual image data with timestamps to obtain timestamp-aligned data; collecting real-time device pose offset data generated by sensor mechanical vibration; and using the device pose offset data to correct the timestamp-aligned data, eliminating spatial coordinate errors, thereby generating synchronous multi-source sensing data.

[0009] Optionally, the step of constructing a real-time dynamic environment model containing target geometric and semantic information based on the synchronous multi-source sensing data includes: extracting three-dimensional geometric features from the lidar point cloud data in the synchronous multi-source sensing data to establish an obstacle grid layer representing the distribution of physical obstacles; identifying target semantic features from the visual image data in the synchronous multi-source sensing data to form a semantic label layer that labels the target category; and performing pixel fusion of the obstacle grid layer and the semantic label layer to assign semantic attributes at geometric locations, thereby constructing an initial environment model.

[0010] Optionally, the step of constructing a real-time dynamic environment model containing target geometric information and semantic information based on the synchronous multi-source sensing data further includes: obtaining a vehicle scheduling plan, calculating the mechanical equipment movement trajectory within a preset time window based on the vehicle scheduling plan, and generating a mechanical equipment movement trend layer; and performing multi-layer data fusion between the initial environment model and the mechanical equipment movement trend layer to obtain a dynamic environment model.

[0011] Optionally, generating a global composite risk field map representing the comprehensive risk distribution includes: identifying all dynamic and static targets in the dynamic environment model and constructing a risk source set; obtaining the motion state and target type of each target in the risk source set and calculating the vectorized risk potential field of each target; and vector superimposing the risk potential fields of each target to generate a global composite risk field map.

[0012] Optionally, the step of calculating the vectorized risk potential field of each target based on the motion state and target type of each target in the risk source set includes: obtaining target state parameters based on the real-time motion speed and target classification attributes of each target in the risk source set; assigning risk weight coefficients according to the target classification attributes of each target to obtain the risk type weight of each target; and calculating the risk potential field of each target by combining the target state parameters and the risk type weight of each target.

[0013] Optionally, the method further includes: dividing and outputting a comprehensive safety risk level based on the field strength values ​​of each region in the composite risk field map; mapping the comprehensive safety risk level to a preset multi-level response strategy set to generate a preliminary control strategy that includes the adjustment of mechanical equipment operating parameters and the triggering of alarm signals; generating a collaborative protection command based on the preliminary control strategy, and adjusting the mechanical equipment operating speed or performing braking in real time based on the collaborative protection command to activate the corresponding audible and visual alarm device.

[0014] Optionally, the method further includes: periodically detecting the integrity and quality of the sensor data stream, identifying and generating a sensor fault signal; when the sensor fault signal is received, using data from a sensor that is still working normally to fill the data gaps in the faulty sensor and generate compensating sensing data; and using the compensating sensing data to correct the synchronous multi-source sensing data.

[0015] Optionally, when the sensor fault signal is received, the step of filling the data gaps of the faulty sensor with data from a sensor that is still operating normally to generate compensated perception data includes: receiving the sensor fault signal; when the sensor fault signal indicates a lidar fault, performing depth estimation using visual image data and combining it with pre-acquired historical point cloud data to reconstruct and generate compensated 3D environment data; when the sensor fault signal indicates a visual sensor fault, performing clustering and texture analysis on the lidar point cloud data to identify and classify targets and generate compensated target label data; wherein, the compensated perception data includes the compensated 3D environment data and the compensated target label data.

[0016] Based on the same inventive concept, this invention also provides a LiDAR-based safety protection system for a three-dimensional parking garage. The system includes: a positioning data acquisition module for acquiring LiDAR point cloud data and visual image data within the three-dimensional parking garage, performing spatiotemporal synchronization calibration of the LiDAR point cloud data and the visual image data to generate synchronous multi-source sensing data; a positioning model construction module for constructing a real-time dynamic environment model containing target geometric and semantic information based on the synchronous multi-source sensing data; and a positioning identification module for identifying dynamic and static targets within the real-time dynamic environment model, calculating the vectorized risk potential field of each target, and generating a global composite risk field map characterizing the comprehensive risk distribution.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] 1. This invention constructs an environment model containing three-dimensional geometric and semantic information by fusing lidar point cloud data and visual image data and performing precise spatiotemporal synchronization. This multi-source perception data fusion method breaks through the perception limitations of a single sensor, enabling the system to not only accurately know the position and shape of obstacles, but also accurately identify their category attributes.

[0019] 2. This invention transforms safety risks into a calculable and quantifiable continuous spatial field, and can perform differentiated modeling based on factors such as the type and motion state of the target; this enables safety assessments to dynamically and intuitively present the risk distribution and gradient throughout the garage, and in particular, can effectively identify potential high-risk areas formed by the intersection of multiple dynamic targets, greatly improving the accuracy and predictability of risk assessments.

[0020] 3. This invention combines the garage's own scheduling plan with real-time environmental perception. By calculating the future movement trajectory of mechanical equipment, it introduces the time dimension of prediction capability into the environmental model, which can transform from passive collision detection to active risk prediction. It can identify potential safety conflicts on the planned path of mechanical equipment in advance, thereby gaining a valuable time window for taking preventive measures and significantly enhancing the active safety of the entire protection system.

[0021] 4. In the event of a sudden failure of some sensors, this invention can reconstruct information using data from heterogeneous sensors that are still working normally, fill in the data gaps, ensure the continuity of core sensing functions, ensure high reliability and robustness under complex working conditions, avoid the risk of paralyzing the entire safety system due to a single point of hardware failure, and ensure the uninterrupted operation of safety protection functions.

[0022] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a method for safety protection of a three-dimensional parking garage based on lidar, according to an embodiment of the present invention.

[0025] Figure 2This is a schematic diagram of a three-dimensional parking garage safety protection system based on lidar according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Reference Figure 1 One embodiment of the present invention proposes a three-dimensional parking garage safety protection method based on lidar. It adopts a spatiotemporally synchronized lidar point cloud and visual image fusion perception technology, and constructs a global composite risk field map that characterizes the comprehensive risk distribution, which can achieve high-precision identification and positioning of dynamic and static targets in the parking garage.

[0028] The method described in this embodiment specifically includes:

[0029] Acquire lidar point cloud data and visual image data in the automated parking garage, perform spatiotemporal synchronization calibration on the lidar point cloud data and the visual image data, and generate synchronous multi-source perception data.

[0030] Based on the synchronous multi-source sensing data, a real-time dynamic environment model containing target geometric information and semantic information is constructed.

[0031] In the real-time dynamic environment model, dynamic and static targets are identified, and the vectorized risk potential field of each target is calculated to generate a global composite risk field map that characterizes the comprehensive risk distribution.

[0032] This invention overcomes the limitations of single-sensor perception by deeply fusing lidar and visual images. It not only detects the presence of obstacles but also accurately identifies the type and attributes of targets, enhancing the depth and breadth of environmental perception. This method transforms the concept of safety risk into a calculable and quantifiable physical field model. By generating a global composite risk field map, it can proactively reveal potential danger zones and risk hotspots, especially the complex risk situation formed by the interaction of multiple dynamic targets. This provides a decision-making basis for subsequent early warning and collaborative control strategies, thereby improving the initiative, foresight, and intelligence level of safety protection in automated parking systems.

[0033] Optionally, the generation of synchronous multi-source sensing data includes:

[0034] Acquire lidar point cloud data and visual image data within the automated parking garage;

[0035] Receive a unified hardware synchronization pulse signal, and perform timestamp alignment between the lidar point cloud data and the visual image data to obtain timestamp aligned data;

[0036] Collect real-time device pose offset data caused by mechanical vibration of the sensor;

[0037] The timestamp alignment data is corrected using the device pose offset data to eliminate spatial coordinate errors, thereby generating synchronous multi-source sensing data.

[0038] Specifically, the system first needs to acquire LiDAR point cloud data and visual image data in parallel within the automated parking garage environment. Since the two sensors operate independently, their internal clocks are off-center, and their installation positions may be affected by high-frequency vibrations from mechanical equipment operation, causing the data to be unable to be directly aligned in time and space. To achieve spatiotemporal synchronization, the system introduces a unified hardware synchronization pulse signal source, such as a GPS pulse-per-second (PPS) signal or a dedicated synchronization controller. This source simultaneously sends high-precision time synchronization pulses to both the LiDAR and the visual sensor. When a sensor receives a synchronization pulse, it assigns a precise timestamp based on the same time reference to its currently acquired data frame (point cloud data frame or visual image data frame), and the timestamps of subsequent frames are recursively derived from this. In this way, data from different sources are aligned in the time dimension, forming timestamp-aligned data. However, time alignment alone is insufficient to guarantee data quality, as mechanical vibrations can cause minute but rapid changes in the physical position and orientation of the sensors during the acquisition process. To address this issue, the system acquires real-time device pose offset data caused by mechanical vibrations, typically achieved through an inertial measurement unit (IMU) rigidly connected to the LiDAR and camera. The IMU can output the angular velocity and acceleration information of the carrier at high frequency. Through integration, the translational and rotational changes of the sensor relative to its initial calibration position can be calculated in real time, i.e., the device pose offset data. Finally, the system uses this device pose offset data to dynamically compensate and correct the timestamp alignment data. Specifically, for each timestamp of point cloud data or image data, a rigid body transformation matrix is ​​constructed based on the device pose offset data at the corresponding time, transforming the data from the sensor's current local coordinate system to the global static coordinate system of the garage. This transformation can be expressed as:

[0039] ;

[0040] in These are the coordinates of the original data points in the sensor coordinate system. It is a homogeneous transformation matrix composed of real-time device pose offset data, containing rotation and translation components. These are the corrected coordinates in the global coordinate system. After this spatial coordinate error elimination step, the final data obtained is synchronous multi-source sensing data that is highly consistent in both spatiotemporal dimensions.

[0041] Optionally, constructing a real-time dynamic environment model containing target geometric and semantic information based on the synchronous multi-source sensing data includes:

[0042] Three-dimensional geometric features are extracted from the lidar point cloud data in the synchronous multi-source sensing data to establish an obstacle grid layer that characterizes the distribution of physical obstacles;

[0043] Identify target semantic features from visual image data in the synchronous multi-source sensing data to form a semantic label layer that labels the target category;

[0044] The obstacle grid layer and the semantic tag layer are pixel-wise fused, and semantic attributes are assigned to their geometric locations to construct an initial environment model.

[0045] Specifically, the system first processes LiDAR and visual information in parallel based on synchronous multi-source sensing data. On one hand, the system extracts LiDAR point cloud data from the synchronous multi-source sensing data to construct an obstacle grid layer. This process discretizes the continuous three-dimensional space of the automated parking garage into a two-dimensional grid map. Specifically, the garage floor is projected as a two-dimensional grid, with each grid cell representing a small physical area. Next, three-dimensional point cloud data is projected onto this two-dimensional grid, and the state of each cell is determined based on its three-dimensional geometric features, such as height and density. For example, if a cell contains point cloud clusters above the ground, the cell is marked as "occupied"; conversely, if there are no point clouds or the point cloud height is close to the ground, it is marked as "passable"; if there is no sensor data coverage, it is marked as "unknown." This generates an obstacle grid layer that only represents the distribution of physical obstacles, accurately depicting the geometric contours and positions of entities in the environment. On the other hand, the system extracts visual image data from the synchronous multi-source sensing data and uses semantic segmentation or object detection network models in deep learning to identify the semantic features of targets. These models, trained on extensive data, are capable of identifying the specific object categories represented by pixels or regions in images, such as people, vehicles, transporters, walls, and pillars. The processing result is a pixel-level label map of the same size as the original image, where each pixel value represents its corresponding semantic category. This label map constitutes the semantic label layer that annotates the target categories. The final step is pixel fusion, which correlates the information from the two layers mentioned above, assigning semantic attributes at geometric locations. Since the LiDAR and camera have undergone spatiotemporal synchronization calibration, the system possesses a precise coordinate transformation relationship between them. Through this transformation relationship, each "occupied" unit in the obstacle grid layer can be back-projected onto the corresponding pixel region in the semantic label layer. By querying the semantic label within that region, it is possible to determine exactly what obstacle occupies that physical space. For example, a region marked as "occupied" by the obstacle grid layer corresponds to the label "person" in the semantic label layer, and the system updates the attribute of that grid unit to "person." By performing this operation on all occupied units, the purely geometric map is ultimately upgraded into an initial environment model with semantic information.

[0046] Optionally, the step of constructing a real-time dynamic environment model containing target geometric and semantic information based on the synchronous multi-source sensing data further includes:

[0047] Obtain the vehicle scheduling plan, and calculate the mechanical equipment movement trajectory within a preset time window based on the vehicle scheduling plan to generate a mechanical equipment movement trend layer;

[0048] The initial environment model is fused with the motion trend layer of the mechanical equipment to obtain a dynamic environment model.

[0049] Specifically, the system first needs to actively acquire the vehicle scheduling plan issued by the central control system of the automated parking garage. This plan is a high-level task instruction that specifies which mechanical equipment, such as elevators or traversing trolleys, will perform what kind of vehicle storage and retrieval task within a certain period of time, including its starting and destination positions. Next, based on this vehicle scheduling plan and combined with pre-established kinematic and dynamic models of the mechanical equipment, the system calculates the trajectory of the mechanical equipment within a preset time window. This calculation process transforms the high-level task instructions into a series of precise spatiotemporal coordinates. For example, for the vertical movement task of an elevator, the system calculates its precise three-dimensional spatial position and attitude at every future moment based on its rated speed, acceleration, and travel distance. This predictive trajectory data constitutes the mechanical equipment movement trend layer. This layer can be viewed as a dynamic information layer that changes over time, pre-marking the path the mechanical equipment will traverse and the space it will occupy in the environmental model. Finally, the system performs multi-layer data fusion, overlaying and integrating this mechanical equipment movement trend layer containing future temporal information with the initial environmental model that describes the current physical and semantic state of the environment. This fusion process is completed within a shared grid coordinate system, ensuring that each grid cell in the initial environment model possesses not only its current occupancy state and semantic attributes, but also a predictive attribute regarding whether it will be occupied by moving machinery in the future. Through this fusion, the system ultimately generates a dynamic environment model that incorporates both the current state of the environment and the system's own intentions.

[0050] Optionally, the generation of the global composite risk field map characterizing the comprehensive risk distribution includes:

[0051] In the dynamic environment model, all dynamic and static targets are identified, and a set of risk sources is constructed.

[0052] Obtain the motion state and target type of each target in the risk source set, and calculate the vectorized risk potential field of each target;

[0053] The risk potential fields of each target are vector-superimposed to generate a global composite risk field map.

[0054] Optionally, calculating the vectorized risk potential field of each target based on the motion state and target type of each target in the risk source set includes:

[0055] The target state parameters are formed by obtaining the real-time movement speed and target classification attributes of each target in the risk source set;

[0056] Risk weight coefficients are assigned to each objective based on its target classification attributes to obtain the risk type weight for each objective.

[0057] By combining the target state parameters of each target with the risk type weights of each target, the risk potential field of each target is calculated.

[0058] Specifically, firstly, in the constructed dynamic environment model, all perceived objects are classified and identified, distinguishing between dynamic targets, such as moving personnel, vehicles entering and exiting, and operating robotic arms, and static targets, such as pillars, walls, and stationary vehicles. All these targets that may affect safety constitute a dynamically updated set of risk sources. Subsequently, for each target in this risk source set, the system calculates in detail the vectorized risk potential field it generates on the surrounding space. The risk potential field is a concept borrowed from physics, used to describe the intangible influence of a single risk source in space. Its calculation depth depends on two core elements: the target's motion state and target type, both of which can be directly obtained from the dynamic environment model. Specifically, the motion state includes the target's real-time velocity and acceleration obtained through a continuous frame tracking algorithm, while the target type is provided by the model's semantic label layer. For each risk source i, its generated risk potential field... It can be represented as a scalar function, which at a point in space value at This can be described by an asymmetric Gaussian model that takes into account the state of motion:

[0059] ;

[0060] in, It is the coordinate vector of any point in space. It is the coordinate vector of the center position of risk source i, which is obtained in real time through target detection and localization. It is a risk type weighting coefficient preset according to the target type. For example, the weight of a person is significantly higher than that of a static column. It is the inverse of the covariance matrix, which describes the shape and direction of the risk field, and is dynamically determined by the target's motion state. For a static target, It can be an isotropic matrix, forming a circular or spherical risk field. For the dynamic target, i.e., risk source i, It is then constructed as a velocity along its path. The directionally stretched ellipsoid extends the risk field further along the direction of the target's motion, thus reflecting the forward collision risk brought about by the motion. Finally, based on the principle of potential field superposition, the system vector-superimposes the risk potential fields generated by all individual targets in the risk source set. At any point in space... Its overall risk potential It is all individual risk potentials The algebraic sum of these yields a scalar risk potential. The vectorized risk potential... This is the negative gradient of the scalar field, i.e.:

[0061] ;

[0062] Its direction points towards the direction of fastest risk growth, i.e., towards the source of risk, while its magnitude represents the intensity of risk at that point. This applies to all points within the entire garage space. By calculating and rendering the magnitude of the risk potential field, a global composite risk field map representing the overall risk distribution is ultimately generated. This map is usually visualized in the form of a heat map, which intuitively shows the safe areas, warning areas, and danger areas within the garage.

[0063] Optionally, the method further includes:

[0064] Based on the field strength values ​​of each region in the composite risk field map, the comprehensive safety risk level is divided and output.

[0065] The comprehensive safety risk level is mapped to a preset multi-level response strategy set to generate a preliminary control strategy that includes the adjustment of mechanical equipment operating parameters and the triggering of alarm signals;

[0066] Based on the preliminary control strategy, a collaborative protection command is generated, and based on the collaborative protection command, the operating speed of the mechanical equipment is adjusted in real time or braking is performed, activating the corresponding audible and visual alarm device.

[0067] Specifically, the system first classifies risk levels based on the field strength values ​​of each region in the generated global composite risk field map. By setting multiple incremental field strength thresholds, all spatial areas within the garage are divided into different comprehensive safety risk levels, such as "safe zone," "warning zone," and "danger zone," and this dynamic risk zoning map is output in real time. Next, the system maps the classified comprehensive safety risk levels to a preset multi-level response strategy set. This strategy set predefines a series of response measures for different risk levels. For example, a "safe zone" corresponds to a "normal operation" strategy; a "warning zone" might correspond to "slow down" and trigger a "yellow warning"; and a "danger zone" corresponds to "emergency braking" and trigger a "red alert." Through this mapping relationship, the system can automatically generate preliminary control strategies, including adjustments to mechanical equipment operating parameters and alarm signal triggering, based on the current risk situation. For example, if the risk field strength value in the area ahead of a moving vehicle's planned path exceeds the warning threshold, the system will generate an instruction requiring the moving vehicle to reduce its operating speed to a preset safe speed. Finally, the system translates this preliminary control strategy into specific, executable collaborative protection commands and sends them to the central control system of the automated parking garage or directly to the relevant execution units. Based on these collaborative protection commands, the central control system will adjust the operating speed of relevant mechanical equipment in real time or decisively execute emergency braking operations when necessary. Simultaneously, corresponding audible and visual alarm devices will be activated, such as illuminating warning lights and sounding buzzers near the risk area to provide clear warning signals to on-site personnel and vehicle drivers, achieving human-machine collaborative safety protection.

[0068] Optionally, the method further includes:

[0069] Periodically check the integrity and quality of sensor data streams, identify and generate sensor fault signals;

[0070] When a fault signal is received from the sensor, the data missing from the faulty sensor is filled with data from the sensor that is still working normally, and compensating sensing data is generated.

[0071] The compensated sensing data is used to correct the synchronous multi-source sensing data.

[0072] Specifically, the system monitors the integrity and quality of data streams from each sensor, such as LiDAR and vision sensors, in real time. Integrity checks focus on whether data is transmitted stably at a predetermined frequency, for example, checking if the timestamp intervals of data frames are normal. Quality checks analyze the rationality of the data content itself, for example, whether LiDAR point cloud data has a large amount of noise or sparse data points, and whether vision image data has anomalies such as pure black screens or frozen frames. Once any condition that does not meet the preset health standards is detected, the system immediately identifies the source of the fault and generates a clear sensor fault signal, which contains the identification information of the faulty sensor. When the system's perception fusion module receives this sensor fault signal, it triggers a data compensation process. The core idea of ​​this process is to utilize the inherent information redundancy of the multi-sensor system, that is, to use data from sensors that are still working normally to infer and fill the data gaps that the faulty sensor should have provided. For example, if the LiDAR malfunctions, the system will primarily rely on visual image data, using monocular or binocular depth estimation algorithms to reconstruct the scene's 3D geometry and generate compensatory 3D environmental data. If the visual sensor malfunctions, the system will utilize the geometric features of the LiDAR point cloud data, such as the size and shape of point cloud clusters, for cluster analysis and attempt to roughly classify the clustered objects, generating compensatory target label data. The alternative data generated in this process is collectively referred to as compensatory perception data. Finally, the system uses this compensatory perception data to correct the synchronous multi-source perception data. Specifically, in the data fusion pipeline, the system replaces the data stream from the faulty sensor with the newly generated compensatory perception data stream, and then fuses it with data from other normal sensors. In this way, subsequent modules such as environmental modeling and risk assessment can receive a structurally complete set of perception data that, while degraded, is still usable, thus continuing to perform their safety protection functions.

[0073] Optionally, the step of filling the data gaps of the faulty sensor with data from a sensor that is still operating normally and generating compensating sensing data when the sensor fault signal is received includes:

[0074] The sensor fault signal was received;

[0075] When the sensor fault signal indicates a lidar fault, depth estimation is performed using visual image data, and combined with pre-acquired historical point cloud data, to reconstruct and generate compensated 3D environment data.

[0076] When the sensor fault signal indicates a visual sensor fault, clustering and texture analysis are performed on the lidar point cloud data to identify and classify targets and generate compensation target label data.

[0077] The compensation perception data includes the compensation 3D environment data and the compensation target label data.

[0078] Specifically, the emergency response mechanism activated by the system upon receiving a sensor fault signal executes different data reconstruction strategies based on the type of faulty sensor. When the received sensor fault signal indicates a LiDAR malfunction, the system loses its ability to directly acquire high-precision 3D geometric information. To fill this data gap, the system utilizes visual image data acquired by a still-functioning vision sensor. By running a pre-trained depth estimation algorithm model, such as a deep learning-based monocular depth estimation network, the system can infer the depth or distance information of each pixel from the 2D image, thereby generating a depth map. However, depth maps generated solely from images may suffer from scale uncertainty and low accuracy. To optimize this result, the system further incorporates pre-acquired historical point cloud data, typically a high-precision static environment map established during system initialization. By aligning and fusing the real-time generated depth map with this historical point cloud data, the scale of the depth map can be corrected, and high-precision static background information can be used to fill and correct the non-dynamic parts of the reconstructed scene, ultimately reconstructing a compensated 3D environment data sufficient to replace the current frame of LiDAR data. Conversely, when a sensor fault signal indicates a malfunction in the vision sensor, the system loses its ability to directly acquire environmental semantic information. In this case, the system relies on point cloud data provided by the still-operating LiDAR. First, the real-time acquired LiDAR point cloud data is processed using clustering algorithms, such as DBSCAN or Euclidean clustering, to segment the discrete point cloud into clusters representing different independent objects. Then, the system performs geometric and texture analysis on each point cloud cluster. Geometric features include the cluster's size, shape, and volume; texture analysis utilizes the reflection intensity information typically contained in the LiDAR point cloud to analyze the variation patterns of reflectivity within the cluster. By comparing these extracted features with a pre-defined object feature library, the system can identify and classify targets. For example, a small, irregularly shaped, moving point cloud cluster might be classified as a "person," while a large, regularly shaped static cluster might be classified as a "pillar" or a "stationary vehicle." The output of this process is a series of targets with category labels, i.e., compensated target label data. Ultimately, depending on the type of fault, the generated compensated 3D environmental data or compensated target label data together constitute the compensated sensing data used for subsequent processing.

[0079] Based on the same inventive concept, such as Figure 2 As shown, the present invention also provides a three-dimensional parking garage safety protection system based on lidar, the system comprising:

[0080] The positioning data acquisition module is used to acquire lidar point cloud data and visual image data in the three-dimensional parking garage, and to perform spatiotemporal synchronization calibration of the lidar point cloud data and the visual image data to generate synchronous multi-source perception data.

[0081] The localization model construction module is used to construct a real-time dynamic environment model containing target geometric information and semantic information based on the synchronous multi-source sensing data.

[0082] The positioning and identification module is used to identify dynamic and static targets in the real-time dynamic environment model, calculate the vectorized risk potential field of each target, and generate a global composite risk field map that characterizes the comprehensive risk distribution.

[0083] To verify the feasibility of this invention in practice, it was applied to a multi-level parking garage. This multi-level parking garage includes multiple elevators and traversing trolleys for automated vehicle storage and retrieval. The operating environment is complex, posing a potential risk of collisions between mechanical equipment, vehicles entering and exiting, and unauthorized personnel.

[0084] In this embodiment, multiple LiDAR and high-definition vision sensors are deployed in key passages and work areas of the garage. The system first performs spatiotemporal synchronization calibration on the acquired LiDAR point cloud data and visual image data. By receiving a unified hardware synchronization pulse signal, all sensor data frames are tagged with precisely aligned timestamps. Simultaneously, inertial measurement units (IMUs) installed on the sensors monitor pose shifts caused by vibrations from the mechanical equipment in real time. The system uses this shift data to dynamically correct the spatial coordinates of each data frame, thereby generating high-precision synchronous multi-source sensing data, providing a solid foundation for subsequent processing.

[0085] During a vehicle entry operation at 10:30 AM on a certain morning, this invention constructed a real-time dynamic environment model containing target geometric and semantic information. The system first used LiDAR point cloud data to generate an obstacle grid layer, accurately depicting the physical outlines of the walls, pillars, and vehicles waiting to enter the garage. Simultaneously, using visual image data, a deep learning model identified the vehicle as a "car" and the operator as a "person," generating a semantic label layer. Through pixel fusion, the system associated geometric positions with semantic attributes, constructing an initial environment model. Subsequently, the system obtained the vehicle dispatch plan from the central dispatch system, "transport the vehicle to parking space 37 on level B2," and calculated the movement trajectories of the elevator and the traverse trolley over the next 30 seconds, forming a mechanical equipment movement trend layer. This layer, fused with the initial environment model, generated a complete dynamic environment model.

[0086] During the aforementioned storage process, a driver, having forgotten some items, attempted to return to retrieve them after the vehicle was loaded onto the trolley, mistakenly entering the mechanical operating area. In the dynamic environment model, the system identified the driver as a "dynamic target (person)," the moving trolley as a "dynamic target (equipment)," and the surrounding pillars as "static targets." The system then calculated a vectorized risk potential field for each target. According to preset rules, the risk type weight for the target classification attribute "person" was set to 1.0, significantly higher than the 0.2 for the static pillars. For the moving driver and trolley, the risk potential field shape was constructed as an ellipsoid stretched along their velocity direction, representing their forward collision risk.

[0087] The system generates a global composite risk field map by vector superimposing the risk potential fields of all targets. In this map, the field strength value rapidly rises to 95 in the area where the car owner and the trolley are about to intersect. The threshold setting rule is that less than 30 is a safe zone, 30-70 is a warning zone, and greater than 70 is a danger zone. Therefore, this area is identified as a "danger zone". Based on this judgment, the system maps the risk level to a preset multi-level response strategy set, generating an initial control strategy of "emergency braking". This strategy is converted into a collaborative protection command and sent to the garage control system. The traversing trolley performs emergency braking within 0.4 seconds, and at the same time, the red audible and visual alarm on its path is activated, issuing a strong warning to the car owner. The car owner stops moving forward, and the trolley comes to a complete stop 2.5 meters away from the car owner, successfully avoiding a potential safety accident.

[0088] In a test one day, a visual sensor failure scenario was simulated. A visual sensor monitoring the elevator entrance output a black screen image because its lens was obstructed by dirt. The system periodically detected the abnormal data stream from this sensor and immediately generated a sensor failure signal. The fault tolerance module was activated, using point cloud data from the still-operating LiDAR to compensate. The system performed DBSCAN clustering analysis on the point cloud of the area, identifying a moving point cloud cluster with dimensions of approximately 1.8m x 0.6m x 0.5m. By comparing it with a preset object feature library, compensation target label data was generated, classifying it as "human". Despite the loss of visual confirmation, the system successfully maintained security monitoring of the area based on the compensated perception data, ensuring the continuity of the protection function.

[0089] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any indirect connection method can be applied to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0090] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A safety protection method for a three-dimensional parking garage based on lidar, characterized in that, The method includes: The method involves acquiring LiDAR point cloud data and visual image data within an automated parking garage, and performing spatiotemporal synchronization calibration on the LiDAR point cloud data and the visual image data to generate synchronized multi-source sensing data. This includes: acquiring LiDAR point cloud data and visual image data within the automated parking garage; receiving a unified hardware synchronization pulse signal to timestamp-align the LiDAR point cloud data and the visual image data to obtain timestamp-aligned data; collecting real-time device pose offset data caused by mechanical vibration of the sensors; and using the device pose offset data to correct the timestamp-aligned data, eliminating spatial coordinate errors, thereby generating synchronized multi-source sensing data. Based on the synchronous multi-source sensing data, a real-time dynamic environment model containing target geometric and semantic information is constructed. This includes: extracting three-dimensional geometric features from the lidar point cloud data in the synchronous multi-source sensing data to establish an obstacle grid layer representing the distribution of physical obstacles; identifying target semantic features from the visual image data in the synchronous multi-source sensing data to form a semantic label layer that labels target categories; performing pixel-by-pixel fusion of the obstacle grid layer and the semantic label layer to assign semantic attributes at geometric locations, thereby constructing an initial environment model; obtaining a vehicle scheduling plan and calculating the mechanical equipment movement trajectory within a preset time window based on the vehicle scheduling plan to generate a mechanical equipment movement trend layer; and performing multi-layer data fusion of the initial environment model and the mechanical equipment movement trend layer to obtain a dynamic environment model. In the real-time dynamic environment model, dynamic and static targets are identified, and the vectorized risk potential field of each target is calculated to generate a global composite risk field map that characterizes the comprehensive risk distribution.

2. The method for safety protection of a three-dimensional parking garage based on lidar according to claim 1, characterized in that, The generated global composite risk field map representing the comprehensive risk distribution includes: In the dynamic environment model, all dynamic and static targets are identified, and a set of risk sources is constructed. Obtain the motion state and target type of each target in the risk source set, and calculate the vectorized risk potential field of each target; The risk potential fields of each target are vector-superimposed to generate a global composite risk field map.

3. The method for safety protection of a three-dimensional parking garage based on lidar according to claim 2, characterized in that, Obtain the motion state and target type of each target in the risk source set, and calculate the vectorized risk potential field of each target, including: The target state parameters are formed by obtaining the real-time movement speed and target classification attributes of each target in the risk source set; Risk weight coefficients are assigned to each objective based on its target classification attributes to obtain the risk type weight for each objective. By combining the target state parameters of each target with the risk type weights of each target, the risk potential field of each target is calculated.

4. The method for safety protection of a three-dimensional parking garage based on lidar according to claim 3, characterized in that, The method further includes: Based on the field strength values ​​of each region in the composite risk field map, the comprehensive safety risk level is divided and output. The comprehensive safety risk level is mapped to a preset multi-level response strategy set to generate a preliminary control strategy that includes the adjustment of mechanical equipment operating parameters and the triggering of alarm signals; Based on the preliminary control strategy, a collaborative protection command is generated, and based on the collaborative protection command, the operating speed of the mechanical equipment is adjusted in real time or braking is performed, activating the corresponding audible and visual alarm device.

5. A safety protection method for a three-dimensional parking garage based on lidar according to claim 1, characterized in that, The method further includes: Periodically check the integrity and quality of sensor data streams, identify and generate sensor fault signals; When a fault signal is received from the sensor, the data missing from the faulty sensor is filled with data from the sensor that is still working normally, and compensating sensing data is generated. The compensated sensing data is used to correct the synchronous multi-source sensing data.

6. A safety protection method for a three-dimensional parking garage based on lidar according to claim 5, characterized in that, When a sensor fault signal is received, the step of filling the data gaps of the faulty sensor with data from sensors that are still functioning normally to generate compensating sensing data includes: The sensor fault signal was received; When the sensor fault signal indicates a lidar fault, depth estimation is performed using visual image data, and combined with pre-acquired historical point cloud data, to reconstruct and generate compensated 3D environment data. When the sensor fault signal indicates a visual sensor fault, clustering and texture analysis are performed on the lidar point cloud data to identify and classify targets and generate compensation target label data. The compensation perception data includes the compensation 3D environment data and the compensation target label data.

7. A lidar-based automated parking garage safety protection system, applied to the lidar-based automated parking garage safety protection method as described in any one of claims 1-6, characterized in that, The system includes: The positioning data acquisition module is used to acquire lidar point cloud data and visual image data in the three-dimensional parking garage, and to perform spatiotemporal synchronization calibration of the lidar point cloud data and the visual image data to generate synchronous multi-source perception data. The localization model construction module is used to construct a real-time dynamic environment model containing target geometric information and semantic information based on the synchronous multi-source sensing data. The positioning and identification module is used to identify dynamic and static targets in the real-time dynamic environment model, calculate the vectorized risk potential field of each target, and generate a global composite risk field map that characterizes the comprehensive risk distribution.

Images