Methods, systems, equipment and storage media for preventing collisions when dropping precast beams.

Abstract

This application relates to a method, system, device, and storage medium for preventing collisions when dropping precast beams using a precast beam lifting equipment. The method includes: correcting distortion in acquired image data using pre-calibrated camera intrinsic parameters; spatially registering point cloud data with image data using pre-calibrated camera intrinsic parameters, LiDAR-camera extrinsic parameters, and the corrected image; cropping the Region of Interest (ROI) of the point cloud data to obtain the cropped ROI region; extracting beam features from the ROI region, calculating beam displacement, and determining the collision risk level; and outputting the beam displacement and collision risk level. This application significantly improves the accuracy of beam boundary extraction. Furthermore, by cropping the point cloud region, background data is reduced, computational redundancy is decreased, and real-time alignment is improved. This embodiment does not require manual observation, greatly reducing safety risks and solving the technical problems of low accuracy and high safety risks in related technologies.

Description

Technical Field

[0001] This application relates to the field of bridge engineering construction technology, specifically to a method, system, equipment, and storage medium for preventing collisions when dropping precast beams using a lifting device. Background Technology

[0002] Currently, bridge construction is an important component of transportation infrastructure, and the handling and erection of precast beams is a key construction step. As bridge construction develops towards larger scale and standardization, the weight and size of precast beams are constantly increasing, placing higher demands on the precision, safety, and efficiency of the handling process.

[0003] In related technologies, current methods for preventing collisions during precast beam handling primarily rely on manual observation or single-sensor sensing. Manual observation involves construction workers visually observing the relative positions of beam segments and using walkie-talkies to direct the trolley driver, relying on experience to judge beam spacing and placement accuracy. Single-sensor sensing involves cameras capturing construction images, using 2D target detection algorithms to identify beam segment outlines, and combining preset camera parameters to calculate relative positions and generate alignment signals.

[0004] However, manual adjustments are inaccurate and inefficient, and are greatly affected by lighting and weather conditions. Collision avoidance judgments rely on subjective experience, posing high safety risks. Visual cameras are affected by perspective distortion and changes in lighting, resulting in insufficient 3D positioning accuracy and an inability to obtain true distances.

[0005] Therefore, it is necessary to design a new method for preventing collisions when dropping precast beams from lifting equipment in order to overcome the above problems. Summary of the Invention

[0006] This application provides a method, system, equipment, and storage medium for preventing collisions when dropping precast beams using a precast beam lifting device, which can solve the technical problems of low accuracy and high safety risks in related technologies.

[0007] In a first aspect, embodiments of this application provide a method for preventing collisions when lowering precast beams using a lifting device, the method comprising: Distortion correction is performed on the acquired image data using pre-calibrated camera intrinsic parameters; Spatial registration of point cloud data and image data is achieved by using pre-calibrated camera intrinsic parameters, LiDAR-camera extrinsic parameters, and calibrated images, and the ROI region of the point cloud data is cropped to obtain the cropped point cloud ROI region; The beam features are extracted from the ROI region of the point cloud, the beam displacement is calculated, and the collision risk level is determined. Output beam displacement and collision risk level.

[0008] In conjunction with the first aspect, in one implementation method, the camera intrinsic parameter calibration method includes: Acquire multiple images of the calibration board in different poses; The acquired calibration board image is input into the calibration algorithm, which solves the problem through corner detection, sub-pixel optimization, and perspective transformation to obtain the camera intrinsic parameter matrix K and distortion coefficient vector D.

[0009] In conjunction with the first aspect, in one embodiment, the calibration method for the lidar-camera extrinsic parameters includes: Acquire multiple sets of calibration board point cloud data and calibration board image data at different locations; The distortion of the calibration board image data is corrected using pre-calibrated camera intrinsic parameters, and the coordinates (u, v) of the corner points in the chessboard are detected. Extract the calibration board plane from the calibration board point cloud data, and calculate the three-dimensional coordinates of the interior corner points of the point cloud based on the checkerboard size and the number of interior corner points. , , ); Based on the coordinates (u, v) of the inner corner points of the image and the three-dimensional coordinates of the inner corner points of the point cloud ( , , Solve for the extrinsic parameters of the laser radar to the camera: rotation matrix R and translation vector T.

[0010] In conjunction with the first aspect, in one implementation, the distortion correction of the acquired image data using pre-calibrated camera intrinsic parameters includes: The acquired image data is distorted using pre-calibrated camera intrinsic parameters to obtain a distortion-free image; the camera intrinsic parameters include the camera intrinsic parameter matrix K and the distortion coefficient vector D. Synchronize point cloud data with distortion-free image data based on timestamps.

[0011] In conjunction with the first aspect, in one implementation, the step of spatially registering point cloud data and image data using pre-calibrated camera intrinsic parameters and LiDAR-camera extrinsic parameters, as well as the corrected image, and cropping the ROI region of the point cloud data to obtain the cropped point cloud ROI region includes: Load the pre-trained beam segment image recognition model; The corrected image is input into the beam segment image recognition model to detect beam segment targets and output 2D detection boxes. The 2D detection box is redundantly expanded, and the corner points of the 2D detection box are projected onto the LiDAR coordinate system using the camera intrinsic parameters and LiDAR-camera extrinsic parameters to obtain the corresponding 3D view frustum space. Based on the point cloud data synchronized after 3D view frustum clipping, background points are filtered out, and the point cloud ROI region containing beam segments and redundant areas is extracted.

[0012] In conjunction with the first aspect, in one implementation, before extracting beam features from the point cloud ROI region, calculating beam displacement, and determining the collision risk level, the method further includes: Extract the maximum and minimum Y-axis values ​​of beam segments within the ROI region of the point cloud; The maximum and minimum values ​​of the beam segment on the Y-axis are stored in the history queue. The extreme value difference in the queue is calculated. When the extreme value difference is greater than the set value, it is determined to be point cloud shaking, and the calculation is skipped.

[0013] In conjunction with the first aspect, in one implementation, the step of extracting beam features from the point cloud ROI region, calculating beam displacement, and determining the collision risk level includes: Extract the 3D boundary and geometric contour features of the beam from the point cloud ROI region, and remove false features to obtain an accurate target feature set; Identify adjacent beam distribution scenarios based on target feature sets; Calculate beam displacement by combining beam boundary points for different adjacent beam distribution scenarios; The boundary difference between the beam and the adjacent beam is calculated based on the 3D boundary and geometric contour features of the beam, and the collision risk level is determined based on the boundary difference.

[0014] Secondly, embodiments of this application provide a precast beam lifting equipment anti-collision system for beam dropping, the precast beam lifting equipment anti-collision system comprising: The data receiving and preprocessing module is used to perform distortion correction on the acquired image data using pre-calibrated camera intrinsic parameters; The registration and cropping module is used to achieve spatial registration of point cloud data and image data using pre-calibrated camera intrinsic parameters and LiDAR-camera extrinsic parameters and corrected images, and to crop the ROI region of the point cloud data to obtain the cropped point cloud ROI region. The calculation module is used to extract beam features from the point cloud ROI region, calculate beam displacement, and determine the collision risk level. The results publishing module is used to output the beam displacement and collision risk level.

[0015] Thirdly, this application provides a precast beam lifting equipment anti-collision device for beam dropping, the precast beam lifting equipment anti-collision device includes a processor, a memory, and a precast beam lifting equipment anti-collision program stored in the memory and executable by the processor, wherein when the precast beam lifting equipment anti-collision program is executed by the processor, the steps of the above-mentioned precast beam lifting equipment anti-collision method for beam dropping are implemented.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a precast beam lifting equipment anti-collision program, wherein when the precast beam lifting equipment anti-collision program is executed by a processor, it implements the steps of the above-described precast beam lifting equipment anti-collision method.

[0017] The beneficial effects of the technical solutions provided in this application include: Image data distortion can be corrected by pre-calibrating camera intrinsic parameters. Using the corrected image data, camera intrinsic parameters, and LiDAR-camera extrinsic parameters, spatial registration of point cloud data and image data can be achieved, laying the foundation for point cloud cropping. By extracting beam features from the cropped point cloud ROI region, beam displacement can be calculated and collision risk level can be determined. The accuracy of beam boundary extraction can be greatly improved. Furthermore, by cropping the point cloud region, background data can be reduced, computational redundancy can be reduced, and real-time alignment can be improved. Moreover, this embodiment does not require manual observation, which greatly reduces safety risks and solves the technical problems of low accuracy and high safety risks in related technologies. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of an embodiment of the method for preventing collisions when lowering precast beams using a lifting device according to this application. Figure 2 This is a schematic diagram of the hardware structure of the precast beam lifting equipment and beam drop anti-collision device involved in the embodiments of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0021] In a first aspect, embodiments of this application provide a method for preventing collisions when lowering precast beams using a precast beam lifting device.

[0022] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the precast beam lifting equipment collision prevention method for lowering beams according to this application. In this embodiment, the lifting equipment can be, for example, a transport machine or a trolley. Figure 1 As shown, the methods for preventing collisions when lowering precast beams using a lifting equipment include: S100: Performs distortion correction on the acquired image data using pre-calibrated camera intrinsic parameters.

[0023] S200: Spatial registration of point cloud data and image data is achieved by using pre-calibrated camera intrinsic parameters, LiDAR-camera extrinsic parameters and corrected images, and the ROI region of the point cloud data is cropped to obtain the cropped point cloud ROI region.

[0024] S300: Extract beam features from the point cloud ROI region, calculate beam displacement, and determine the collision risk level.

[0025] S400: Outputs beam displacement and collision risk level.

[0026] In this embodiment, parameter calibration is performed before step S100, including camera intrinsic parameter calibration and LiDAR-camera extrinsic parameter calibration. The X-axis, Y-axis, and Z-axis are defined. The X-axis, Y-axis, and Z-axis used in this application are consistent with the x, y, and z axes inherent in the LiDAR.

[0027] In one embodiment, the camera intrinsic parameter calibration method includes: acquiring multiple calibration board images in different poses; then inputting the acquired calibration board images into a calibration algorithm, and solving them through corner detection, sub-pixel optimization, and perspective transformation to obtain the camera intrinsic parameter matrix K and distortion coefficient vector D.

[0028] The calibration steps for camera intrinsic parameters are as follows (based on Zhang Zhengyou's calibration method): Step 1.1 Prepare the calibration plate: Select a checkerboard calibration plate and ensure that the surface of the calibration plate is flat and non-reflective.

[0029] Step 1.2 Equipment Setup: Fix the calibration plate on the beam simulation support in the construction area, ensuring that the calibration plate can be completely photographed by the camera, and that the angle between the plane of the calibration plate and the optical axis of the camera is between 30° and 60°.

[0030] Step 1.3 Data Acquisition: Control the camera to capture at least 15 images of the calibration board in different poses (including translation and rotation at different angles) to ensure that the calibration board is clear, unblurred, and unobstructed in the images.

[0031] Step 1.4 Intrinsic parameter calculation: Input the acquired image into the calibration algorithm, and solve it through corner detection, sub-pixel optimization, and perspective transformation to obtain the camera intrinsic parameter matrix K and distortion coefficient vector D.

[0032] Step 1.5 Accuracy Verification: Calculate the reprojection error of each image. If the average reprojection error is ≤0.5 pixels, the calibration is qualified; otherwise, re-acquire images and calibrate.

[0033] Further, in one embodiment, the calibration method for the lidar-camera extrinsic parameters includes: first acquiring multiple sets of calibration board point cloud data and calibration board image data at different locations; then correcting distortion in the calibration board image data using pre-calibrated camera intrinsic parameters and detecting the coordinates (u, v) of the checkerboard corner points; then extracting the calibration board plane from the calibration board point cloud data and calculating the three-dimensional coordinates of the point cloud corner points based on the checkerboard size and the number of corner points. , , ); and then based on the coordinates of the inner corner points (u, v) of the image and the three-dimensional coordinates of the inner corner points of the point cloud ( , , Solve for the extrinsic parameters of the laser radar to the camera: rotation matrix R and translation vector T.

[0034] Calibration steps for LiDAR-camera extrinsic parameters (based on calibration board method): Step 2.1 Equipment Fixing: Fix the LiDAR and vision camera on the same rigid bracket, ensuring that their relative positions remain unchanged. The bracket is installed at the preset position of the trolley to ensure that the field of view covers the construction area.

[0035] Step 2.2 Calibration plate arrangement: Place the calibration plate within the common field of view of the lidar and the camera, with the plane of the calibration plate perpendicular to the optical axis of both, and at a distance of 2~5m from the equipment.

[0036] Step 2.3 Synchronous Acquisition: Start the synchronous acquisition function of LiDAR and camera, ensure that the data timestamps are consistent (error ≤10ms), and acquire 3 sets of calibration board point cloud and image data at different locations.

[0037] Step 2.4 Feature Extraction: Image processing: The acquired calibration board image is used to correct distortion using the intrinsic parameters (K and D) obtained in step 1.4, and the coordinates (u, v) of the corner points in the chessboard are detected.

[0038] Point cloud: The calibration board plane is extracted from the collected point cloud using a plane segmentation algorithm. Then, based on the chessboard grid size and the number of interior corner points, the three-dimensional coordinates of the interior corner points in the point cloud are calculated. , , ).

[0039] Step 2.5 Extrinsic parameter calculation: Based on the coordinates of the interior corner points (u, v) at the image end and the coordinates of the interior corner points (u, v) at the point cloud end. , , The extrinsic parameters (rotation matrix R, translation vector T) of the lidar to the camera are solved using the least squares method, satisfying the formula: .

[0040] This embodiment combines Zhang Zhengyou's calibration method for camera intrinsic parameters with the calibration board method for LiDAR-camera extrinsic parameters, including parameter calculation, accuracy verification, and storage mechanisms to ensure calibration accuracy and robustness.

[0041] This embodiment can correct image data distortion through pre-calibrated camera intrinsic parameters. Using the corrected image data, along with the camera intrinsic and LiDAR-camera extrinsic parameters, spatial registration of point cloud data and image data can be achieved, laying the foundation for point cloud cropping. By extracting beam features from the cropped point cloud ROI region, beam displacement can be calculated and collision risk levels determined. The accuracy of beam boundary extraction can be greatly improved. Furthermore, by cropping the point cloud region, background data can be reduced, computational redundancy can be decreased, and real-time alignment can be improved. This embodiment does not require manual observation, significantly reducing safety risks and solving the technical problems of low accuracy and high safety risks in related technologies. It is suitable for precast beam handling construction scenarios that rely on the collaborative work of LiDAR and vision cameras, especially for automated construction needs requiring high-precision alignment and collision detection.

[0042] Furthermore, in some embodiments, before step S100, the system includes steps of system initialization, calibration parameter loading and verification, and task instruction parsing and distribution.

[0043] The system initialization process includes: starting the ROS core node and the corresponding node (liangpian_mission_node) in this embodiment, initializing the status variables of each module (task_running=false, is_detection_done=false, etc.), subscribing to topics such as LiDAR point cloud, visual images, and task instructions, and creating a result publishing topic and a 10Hz timer.

[0044] Calibration parameter loading and verification: Load camera intrinsic parameters (K, D), LiDAR-camera extrinsic parameters (R, T), and calibration accuracy data from the ROS parameter server; calculate the reprojection error of the current extrinsic parameters. If the error is ≤1.0 pixel, the calibration is considered valid; if the error exceeds the standard, issue a calibration failure alarm and prompt manual recalibration.

[0045] Task instruction parsing and distribution: Receive task instructions sent by the backend, parse parameters such as task ID, detection type (2=carriage alignment, 3=one-click beam drop), and travel direction; store task data after deduplication by ID; if it is a new task type, stop the current task, reset variables such as point cloud cache and historical data queue, and enable the corresponding data processing flow.

[0046] This embodiment adds a calibration verification and storage mechanism, allowing calibration results to be reused and enabling rapid recalibration after slight equipment displacement, thus improving system robustness.

[0047] Furthermore, in one embodiment, the distortion correction of the acquired image data using pre-calibrated camera intrinsic parameters includes: S101: The acquired image data is distorted using pre-calibrated camera intrinsic parameters to obtain a distortion-free image; wherein, the camera intrinsic parameters include the camera intrinsic parameter matrix K and the distortion coefficient vector D. In this embodiment, the distortion-free image is also the distorted image.

[0048] S102: Synchronize point cloud data with distortion-free image data based on timestamps.

[0049] In this embodiment, the LiDAR acquires raw point cloud data, converts it to PCL format, filters out invalid points, and accumulates them in a cache queue (≥100 points); the visual camera acquires image data, uses intrinsic parameters (K, D) to correct distortion, and obtains a distortion-free image; then, the point cloud and the distortion-free image are synchronized based on timestamps to ensure data temporal consistency. This embodiment uses precise intrinsic parameters to correct image distortion, eliminates the influence of lens distortion, and prepares for registration.

[0050] Further, in one embodiment, the step of spatially registering point cloud data and image data using pre-calibrated camera intrinsic parameters and LiDAR-camera extrinsic parameters, as well as the corrected image, and cropping the ROI region of the point cloud data to obtain the cropped point cloud ROI region includes: S201: Load the pre-trained beam segment image recognition model.

[0051] S202: Input the corrected image into the beam segment image recognition model, detect the beam segment target, and output a 2D detection box.

[0052] S203: Redundantly expand the 2D detection box, and simultaneously project the corner points of the 2D detection box onto the LiDAR coordinate system using camera intrinsic parameters and LiDAR-camera extrinsic parameters to obtain the corresponding 3D view frustum space.

[0053] S204: Based on the point cloud data after 3D view frustum space clipping synchronization, filter background points and extract the point cloud ROI region containing beam segments and redundant areas.

[0054] This embodiment first loads a pre-trained YOLO model for beam segment image recognition; then, it inputs a distortion-free image into the model to detect beam segments and output initial 2D detection boxes; next, it redundantly expands the 2D detection boxes, and simultaneously projects the four corner points of the 2D detection boxes onto the LiDAR coordinate system using extrinsic and camera intrinsic parameters to obtain the corresponding 3D view frustum; finally, based on the point cloud data synchronized after 3D view frustum clipping, background points are filtered to extract point cloud ROI regions containing only beam segments and redundant areas. This embodiment's dynamic ROI clipping can reduce background data by 60%~80%, decrease computational redundancy, improve system real-time performance by 30%, and meet the 10Hz real-time alignment requirement.

[0055] Furthermore, in some embodiments, before extracting beam features from the point cloud ROI region, calculating beam displacement, and determining the collision risk level, the process may further include: Step a: Extract the maximum and minimum values ​​of the beam segments along the Y-axis in the ROI region of the point cloud.

[0056] Step b: Store the maximum and minimum values ​​of the beam segment's Y-axis into the historical queue, calculate the extreme value difference within the queue, and if the extreme value difference is greater than the set value, it is determined to be point cloud swaying, and the calculation is skipped.

[0057] In this embodiment, after the cropping in step S200 and before the calculation in step S300, point cloud stability detection is performed first. That is, the maximum / minimum Y-axis values ​​(curr_fu_y_max / curr_fu_y_min) of the beam segments in the ROI point cloud (i.e., the point cloud ROI region) are extracted first; then, the maximum / minimum Y-axis values ​​of the beam segments are stored in a historical queue (maximum 30 frames are retained), and the extreme value difference within the queue is calculated; if the extreme value difference is >3 meters, it is determined to be point cloud swaying, and the calculation is skipped. This embodiment uses a point cloud swaying detection mechanism to filter equipment vibration and environmental interference, with a displacement calculation error ≤2cm, far superior to manual alignment and single sensor solutions; the multi-scenario displacement calculation logic adapts to complex construction conditions, with an alignment success rate ≥98%. This embodiment's point cloud swaying detection mechanism based on 30 frames of historical data can judge data stability through boundary feature extreme value differences, filter interference, and improve alignment accuracy.

[0058] Furthermore, in one embodiment, the step of extracting beam features from the point cloud ROI region, calculating beam displacement, and determining the collision risk level may include: S301: Extract the 3D boundary and geometric contour features of the beam from the point cloud ROI region, and remove false features to obtain an accurate target feature set; S302: Identify adjacent beam distribution scenarios based on target feature sets; S303: Calculate beam displacement by combining beam boundary points for different adjacent beam distribution scenarios; S304: Calculate the boundary difference between the beam segment and adjacent beams based on the 3D boundary and geometric contour features of the beam segment, and determine the collision risk level based on the boundary difference.

[0059] In this embodiment, the 3D boundary and geometric contour features of the beam segments are extracted from the cropped point cloud, and false features are removed to obtain an accurate target feature set. The distribution scenarios of adjacent beams are identified based on the target feature set. These scenarios include no adjacent beams on both sides, an adjacent beam only on the left side, an adjacent beam only on the right side, and adjacent beams on both sides. Then, for different adjacent beam scenarios, based on a safety distance threshold of 30-50cm, and combined with the beam segment boundary points (which should be understood as being obtained based on the 3D boundary and geometric contour features of the beam segment), the beam segment displacement (delta_Y) is calculated (where the beam segment is aligned by the trolley movement). The boundary difference between the beam segment and adjacent beams (diff_l / diff_r) is calculated as follows: diff_l = left boundary of beam segment – ​​right boundary of left adjacent beam; diff_r = left boundary of right adjacent beam – right boundary of beam segment. Then, the collision risk level is determined. If it is a trolley alignment task, it is determined whether the displacement meets the safety distance requirements. If it does, the task is marked as complete (is_detection_done=true). This embodiment achieves precise registration of point cloud and image based on accurate calibration parameters. Leveraging the geometric positioning advantages of LiDAR, it improves beam boundary extraction accuracy by over 40%. The multi-scenario displacement calculation logic in this embodiment can adapt to four adjacent beam distribution scenarios, accurately calculating displacement based on safe distance thresholds, meeting the needs of complex construction conditions.

[0060] Preferably, in step S400 above, data such as beam displacement, collision risk level, and detection status are first encapsulated and published at a frequency of 10Hz. If the trolley alignment task is completed, the results are published 10 times and then the task stops. If it is a one-click beam lowering task, the detection results are continuously updated and published. After receiving the results, the backend control system controls the trolley to move and align, and then sends a task completion signal. This embodiment's stable result publication and anomaly alarm mechanism can publish at a high frequency of 10Hz and push repeatedly after the task is completed, ensuring reliable data reception. It also provides timely alarms when calibration fails, improving system security. Furthermore, the entire process from calibration to alignment is automated, requiring no manual intervention, increasing construction efficiency by more than 50% and shortening the construction cycle. The graded collision detection and anomaly alarm mechanism reduces construction safety risks and avoids collision accidents.

[0061] This embodiment addresses the core issues of "high reliance on manual labor, insufficient accuracy of single sensors, and poor adaptability to working conditions" in related technologies by standardizing camera intrinsic parameter calibration and LiDAR-camera extrinsic parameter calibration processes, combined with dynamic ROI configuration, point cloud sway detection, and multi-scene displacement calculation technology. This enables automated, high-precision, and high-stability beam handling, as well as collision avoidance detection.

[0062] Secondly, embodiments of this application also provide a precast beam lifting equipment anti-collision system for beam dropping.

[0063] In one embodiment, the precast beam lifting equipment's anti-collision system for dropping beams includes: a data receiving and preprocessing module, used to correct distortion in acquired image data using pre-calibrated camera intrinsic parameters; a registration and cropping module, used to spatially register point cloud data and image data using pre-calibrated camera intrinsic parameters, LiDAR-camera extrinsic parameters, and the corrected image, and to crop the ROI region of the point cloud data to obtain the cropped point cloud ROI region; a calculation module, used to extract beam features from the point cloud ROI region, calculate the beam displacement, and determine the collision risk level; and a result publishing module, used to output the beam displacement and collision risk level. The modules interact via ROS topics. This embodiment's modular design makes each functional unit independent, facilitating debugging and upgrades; calibration parameters can be modified through configuration files to adapt to different beam types and construction scenarios, demonstrating strong versatility.

[0064] Furthermore, in one embodiment, the precast beam lifting equipment's anti-collision system for lowering beams also includes a calibration module. This module is responsible for the accurate calibration and verification of camera intrinsic parameters and LiDAR-camera extrinsic parameters, providing fundamental parameter support for subsequent registration. The calibration module includes a camera intrinsic parameter calibration unit and a LiDAR-camera extrinsic parameter calibration unit. The camera intrinsic parameter calibration unit uses the Zhang Zhengyou calibration method, acquiring images of the calibration board at different poses to calculate the camera's focal length (fx, fy), principal point coordinates (cx, cy), and distortion coefficients (k1, k2, p1, p2, k3), generating the camera intrinsic parameter matrix K and distortion coefficient vector D. The LiDAR-camera extrinsic parameter calibration unit uses the calibration board method, simultaneously acquiring point cloud data and image data from the calibration board, calculating the rotation matrix R and translation vector T from the LiDAR coordinate system to the camera coordinate system, achieving spatial registration between the two. This embodiment adopts the Zhang Zhengyou calibration method and the calibration plate method, with the camera intrinsic parameter reprojection error ≤0.5 pixels and the lidar-camera extrinsic parameter reprojection error ≤1.0 pixel, ensuring accurate cross-modal data registration; a new calibration verification and storage mechanism is added, the calibration results can be reused, and the equipment can be quickly recalibrated after slight displacement, improving the system robustness.

[0065] Furthermore, in one embodiment, the aforementioned data receiving and preprocessing module is responsible for the synchronous reception, format conversion, and optimization of multi-sensor data. This data receiving and preprocessing module includes a LiDAR data receiving unit, a visual data receiving unit, a task instruction receiving unit, a data synchronization unit, a point cloud preprocessing unit, and an image preprocessing unit. The LiDAR data receiving unit subscribes to the raw point cloud data published by the LiDAR, which includes x / y / z three-dimensional coordinate information. Visual data receiving unit: Subscribes to image topics published by the visual camera and receives RGB image data; Task instruction receiving unit: Subscribes to task instruction topics published by the backend and parses the task type (determines whether to execute a detection task, the direction of vehicle movement, etc.); Data synchronization unit: Aligns the LiDAR point cloud and visual image based on timestamps to ensure time consistency of cross-modal data; Point cloud preprocessing unit: Converts radar data of different formats into PCL (Point Cloud Library) standard format, filters invalid points (non-finite value points, points that are too far / too close), and accumulates point cloud frames to the minimum processing threshold; Image preprocessing unit: Uses the intrinsic parameter matrix K and distortion coefficient vector D obtained by the calibration module to perform distortion correction on the image, eliminate the influence of lens distortion, and prepare for registration.

[0066] Preferably, the above-mentioned registration and cropping module includes: a loading unit for loading a pre-trained beam segment image recognition model; a point cloud-image registration unit for inputting the corrected image into the beam segment image recognition model, detecting beam segment targets and outputting 2D detection boxes; and redundancy expansion of the 2D detection boxes, while using camera intrinsic parameters and LiDAR-camera extrinsic parameters to project the corner points of the 2D detection boxes to the LiDAR coordinate system to obtain the corresponding 3D view frustum space; and a point cloud ROI cropping unit for cropping synchronized point cloud data based on the 3D view frustum space, filtering background points, and extracting point cloud ROI regions containing beam segments and redundant areas.

[0067] Furthermore, in one embodiment, the precast beam lifting equipment's anti-collision system for lowering beams also includes a point cloud stability detection module. This module filters out interference data and improves alignment stability. Specifically, the point cloud stability detection module includes a historical data caching unit and a sway judgment unit. The historical data caching unit maintains a 30-frame historical queue of beam segment boundary features, storing the maximum / minimum Y-axis values ​​of the beam segment extracted from each frame of the point cloud. The sway judgment unit calculates the extreme value difference of the boundary features in the historical queue. If the difference exceeds a sway threshold (e.g., 3 meters), it is determined to be point cloud sway, and the calculation is skipped. If the difference is ≤ the threshold or the data is less than 30 frames, it is determined to be stable or the data is insufficient, and subsequent processing begins.

[0068] Furthermore, in one embodiment, the calculation module mainly implements feature extraction, displacement calculation, and collision avoidance detection. The calculation module includes a feature extraction unit, a multi-scene displacement calculation unit, and a hierarchical collision avoidance detection unit. The feature extraction unit can perform point cloud feature extraction. Point cloud feature extraction: Extract the 3D boundary and geometric contour features of the beam segment from the point cloud ROI region, and remove false features, retaining accurate target features. The multi-scene displacement calculation unit can perform adjacent beam scene recognition and displacement calculation. Adjacent beam scene recognition: Based on the target feature set, determine the distribution of adjacent beams, and classify them into four scenarios: "no adjacent beams on both sides, adjacent beams only on the left side, adjacent beams only on the right side, and adjacent beams on both sides"; Displacement calculation: Based on a safety distance threshold of 30~50cm, combined with the beam segment boundary points, calculate the beam segment displacement (delta_Y, negative on the left and positive on the right, unit: m). The graded collision avoidance detection unit is used to calculate the boundary difference between the beam segment and the adjacent beam (diff_l = left boundary of the beam segment – ​​right boundary of the adjacent beam on the left; diff_r = left boundary of the adjacent beam on the right – right boundary of the beam segment). The risk level is determined based on the difference: if diff_l > -0.3m or diff_r > -0.3m, it is a parking area (risk level 3); otherwise, it is an area without obstacles (risk level 0).

[0069] Furthermore, in one embodiment, the aforementioned result publishing module is used to ensure stable and reliable transmission of alignment results. The result publishing module includes a data encapsulation unit, a stable publishing unit, and an anomaly alarm unit. The data encapsulation unit encapsulates data such as beam displacement, collision risk level, detection status (0=default, 1=detecting in progress, 2=detection completed, 3=detection failed), and calibration parameter validity markers into standardized comma-separated strings. The stable publishing unit continuously publishes the detection results to the topic ( / detection_result) at a frequency of 10Hz; if the trolley alignment task is completed, the results are repeated 10 times (with an interval of 0.1 seconds) to ensure reliable reception by the backend control system. The anomaly alarm unit issues an alarm signal if calibration parameters fail (e.g., reprojection error exceeds the limit) or sensor data is interrupted, prompting manual intervention.

[0070] The functions of each module in the above-mentioned precast beam lifting equipment anti-collision system correspond to the steps in the above-mentioned precast beam lifting equipment anti-collision method embodiment, and their functions and implementation processes will not be described in detail here.

[0071] The hardware environment of this application includes: 1. LiDAR: Livox MID-360 LiDAR was selected.

[0072] 2. Vision camera: Senyun industrial camera was selected.

[0073] 3. Calibration board: checkerboard calibration board (5cm×5cm, 8×6 inner corner points), made of matte white material to avoid glare.

[0074] 4. Computing equipment: Industrial computer (CPU: Intel i7-12700H, GPU: NVIDIA RTX 3060, 16GB memory), pre-installed with Ubuntu 20.04 LTS operating system and ROS Noetic.

[0075] 5. Rigid bracket: Custom aluminum alloy bracket is used to fix the LiDAR and camera, ensuring that their relative positions remain unchanged, and the bracket vibration frequency is ≤5Hz.

[0076] The software configuration of this application includes: 1. Basic dependency libraries: ROS core library: used for topic communication and parameter server management; PCL 1.12: Used for point cloud processing (cropping, filtering, feature extraction); OpenCV 4.5: Used for image distortion correction, corner detection, and calibration algorithm implementation; Eigen 3.4: Used for matrix operations (calibration parameter calculation, point cloud projection).

[0077] 2. Core Algorithm Library: Camera intrinsic parameter calibration: Zhang Zhengyou's calibration method is implemented based on the cv::calibrateCamera function of OpenCV; 2D Object Detection: The YOLOv8 algorithm is integrated to extract 2D features of beam segments in the image.

[0078] 3. Parameter configuration: Calibration parameters: Default value of intrinsic parameter matrix K (based on camera model), initial value of extrinsic parameter R / T (based on installation location measurement); Other parameters: minimum point cloud processing threshold of 100 points, shaking threshold of 3 meters, result publishing frequency of 10Hz, and repeated publishing 10 times after the task is completed.

[0079] The specific implementation steps of this application are as follows: Step 1: Hardware Installation and Debugging 1. Fix the lidar and camera on a rigid bracket, and adjust the installation angle to ensure that the field of view of both covers the construction area (beams and adjacent beams). 2. Install the bracket on the front end of the top of the vehicle and secure it firmly to prevent it from loosening during construction; 3. Connect the lidar, camera, and industrial computer to test the communication status of the equipment and ensure stable data transmission.

[0080] Step 2: Calibration and Execution 1. Start the ROS core node (roscore) and run the calibration program (based on OpenCV and PCL). 2. Perform camera intrinsic parameter calibration as per step 1.1, acquire 15 calibration board images, calculate intrinsic parameter K and distortion coefficient D, and store the images after verifying that the reprojection error is ≤0.5 pixels; 3. Perform extrinsic parameter calibration as per step 1.2, collect synchronous point cloud and image data from 3 sets of calibration boards, calculate extrinsic parameters R and T, and store the data after verifying that the reprojection error is ≤1.0 pixel; 4. Restart the alignment node, load the calibration parameters, and confirm that the calibration is valid (the node log will output "Calibrationvalid").

[0081] Step 3: Construction alignment and execution 1. Start the lidar, camera, and alignment node. After the node initialization is complete, output "Beam segment detection task node initialization complete"; 2. The backend system sends the vehicle alignment task instruction (checkType=2), and after node parsing, it outputs "Start Task 2 (Vehicle Alignment)" to enable the point cloud processing flow; 3. The LiDAR and camera acquire data synchronously, and the nodes complete data preprocessing, dynamic ROI cropping, and point cloud stability detection; 4. Extract node features, identify adjacent beam scenarios (such as adjacent beams on both sides), and calculate the displacement of the trolley (i.e., the displacement of the beam segment) (e.g., delta_Y=0.2m). 5. The node publishes the detection results at 10Hz, and the back-end control system receives them and controls the trolley to move 0.2m. 6. After the displacement adjustment is completed, the node detects that the beam spacing meets the safety threshold of 30~50cm, marks the task as completed, repeats the result publication 10 times, and then stops the task; 7. If switched to the one-click beam drop task (checkType=3), the node loads the corresponding ROI parameters and continuously performs collision avoidance detection. The risk level is output normally for the first 30 seconds, and the detection results are hidden after 30 seconds (test scenario adaptation).

[0082] Step 4: Calibration and Maintenance 1. Before each day's construction, run the calibration verification program to check the reprojection error of the calibration parameters. If the error is ≤1.0 pixel, continue to use it; otherwise, recalibrate. 2. If the equipment is involved in a collision or the support becomes loose, the calibration process must be repeated to ensure the accuracy of subsequent registration and calculation.

[0083] Thirdly, this application provides a precast beam lifting equipment anti-collision device for dropping beams. The precast beam lifting equipment anti-collision device can be a personal computer (PC), laptop computer, server or other device with data processing function.

[0084] Reference Figure 2 , Figure 2 This is a schematic diagram of the hardware structure of the anti-collision device for the precast beam lifting equipment involved in the embodiments of this application. In this embodiment, the anti-collision device for the precast beam lifting equipment may include a processor, a memory, a communication interface, and a communication bus.

[0085] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0086] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces are used to interconnect components within the precast beam lifting and anti-collision device, as well as to interconnect the precast beam lifting and anti-collision device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0087] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0088] The processor can be a general-purpose processor, which can call the anti-collision program for the precast beam lifting equipment stored in the memory and execute the anti-collision method for the precast beam lifting equipment provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the anti-collision program for the precast beam lifting equipment is called can be referred to in the various embodiments of the anti-collision method for the precast beam lifting equipment provided in this application, and will not be repeated here.

[0089] Those skilled in the art will understand that Figure 2The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0090] Fourthly, embodiments of this application also provide a readable storage medium.

[0091] The present application has a readable storage medium storing a precast beam lifting equipment anti-collision program, wherein when the precast beam lifting equipment anti-collision program is executed by a processor, it implements the steps of the precast beam lifting equipment anti-collision method as described above.

[0092] The method implemented when the precast beam lifting equipment anti-collision procedure is executed can be referred to in the various embodiments of the precast beam lifting equipment anti-collision method of this application, and will not be repeated here.

[0093] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0094] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0095] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0096] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0097] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0099] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for preventing collisions when lowering precast beams using a lifting device, characterized in that, The method for preventing collisions when dropping precast beams using the lifting equipment includes: Distortion correction is performed on the acquired image data using pre-calibrated camera intrinsic parameters; Spatial registration of point cloud data and image data is achieved by using pre-calibrated camera intrinsic parameters, LiDAR-camera extrinsic parameters, and calibrated images, and the ROI region of the point cloud data is cropped to obtain the cropped point cloud ROI region; The beam features are extracted from the ROI region of the point cloud, the beam displacement is calculated, and the collision risk level is determined. Output beam displacement and collision risk level.

2. The method for preventing collisions when lowering precast beams using a lifting device as described in claim 1, characterized in that, Methods for calibrating camera intrinsic parameters include: Acquire multiple images of the calibration board in different poses; The acquired calibration board image is input into the calibration algorithm, which solves the problem through corner detection, sub-pixel optimization, and perspective transformation to obtain the camera intrinsic parameter matrix K and distortion coefficient vector D.

3. The method for preventing collisions when lowering precast beams using a precast beam lifting device as described in claim 1, characterized in that, The calibration method for the extrinsic parameters of the lidar-camera system includes: Acquire multiple sets of calibration board point cloud data and calibration board image data at different locations; The distortion of the calibration board image data is corrected using pre-calibrated camera intrinsic parameters, and the coordinates (u,v) of the corner points in the chessboard are detected. Extract the calibration board plane from the calibration board point cloud data, and calculate the three-dimensional coordinates of the interior corner points of the point cloud based on the checkerboard size and the number of interior corner points. , , ); Based on the coordinates (u, v) of the inner corner points of the image and the three-dimensional coordinates of the inner corner points of the point cloud ( , , Solve for the extrinsic parameters of the laser radar to the camera: rotation matrix R and translation vector T.

4. The method for preventing collisions when lowering precast beams using a precast beam lifting device as described in claim 1, characterized in that, The distortion correction of the acquired image data using pre-calibrated camera intrinsic parameters includes: The acquired image data is distorted using pre-calibrated camera intrinsic parameters to obtain a distortion-free image; the camera intrinsic parameters include the camera intrinsic parameter matrix K and the distortion coefficient vector D. Synchronize point cloud data with distortion-free image data based on timestamps.

5. The method for preventing collisions when lowering precast beams using a precast beam lifting device as described in claim 1, characterized in that, The process of spatially registering point cloud data with image data using pre-calibrated camera intrinsic parameters and LiDAR-camera extrinsic parameters, along with corrected images, and cropping the ROI region of the point cloud data to obtain the cropped point cloud ROI region includes: Load the pre-trained beam segment image recognition model; The corrected image is input into the beam segment image recognition model to detect beam segment targets and output 2D detection boxes. The 2D detection box is redundantly expanded, and the corner points of the 2D detection box are projected onto the LiDAR coordinate system using the camera intrinsic parameters and LiDAR-camera extrinsic parameters to obtain the corresponding 3D view frustum space. Based on the point cloud data synchronized after 3D view frustum clipping, background points are filtered out, and the point cloud ROI region containing beam segments and redundant areas is extracted.

6. The method for preventing collisions when lowering precast beams using a precast beam lifting device as described in claim 1, characterized in that, Before extracting beam features from the point cloud ROI region, calculating beam displacement, and determining the collision risk level, the process also includes: Extract the maximum and minimum Y-axis values ​​of beam segments within the ROI region of the point cloud; The maximum and minimum values ​​of the beam segment on the Y-axis are stored in the history queue. The extreme value difference in the queue is calculated. When the extreme value difference is greater than the set value, it is determined to be point cloud shaking, and the calculation is skipped.

7. The method for preventing collisions when lowering precast beams using a precast beam lifting device as described in claim 1, characterized in that, The process of extracting beam features from the point cloud ROI region, calculating beam displacement, and determining the collision risk level includes: Extract the 3D boundary and geometric contour features of the beam from the point cloud ROI region, and remove false features to obtain an accurate target feature set; Identify adjacent beam distribution scenarios based on target feature sets; Calculate beam displacement by combining beam boundary points for different adjacent beam distribution scenarios; The boundary difference between the beam and the adjacent beam is calculated based on the 3D boundary and geometric contour features of the beam, and the collision risk level is determined based on the boundary difference.

8. A precast beam lifting equipment anti-collision system for lowering beams, characterized in that, The precast beam lifting equipment's anti-collision system for dropping beams includes: The data receiving and preprocessing module is used to perform distortion correction on the acquired image data using pre-calibrated camera intrinsic parameters; The registration and cropping module is used to achieve spatial registration of point cloud data and image data using pre-calibrated camera intrinsic parameters and LiDAR-camera extrinsic parameters and corrected images, and to crop the ROI region of the point cloud data to obtain the cropped point cloud ROI region. The calculation module is used to extract beam features from the point cloud ROI region, calculate beam displacement, and determine the collision risk level. The results publishing module is used to output the beam displacement and collision risk level.

9. A precast beam lifting equipment anti-collision device for lowering beams, characterized in that, The precast beam lifting equipment anti-collision device includes a processor, a memory, and a precast beam lifting equipment anti-collision program stored in the memory and executable by the processor, wherein when the precast beam lifting equipment anti-collision program is executed by the processor, it implements the steps of the precast beam lifting equipment anti-collision method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a precast beam lifting equipment anti-collision program, wherein when the precast beam lifting equipment anti-collision program is executed by a processor, it implements the steps of the precast beam lifting equipment anti-collision method as described in any one of claims 1 to 7.

Images