Automatic unloading guide system based on multiple sensors and crown block scheduling method

Abstract

The invention discloses an automatic unloading guiding system based on multiple sensors and a crown block scheduling method. The automatic unloading guiding system comprises a line laser radar array, a high-precision industrial camera, an edge computing terminal and a man-machine interaction device. 32-line laser radars arranged on steel beams on the two sides of the workshop in the length direction detect the horizontal deflection angle (the precision is + / -0.1 degree), the longitudinal deviation (+ / -1.5 cm) and carbon block layer height data of a transport vehicle in real time through the point cloud layering analysis technology, and the carbon block missing state (the detection rate is larger than or equal to 99.3%) is synchronously recognized; a carbon block three-dimensional model is reconstructed, and the offset (smaller than or equal to 5 mm) of the center line of the single carbon block and the safe distance (the threshold value is 25 cm) between the single carbon block and a vehicle fence are accurately calculated; the edge computing terminal fuses multi-source data through a volume Kalman filter (CKF) algorithm, a correction instruction is dynamically generated, the LED guide screen guides a driver to adjust in an AR virtual marked line superposition mode, and abnormal carbon blocks guide ground personnel to adjust through AR mark navigation of an industrial PAD.

Description

Technical Field

[0001] The invention relates to an automatic unloading guidance system based on multiple sensors and an overhead crane dispatching method, belonging to the technical field of industrial automation control. Background Art

[0002] In the electrolytic aluminum industry, the level of automation in anode carbon block unloading operations directly affects production efficiency and operating costs. Flatbed trucks and sidecar trucks are mostly used to transport carbon blocks. The traditional unloading method is a "ground command + driver collaboration" mode of unloading: ground workers need to be present throughout the process, communicating through gesture commands and intercom voice, guiding the truck driver to position the vehicle and the overhead crane driver to operate the truck's travel mechanism and clamps to clamp the carbon blocks. This mode has significant limitations: first, the human-computer interaction efficiency is low, and single positioning requires repeated adjustments, taking an average of 180 seconds per time. In addition, the manually estimated carbon block centerline offset often deviates by more than ±15mm, which can easily cause clamp collisions. Second, full-time ground command personnel are required (usually 2-3 people per shift), which results in high labor costs. Third, the entire operation relies on manual judgment and operation, making unmanned operation of the overhead crane impossible. Misjudgment or operational errors may also lead to safety accidents.

[0003] Traditional overhead crane operations lack real-time carbon block posture data feedback, making it impossible to form a "perception-decision-execution" closed-loop control, which restricts the development of unmanned operations throughout the entire process. The present invention uses multi-sensor fusion technology to analyze the three-dimensional coordinates of carbon blocks in real time (positioning accuracy ±3mm), and is deeply integrated with the scheduling system to directly drive the overhead crane to complete autonomous clamping, reducing manual intervention scenarios to special situations such as abnormal carbon block stacking (incidence rate <5%). After the system was implemented, the single operation time was shortened to 45 seconds, the ground staffing was reduced by 90%, and the safety hazards of manual operation were completely avoided through precise anti-collision control (safety distance monitoring accuracy ±3mm). Compared with the traditional model, this system not only realizes the unmanned unloading process, but also promotes the automation upgrade of the entire carbon block transportation process, providing key technical support for the intelligent transformation of the electrolytic aluminum industry. Summary of the Invention

[0004] The purpose of the present invention is to provide an automatic unloading guidance system and overhead crane scheduling method based on multiple sensors, so as to solve the problems in the prior art of low efficiency, poor precision, high safety risks of traditional manual unloading in the transportation scenarios of aluminum anode carbon blocks on flatbed and sidecar trucks, insufficient adaptability of existing solutions to dynamic changes in the height of the carriage sideboards, lack of three-dimensional point cloud features caused by reflective interference from metal embedded parts on the surface of the carbon blocks and dust shielding effects in industrial scenarios, and difficulty in meeting high-precision detection requirements.

[0005] The technical solution of the present invention is a multi-sensor based automatic unloading guidance system, which includes a multi-sensor detection unit, an edge computing terminal, a human-computer interaction device and an overhead crane collaborative control module, wherein:

[0006] The multi-sensor detection unit includes a laser radar detection module, which is used to obtain three-dimensional point cloud data of the anode carbon block stack in the car in real time, detect the vehicle's horizontal inclination, longitudinal offset and carbon block layer height, a machine vision detection module, which is used to capture the carbon block surface image and identify the geometric center offset of a single carbon block, and a redundant ranging module, which is used to perform redundant verification of the safe distance between the carbon block and the vehicle guardrail;

[0007] The edge computing terminal is used to perform spatiotemporal synchronization and fusion calculations on multi-sensor data to generate vehicle adjustment instructions and carbon block positioning coordinates;

[0008] The human-computer interaction device is used to send visual adjustment instructions to the driver and ground personnel;

[0009] The overhead crane collaborative control module is used to send the coordinates of the carbon block to the overhead crane scheduling system to control the overhead crane to perform the grasping operation.

[0010] Furthermore, the laser radar detection module is a multi-line laser radar array with a horizontal field of view of 360°, a vertical field of view of 40°, a point cloud density of ≥160,000 points / second, and the dual radar point clouds are fused by the NDT point cloud registration algorithm, and the RANSAC algorithm is used to fit the surface plane equation of the carbon block pile to calculate the horizontal inclination angle. , when θ>0.5°, the vehicle adjustment command is triggered.

[0011] Furthermore, the machine vision inspection module is composed of a group of industrial cameras with multi-perspective settings, and the resolution of the industrial cameras is ≥20 million pixels. The YOLOv5 model is used to detect the carbon block bounding box, and the Harris-ZNCC sub-pixel corner detection algorithm is combined to extract the carbon block corners. The EPnP algorithm is used to solve the three-dimensional pose of the carbon block, and the center coordinate positioning error is ≤5mm.

[0012] Furthermore, the industrial camera group includes 8 column-deployed cameras and 3 top beam-deployed cameras.

[0013] Furthermore, the hardware of the edge computing terminal adopts the NVIDIA Jetson AGX Xavier platform, with a built-in multi-source data synchronization module, and achieves sensor spatiotemporal alignment through the PTP protocol, with a clock deviation of ≤±0.5ms. The data processing process includes:

[0014] a) Perform voxel grid filtering and statistical outlier removal on point cloud data;

[0015] b) Perform CLAHE enhancement and MSRCR reflection suppression on the image data;

[0016] c) The point cloud and visual data are fused using a volumetric Kalman filter (CKF), where the state vector contains the position and attitude angle of the carbon block.

[0017] Furthermore, the human-computer interaction device includes:

[0018] An LED guide screen (301) uses augmented reality (AR) technology to superimpose virtual markings on the vehicle adjustment path, displaying a direction arrow and displacement;

[0019] Industrial PAD (302), locates abnormal carbon blocks through AR marking, and supports touch-screen access to the three-dimensional point cloud model of the carbon blocks;

[0020] The sound and light alarm is used to trigger a 90dB buzzer and red strobe warning when an abnormality is detected.

[0021] Furthermore, the overhead crane collaborative control module includes:

[0022] Multi-protocol communication, which supports OPC UA, ISO / IEC 9506 MMS, and Web-Socket;

[0023] A data packet, comprising a carbon block ID, three-dimensional coordinates, a timestamp, and a check code;

[0024] The overhead crane clamp has a positioning error of ≤2cm and a communication cycle of ≤100ms.

[0025] A method for dispatching an overhead crane based on multi-sensor automatic unloading comprises the following steps:

[0026] S1. Use LiDAR and industrial cameras to simultaneously collect 3D point cloud data and carbon block image data from the carriage.

[0027] S2. Perform registration and feature extraction on the point cloud data to calculate the vehicle's horizontal inclination angle and the carbon block layer height;

[0028] S3. Perform image data enhancement processing and deep learning detection to calculate the 3D pose of the carbon block;

[0029] S4. Use the CKF algorithm to fuse multi-source data and generate vehicle adjustment instructions and carbon block coordinates;

[0030] S5. Use LED screens and industrial PADs to guide personnel in adjusting the vehicle and carbon block positions;

[0031] S6. When the test meets the standards, the coordinates of the carbon block are sent to the overhead crane dispatch system, which controls the overhead crane to complete the grab. The overhead crane dispatch instruction contains a JSON data packet of the carbon block coordinates, and the check code uses the CRC32 algorithm.

[0032] Furthermore, in step S4:

[0033] (1) The vehicle horizontal tilt angle threshold is set to 0.5°, and the carbon block center offset threshold is set to 5 cm;

[0034] (2) A hierarchical verification mechanism is used to prioritize the correction of the overall vehicle posture and then locate abnormal carbon blocks.

[0035] Furthermore, in step S5:

[0036] (1) The driver's adjustment instructions are displayed as an AR virtual line and arrow vector superimposed;

[0037] (2) Ground personnel navigate to the location of the abnormal carbon block through AR markers and make adjustments based on the PAD three-dimensional model demonstration path.

[0038] Compared with existing technologies, this invention offers the following advantages: It utilizes a multi-line LiDAR array, a high-precision industrial camera, an edge computing terminal, and a human-computer interaction device. The multi-line LiDAR and industrial camera integrate sensing capabilities, combined with a two-stage detection algorithm, to achieve millimeter-level 3D modeling and sub-centimeter-level positioning of anode carbon block stacks. This improves detection efficiency by five times compared to manual methods, with a positioning error of ≤1.5cm. Field measurements at an aluminum smelter (operating conditions: dust concentration ≥50mg / m³, ambient illumination ≤100lux) demonstrated that the system takes ≤8 seconds for a single inspection (compared to an average of 42 seconds for traditional manual inspections), with an average carbon block center positioning error of 1.2cm (σ = 0.3cm). The system employs a layered verification mechanism, first performing overall leveling correction (tilt threshold 0.5°), followed by precise individual detection (center deviation >5cm triggers an alarm). Simultaneously, the system dynamically guides vehicles using LED screens and an industrial PAD identifies abnormal carbon blocks, forming a collaborative closed-loop system. The system maintains a 99.3% recognition accuracy even in dusty and vibrating environments, significantly improving unloading safety and overhead crane dispatching efficiency.

[0039] At the same time, 32-line LiDARs deployed on the steel beams along the length of the workshop use point cloud layered analysis technology to detect the horizontal deflection angle (accuracy ±0.1°), longitudinal offset (±1.5cm), and carbon block layer height of transport vehicles in real time, and simultaneously identify missing carbon blocks (detection rate ≥99.3%). Eleven 20-megapixel industrial cameras (eight deployed on columns and three on the roof) use a multi-view stereo matching algorithm to reconstruct the three-dimensional model of the carbon blocks, accurately calculating the centerline offset of each carbon block (≤5mm) and the safe distance from the vehicle railing (threshold 25cm). The edge computing terminal uses the volumetric Kalman filter (CKF) algorithm to fuse multi-source data and dynamically generate correction instructions. The LED guide screen uses AR virtual markings to guide the driver's adjustments. Abnormal carbon blocks are navigated by AR markers on the industrial PAD to guide ground personnel to make adjustments. Compared with traditional solutions, a single detection takes ≤8 seconds, reducing the frequency of manual intervention by 90%. It operates stably under dust concentrations ≥50mg / m³ and vibration acceleration 0.3g RMS. The system can be seamlessly connected to the overhead crane dispatching system, with a gripper positioning error of ≤2cm, significantly improving the operating efficiency and safety of the anode assembly workshop and possessing wide application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a system architecture diagram of an embodiment of the invention;

[0041] Figure 2 is a flow chart of a carbon block detection algorithm according to an embodiment of the invention;

[0042] Figure 3 This is a schematic diagram of a hierarchical guidance logic according to an embodiment of the invention;

[0043] Figure 4 is a schematic top view of the deployment of various detection units and components of an embodiment of the invention;

[0044] Figure 5 This is a schematic front view of the deployment of various detection units and components of an embodiment of the invention;

[0045] Figure 6 It is a schematic diagram of the system network communication structure of an embodiment of the invention.

[0046] In the figure, the corresponding relationship between the component names and the drawing numbers is as follows:

[0047] 100-Multi-sensor detection unit, 101-LiDAR detection module, 102-Machine vision detection module, 103-Redundant ranging module, 200-Edge computing terminal, 201-Edge computing server, 202-Data fusion switch, 300-Human-computer interaction device, 301-LED guide screen, 302-Industrial PAD, 303-Sound and light alarm, 304-Industrial wireless router, 400-Crown crane collaborative control module, 401-Crown crane dispatching system server, 402-Industrial router, 403-Crown crane equipment, 501-Coal block unloading area. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0049] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0050] In addition, the descriptions of "first", "second", etc. in the present invention are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0051] Example 1: System hardware deployment and sensor networking

[0052] See also Figure 1 ,This embodiment provides a complete system layered architecture diagram, which is divided into perception layer, edge computing layer, interaction layer, and execution layer according to functions and data flow.

[0053] See also Figure 4 and Figure 5 , this embodiment provides a top view and a front view of the deployment of each detection unit and component, indicating the installation position of each detection unit relative to the unloading area.

[0054] See Figure 1-6 In the present invention, a multi-sensor based automatic unloading guidance system includes a multi-sensor detection unit 100, an edge computing terminal 200, a human-computer interaction device 300 and an overhead crane collaborative control module 400, wherein:

[0055] The multi-sensor detection unit 100 includes a laser radar detection module 101, which is composed of a laser radar array deployed symmetrically on both sides and deployed in the safe neutral zone of the overhead crane. It is used to obtain three-dimensional point cloud data of the anode carbon block stack in the car in real time and detect the vehicle's horizontal inclination, longitudinal offset and carbon block layer height; a machine vision detection module 102, which is composed of a group of industrial cameras deployed from multiple perspectives, including 11 industrial cameras, covering the entire and local areas of the car, and is used to capture the surface image of the carbon blocks and identify the geometric center offset of individual carbon blocks; and a redundant ranging module 103, which is composed of laser ranging sensors deployed at the four corners of the car. It is set to a safety distance threshold of 25 cm and is used to perform redundant verification of the safety distance between the carbon blocks and the vehicle railing; the edge computing terminal 200 is used to perform spatiotemporal synchronization and fusion calculations on the multi-sensor data to generate vehicle adjustment instructions and carbon block positioning coordinates; the human-computer interaction device 300 is used to send visual adjustment instructions to the driver and ground personnel; and the overhead crane collaborative control module 400 is used to send the carbon block coordinates to the overhead crane scheduling system to control the overhead crane to perform grasping operations.

[0056] The system adopts a star-shaped architecture. The LiDAR detection module 101, machine vision detection module 102, and redundant ranging module 103 are connected to the data fusion switch 202 via a network cable. This data is then transmitted to the edge computing server 301 for fusion analysis and calculation. The posture adjustment data of the truck and charcoal blocks is transmitted to the human-computer interaction device via the data fusion switch 202. The data fusion switch 202 is connected to the LED guide screen 301, the sound and light alarm 303, the industrial wireless router 304, and the industrial router 402 via a network cable. The industrial PAD 302 uses the industrial Wi-Fi provided by the industrial wireless router 304 to exchange data with the edge computing terminal. The industrial router 402 is also connected to the overhead crane dispatch system server 401 and the overhead crane equipment 403 via a network cable. Once the postures of the truck and charcoal blocks meet the standards, the overhead crane collaborative control code module 400 calculates the charcoal block coordinate data and transmits it to the overhead crane dispatch system server 401 via the data fusion switch 202 and the industrial router 402. The dispatch system then controls the overhead crane to perform the gripping operation. According to actual networking needs, network devices can be added between some components, and the connection can be changed to fiber optic communication and industrial wireless communication.

[0057] The following describes in detail the deployment and communication networking of each detection unit in conjunction with the on-site situation of this embodiment.

[0058] This system is implemented in the anode carbon block unloading area 501 of the carbon block warehouse. The unloading area 501 is 15m long and 4m wide. The specific implementation steps are as follows:

[0059] 1. Multi-sensor detection unit installation

[0060] (1) LiDAR array: The LiDAR array is installed with a pitch angle of 35° downward on the steel beams on both sides of the workshop length. Two Velodyne HDL-32E multi-line LiDARs (horizontal angular resolution 0.1°, vertical field of view angle +10° to -30°, point cloud density ≥160,000 points / second) are symmetrically deployed on the steel beams on both sides of the workshop length (X=±8.25m). The installation height is 8.6m from the ground and the pitch angle is adjusted to 35° downward. After the dual radar point clouds are fused, a three-dimensional detection area covering the entire length of the car is formed.

[0061] (2) Industrial vision system: 4 Basler ace acA4112-20um industrial cameras (11 in total, resolution 4096×3000, frame rate 5fps) are installed on the columns on both sides of the unloading area (installation height 4.8m, pitch angle 15° downward) and 3 on the top beam (installation height 10.5m, pitch angle 30° downward). They are equipped with electric zoom lenses with a focal length of 16mm (column deployment) and 20mm (top deployment), and are equipped with ring LED fill lights (color temperature 5600K, illumination ≥5000lux). The camera posture horizontal error is maintained at ≤0.1° through the anti-shake gimbal.

[0062] (3) Redundant distance measurement module: Four sets of SICK DT50-Hi laser rangefinders (range 0.2-50m, accuracy ±1mm) are installed 1m outside the guardrails at the four corners of the carriage (height 1.8m). The safety distance threshold is set to 25cm. If the limit is exceeded, an audible and visual alarm will be triggered. Four sets of laser distance measurement sensors are deployed at the four corners of the carriage for redundant verification of the height of the carbon blocks and the vehicle.

[0063] Furthermore, an edge computing terminal is set up, including an industrial computer hardware device and a fusion computing software system, and deployed in the control room. The industrial computer uses the NVIDIA Jetson AGX Xavier platform, equipped with a 256-core GPU, equipped with a TensorRT8.0 acceleration engine, supports INT8 quantitative reasoning, has a built-in multi-source data synchronization module, and realizes multi-sensor spatiotemporal calibration through the PTP precision clock protocol (IEEE1588), with a spatiotemporal alignment error of ≤±0.5ms. The fusion computing software system is divided into point cloud data processing subsystem, image data processing subsystem, fusion computing subsystem, and data interaction subsystem according to its function. The point cloud data processing subsystem uses the NDT point cloud registration algorithm (Normal Distributions) based on feature descriptors. Transform) to perform point cloud registration, eliminate vehicle vibration noise, fit the upper surface plane of the carbon block pile through the RANSAC (random sampling consensus) algorithm, calculate the horizontal inclination, extract the edge point cloud of the carbon block pile, and calculate the minimum Euclidean distance to the car body guardrail; the image data processing subsystem enhances image contrast based on adaptive histogram equalization (CLAHE), uses an improved YOLOv5 model (input resolution 1280×1024) to detect the bounding box of a single carbon block, and combines the Canny edge detection algorithm to extract the pixel coordinates of the carbon block centerline; the fusion calculation and analysis subsystem uses volumetric Kalman filtering to fuse multimodal data The CKF integrates multi-source data, dynamically generates vehicle adjustment instructions and overhead crane positioning coordinates, and achieves millisecond-level decision response; the data interaction module provides data receiving and sending interfaces. The data receiving interface is responsible for receiving point cloud and image data. The lidar and industrial camera achieve spatiotemporal alignment through hardware synchronization signals (PPS pulses). The point cloud data (10Hz) and image data (5fps) are transmitted to the edge terminal via Gigabit Ethernet. After data preprocessing, they are sent to the point cloud data processing subsystem and the image data processing subsystem for analysis and processing. The data sending interface is responsible for sending the analysis results to the human-computer interaction device and the overhead crane collaborative control module.

[0064] 2. Edge computing terminal configuration

[0065] (1) The hardware uses the NVIDIA Jetson AGX Xavier industrial computer, equipped with a 256-core Volta GPU and 32GB DDR4 memory, running the Ubuntu 18.04 system and the ROS Melodic framework.

[0066] (2) Sensor networking: The lidar is connected to the industrial computer gigabit switch (Moxa EDS-405A) via Ethernet, and the industrial camera transmits image data through a 6-way CoaXPress 2.0 interface. All devices are time synchronized through the PTP protocol (IEEE 1588v2), with a clock deviation of ≤±0.5ms.

[0067] Example 2: Data processing and algorithm implementation;

[0068] Specifically, the machine vision inspection module consists of a multi-perspective industrial camera group with a resolution of ≥20 megapixels. It uses the YOLOv5 model to detect the carbon block bounding box, combined with the Harris-ZNCC sub-pixel corner detection algorithm to extract the carbon block corners, and the EPnP algorithm to solve the carbon block's three-dimensional pose, with a center coordinate positioning error of ≤5mm. The resulting machine vision inspection module consists of eight high-precision industrial cameras with a resolution of ≥20 megapixels, a ring-shaped LED fill light, and an anti-shake gimbal with adjustable tilt. These cameras are deployed on the columns on both sides of the unloading area (4 on each side, 4.8m high) and on the roof beams of the workshop (3, 10.5m high). The cameras capture high-definition images of the carbon block surface texture and stacking gaps, calculate the carbon block centerline offset using the sub-pixel corner detection algorithm (Harris-ZNCC), and achieve three-dimensional pose solution using multi-perspective stereo matching.

[0069] See also Figure 2 This embodiment provides a complete carbon block detection data processing flow, which includes data collection, preprocessing, and fusion calculation processes.

[0070] 1. Point cloud processing flow

[0071] (1) Data preprocessing: The point cloud data were downsampled using voxel grid filtering (voxel size 3 cm³), and the statistical outlier removal algorithm (SOR, neighborhood number 50, standard deviation threshold 1.5) was used to remove noise.

[0072] (2) Point cloud registration: The dual radar point clouds are registered based on the NDT algorithm (grid resolution 10 cm), and the coordinate transformation matrix is ​​added to correct the installation height deviation (ΔH = 8.6 m - 6.5 m = 2.1 m). The registration error RMS is ≤ 2 cm.

[0073] In this city's example, more specifically, the laser radar detection module is a multi-line laser radar array with a horizontal field of view of 360°, a vertical field of view of 40°, and a point cloud density of ≥160,000 points / second. The dual radar point clouds are fused using the NDT point cloud registration algorithm, and the RANSAC algorithm is used to fit the plane equation of the upper surface of the carbon block pile to calculate the horizontal inclination angle. When θ>0.5°, a vehicle adjustment command is triggered. The system consists of dual 32-line lidar arrays (Velodyne HDL-32E) symmetrically deployed on either side of the longitudinal steel beam above the workshop. The system is installed at a height of 8.6 meters, with a downward pitch angle of 35°, a horizontal field of view of 360° and a vertical field of view of 40°. It maintains a point cloud density of ≥160,000 points per second. Through multi-beam laser pulse transmission and reception, it measures the 3D coordinates (X, Y, Z) of each reflection point, forming 3D spatial information about the anode carbon block stack within the carriage. This system constructs a millimeter-level precision point cloud model. By layering and analyzing the transport vehicle's point cloud data, it detects the transport vehicle's horizontal deflection, longitudinal offset, and carbon block layer height in real time, and simultaneously identifies missing carbon blocks.

[0074] (3) Feature extraction: RANSAC algorithm is used to fit the plane equation of the upper surface of the carbon block pile ax + by + cz + d = 0, and the horizontal inclination angle is calculated. , when θ>0.5°, the vehicle adjustment command is triggered.

[0075] 2. Visual detection algorithm

[0076] (1) Image enhancement: The original image is processed with CLAHE (tile grid 8×8, contrast limit 2.0), and the MSRCR algorithm is used to eliminate metal reflections.

[0077] (2) Carbon block positioning: Use the improved YOLOv5 model (Backbone is replaced by ConvNeXt-Tiny, input size is 1280×1024) to detect the carbon block bounding box and output the detection result with a confidence level ≥ 0.95.

[0078] (3) Posture solution: The corner points of the carbon block are extracted based on the Harris-ZNCC sub-pixel corner detection algorithm, and the three-dimensional pose of the carbon block is calculated using the EPnP algorithm, with the center coordinate error ≤5mm.

[0079] 3. Multimodal data fusion

[0080] The point cloud and visual data are fused using the volumetric Kalman filter (CKF), and the state vector

[0081] X = [x, z, θx, θz]T contains the position and attitude angle of the carbon block, and the observation equation is:

[0082] ,

[0083] where the process noise Q = diag(0.1², 0.1², 0.05², 0.05²) and the observation noise R = diag(0.02², 0.02²).

[0084] Example 3: Human-machine collaborative guidance and overhead crane dispatching

[0085] See also Figure 3 ,This embodiment demonstrates the interactive relationship and priority rules of the three-level ,guidance system (vehicle / carbon block / system) of the unloading guidance system, ,reflecting the guidance priority and interactive logic of "vehicle→carbon ,block→system".

[0086] 1. Vehicle posture guidance

[0087] When the vehicle's horizontal deviation angle is detected to be out of limit, the edge computing terminal sends an adjustment command to the LED guidance screen (LG 55LS5B, resolution 1920×1080) via the Modbus TCP protocol. The screen displays a red arrow vector (8cm left to right) and an AR virtual marking (with a superimposed distance scale), and simultaneously triggers a buzzer (frequency 2kHz, sound pressure level 90dB) to alert the driver.

[0088] 2. Carbon block posture adjustment

[0089] Ground personnel hold an industrial PAD (Samsung Galaxy Tab Active4 Pro). When it detects that the center deviation of a carbon block is greater than 5cm, the PAD displays a heat map and locates the abnormal carbon block through AR marking. Clicking the abnormal carbon block can call up a 3D point cloud model, and the path adjustment demonstration can be demonstrated by dragging the virtual gripper.

[0090] 3. Overhead Crane Collaborative Operation

[0091] When all carbon block positions meet the standards, the edge computing terminal sends a carbon block coordinate data packet (JSON format, including ID, X / Y / Z coordinates, timestamp, and CRC32 checksum) to the overhead crane dispatching system through the OPC UA protocol (communication cycle 100ms). The overhead crane plans the grasping path based on the coordinates, and the gripper positioning error is ≤2cm.

[0092] Specifically, the industrial camera group includes 8 column-deployed cameras and 3 top beam-deployed cameras.

[0093] Specifically, the hardware of the edge computing terminal uses the NVIDIA Jetson AGX Xavier platform, with a built-in multi-source data synchronization module. It uses the PTP protocol to achieve sensor spatiotemporal alignment, with a clock deviation of ≤±0.5ms. The data processing process includes:

[0094] a) Perform voxel grid filtering and statistical outlier removal on point cloud data;

[0095] b) Perform CLAHE enhancement and MSRCR reflection suppression on the image data;

[0096] c) The point cloud and visual data are fused using a volumetric Kalman filter (CKF), where the state vector contains the position and attitude angle of the carbon block.

[0097] Specifically, the human-machine interaction device includes an LED guidance screen 301, which uses augmented reality (AR) technology to overlay virtual markings on the vehicle's adjustment path, displaying direction arrows and displacement. An industrial PAD 302 uses AR markers to locate abnormal briquettes and supports touch-screen access to a 3D point cloud model of the briquettes. An audio-visual alarm 303 triggers a 90dB beep and a red flashing warning when an anomaly is detected. The industrial large screen is a vertical LED guidance screen that receives control commands from the edge computing terminal via the Modbus TCP protocol. It displays the required vehicle movement direction (← / → arrows) and distance (e.g., "move left 12cm"), guiding the driver with adjustments by overlaying arrow vector diagrams with numerical values. Using augmented reality (AR) overlay display technology, the virtual markings on the vehicle's adjustment path are projected in real time. The industrial PAD guides ground workers in adjusting misaligned briquettes, displays a heat map of the briquette pile (abnormal briquettes with deviations greater than 5cm are marked in red), and supports touch-screen access to a detailed view of the 3D model. The audio-visual alarm device triggers simultaneous audio and visual alerts when an anomaly is detected.

[0098] Specifically, the overhead crane collaborative control module includes multi-protocol communication, which supports OPC UA, ISO / IEC 9506 MMS, and Web-Socket; data packets, which include carbon block ID, three-dimensional coordinates, timestamp, and check code; overhead crane grippers, whose positioning error is ≤2cm and communication cycle is ≤100ms. When the edge computing terminal completes the detection and the carbon block posture meets the automatic clamping requirements, the precise carbon block coordinates are automatically sent to the scheduling system. The overhead crane scheduling system controls the overhead crane according to the carbon block coordinates to complete the unloading operation.

[0099] Specifically, a method for dispatching an overhead crane based on multi-sensor automatic unloading includes a multi-sensor automatic unloading guidance system, and the guidance steps are as follows:

[0100] S1. Use LiDAR and industrial cameras to simultaneously collect 3D point cloud data and carbon block image data from the carriage.

[0101] S2. Perform registration and feature extraction on the point cloud data to calculate the vehicle's horizontal inclination angle and the carbon block layer height;

[0102] S3. Perform image data enhancement processing and deep learning detection to calculate the 3D pose of the carbon block;

[0103] S4. Use the CKF algorithm to fuse multi-source data and generate vehicle adjustment instructions and carbon block coordinates;

[0104] S5. Use LED screens and industrial PADs to guide personnel in adjusting the vehicle and carbon block positions;

[0105] S6. When the test meets the standards, the coordinates of the carbon block are sent to the overhead crane dispatching system to control the overhead crane to complete the grabbing.

[0106] Specifically, in step S4:

[0107] (1) The vehicle horizontal tilt angle threshold is set to 0.5°, and the carbon block center offset threshold is set to 5 cm;

[0108] (2) A hierarchical verification mechanism is adopted to prioritize the correction of the overall vehicle posture and then locate the abnormal carbon block. The overhead crane dispatch instruction contains a JSON data packet of the carbon block coordinates, and the verification code uses the CRC32 algorithm.

[0109] Specifically, in step S5:

[0110] (1) The driver's adjustment instructions are displayed as an AR virtual line and arrow vector superimposed;

[0111] (2) Ground personnel navigate to the location of the abnormal carbon block through AR markers and make adjustments based on the PAD three-dimensional model demonstration path.

[0112] In addition to the above-mentioned preferred embodiments, the present invention has other implementation modes. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection requested by the present invention.

Claims

1. An automatic unloading guidance system based on multiple sensors, characterized in that: It comprises a multi-sensor detection unit (100), an edge computing terminal (200), a human-machine interaction device (300) and an overhead crane collaborative control module (400), wherein: The multi-sensor detection unit (100) includes a laser radar detection module (101), which is used to obtain three-dimensional point cloud data of the anode carbon block stack in the carriage in real time, detect the vehicle horizontal inclination, longitudinal offset and carbon block layer height, a machine vision detection module (102), which is used to capture the carbon block surface image and identify the geometric center offset of a single carbon block, and a redundant distance measurement module (103), which is used to perform redundant verification of the safety distance between the carbon block and the vehicle railing; The edge computing terminal (200) is used to perform spatiotemporal synchronization and fusion calculation on multi-sensor data to generate vehicle adjustment instructions and carbon block positioning coordinates; The human-machine interaction device (300) is used to send visual adjustment instructions to the driver and ground personnel; The overhead crane collaborative control module (400) is used to send the coordinates of the carbon block to the overhead crane dispatching system, and control the overhead crane to perform a grabbing operation.

2. The multi-sensor based automatic unloading guidance system according to claim 1, characterized in that: The laser radar detection module is a multi-line laser radar array with a horizontal field of view of 360°, a vertical field of view of 40°, a point cloud density of ≥160,000 points / second, and the dual radar point clouds are fused using the NDT point cloud registration algorithm. The RANSAC algorithm is used to fit the plane equation of the carbon block pile surface and calculate the horizontal inclination angle. , when θ>0.5°, the vehicle adjustment command is triggered.

3. The multi-sensor based automatic unloading guidance system according to claim 1 or 2, characterized in that: The machine vision inspection module is composed of a group of industrial cameras with multi-perspective settings, and the resolution of the industrial cameras is ≥20 million pixels. The YOLOv5 model is used to detect the carbon block bounding box, and the Harris-ZNCC sub-pixel corner detection algorithm is combined to extract the carbon block corners. The EPnP algorithm is used to solve the three-dimensional pose of the carbon block, and the center coordinate positioning error is ≤5mm.

4. The multi-sensor based automatic unloading guidance system according to claim 1, characterized in that: The edge computing terminal hardware uses the NVIDIA Jetson AGX Xavier platform, with a built-in multi-source data synchronization module. It uses the PTP protocol to achieve sensor spatiotemporal alignment, with a clock deviation of ≤±0.5ms. The data processing process includes: a) Perform voxel grid filtering and statistical outlier removal on point cloud data; b) Perform CLAHE enhancement and MSRCR reflection suppression on the image data; c) The point cloud and visual data are fused using a volumetric Kalman filter (CKF), where the state vector contains the position and attitude angle of the carbon block.

5. The multi-sensor based automatic unloading guidance system according to claim 3, characterized in that: The industrial camera group includes a camera installed on a column and a camera installed on a top beam.

6. The multi-sensor based automatic unloading guidance system according to claim 1, characterized in that: The human-computer interaction device comprises: An LED guide screen (301) uses augmented reality (AR) technology to superimpose virtual markings on the vehicle adjustment path, displaying a direction arrow and displacement; Industrial PAD (302), locates abnormal carbon blocks through AR marking, and supports touch-screen access to the three-dimensional point cloud model of the carbon blocks; The sound and light alarm (303) is used to trigger a 90dB buzzer and a red strobe warning when an abnormality is detected.

7. The multi-sensor based automatic unloading guidance system according to claim 1, characterized in that: The overhead crane collaborative control module includes: Multi-protocol communication, which supports OPC UA, ISO / IEC 9506 MMS, and Web-Socket; A data packet, comprising a carbon block ID, three-dimensional coordinates, a timestamp, and a check code; The overhead crane clamp has a positioning error of ≤2cm and a communication cycle of ≤100ms.

8. A method for dispatching an overhead crane based on multi-sensor automatic unloading, characterized in that: An automatic unloading guidance system comprising a multi-sensor according to any one of claims 1 to 7, comprising the following steps: S1. Use LiDAR and industrial cameras to simultaneously collect 3D point cloud data and carbon block image data from the carriage. S2. Perform registration and feature extraction on the point cloud data to calculate the vehicle's horizontal inclination angle and the carbon block layer height; S3. Perform image data enhancement processing and deep learning detection to calculate the 3D pose of the carbon block; S4. Use the CKF algorithm to fuse multi-source data and generate vehicle adjustment instructions and carbon block coordinates; S5. Use LED screens and industrial PADs to guide personnel in adjusting the vehicle and carbon block positions; S6. When the test meets the standards, the coordinates of the carbon block are sent to the overhead crane dispatch system, which controls the overhead crane to complete the grab. The overhead crane dispatch instruction contains a JSON data packet of the carbon block coordinates, and the check code uses the CRC32 algorithm.

9. The method for dispatching an overhead crane based on multi-sensor automatic unloading according to claim 8, characterized in that: In step S4: (1) The vehicle horizontal tilt angle threshold is set to 0.5°, and the carbon block center offset threshold is set to 5 cm; (2) A hierarchical verification mechanism is used to prioritize the correction of the overall vehicle posture and then locate abnormal carbon blocks.

10. The method for dispatching an overhead crane based on multi-sensor automatic unloading according to claim 9, characterized in that: In step S5: (1) The driver's adjustment instructions are displayed as an AR virtual line and arrow vector superimposed; (2) Ground personnel navigate to the location of the abnormal carbon block through AR markers and make adjustments based on the PAD three-dimensional model demonstration path.

Images