Ship unloading method and system for bulk cargo wharf automatic grab ship unloader

Abstract

The invention discloses a bulk cargo wharf automatic grab bucket ship unloader ship unloading method and system, and relates to the technical field of bulk cargo wharf automatic ship unloading, and the method comprises the steps: carrying out the data collection of a cabin structure and bulk cargo distribution through a laser radar, a camera and an ultrasonic sensor; data acquired by the laser radar, the camera and the ultrasonic sensor are fused to generate a three-dimensional grid model; task allocation is carried out on the ship unloader based on a block chain platform; generating an unloading path based on the three-dimensional grid model and the distributed task; the unloading path is updated through the state change of bulk cargoes in the cargo ship in the operation process; according to the positions and states of different ship unloaders, the unloading paths of the ship unloaders are coordinated in real time; according to the ship unloader, collaborative operation of the multiple ship unloaders is achieved, conflicts and collisions are avoided, and the overall unloading efficiency and safety are further improved.

Description

Technical Field

[0001] The invention relates to the technical field of automatic ship unloading at a bulk cargo terminal, and in particular to a ship unloading method and system of an automatic grab ship unloader at a bulk cargo terminal. Background Art

[0002] Automated unloading operations at bulk cargo terminals are an important part of port logistics. With the growth of global trade and the increasing requirements for efficiency and safety, traditional manual operations are gradually being replaced by automated systems.

[0003] Early automated ship unloaders mainly relied on preset path planning and fixed operating modes, which were suitable for situations where cargo distribution was relatively uniform and had little variation. However, with the increase in ship size and the diversification of cargo types, traditional automated systems are unable to cope with the complex and changeable cabin environment. When multiple ship unloaders operate in the same area, there is a lack of effective coordination mechanism, which makes collisions or conflicts prone to occur, affecting the safety and continuity of operations. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a ship unloading method of an automatic grab ship unloader at a bulk cargo terminal to solve the problem of coordinating multiple ship unloaders to operate synchronously in the same area.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for unloading a ship using an automatic grab ship unloader at a bulk cargo terminal, comprising: Use lidar, cameras and ultrasonic sensors to collect data on the ship's cabin structure and bulk cargo distribution; Generate a 3D mesh model by fusing data collected by lidar, camera and ultrasonic sensor; Based on the blockchain platform, tasks are assigned to the ship unloader; Generate unloading routes based on the 3D mesh model and assigned tasks; Update the unloading path according to the change of bulk cargo status in the cargo ship during the operation; According to the location and status of different ship unloaders, the unloading paths of each ship unloader are coordinated in real time.

[0007] As a preferred solution of the ship unloading method of the automatic grab ship unloader of the bulk cargo terminal of the present invention, wherein: the laser radar, camera and ultrasonic sensor are used to collect data on the cabin structure and bulk cargo distribution, and the specific steps are as follows: Use a checkerboard calibration board to collect data from the lidar and camera at the same time for joint calibration; The external parameter matrix between the lidar and the camera is calculated through the calibration algorithm; Use ultrasonic sensors to scan standard objects of known distance for distance calibration; The laser radar scans the cabin at all angles at a fixed frequency to collect point cloud data; The voxel filtering method is used to filter the collected point cloud data to remove noise points and outliers; While the LiDAR is scanning, the camera simultaneously collects image data of the cabin and uses a deep learning algorithm to segment the image to identify bulk cargo, cabin walls and other obstacles. The ultrasonic sensor collects the distance data between the cabin and the ultrasonic sensor to monitor the position of obstacles around the ship unloader in real time, and uses moving average filtering to smooth the distance data.

[0008] As a preferred solution of the ship unloading method of the automatic grab ship unloader of the bulk cargo terminal of the present invention, wherein: the data collected by the laser radar, the camera and the ultrasonic sensor are fused to generate a three-dimensional grid model, and the specific steps are as follows: Using the hardware synchronization method, the timestamps of the processed point cloud data, image data, and distance data are aligned; Based on the external parameter matrix, the image data collected by the camera is aligned with the point cloud data of the lidar. Each pixel in the image corresponds to a point cloud data, and the point cloud data with image information is obtained. The positions of obstacles around the ship unloader monitored by the ultrasonic sensor are converted into three-dimensional points and synchronized with the point cloud data with image information to obtain fused point cloud data; Rasterize the fused point cloud data and divide the entire three-dimensional space into several voxels; By connecting the representative points in the voxels, triangular patches are formed to generate a three-dimensional mesh model; The generated three-dimensional mesh model is optimized using the QEM algorithm; By continuously collecting new fused point cloud data and integrating it with the existing 3D mesh model, the 3D mesh model is updated in real time and uploaded to the blockchain platform; The ship unloader will upload the updated three-dimensional mesh model and working status to the blockchain platform in real time through the control center.

[0009] As a preferred solution of the ship unloading method of the automatic grab ship unloader of the bulk cargo terminal of the present invention, wherein: the task allocation of the ship unloader is performed based on the blockchain platform, and the specific steps are as follows: The port dispatch center uploads the cargo ship arrival plan and bulk cargo list to the blockchain platform; The blockchain platform assigns unloading tasks to each ship unloader based on the real-time 3D mesh model and real-time working status uploaded by the ship unloader; The information is sent to the ship unloader control center through the safety channel, and the ship unloader automatically unloads the ship according to the received unloading task.

[0010] As a preferred solution of the ship unloading method of the automatic grab ship unloader of the bulk cargo terminal of the present invention, wherein: the unloading path is generated based on the three-dimensional grid model and the assigned tasks, and the specific steps are as follows: Mark the inaccessible voxels according to the cargo distribution, cabin structure and obstacle locations in the 3D grid model; According to the unloading tasks issued by the blockchain platform to the ship unloader, the unloading path is planned using the hybrid path planning algorithm of A-Star combined with deep reinforcement learning; The hybrid path planning algorithm uses the Euclidean distance as a heuristic function to estimate the cost from the initial position of the ship unloader grab to the target position; According to the voxel division in the three-dimensional grid model, starting from the voxel where the grab bucket of the ship unloader is located, the adjacent voxels are gradually expanded, and the inaccessible voxels are bypassed until the voxel where the target position is located is reached; Based on the minimum cost principle, the voxel with the minimum cost is selected for expansion along the target direction; When the target position is found, the route from the target position to the initial position of the ship unloader grab is traced in reverse to generate the initial path.

[0011] As a preferred solution of the ship unloading method of the automatic grab ship unloader of the bulk cargo terminal of the present invention, wherein: the unloading path is updated according to the change of the state of the bulk cargo in the cargo ship during the operation, and the specific steps are as follows: Based on the real-time updated 3D mesh model, the unloader grab re-marks the inaccessible voxels every time it grabs bulk cargo; According to the height coordinates of bulk cargo in the 3D grid model, the bulk cargo accumulation state is judged, and the position with the highest height coordinate is preferentially set as the target position of the grab bucket of the ship unloader. Estimate the cost from the initial position of the ship unloader grab to the target position, update the initial path, and generate a new unloading path.

[0012] As a preferred solution of the ship unloading method of the automatic grab ship unloader of the bulk cargo terminal of the present invention, wherein: the unloading paths of the ship unloaders are coordinated in real time according to the positions and states of the different ship unloaders, and the specific steps are as follows: The ship unloader control center uploads the real-time unloading path information to the port dispatch center at a fixed frequency; According to the maximum opening radius of the grab buckets of adjacent ship unloaders, the safety spacing threshold between the grab buckets of the ship unloaders is set; Set the safety time threshold according to the rotation speed of the ship unloader and the collection speed of the ship unloader grab; The real-time unloading path information is summarized, and the real-time unloading paths of adjacent ship unloaders are cross-compared. When the distance between the grab buckets of adjacent ship unloaders is less than the safety distance threshold, it is determined that there is a collision risk, otherwise there is no collision risk; The time cross-comparison of two unloading paths with collision risk is carried out. When the time difference between the grab buckets of adjacent unloaders reaching the collision risk position is less than the safety time threshold, it is confirmed that there is a collision risk. Otherwise, there is no collision risk. The port dispatch center issued a safety warning notice to the control center of the ship unloader with a collision risk, and one ship unloader stopped moving and started again after the other ship unloader passed the collision risk position; When the unloading path is updated, the voxels where there is a collision risk are marked as inaccessible voxels.

[0013] In a second aspect, the present invention provides a bulk cargo terminal automatic grab ship unloader ship unloading system, comprising a data acquisition module, a model building module, a task allocation module, a path generation module, a path update module and a path coordination module. The data acquisition module is used to collect data on the cabin structure and bulk cargo distribution using a laser radar, a camera and an ultrasonic sensor; The model building module is used to fuse the data collected by the laser radar, camera and ultrasonic sensor to generate a three-dimensional grid model; The task allocation module is used to allocate tasks to the ship unloader based on the blockchain platform; The path generation module is used to generate a discharge path based on the three-dimensional grid model and the assigned tasks; The path updating module is used to update the unloading path according to the change of the bulk cargo state in the cargo ship during the operation; The path coordination module is used to coordinate the unloading paths of the ship unloaders in real time according to the positions and states of the different ship unloaders.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the ship unloading method of the automatic grab ship unloader for a bulk terminal as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the ship unloading method of the automatic grab ship unloader for a bulk terminal as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: through the fusion of lidar, camera and ultrasonic sensor, the system realizes high-precision environmental perception, ensures the accuracy of the three-dimensional grid model, and provides reliable basic data for subsequent path planning. The task allocation system based on blockchain avoids human intervention and operational errors, improves the fairness and traceability of task allocation, and the hybrid path planning algorithm automatically generates unloading paths in static and dynamic environments, adapts to changes in cargo distribution, and avoids operation delays caused by path failure. The real-time path update mechanism dynamically adjusts the path according to the latest cargo distribution, avoids repeated grabbing and unnecessary movement, and significantly improves the unloading efficiency. Through the global coordination of the port dispatching center, the collaborative operation of multiple unloaders is realized, conflicts and collisions are avoided, and the overall unloading efficiency and safety are further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a flow chart of the ship unloading method of the automatic grab ship unloader at a bulk cargo terminal in Example 1.

[0019] Figure 2 This is a module diagram of the ship unloading system of the automatic grab ship unloader of the bulk cargo terminal in Example 1. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, reference Figure 1 and Figure 2, which is the first embodiment of the present invention, provides a method for unloading a ship by an automatic grab ship unloader at a bulk cargo terminal, comprising the following steps: S1. Use lidar, cameras and ultrasonic sensors to collect data on the cabin structure and bulk cargo distribution.

[0024] Install the laser radar and camera on the top of the ship unloader to facilitate scanning of the cargo ship's hold and bulk cargo. The installation position of the ultrasonic sensor should cover the front, rear, left and right directions of the ship unloader to ensure that cargo ships approaching the ship unloader can be detected; Use a checkerboard calibration board to collect data from the lidar and camera at the same time for joint calibration; Through the calibration algorithm (such as Zhang Zhengyou calibration method), the external parameter matrix (i.e. rotation matrix and translation vector) between the lidar and the camera is calculated to ensure the consistency of the coordinate systems of the two. Use ultrasonic sensors to scan standard objects of known distance for distance calibration; Specifically, based on the known geometric structure of the calibration plate, combined with the data collected by the lidar and camera, the extrinsic matrix between the two (i.e., the rotation matrix and translation vector) is calculated; Select several feature points with known positions on the calibration plate (such as the corner points of the calibration plate), and measure the 3D coordinates of these points using the laser radar and camera. Compare the measurement results of the two and calculate the error value. If the error is within the allowable range (such as ±1cm), the calibration is successful; otherwise, recalibration is required; The laser radar scans the cabin at all angles at a fixed frequency to collect point cloud data. The point cloud data generated by each scan includes XYZ coordinates and intensity values; The voxel filtering method is used to filter the collected point cloud data to remove noise points and outliers; While the LiDAR is scanning, the camera simultaneously collects image data of the cabin and uses deep learning algorithms (such as Mask R-CNN) to segment the image and identify bulk cargo, cabin walls and other obstacles. Among them, other obstacles include lighting equipment, cables and wires, and safety signs; The ultrasonic sensor collects the distance data between the cabin and the ultrasonic sensor to monitor the position of obstacles around the ship unloader in real time, and uses moving average filtering to smooth the distance data to reduce fluctuations caused by environmental noise.

[0025] S2. Generate a three-dimensional mesh model by fusing the data collected by the lidar, camera and ultrasonic sensor.

[0026] Using the hardware synchronization method, the timestamps of the processed point cloud data, image data, and distance data are aligned; Based on the external parameter matrix, the image data collected by the camera is aligned with the point cloud data of the lidar. Each pixel in the image corresponds to a point cloud data, and the point cloud data with image information is obtained. The positions of obstacles around the ship unloader monitored by the ultrasonic sensor are converted into three-dimensional points and synchronized with the point cloud data with image information to obtain fused point cloud data; Rasterize the fused point cloud data and divide the entire three-dimensional space into several voxels; By connecting the representative points in the voxels, triangular patches are formed to generate a three-dimensional mesh model; The generated three-dimensional mesh model is optimized using the QEM algorithm to reduce the complexity of the three-dimensional mesh model; As the unloader operates, the bulk cargo in the hold gradually decreases. By continuously collecting new fused point cloud data and integrating it with the existing 3D mesh model, the 3D mesh model is updated in real time and uploaded to the blockchain platform. Specifically, according to the size of the cabin and the distribution range of the point cloud data, the boundary of the voxel grid is determined, and the entire three-dimensional space is divided into uniform voxels. Through the voxel grid, the redundancy of the point cloud data is effectively reduced while maintaining the details of the three-dimensional grid model; Distribute the fused point cloud data into voxels, each voxel contains multiple point cloud data, and take the average value of all point clouds in the voxel as the representative point; Use triangulation methods (such as Delaunay triangulation) to form triangular patches by connecting representative points in voxels to generate a three-dimensional mesh model; The QEM algorithm is used to optimize the 3D mesh model by minimizing the error after vertex merging. The quadratic error matrix is ​​calculated for each vertex of the triangular face to describe the geometric relationship between the vertex and its neighboring vertices. Traverse all vertex pairs, calculate the error value of each pair of vertices after merging, select the vertex pair with the smallest error to merge, and update the error matrix of adjacent vertices; It should be noted that the QEM algorithm can preserve the geometric features to the maximum extent while simplifying the model; Incremental fusion is used to fuse the newly collected data, and the ICP (Iterative Closest Point) algorithm is used to align the newly fused point cloud data with the existing 3D mesh model; Add the newly registered fused point cloud data to the voxel grid, calculate the point cloud density and point cloud geometric features within each voxel, and detect the voxels with density and geometric features changes. For example, a significant decrease in the point cloud density within a voxel indicates that the bulk cargo in that area has been removed. Prioritize updating the voxels that have been detected to have changed, and do not process the voxels that have not changed; The ship unloader will upload the real-time updated three-dimensional mesh model and working status to the blockchain platform through the control center, which will serve as an important basis for the blockchain platform to allocate tasks.

[0027] S3. Based on the blockchain platform, tasks are assigned to the ship unloader.

[0028] The port dispatch center uploads the cargo ship arrival plan and bulk cargo list to the blockchain platform; The blockchain platform assigns unloading tasks to each ship unloader based on the real-time 3D mesh model and real-time working status uploaded by the ship unloader; Among them, the unloading task includes unloading priority, ship name, bulk cargo type (such as coal, ore, grain, etc.), unloading volume, estimated operation time, and unloader ID and number; The generated unloading task will be checked through a series of validation rules to ensure that all fields meet the predefined standards, such as the unloading quantity cannot exceed the maximum deadweight of the ship, the operation time window cannot conflict with other tasks, etc. It is sent to the ship unloader control center through a secure channel (such as TLS / SSL), and the ship unloader automatically unloads the ship according to the received unloading task.

[0029] S4. Generate the unloading path based on the 3D mesh model and the assigned tasks.

[0030] Mark the inaccessible voxels according to the cargo distribution, cabin structure and obstacle locations in the 3D grid model; According to the unloading tasks issued by the blockchain platform to the ship unloader, the unloading path is planned using the hybrid path planning algorithm of A-Star combined with deep reinforcement learning (DRL); The hybrid path planning algorithm uses the Euclidean distance as a heuristic function to estimate the cost from the initial position of the ship unloader grab to the target position. The calculation formula is as follows: ; in, is the Euclidean distance from the grab bucket of the ship unloader to the target position, are the three-dimensional coordinate values ​​of the ship unloader grab in the three-dimensional grid model, The three-dimensional coordinate values ​​of the target positions in the three-dimensional grid model respectively; According to the voxel division in the three-dimensional grid model, starting from the voxel where the grab bucket of the ship unloader is located, the adjacent voxels are gradually expanded, and the inaccessible voxels are bypassed until the voxel where the target position is located is reached; Based on the minimum cost principle, the voxel with the minimum cost is selected for expansion along the target direction; When the target position is found, the route from the target position to the initial position of the ship unloader grab is traced in reverse to generate the initial path.

[0031] S5. Update the unloading path according to the change of bulk cargo status in the cargo ship during the operation.

[0032] Based on the real-time updated 3D mesh model, the unloader grab re-marks the inaccessible voxels every time it grabs bulk cargo; When the cargo ship just arrives at the port, the bulk cargo in the cabin is evenly piled up, and the grab bucket of the ship unloader can grab the cargo randomly. After grabbing, the surface of the bulk cargo pile will fluctuate. If the cargo is grabbed at one place, it may affect the center of gravity of the cargo ship and cause the ship to capsize. At the same time, it is not conducive to the unloading efficiency. According to the height coordinates of bulk cargo in the 3D grid model, the bulk cargo accumulation state is judged, and the position with the highest height coordinate is preferentially set as the target position of the grab bucket of the ship unloader. Estimate the cost from the initial position of the ship unloader grab to the target position, update the initial path, and generate a new unloading path.

[0033] S6. Coordinate the unloading paths of each ship unloader in real time according to the location and status of different ship unloaders.

[0034] The ship unloader control center uploads the real-time unloading path information to the port dispatch center at a fixed frequency (e.g., once per second); The unloading path information includes the current position of the unloader, the target position, the unloading path, the activation time of the unloading path, and the estimated arrival and return time; According to the maximum opening radius of the grab buckets of adjacent ship unloaders, the safety spacing threshold between the grab buckets of the ship unloaders is set to twice the maximum radius of the grab buckets; Set the safety time threshold according to the rotation speed of the ship unloader and the collection speed of the ship unloader grab; Furthermore, the real-time unloading path information is summarized, and the real-time unloading paths of adjacent ship unloaders are cross-compared. When the distance between the grab buckets of adjacent ship unloaders is less than the safety distance threshold, it is determined that there is a collision risk, otherwise there is no collision risk; The time cross-comparison of two unloading paths with collision risk is carried out. When the time difference between the grab buckets of adjacent unloaders reaching the collision risk position is less than the safety time threshold, it is confirmed that there is a collision risk. Otherwise, there is no collision risk. The port dispatch center issued a safety warning notice to the control center of the ship unloader with a collision risk, and one ship unloader stopped moving and started again after the other ship unloader passed the collision risk position; Specifically, by comparing the path lengths from the grab buckets of the two ship unloaders to the collision risk position, the ship unloader with the longer path is paused and the ship unloader with the shorter path is allowed to pass after the grab bucket has passed, to ensure safety and efficiency. When the unloading path is updated, the voxels where there is a collision risk are marked as inaccessible voxels.

[0035] This embodiment also provides a bulk cargo terminal automatic grab ship unloader ship unloading system, including: a data acquisition module, a model building module, a task allocation module, a path generation module, a path update module and a path coordination module. Data acquisition module, used to collect data on the cabin structure and bulk cargo distribution using lidar, cameras and ultrasonic sensors; The model building module is used to fuse the data collected by the lidar, camera and ultrasonic sensor to generate a three-dimensional mesh model; The task allocation module is used to allocate tasks to the ship unloader based on the blockchain platform; A path generation module, used to generate a unloading path based on a three-dimensional mesh model and assigned tasks; A path updating module is used to update the unloading path according to the change of bulk cargo status in the cargo ship during the operation; The path coordination module is used to coordinate the unloading paths of each ship unloader in real time according to the position and status of different ship unloaders.

[0036] This embodiment also provides a computer device, which is applicable to the case of a ship unloading method of an automatic grab ship unloader at a bulk cargo terminal, comprising: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the ship unloading method of the automatic grab ship unloader at a bulk cargo terminal as proposed in the above embodiment.

[0037] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0038] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for unloading a ship by an automatic grab ship unloader at a bulk terminal as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a disk or an optical disk.

[0039] In summary, the present invention achieves high-precision environmental perception through the fusion of lidar, camera and ultrasonic sensor, ensures the accuracy of the three-dimensional grid model, and provides reliable basic data for subsequent path planning. The task allocation system based on blockchain avoids human intervention and operational errors, improves the fairness and traceability of task allocation, and the hybrid path planning algorithm automatically generates unloading paths in static and dynamic environments to adapt to changes in cargo distribution and avoid operation delays caused by path failure. The real-time path update mechanism dynamically adjusts the path according to the latest cargo distribution, avoids repeated grabbing and unnecessary movement, and significantly improves the unloading efficiency. Through the global coordination of the port dispatching center, the collaborative operation of multiple ship unloaders is realized, conflicts and collisions are avoided, and the overall unloading efficiency and safety are further improved.

[0040] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A bulk cargo terminal automatic grab ship unloader ship unloading method, characterized in that: include, Use lidar, cameras and ultrasonic sensors to collect data on the ship's cabin structure and bulk cargo distribution; Generate a 3D mesh model by fusing data collected by lidar, camera and ultrasonic sensor; Based on the blockchain platform, tasks are assigned to the ship unloader; Generate unloading paths based on the 3D mesh model and assigned tasks; Update the unloading path according to the change of bulk cargo status in the cargo ship during the operation; According to the location and status of different ship unloaders, the unloading paths of each ship unloader are coordinated in real time.

2. The ship unloading method of the automatic grab ship unloader at a bulk cargo terminal according to claim 1, characterized in that: The laser radar, camera and ultrasonic sensor are used to collect data on the cabin structure and bulk cargo distribution. The specific steps are as follows: Use a checkerboard calibration board to collect data from the lidar and camera at the same time for joint calibration; The external parameter matrix between the lidar and the camera is calculated through the calibration algorithm; Use ultrasonic sensors to scan standard objects of known distance for distance calibration; The laser radar scans the cabin at all angles at a fixed frequency to collect point cloud data; The voxel filtering method is used to filter the collected point cloud data to remove noise points and outliers; While the LiDAR is scanning, the camera simultaneously collects image data of the cabin and uses a deep learning algorithm to segment the image to identify bulk cargo, cabin walls and other obstacles. The ultrasonic sensor collects the distance data between the cabin and the ultrasonic sensor to monitor the position of obstacles around the ship unloader in real time, and uses moving average filtering to smooth the distance data.

3. The ship unloading method of the automatic grab ship unloader at a bulk cargo terminal according to claim 2, characterized in that: The three-dimensional mesh model is generated by fusing the data collected by the laser radar, camera and ultrasonic sensor. The specific steps are as follows: Using the hardware synchronization method, the timestamps of the processed point cloud data, image data, and distance data are aligned; Based on the external parameter matrix, align the image data collected by the camera with the point cloud data of the lidar; Each pixel in the image corresponds to a point cloud data, and the point cloud data with image information is obtained; The positions of obstacles around the ship unloader monitored by the ultrasonic sensor are converted into three-dimensional points and synchronized with the point cloud data with image information to obtain fused point cloud data; Rasterize the fused point cloud data and divide the entire three-dimensional space into several voxels; By connecting the representative points in the voxels, triangular patches are formed to generate a three-dimensional mesh model; The generated three-dimensional mesh model is optimized using the QEM algorithm; By continuously collecting new fused point cloud data and integrating it with the existing 3D mesh model, the 3D mesh model is updated in real time and uploaded to the blockchain platform; The ship unloader will upload the updated three-dimensional mesh model and working status to the blockchain platform in real time through the control center.

4. The ship unloading method of the automatic grab ship unloader at a bulk cargo terminal according to claim 3, characterized in that: The specific steps of assigning tasks to the ship unloader based on the blockchain platform are as follows: The port dispatch center uploads the cargo ship arrival plan and bulk cargo list to the blockchain platform; The blockchain platform assigns unloading tasks to each ship unloader based on the real-time 3D mesh model and real-time working status uploaded by the ship unloader; The information is sent to the ship unloader control center through the safety channel, and the ship unloader automatically unloads the ship according to the received unloading task.

5. The ship unloading method of the automatic grab ship unloader at a bulk cargo terminal according to claim 4, characterized in that: The specific steps of generating the unloading path based on the three-dimensional grid model and the assigned tasks are as follows: Mark the inaccessible voxels according to the cargo distribution, cabin structure and obstacle locations in the 3D grid model; According to the unloading tasks issued by the blockchain platform to the ship unloader, the unloading path is planned using the hybrid path planning algorithm of A-Star combined with deep reinforcement learning; The hybrid path planning algorithm uses the Euclidean distance as a heuristic function to estimate the cost from the initial position of the ship unloader grab to the target position; According to the voxel division in the three-dimensional grid model, starting from the voxel where the grab bucket of the ship unloader is located, the adjacent voxels are gradually expanded, and the inaccessible voxels are bypassed until the voxel where the target position is located is reached; Based on the minimum cost principle, the voxel with the minimum cost is selected for expansion along the target direction; When the target position is found, the route from the target position to the initial position of the ship unloader grab is traced in reverse to generate the initial path.

6. The ship unloading method of the automatic grab ship unloader at a bulk cargo terminal according to claim 5, characterized in that: The specific steps of updating the unloading path by the change of bulk cargo state in the cargo ship during the operation are as follows: Based on the real-time updated 3D mesh model, the unloader grab re-marks the inaccessible voxels every time it grabs bulk cargo; According to the height coordinates of bulk cargo in the 3D grid model, the bulk cargo accumulation state is judged, and the position with the highest height coordinate is preferentially set as the target position of the grab bucket of the ship unloader. Estimate the cost from the initial position of the ship unloader grab to the target position, update the initial path, and generate a new unloading path.

7. The ship unloading method of the automatic grab ship unloader at a bulk cargo terminal according to claim 6, characterized in that: The specific steps of coordinating the unloading paths of the ship unloaders in real time according to the positions and states of the ship unloaders are as follows: The ship unloader control center uploads the real-time unloading path information to the port dispatch center at a fixed frequency; According to the maximum opening radius of the grab buckets of adjacent ship unloaders, the safety spacing threshold between the grab buckets of the ship unloaders is set; Set the safety time threshold according to the rotation speed of the ship unloader and the collection speed of the ship unloader grab; The real-time unloading path information is summarized, and the real-time unloading paths of adjacent ship unloaders are cross-compared. When the distance between the grab buckets of adjacent ship unloaders is less than the safety distance threshold, it is determined that there is a collision risk, otherwise there is no collision risk; The time cross-comparison of two unloading paths with collision risk is carried out. When the time difference between the grab buckets of adjacent unloaders reaching the collision risk position is less than the safety time threshold, it is confirmed that there is a collision risk. Otherwise, there is no collision risk. The port dispatch center issued a safety warning notice to the control center of the ship unloader with a collision risk, and one ship unloader stopped moving and started again after the other ship unloader passed the collision risk position; When the unloading path is updated, the voxels where there is a collision risk are marked as inaccessible voxels.

8. A bulk cargo terminal automatic grab bucket ship unloader ship unloading system, based on the bulk cargo terminal automatic grab bucket ship unloader ship unloading method according to any one of claims 1 to 7, characterized in that: It includes data acquisition module, model building module, task allocation module, path generation module, path update module and path coordination module. The data acquisition module is used to collect data on the cabin structure and bulk cargo distribution using a laser radar, a camera and an ultrasonic sensor; The model building module is used to fuse the data collected by the laser radar, camera and ultrasonic sensor to generate a three-dimensional grid model; The task allocation module is used to allocate tasks to the ship unloader based on the blockchain platform; The path generation module is used to generate a discharge path based on the three-dimensional grid model and the assigned tasks; The path updating module is used to update the unloading path according to the change of the bulk cargo state in the cargo ship during the operation; The path coordination module is used to coordinate the unloading paths of the ship unloaders in real time according to the positions and states of the different ship unloaders.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the ship unloading method of the automatic grab ship unloader for bulk cargo terminal described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the ship unloading method of the automatic grab ship unloader for a bulk cargo terminal as described in any one of claims 1 to 7 are implemented.

Images