Ship-engine-shore collaborative positioning method, device and electronic equipment for bulk carrier tank clearing operations

Abstract

The present invention relates to a ship-machine-shore collaborative positioning method, device and electronic equipment for bulk carrier hold clearing operations. The method comprises: obtaining three-dimensional hatch data of the bulk carrier and first position data of the loader relative to the bulk carrier hatch through the sensor of the ship unloader, and obtaining a loader coordinate system and a hatch coordinate system; obtaining cabin space data of the bulk carrier through the sensor of the loader, and obtaining cabin point cloud data through laser radar point cloud processing; obtaining a local map of the cabin and local positioning information based on the cabin point cloud data, and sending the local map and local positioning information to the ship unloader control end; obtaining second position data of the ship unloader sensor relative to the hatch, and obtaining a ship unloader coordinate system; unifying the ship unloader coordinate system and the loader coordinate system through the control end of the ship unloader, obtaining relative position information between the associated ship unloader and loader, and obtaining collaborative positioning information of the ship unloader and loader.

Description

Technical Field

[0001] The present invention relates to the technical field of automated operations for bulk carriers in ports, and in particular to a ship-engine-shore collaborative positioning method, device and electronic equipment for bulk carrier tank clearing operations. Background Art

[0002] A gantry grab ship unloader is a lifting and transporting machine used to load and unload materials from bulk carriers and containers during bulk carrier unloading operations at ports. A gantry crane is a crucial shoreside equipment for port operations. Its working process involves using a grab bucket to unload bulk cargo (coal, grain, etc.) from a bulk carrier to a designated area on the dock, or to perform reverse loading operations. Due to the unloader driver's perspective, it is difficult to understand the status of the ship's hold during loading operations. This typically requires installing a camera above the grab bucket to increase the driver's field of view, or having a conductor on the bulk carrier guide the driver's operations. Both methods clearly require manual operation. To adapt to the rapid growth of world trade, the fully manual and semi-manual, semi-automatic operating modes of terminal ship unloaders no longer meet the requirements for automated and efficient operations. Achieving fully automated operation of large bulk carriers and improving the intelligence of equipment has become a pressing research direction.

[0003] During the operation of traditional bulk carriers, since the cab is located on the landside, the operators can only see one side of the cabin. As the operation continues, the ship's draft continues to decrease, and the visual blind spot will become larger and larger. Especially during night operations and cabin cleaning stages, there are major safety hazards such as collisions.

[0004] To address these issues, Wuhan Gangdi Intelligent Technology Co., Ltd. has developed a 3D LiDAR-based method for modeling the cargo hold and materials within a bulk ship unloader. This method uses two 3D LiDARs to construct the unloader coordinate system and the cargo hold coordinate system, and then performs coordinate system conversion through point cloud registration to achieve cargo hold scene modeling. Separately, MCC Baosteel Technology Services Co., Ltd. has developed a collaborative operation method for tank cleaning machinery suitable for large bulk carriers. The method details the collaborative operation of the unloader, tank cleaning personnel, and loader. However, this method poses a significant risk of collision between personnel and loading equipment. Therefore, the existing technical solutions are lacking in the research of ship-engine-shore collaborative positioning methods and systems in bulk carrier tank cleaning scenarios. The reason is that the bulk carrier tank cleaning scenario has obvious particularities, which makes it difficult to apply general positioning methods. Specific problems include: the traditional bulk carrier tank cleaning process is guided by the operator, and the position of the ship unloader relative to the material is manually positioned by the operator, which greatly reduces the efficiency of material loading and unloading. The harsh environment in the cabin, such as low illumination and high dust and oil pollution, affects the positioning effect of the shore-based ship unloader using its own sensors to locate the material in the cabin. Although the internal information can be observed from the top of the bulk carrier cabin, the special structure of the hatch makes it impossible to achieve blind spot observation inside the cabin. Relying solely on the loader sensor or the ship unloader sensor cannot achieve blind spot-free and accurate perception and positioning during the tank cleaning operation. Therefore, the ship-engine-shore collaborative positioning method needs to be studied. In short, the research on using robots to realize autonomous bulk carrier tank cleaning is insufficient, especially the research on the collaborative positioning technology of the loader and ship unloader during autonomous tank cleaning is insufficient. The applicability has not yet been studied in combination with the characteristics of bulk carrier tank unloading and ship-engine positioning. Summary of the Invention

[0005] In view of this, it is necessary to provide a ship-engine-shore collaborative positioning method, device and storage medium for bulk carrier tank cleaning operations to solve the technical problem of ship-engine-shore collaborative positioning in bulk carrier tank cleaning scenarios.

[0006] In order to achieve the above object, the present invention provides a method comprising:

[0007] In a first aspect, the present invention provides a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations, the method being applied to a ship-engine-shore collaborative positioning system, the system comprising a ship unloader and a loader, each of the ship unloader and the loader being provided with corresponding sensors, the method comprising:

[0008] Acquire three-dimensional hatch data of the bulk carrier and first pose data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data;

[0009] Acquiring cabin space data of the bulk carrier through a sensor of the loader, processing the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtaining a local map and local positioning information within the cabin based on the cabin point cloud data;

[0010] Sending the local map and the local positioning information to the ship unloader control terminal;

[0011] acquiring second posture data of a sensor of the ship unloader relative to the hatch, and obtaining a coordinate system of the ship unloader based on the second posture data;

[0012] unifying the ship unloader coordinate system and the loader coordinate system through the control end of the ship unloader to obtain the relative position information between the ship unloader and the loader after association;

[0013] The coordinated positioning information of the ship unloader and the loader is obtained according to the relative position information.

[0014] Furthermore, the sensor on the ship unloader is a three-dimensional laser radar.

[0015] Furthermore, the sensor on the loader is a three-dimensional laser radar, and the loader is also provided with reflective stickers.

[0016] Furthermore, the reflective tape on the loader and the three-dimensional laser radar on the sensor of the loader are respectively installed on the top plane of the loader, the plane of each reflective tape on the loader completely coincides with the top plane of the loader, and the center point of the plane of each reflective tape on the loader needs to be located on a straight line, and the bottom plane of the three-dimensional laser radar on the sensor of the loader completely coincides with the top plane of the loader, and the position of the three-dimensional laser radar on the sensor of the loader is higher than the position of the reflective tape of the loader.

[0017] Furthermore, obtaining the first position data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader includes:

[0018] Acquiring the reflection degree information of the reflective tape of the loader through the sensor of the ship unloader to obtain an intensity map;

[0019] Determining, based on the intensity map, the planar position information of the reflective tape of the loader in the intensity map, obtaining the position information of the planar center point of each reflective tape of the loader, and obtaining the first pose data based on the position information of the planar center point of each reflective tape of the loader;

[0020] The expression of the degree graph is:

[0021]

[0022] Where: D is the intensity map, E1 is the preset first constant parameter, E2 is the preset second constant parameter, E3 is the preset third constant parameter, {x ij ,(i,j)}∈I 2 is the value of each point in the digital image after the digital image is processed by mean filtering, A is the filtering window, is the dimension of the intensity map.

[0023] Furthermore, obtaining three-dimensional hatch data of the bulk carrier through the sensor of the ship unloader includes:

[0024] Acquiring point cloud data of the hatch through the sensor of the ship unloader, and obtaining a plurality of plane point cloud data after segmenting the point cloud data of the hatch through a plane feature extraction method using voxel growth;

[0025] By respectively counting the plurality of plane point cloud data of the hatch, an average point density of each plane of the hatch is obtained, the average point density of each plane of the hatch is used as the projection resolution, and the three-dimensional plane data of the hatch is projected into a two-dimensional binary image, and the two-dimensional binary image is preprocessed to obtain preprocessed data;

[0026] The pre-processed data is back-projected onto the three-dimensional plane of the hatch to obtain three-dimensional data of the hatch.

[0027] Furthermore, the bulk carrier's cabin space data includes:

[0028] The structural characteristic data of the bulk carrier's interior and the material data of the bulk carrier;

[0029] The expression for the smoothness of the bulk carrier's cabin structural characteristic data is:

[0030]

[0031] Where: l is the smoothness of the bulk carrier's cabin structure characteristic data, P represents a part of the point set in a certain frame point cloud, Indicates the current point, Indicates the nearest point;

[0032] The material characteristic data of the bulk carrier is obtained by using a sample clustering method.

[0033] In a second aspect, the present invention further provides a ship-engine-shore collaborative positioning device for bulk carrier tank clearing operations, which is applied to a ship-engine-shore collaborative positioning system, wherein the system includes a ship unloader and a loader, and each of the ship unloader and the loader is provided with corresponding sensors, including:

[0034] a first acquisition module, configured to acquire three-dimensional data of the bulk carrier hatch and first pose data of the loader relative to the bulk carrier hatch through sensors of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data;

[0035] a second acquisition module, configured to acquire cabin space data of the bulk carrier through a sensor of the loader, process the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtain a local map and local positioning information within the cabin based on the cabin point cloud data;

[0036] Sending the local map and the local positioning information to the ship unloader control terminal;

[0037] a third acquisition module, configured to acquire second posture data of the sensor of the ship unloader relative to the hatch, and obtain a coordinate system of the ship unloader based on the second posture data;

[0038] an association module, configured to unify the coordinate system of the ship unloader and the coordinate system of the loader through the control terminal of the ship unloader, and obtain the relative position information between the ship unloader and the loader after association;

[0039] The collaborative positioning module is used to obtain collaborative positioning information of the ship unloader and the loader based on the relative position information.

[0040] In a third aspect, the present invention further provides an electronic device for executing the program stored in the memory to implement the steps in a ship-machine-shore collaborative positioning method for bulk carrier tank clearing operations as described in any of the above implementations.

[0041] In a fourth aspect, the present invention further provides a storage medium for storing a computer program capable of implementing the steps in a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations in any of the above-mentioned implementation methods.

[0042] The present invention provides a ship-machine-shore collaborative positioning method, device, electronic device and storage medium for bulk carrier tank cleaning operations, which are applied to a ship-machine-shore collaborative positioning system, wherein the system includes a ship unloader and a loader, and the ship unloader and the loader are both provided with corresponding sensors, including: obtaining three-dimensional data of the bulk carrier's hatch and first position data of the loader relative to the bulk carrier's hatch through the sensor of the ship unloader, obtaining a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first position data, obtaining cabin space data of the bulk carrier through the sensor of the loader, and transmitting the cabin space data through a laser beam. The radar point cloud is processed to obtain cabin point cloud data, and based on the cabin point cloud data, a local map of the cabin and local positioning information are obtained. The local map of the cabin and local positioning information are sent to the ship unloader control terminal, and the second position data of the ship unloader sensor relative to the hatch are obtained. Based on the second position data, the ship unloader coordinate system is obtained. The ship unloader control terminal unifies the ship unloader coordinate system with the loader coordinate system to obtain the relative position information between the associated ship unloader and the loader. Based on the relative position information, the coordinated positioning information of the ship unloader and the loader is obtained. Compared with the existing technology, the present invention solves the technical problem of ship-machine-shore coordinated positioning in the bulk cargo ship clearing scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A method flow chart of an embodiment of a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention;

[0045] Figure 2 A top view of a loader according to an embodiment of a ship-machine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention;

[0046] Figure 3 A structural schematic diagram of an embodiment of a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention;

[0047] Figure 4 A schematic diagram of the device structure of an embodiment of a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention;

[0048] Figure 5 This is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more.

[0051] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products or devices.

[0052] The naming or numbering of the steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0053] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0054] The present invention provides a ship-engine-shore collaborative positioning method, device, electronic equipment and storage medium for bulk carrier tank clearing operations, which are described below respectively.

[0055] Figure 1 A method flow chart of an embodiment of a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention includes:

[0056] S110, obtaining three-dimensional data of a bulk carrier hatch and first pose data of the loader relative to the bulk carrier hatch through sensors of the ship unloader, and obtaining a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data;

[0057] S120. Acquire cabin space data of the bulk carrier through a sensor of the loader, process the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtain a local map and local positioning information within the cabin based on the cabin point cloud data;

[0058] S130, sending the local map and the local positioning information to the ship unloader control terminal;

[0059] S140, acquiring second posture data of the sensor of the ship unloader relative to the hatch, and obtaining a coordinate system of the ship unloader based on the second posture data;

[0060] S150, unifying the ship unloader coordinate system and the loader coordinate system through the control terminal of the ship unloader to obtain the associated relative position information between the ship unloader and the loader;

[0061] S160. Obtaining collaborative positioning information of the ship unloader and the loader based on the relative position information.

[0062] It can be understood that, compared with the prior art, the present invention solves the technical problem of ship-engine-shore collaborative positioning in the bulk carrier tank clearing scenario.

[0063] In step S110, three-dimensional data of the bulk carrier hatch and first pose data of the loader relative to the bulk carrier hatch are obtained through sensors of the ship unloader, and a loader coordinate system and a hatch coordinate system are obtained based on the three-dimensional hatch data and the first pose data;

[0064] Furthermore, the sensor on the ship unloader and the loader is respectively a three-dimensional laser radar;

[0065] Furthermore, the three-dimensional data of the hatch is obtained based on the following steps:

[0066] The hatch point cloud data is acquired by the sensor of the ship unloader, and a plurality of plane point cloud data after the hatch point cloud data is segmented are obtained by adopting a plane feature fine extraction method of voxel growth.

[0067] On the basis of the above scheme, preferably, the planar feature fine extraction method of voxel growth includes: first, using an octree structure to segment the point cloud into a series of voxels according to the density and thickness distribution of the point cloud; second, finding seed voxels close to the planar distribution and stratifying them according to their planar distribution probability; third, obtaining the growth conditions by statistics of the growth thresholds of the voxels in each layer; finally, performing growth merging in units of voxels to obtain the final planar segmentation result.

[0068] By separately counting the multiple plane point cloud data of the hatch, the average point density of each plane of the hatch is obtained, and the average point density of each plane of the hatch is used as the projection resolution to project the three-dimensional plane data of the hatch into a two-dimensional binary image, and the two-dimensional binary image is preprocessed to obtain preprocessed data.

[0069] On the basis of the above scheme, preferably, the projected two-dimensional binary image is preprocessed by using an improved mathematical morphological operation, line segments are extracted from the binary image by using an LSD algorithm, and complete edge points of the plane projection are identified by using a simple edge determination operator.

[0070] The pre-processed data is back-projected onto the three-dimensional plane of the hatch to obtain the three-dimensional data of the hatch.

[0071] Based on the above solution, preferably, the two-dimensional line segments and edge points are reversely projected back to the three-dimensional space of the hatch, and the final edge segment extraction results are obtained through fitting calculation and optimization processes to obtain accurate three-dimensional information of the hatch coordinates.

[0072] It can be understood that multiple coordinate systems are established, including the hatch coordinate system, the ship unloader coordinate system, the grab hook coordinate system and the loader coordinate system, wherein the hatch coordinate system takes the center of the hatch as the origin, the Z axis is perpendicular to the horizontal plane and upward, the Y axis is perpendicular to the dock shoreline and pointing to the water side, and the X axis direction is determined according to the right-hand rule. The loader coordinate system coincides with the three-dimensional lidar coordinate system on the loader's sensor, the Y axis is perpendicular to the loader's moving direction, and the X axis direction is determined according to the right-hand rule. The ship unloader coordinate system coincides with the three-dimensional lidar coordinate system on the ship unloader's sensor, the Y axis always points in the direction of the boom, and the X axis direction is determined according to the right-hand rule. The three-dimensional point cloud information of the hatch is obtained by the three-dimensional lidar on the ship unloader's sensor, and the hatch coordinate system is obtained by the plane and edge straight line fitting method. Since the three-dimensional lidar on the ship unloader's sensor can observe the loader, the hatch plane can be mapped to the loader plane when the bulk carrier's hold height is known, and the lateral coordinate Xr and longitudinal coordinate Yr of the loader relative to the hatch are obtained.

[0073] Figure 2 A top view of a loader according to an embodiment of a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention includes:

[0074] The loader is also provided with two rectangular reflective stickers of different sizes;

[0075] Furthermore, the reflective tape on the loader and the three-dimensional laser radar on the loader's sensor are respectively installed on the top plane of the loader, the plane of each reflective tape on the loader completely overlaps with the top plane of the loader, and the center point of the plane of each reflective tape on the loader needs to be located on a straight line, and the bottom plane of the three-dimensional laser radar on the loader's sensor completely overlaps with the top plane of the loader, and the position of the three-dimensional laser radar on the loader's sensor is higher than the position of the reflective tape on the loader. Preferably, the reflective tape is respectively installed in the A1, B1, C1, and D1 areas and the A2, B2, C2, and D2 areas of the loader, with the center points of the reflective tape being H1 and H2 respectively. The three-dimensional laser radar on the loader's sensor is placed in the middle area, and the reflective tape is placed to block it.

[0076] Acquiring the first position data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, including:

[0077] Acquiring the reflection degree information of the reflective tape of the loader through the sensor of the ship unloader to obtain an intensity map;

[0078] Determining, based on the intensity map, the planar position information of the reflective tape of the loader in the intensity map, obtaining the position information of the planar center point of each reflective tape of the loader, and obtaining the first pose data based on the position information of the planar center point of each reflective tape of the loader;

[0079] The expression of the process intensity diagram is:

[0080]

[0081] Wherein: D is the intensity map, E1 is the preset first constant parameter, E1 value is 2, E2 is the preset second constant parameter, E2 value is 1, E3 is the preset third constant parameter, E3 value is 2, {x ij ,(i,j)}∈I 2 is the value of each point in the digital image after the digital image is processed by mean filtering, A is the filtering window, is the dimension of the intensity map.

[0082] It can be understood that when the bulk carrier reaches the final stage of clearing the hold, the loader is placed directly under the hatch by the unloader, and the three-dimensional laser radar on the sensor of the unloader scans the reflective sticker of the loader through the hatch of the bulk carrier, obtains the reflection intensity information of the two reflective stickers in front and behind, and generates the degree map; in order to solve the noise influence caused by the reflection distance and the complex environment in the bulk carrier cabin, a mean filter is used to replace the value of each point in the digital image with the mean of the grayscale values ​​of each pixel in its field to remove the noise points, and then the degree map is fitted with a fixed-scale rectangular constraint to obtain the planar position information of the reflective sticker, and then the position of the center point H1 and H2 is obtained; since the distance from the bulk carrier hatch to the bottom of the hold, the layout height of the reflective sticker on the loader, and the installation position of the three-dimensional laser radar on the loader's sensor are known, the initial position (Xr, Yr) of the loader coordinate system relative to the bulk carrier's hatch coordinate system and the coordinate transformation matrix TRL1=[R t], the yaw angle of the loader relative to the hatch of the bulk carrier is determined by the angle between the straight line H1H2 and the Y axis of the hatch coordinate system.

[0083] In step S120, the cabin space data of the bulk carrier is acquired through the sensor of the loader, the cabin space data is processed through a lidar point cloud to obtain cabin point cloud data, and a local map and local positioning information of the cabin are obtained based on the cabin point cloud data;

[0084] Furthermore, the bulk carrier's cabin space data includes:

[0085] The structural characteristic data of the bulk carrier's interior and the material data of the bulk carrier;

[0086] The expression for the smoothness of the bulk carrier's cabin structural characteristic data is:

[0087]

[0088] Where: l is the smoothness of the bulk carrier's cabin structure characteristic data, P represents a part of the point set in a certain frame point cloud, Indicates the current point, Indicates the nearest point;

[0089] The material characteristic data of the bulk carrier is obtained by a sample clustering method. Preferably, the material characteristic data is the centroid position data of the bulk carrier's material and the point set range of the bulk carrier's material.

[0090] Furthermore, a high-precision map of the bulk carrier's interior is obtained through robotic positioning and mapping technology, and the coordinates of a coordinate system centered on the hatch are output. The specific steps include:

[0091] S121, pre-processing the 3D laser point cloud. Preferably, noise points and invalid points contained in the 3D laser point cloud are removed, which are usually manifested as abnormal distance values ​​between the 3D laser radar and a certain point, or abnormal reflections on the surface of an object.

[0092] S122, extracting point cloud features in the bulk carrier hold. Preferably, the 3D laser radar scans to acquire bulk carrier hold spatial data, and extracts wire features, point and surface features, and material features of the bulk carrier hold structure.

[0093] S123. Point cloud matching in the bulk carrier hold, preferably, is performed using a method based on point-line features and point-surface features.

[0094] S124. Map representation of the bulk carrier's cabin. Preferably, since the moving direction and height of the unloader are known, and the moving height of the loader generally does not change, the present invention uses a two-dimensional grid map to represent the bulk carrier's cabin environment.

[0095] S125, updating the map in the bulk carrier's hold. Preferably, during the bulk carrier's hold cleaning process, the bulk carrier's hold material information is detected in real time, and the position update is completed in the grid map.

[0096] It can be understood that the extraction of point-line features and point-surface features is mainly represented by the smoothness of the structure in the bulk carrier cabin. If the smoothness value of the structure in the bulk carrier cabin is large, it means that the gap between the current point and the surrounding points is large, the curvature is high, and it is represented as an edge point feature. If the smoothness value of the structure in the bulk carrier cabin is small, it means that the gap between the current point and the surrounding points is small, the curvature is low, and it is represented as a plane point feature. Define the cluster sample kW of the bulk carrier's material, and then traverse all the three-dimensional lidar point cloud data. For each possible sample K, calculate the Euclidean distance between it and the kW sample. When the distance meets the threshold, it is classified as the category of the bulk carrier's material. Iteratively calculate the sample centroid position P of the category of the bulk carrier's material w (x, y, z) and the point set range r; the scale of the grid map is adjusted according to the bulk carrier's hold and the size of the bulk carrier's materials. At the same time, the grid map contains the result information of the bulk carrier, the material location, and the material range information, which facilitates obstacle avoidance of the grab bucket and path planning of the loader. In particular, since the change matrices of the hatch coordinate system and the loader coordinate system are known, the grid map coordinate system changes with the hatch coordinate system as the global coordinate.

[0097] In step S130, the local map and the local positioning information are sent to the ship unloader control terminal;

[0098] It can be understood that after the posture is initialized, the loader starts to build a high-precision map of the bulk carrier's cabin. During the collaborative operation stage, the loader updates its own positioning information in real time according to the map and sends it to the unloader control end for collaborative positioning.

[0099] In step S140, second posture data of the sensor of the ship unloader relative to the hatch is obtained, and a coordinate system of the ship unloader is obtained based on the second posture data;

[0100] It can be understood that the three-dimensional laser radar on the sensor of the ship unloader is fixed on the telescopic arm of the ship unloader, the ship unloader coordinate system can be obtained by translation of the three-dimensional laser radar coordinate system, and the three-dimensional data of the bulk carrier's hatch can be obtained through step S110. The position of the ship unloader changes with the radial extension and contraction of the connecting rod arm, so the coordinate change matrix of the ship unloader coordinate system relative to the bulk carrier's hatch coordinate system is dynamically changing, and its precise value needs to be repeatedly calculated each time the ship unloader moves. The coordinate transformation matrix TRL2 = [R2 t2] between the ship unloader coordinate system and the bulk carrier's hatch coordinate system is determined according to the absolute value of the distance moved by the ship unloader.

[0101] In step S150, the ship unloader coordinate system and the loader coordinate system are unified by the control terminal of the ship unloader to obtain the relative position information between the ship unloader and the loader after association;

[0102] It can be understood that the three-dimensional laser radar on the sensor of the ship unloader performs a 3D scan on the hatch of the bulk carrier to obtain the hatch point cloud information of the bulk carrier and the first initial position data of the loader, and the coordinate transformation matrix TRL1=[R1 t1] of the loader relative to the hatch of the bulk carrier is obtained through the mapping relationship between the hatch of the bulk carrier and the bottom of the bulk carrier, and the transformation matrix TRL2=[R2t2] of the ship unloader relative to the hatch of the bulk carrier is obtained in step S140, with the hatch coordinate system of the bulk carrier as the global coordinate, to obtain the coordinate system transformation matrix TRL=[R t] of the ship unloader and the loader.

[0103] In step S160, the coordinated positioning information of the ship unloader and the loader is obtained according to the relative position information;

[0104] It can be understood that the safe range L of movement is set according to the relative position information of the loader and the ship unloader, and the ship unloader plans a suitable path based on the safe distance and the material information in the map to avoid collision with the loader while the ship unloader performs the cabin clearing operation.

[0105] Figure 3 A schematic structural diagram of an embodiment of a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the present invention includes:

[0106] In the structural diagram of the embodiment, 1 is a grab bucket, 2 is a three-dimensional laser radar on the sensor of the ship unloader, 3 is the hatch of the bulk carrier, 4 is a three-dimensional laser radar on the sensor of the loader, 5 is the loader, and 6 is the material on the bulk carrier.

[0107] It can be understood that the three-dimensional laser radar and reflective stickers on the loader's sensor are installed on the top of the robot, and the three-dimensional laser radar on the ship unloader's sensor is installed near the telescopic arm to scan an unobstructed area of ​​the working area. The three-dimensional laser radar on the ship unloader's sensor scans downward, and the three-dimensional laser radar on the loader's sensor scans horizontally.

[0108] In order to better implement a ship-engine-shore collaborative positioning method for bulk carrier tank cleaning operations in an embodiment of the present invention, based on this method, correspondingly, please refer to Figure 4 , Figure 4 This is a schematic diagram of an embodiment of the device provided by the present invention, which is applied to a ship-engine-shore collaborative positioning system. The system includes a ship unloader and a loader. The ship unloader and the loader are both provided with corresponding sensors, including:

[0109] A first acquisition module 401 is configured to acquire three-dimensional hatch data of the bulk carrier and first pose data of the loader relative to the hatch of the bulk carrier through sensors of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data;

[0110] a second acquisition module 402 for acquiring cabin space data of the bulk carrier through sensors of the loader, processing the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtaining a local map and local positioning information of the cabin based on the cabin point cloud data;

[0111] Sending the local map and the local positioning information to the ship unloader control terminal;

[0112] A third acquisition module 403 is configured to acquire second posture data of the sensor of the ship unloader relative to the hatch, and obtain a coordinate system of the ship unloader based on the second posture data;

[0113] An association module 404 is configured to unify the coordinate system of the ship unloader and the coordinate system of the loader through the control terminal of the ship unloader to obtain the relative position information between the ship unloader and the loader after association;

[0114] The collaborative positioning module 405 is configured to obtain collaborative positioning information of the ship unloader and the loader based on the relative position information.

[0115] The ship-engine-shore collaborative positioning device for bulk carrier tank cleaning operations provided in the above embodiment can realize the technical solution described in the above embodiment of the ship-engine-shore collaborative positioning method for bulk carrier tank cleaning operations. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above embodiment of the ship-engine-shore collaborative positioning method for bulk carrier tank cleaning operations, which will not be repeated here.

[0116] like Figure 5 As shown, the present invention also provides an electronic device 500. The electronic device 500 includes a processor 501, a memory 502 and a display 503. Figure 5 Only some of the components of the electronic device 500 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0117] In some embodiments, the memory 502 may be an internal storage unit of the electronic device 500, such as a hard disk or memory of the electronic device 500. In other embodiments, the memory 502 may also be an external storage device of the electronic device 500, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 500.

[0118] Furthermore, the memory 502 may include both an internal storage unit of the electronic device 500 and an external storage device. The memory 502 is used to store application software installed in the electronic device 500 and various data.

[0119] In some embodiments, the processor 501 can be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 502, such as a ship-engine-shore collaborative positioning method for bulk carrier tank cleaning operations in the present invention.

[0120] In some embodiments, the display 503 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 503 is used to display information on the electronic device 500 and to display a visual user interface. Components 501-503 of the electronic device 500 communicate with each other via a system bus.

[0121] In some embodiments of the present invention, when the processor 501 executes the ship-engine-shore collaborative positioning program for bulk carrier tank clearing operations in the memory 502, the following steps may be implemented:

[0122] Acquire three-dimensional hatch data of the bulk carrier and first pose data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data;

[0123] Acquiring cabin space data of the bulk carrier through a sensor of the loader, processing the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtaining a local map and local positioning information within the cabin based on the cabin point cloud data;

[0124] Sending the local map and the local positioning information to the ship unloader control terminal;

[0125] acquiring second posture data of a sensor of the ship unloader relative to the hatch, and obtaining a coordinate system of the ship unloader based on the second posture data;

[0126] unifying the coordinate system of the ship unloader and the coordinate system of the loader through the control end of the ship unloader to obtain the relative position information between the ship unloader and the loader after association;

[0127] Based on the relative position information, the coordinated positioning information of the ship unloader and the loader is obtained. It should be understood that when the processor 501 executes the ship-machine-shore coordinated positioning program for bulk carrier tank clearing operations in the memory 502, in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0128] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 500 mentioned. The electronic device 500 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, or the like. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices equipped with iOS, Android, Microsoft, or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 500 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0129] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program is implemented to perform a ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations provided by the above methods. The method is applied to a ship-engine-shore collaborative positioning system, the system including a ship unloader and a loader, each of the ship unloader and the loader being provided with corresponding sensors. The method includes:

[0130] Acquire three-dimensional hatch data of the bulk carrier and first pose data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data;

[0131] Acquiring cabin space data of the bulk carrier through a sensor of the loader, processing the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtaining a local map and local positioning information within the cabin based on the cabin point cloud data;

[0132] Sending the local map and the local positioning information to the ship unloader control terminal;

[0133] acquiring second posture data of a sensor of the ship unloader relative to the hatch, and obtaining a coordinate system of the ship unloader based on the second posture data;

[0134] unifying the coordinate system of the ship unloader and the coordinate system of the loader through the control end of the ship unloader to obtain the relative position information between the ship unloader and the loader after association;

[0135] The coordinated positioning information of the ship unloader and the loader is obtained according to the relative position information.

[0136] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0137] The above is a detailed introduction to the ship-machine-shore collaborative positioning method, device, electronic equipment and storage medium for bulk carrier tank clearing operations provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A ship-engine-shore collaborative positioning method for bulk carrier tank cleaning operations, characterized in that: The method is applied to a ship-engine-shore collaborative positioning system, which includes a ship unloader and a loader. The ship unloader and the loader are both provided with corresponding sensors. The sensor on the loader is a three-dimensional laser radar, and the loader is also provided with reflective stickers. The method includes: Acquire three-dimensional hatch data of the bulk carrier and first pose data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data; Acquiring cabin space data of the bulk carrier through a sensor of the loader, processing the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtaining a local map and local positioning information within the cabin based on the cabin point cloud data; Sending the local map and the local positioning information to the ship unloader control terminal; acquiring second posture data of a sensor of the ship unloader relative to the hatch, and obtaining a coordinate system of the ship unloader based on the second posture data; unifying the ship unloader coordinate system and the loader coordinate system through the control end of the ship unloader to obtain the relative position information between the ship unloader and the loader after association; obtaining collaborative positioning information of the ship unloader and the loader according to the relative position information; Acquiring the first position data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, including: Acquiring the reflection degree information of the reflective tape of the loader through the sensor of the ship unloader to obtain an intensity map; Determining, based on the intensity map, the planar position information of the reflective tape of the loader in the intensity map, obtaining the position information of the planar center point of each reflective tape, and obtaining the first pose data based on the position information of the planar center point of each reflective tape; The expression of the intensity map is: ； Where: D is the intensity map, E1 is the preset first constant parameter, E2 is the preset second constant parameter, and E3 is the preset third constant parameter. is the value of each point in the digital image after the digital image is processed by mean filtering, A is the filter window, is the dimension of the intensity map.

2. The ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations according to claim 1, characterized in that: The sensor on the ship unloader is a three-dimensional laser radar.

3. The ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations according to claim 1, characterized in that: The reflective tape on the loader and the three-dimensional laser radar on the sensor of the loader are respectively installed on the top plane of the loader, each reflective tape plane on the loader completely coincides with the top plane of the loader, and the plane center point of each reflective tape on the loader needs to be located on a straight line, and the bottom plane of the three-dimensional laser radar on the sensor of the loader completely coincides with the top plane of the loader, and the position of the three-dimensional laser radar on the sensor of the loader is higher than the position of the reflective tape of the loader.

4. The ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations according to claim 1, characterized in that: The three-dimensional data of the bulk carrier's hatch is obtained by the sensor of the ship unloader, including: The point cloud data of the hatch is acquired by the sensor of the ship unloader, and a plane feature extraction method of voxel growth is adopted to obtain a plurality of plane point cloud data after the point cloud data of the hatch is segmented; The method comprises: obtaining an average point density of each plane of the hatch by respectively counting a plurality of plane point cloud data of the hatch, projecting the three-dimensional plane data of the hatch into a two-dimensional binary image using the average point density of each plane of the hatch as a projection resolution, and preprocessing the two-dimensional binary image to obtain preprocessed data; The pre-processed data is back-projected onto the three-dimensional plane of the hatch to obtain the three-dimensional data of the hatch.

5. The ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations according to claim 1, characterized in that: The bulk carrier's hold space data includes: The structural characteristic data of the bulk carrier's interior and the material data of the bulk carrier; The expression for the smoothness of the bulk carrier's cabin structural characteristic data is: ； Where: is the smoothness of the bulk carrier's cabin structural feature data, P represents a set of points in a certain frame of point cloud, Indicates the current point, Indicates the nearest point; The material characteristic data of the bulk carrier is obtained by using a sample clustering method.

6. A ship-engine-shore collaborative positioning device for bulk carrier tank cleaning operations, characterized in that: Applied to the ship-engine-shore collaborative positioning system, the system includes a ship unloader and a loader, both of which are equipped with corresponding sensors. The sensor on the loader is a three-dimensional laser radar, and the loader is also equipped with reflective stickers, including: a first acquisition module, configured to acquire three-dimensional data of the bulk carrier hatch and first pose data of the loader relative to the bulk carrier hatch through sensors of the ship unloader, and obtain a loader coordinate system and a hatch coordinate system based on the three-dimensional hatch data and the first pose data; a second acquisition module, configured to acquire cabin space data of the bulk carrier through a sensor of the loader, process the cabin space data through a lidar point cloud to obtain cabin point cloud data, and obtain a local map and local positioning information within the cabin based on the cabin point cloud data; Sending the local map and local positioning information in the cabin to the ship unloader control terminal; a third acquisition module, configured to acquire second posture data of the sensor of the ship unloader relative to the hatch, and obtain a coordinate system of the ship unloader based on the second posture data; an association module, configured to unify the coordinate system of the ship unloader and the coordinate system of the loader through the control terminal of the ship unloader, and obtain the relative position information between the ship unloader and the loader after association; a collaborative positioning module, configured to obtain collaborative positioning information of the ship unloader and the loader based on the relative position information; Acquiring the first position data of the loader relative to the hatch of the bulk carrier through the sensor of the ship unloader, including: Acquiring the reflection degree information of the reflective tape of the loader through the sensor of the ship unloader to obtain an intensity map; Determining, based on the intensity map, the planar position information of the reflective tape of the loader in the intensity map, obtaining the position information of the planar center point of each reflective tape, and obtaining the first pose data based on the position information of the planar center point of each reflective tape; The expression of the intensity map is: ； Where: D is the intensity map, E1 is the preset first constant parameter, E2 is the preset second constant parameter, and E3 is the preset third constant parameter. is the value of each point in the digital image after the digital image is processed by mean filtering, A is the filter window, is the dimension of the intensity map.

7. An electronic device, characterized in that: comprising a memory and a processor, wherein, The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations as described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the ship-engine-shore collaborative positioning method for bulk carrier tank clearing operations according to any one of claims 1 to 5 is implemented.

Images