Ship bulk cargo volume measurement method and electronic equipment

Through a combination of deep learning and SLAM algorithm, the hatch reference plane is automatically identified and three-dimensional point cloud data is optimized, which solves the difficulty of bulk cargo volume measurement in the cabin and achieves high-precision and efficient volume measurement.

CN118936318APending Publication Date: 2024-11-12SHANGHAI DAHUA SURVEYING & MAPPING TECHNOLOGY CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411413249.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

There are difficulties in measuring bulk cargo volume in the cabin, especially because the hatch position is difficult to determine and the bulk cargo volume is difficult to obtain, and the traditional measurement methods are greatly affected by the irregular movement of the ship, resulting in low accuracy.

Method used

Deep learning algorithm is used to automatically identify the target ball heart fit the hatch reference plane, and obtain three-dimensional point cloud data through dual-globe laser scanning device and SLAM algorithm. Combined with factor graph maintenance and closed-loop detection, optimized processing is carried out to determine bulk volume .

Benefits of technology

It realizes accurate measurement of bulk cargo volume in the cabin, reduces measurement errors, improves measurement accuracy and efficiency, and solves the problems of difficulty in determining hatch position and obtaining cargo volume.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118936318A_ABST
    Figure CN118936318A_ABST
Patent Text Reader

Abstract

The invention provides a ship bulk cargo volume measurement method and electronic equipment. The method comprises the following steps: acquiring a hatch reference plane of a ship; obtaining three-dimensional point cloud data of the bulk cargo, and optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; and determining the volume of the bulk cargo according to the hatch reference plane and the optimized three-dimensional point cloud data. According to the scheme provided by the invention, the target ball is arranged at the cabin opening corner, the cabin opening reference plane is automatically identified and fitted by adopting a deep learning method, and the three-dimensional point cloud data is analyzed and managed by utilizing professional measurement software, so that the problems that the position of the cabin opening is difficult to determine and the bulk cargo volume is difficult to measure are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of ship transportation, and in particular to a method and electronic equipment for measuring the volume of bulk cargo on a ship. Background Art

[0002] Bulk cargoes such as sand and grain are usually delivered to the destination by bulk carriers. How to quickly and accurately measure the volume of bulk cargo is one of the keys to controlling costs in the project. Accurately judging the location of bulk cargo in the cabin is also related to transportation efficiency and safety. Bulk cargoes in the cabin are stacked in different shapes and sizes, and the environment is relatively complex. In addition, since the bulk carrier being measured has a deep draft, it is not realistic to measure it by berthing on the shore. It is necessary to rely on a transfer ship to measure on the water. Therefore, the measurement results are greatly affected by the irregular movement of the two ships, which is easy to cause scanning matching errors. At present, it is difficult to accurately measure the volume of bulk materials in the cabin. Summary of the invention

[0003] The embodiments of the present application provide a method and electronic equipment for measuring the bulk cargo volume of a ship, which solves the problem that the hatch position is difficult to determine and the bulk cargo volume is difficult to measure.

[0004] An embodiment of the present application provides a method for measuring the volume of bulk cargo on a ship, comprising the following steps: obtaining a hatch reference plane of the ship; obtaining three-dimensional point cloud data of the bulk cargo, and optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; and determining the volume of the bulk cargo based on the hatch reference plane and the optimized three-dimensional point cloud data.

[0005] In one embodiment, the obtaining of the hatch reference plane of the ship comprises: based on target balls arranged at the hatch corners of the ship, using a deep learning algorithm to perform recognition training on target balls of different shapes to automatically identify the center of the target balls; and performing fitting based on the center of the target balls to obtain the hatch reference plane.

[0006] In one embodiment, the obtaining of the three-dimensional point cloud data of the bulk cargo includes: obtaining point cloud information of the bulk cargo through a dual-pan laser scanning device; using a simultaneous positioning and map building SLAM algorithm to establish a three-dimensional topographic map of the environment based on the point cloud information; in the process of establishing the three-dimensional topographic map of the environment, correcting the estimated position and posture of the bulk cargo to obtain the three-dimensional point cloud data of the bulk cargo.

[0007] In one embodiment, the dual-pan laser scanning device includes a sensor scanning device and a mobile control device; the sensor scanning device includes a sensor integrated control box and a laser radar, and the laser radar is installed inside the sensor integrated control box; the mobile control device includes a control box fixed base, a slider base and a horizontal guide rail, the sensor integrated control box is installed on the control box fixed base, the control box fixed base is installed on the slider base, and the slider base is installed on the horizontal guide rail; the slider base includes a slide rail servo motor, the end of the slide rail servo motor has a gear, which is meshed with the rack on the horizontal guide rail, and the slide rail servo motor drives the gear to rotate on the rack so that the slider base moves horizontally on the horizontal guide rail; the control box fixed base includes a horizontal base servo motor and a vertical base servo motor.

[0008] In one embodiment, the point cloud information of the bulk cargo is obtained by using a dual-pan laser scanning device, including: driving the sensor integrated control box to rotate around the vertical axis through the horizontal base servo motor, so that the laser radar obtains point cloud information with an increased horizontal field of view angle; driving the sensor integrated control box to rotate around the horizontal axis through the vertical base servo motor, so that the laser radar obtains point cloud information with an increased vertical field of view angle; using the point cloud information with the increased horizontal field of view angle and the point cloud information with the increased vertical field of view angle as the point cloud information of the bulk cargo.

[0009] In one embodiment, the SLAM algorithm is used to establish a three-dimensional topographic map of the environment based on the point cloud information, including: processing each frame of point cloud information to obtain feature points, feature lines and feature surfaces; comparing the coordinate changes of the same feature points in the point cloud information of two adjacent frames, and obtaining the three-dimensional spatial relative displacement and relative three-dimensional attitude angle change between the point cloud information of two adjacent frames based on the laser scanning visual measurement principle; through coordinate system conversion, the point cloud information of the latter frame is aligned with the point cloud information of the previous frame. Through continuous alignment processing, the point cloud information of each frame can be aligned in the reference coordinate system of the starting point of the laser scanning measurement to obtain a three-dimensional topographic map of the environment.

[0010] In one embodiment, in the process of establishing the three-dimensional topographic map of the environment, the pose estimation value of the bulk cargo is corrected to obtain the three-dimensional point cloud data of the bulk cargo, including: determining the initial three-dimensional point cloud data of the bulk cargo according to the three-dimensional topographic map of the environment; and correcting the pose estimation value of the bulk cargo according to the initial three-dimensional point cloud data of the bulk cargo by utilizing factor graph maintenance, lidar failure detection and closed-loop detection to obtain the three-dimensional point cloud data of the bulk cargo.

[0011] In one embodiment, the optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: performing a denoising operation and a density increasing operation on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

[0012] In one embodiment, the optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: performing translation and rotation operations on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

[0013] In one embodiment, the optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: performing a refined editing operation and a data removal operation on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

[0014] An embodiment of the present application further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the above-mentioned method for measuring the bulk cargo volume of a ship.

[0015] The solution provided in the above-mentioned embodiment of the present application arranges target balls in the corners of the hatch, uses a deep learning method to automatically identify and fit the hatch reference plane, and uses professional measurement software to analyze and manage the three-dimensional point cloud data, thereby solving the problems of difficulty in determining the hatch position and difficulty in measuring bulk cargo. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the embodiments of the present application, the drawings required for use in the embodiments of the present application are briefly introduced below.

[0017] Figure 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application; Figure 2 It is a flow chart of a method for measuring the volume of bulk cargo on a ship provided in an embodiment of the present application; Figure 3 It is a top view schematic diagram of the scanning and mapping stage provided in an embodiment of the present application; Figure 4 It is a side view schematic diagram of the scanning and mapping stage provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0019] Similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0020] As mentioned above, bulk cargoes such as sand and grain are generally delivered to the destination by bulk carriers, but it is currently difficult to accurately measure the volume of bulk cargoes in the cabin. The traditional measurement method usually fixes sensors such as lidar on the truss of the transfer ship platform, but the sensor has a small scanning range, there are blind spots at both ends of the cabin and where the bulkhead is blocked, and it is easily affected by the irregular movement of the ship, and the accuracy is low. Or manual handheld lidar is used for measurement, but it requires going back and forth between the two ships, which is high-risk and time-consuming and labor-intensive, and it is especially inconvenient to carry out measurement work at night.

[0021] In order to overcome the above-mentioned bulk cargo scanning and measurement difficulties in the cabin, the embodiment of the present application proposes a method for measuring the bulk cargo volume of a ship. By integrating the laser radar sensor scanning and measurement module with the horizontal guide rail, the dual rotating pan-tilt platform, etc., and installing it on the transfer ship truss, it is possible to extract point cloud information in a large range and without blind spots. By adopting the SLAM algorithm, the ship's motion posture is compensated in real time to achieve high-precision three-dimensional point cloud data extraction. The hatch reference plane is fitted by a deep learning method, and professional measurement software analyzes and manages the three-dimensional point cloud data to achieve accurate measurement of the bulk cargo volume.

[0022] Figure 1 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 100 can be used to execute the method for measuring the bulk volume of a ship provided in an embodiment of the present application. Figure 1 As shown, the electronic device 100 includes: one or more processors 102, and one or more memories 104 storing instructions executable by the processors. The processor 102 is configured to execute the method for measuring the volume of bulk cargo of a ship provided in the following embodiment of the present application.

[0023] The processor 102 can be a gateway, a smart terminal, or a device including a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or instruction execution capabilities. It can process data of other components in the electronic device 100 and control other components in the electronic device 100 to perform desired functions.

[0024] The memory 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may run the program instructions to implement the method for measuring the bulk volume of a ship described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0025] In one embodiment, Figure 1 The electronic device 100 shown may also include an input device 106, an output device 108, and a data acquisition device 110, which are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). Figure 1 The components and structures of the electronic device 100 shown are merely exemplary and non-limiting. The electronic device 100 may also have other components and structures as required.

[0026] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, etc. The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, etc. The data acquisition device 110 may acquire hatch reference planes, three-dimensional point cloud data, etc., and store the acquired data in the memory 104 for use by other components. Exemplarily, the data acquisition device 110 may be a camera, a sensor, etc.

[0027] In one embodiment, the various components in the example electronic device 100 for implementing the method for measuring the volume of bulk cargoes of a ship according to an embodiment of the present application may be integrated or dispersed, such as integrating the processor 102, the memory 104, the input device 106 and the output device 108 into one, while separating the data acquisition device 110.

[0028] In one embodiment, the exemplary electronic device 100 for implementing the method for measuring the bulk volume of a ship according to an embodiment of the present application may be implemented as an intelligent terminal such as a smart phone, a tablet computer, a desktop computer, a server, a shipboard device, or the like.

[0029] Figure 2 FIG. 1 is a flow chart of a method for measuring the bulk cargo volume of a ship according to an embodiment of the present application. Figure 2 As shown, the method includes the following steps 210-230.

[0030] Step 210: Acquire the hatch reference plane of the ship.

[0031] According to the target balls arranged at the hatch corners of the ship, a deep learning algorithm is used to perform recognition training on target balls of different shapes to realize automatic recognition of the target ball center; according to the target ball center, fitting is performed to obtain the hatch reference plane.

[0032] Step 220: Acquire the three-dimensional point cloud data of the bulk cargo, and optimize the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

[0033] With respect to obtaining the three-dimensional point cloud data of the bulk cargo, the method specifically includes: obtaining the point cloud information of the bulk cargo through a dual-pan laser scanning device; using a simultaneous positioning and map construction (SLAM) algorithm to establish a three-dimensional topographic map of the environment according to the point cloud information; in the process of establishing the three-dimensional topographic map of the environment, correcting the estimated value of the posture of the bulk cargo to obtain the three-dimensional point cloud data of the bulk cargo.

[0034] The dual-pan laser scanning device includes a sensor scanning device and a mobile control device; the sensor scanning device includes a sensor integrated control box and a laser radar, and the laser radar is installed inside the sensor integrated control box; the mobile control device includes a control box fixed base, a slider base and a horizontal guide rail, the sensor integrated control box is installed on the control box fixed base, the control box fixed base is installed on the slider base, and the slider base is installed on the horizontal guide rail; the slider base includes a slide rail servo motor, the end of the slide rail servo motor has a gear, which is engaged with the rack on the horizontal guide rail, and the slide rail servo motor drives the gear to rotate on the rack so that the slider base moves horizontally on the horizontal guide rail; the control box fixed base includes a horizontal base servo motor and a vertical base servo motor.

[0035] The method of obtaining the point cloud information of the bulk cargo by using a dual-pan laser scanning device includes: driving the sensor integrated control box to rotate around the vertical axis by using the horizontal base servo motor, so that the laser radar obtains point cloud information with an increased horizontal field of view angle; driving the sensor integrated control box to rotate around the horizontal axis by using the vertical base servo motor, so that the laser radar obtains point cloud information with an increased vertical field of view angle; and using the point cloud information with the increased horizontal field of view angle and the point cloud information with the increased vertical field of view angle as the point cloud information of the bulk cargo.

[0036] Specifically, the sensor scanning device includes a sensor integrated control box and a sensor. The sensor integrated control box contains some sensors and can play a certain role in water splashing. The laser radar (such as a multi-line laser radar) is installed and fixed inside the sensor integrated control box to ensure that the radar scanning field of view is not blocked. It is used to send lasers into the cabin and measure the distance and reflection intensity information of the laser emission direction during the movement, and obtain the point cloud information of the bulk cargo pile in the cabin. The sensor integrated control box contains a control unit in various forms such as an industrial computer or a development board. The laser radar transmits data to the control unit through a cable. The sensor integrated control box has a communication module inside, which is responsible for transmitting the sensor information received by the control unit and the positioning information obtained after it runs the algorithm to the remote control device. The transmission method can be through a cable connection or wireless network transmission. The sensor integrated control box has a power module inside, which can be a built-in rechargeable battery or powered by the ship's cable to independently power the control unit and the laser radar.

[0037] The mobile control device includes a control box fixed base and a horizontal guide rail. The sensor integrated control box is installed on the control box fixed base. The control box fixed base is equipped with a horizontal base servo motor and a vertical servo motor. The horizontal base servo motor can drive the sensor integrated control box to rotate around the vertical axis to increase the horizontal field of view of the sensor scanning; the vertical base servo motor can drive the sensor integrated control box to rotate around the horizontal axis to increase the vertical field of view of the sensor. The double base servo motors constitute a double rotating pan-tilt structure. The control box fixed base is fixed on the slider base as a whole, and the slider base is installed on the horizontal guide rail. The slider base is equipped with a slide rail servo motor. The end of the slide rail servo motor is equipped with a gear, which is meshed with the rack on the horizontal guide rail. The servo motor drives the gear to rotate on the rack, so that the slider base can be horizontally moved on the horizontal guide rail, and the control box fixed base and the sensor integrated control box can be driven to move along the horizontal guide rail. The device is generally arranged near the cabin of the bulk carrier, and its stroke can be set according to the length of the cabin. The sensor scanning device can be fixed on the mobile control device.

[0038] The SLAM algorithm is used to establish a three-dimensional topographic map of the environment based on the point cloud information, including: processing each frame of point cloud information to obtain feature points, feature lines and feature surfaces; comparing the coordinate changes of the same feature points in the point cloud information of two adjacent frames, and obtaining the three-dimensional spatial relative displacement and relative three-dimensional posture angle change between the point cloud information of two adjacent frames based on the laser scanning visual measurement principle; through coordinate system conversion, the point cloud information of the latter frame is aligned with the point cloud information of the previous frame, and through continuous alignment processing, the point cloud information of each frame can be aligned in the reference coordinate system of the starting point of the laser scanning measurement to obtain a three-dimensional topographic map of the environment.

[0039] In the process of establishing the three-dimensional topographic map of the environment, the estimated value of the posture of the bulk cargo is corrected to obtain the three-dimensional point cloud data of the bulk cargo, including: determining the initial three-dimensional point cloud data of the bulk cargo according to the three-dimensional topographic map of the environment; and correcting the estimated value of the posture of the bulk cargo according to the initial three-dimensional point cloud data of the bulk cargo by using factor graph maintenance, laser radar failure detection and closed-loop detection to obtain the three-dimensional point cloud data of the bulk cargo.

[0040] The optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: performing a denoising operation and a density increasing operation on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data. Performing a translation operation and a rotation operation on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data. Performing a refined editing operation and a data culling operation on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

[0041] Step 203: Determine the volume of the bulk cargo according to the hatch reference plane and the optimized three-dimensional point cloud data.

[0042] In another embodiment of the present application, the method for measuring the volume of bulk cargo on a ship includes three processes, namely, a positioning stage, a scanning and mapping stage, and a volume measurement stage. The three stages are specifically as follows: Phase 1: Positioning phase.

[0043] According to the actual size of the bulk carrier cabin being tested, the algorithm's scanning range and field of view are set to limit the scanning range to the bulk material in the cabin, and try to avoid irrelevant interference from the hull. According to the actual size of the wind and waves, the algorithm's stability is adjusted.

[0044] Turn on the LiDAR, and the laser sensor acquires the geometric information of the environment and performs motion estimation. Enable the multi-line laser SLAM algorithm to process the point cloud data of the LiDAR to achieve autonomous positioning and map construction. The algorithm as a whole includes the LiDAR odometer module.

[0045] Initial value estimation. The initial value is very important for continuous scan matching, especially in scenes with violent motion or irregular trajectories. Accurate initial values ​​can significantly improve the success rate of subsequent inter-frame matching. At the same time, inter-frame matching is also the most time-consuming step in the lidar odometry module. Before the lidar odometry system is initialized, the position and attitude information of the two radar key frames are used as the initial values ​​for inter-frame matching. This method can initialize the system for scenes with small initial velocity and angular velocity. This process can improve the accuracy and robustness of initial value prediction in both texture-rich and texture-poor environments, while increasing the speed of inter-frame matching.

[0046] Initialization. The LiDAR odometer system uses the first frame of information received as the world coordinate system, aligns the inertial system with the world coordinate system for initialization. The depth information observed by the LiDAR odometer system is used to estimate the state of the system, which can effectively improve the speed and robustness of initialization.

[0047] Factor graph maintenance. When the pose change exceeds a threshold, a new lidar keyframe is selected, and the normal lidar frames between the keyframes are discarded. After selecting a new keyframe, the new state is added to the factor graph as a node. Adding keyframes in this way not only achieves a balance between memory consumption and map density, but also maintains high efficiency, which helps maintain a relatively sparse factor graph for real-time optimization.

[0048] LiDAR failure detection. Although LiDAR can obtain detailed information of scenes at a longer range, it may fail to scan and match in some flat and open environments lacking feature structures, so it needs to be detected. The nonlinear optimization problem in inter-frame matching can be expressed in the form of linear least squares. When the minimum eigenvalue is lower than the threshold of the first optimization iteration, the LiDAR odometry module will report failure, and the LiDAR odometry constraint will no longer be added to the factor graph.

[0049] Closed loop detection. When the sensor module scans the same location at different times, constraints are established to eliminate the error accumulation caused by the odometer. The algorithm uses the bag-of-words model (DBoW2) for closed loop detection, detects feature descriptors for each new image keyframe, and matches them with the original extracted descriptors. The timestamps of the closed loop candidate frames detected by DBoW2 are provided to the lidar odometer system for further verification.

[0050] After the above processing, the lidar odometer constraint and loop closure constraint are finally jointly optimized to achieve state estimation. Multi-line laser can adapt to various complex environments, effectively reduce the impact of motion distortion and environmental noise on the SLAM system, and compensate for the ship's motion posture to obtain high-precision three-dimensional point cloud scanning information.

[0051] The second stage: scanning and mapping stage.

[0052] Figure 3 This is a top-down diagram of the scanning and mapping phase. The scanning device starts working from one end of the horizontal rail. Since the length of the horizontal rail is shorter than the length of the cabin, the horizontal field of view of the laser radar is limited, and direct scanning will have blind spots. Through the remote control device, the horizontal base servo motor is started to rotate the sensor integrated control box horizontally around the axis to obtain point cloud information with a larger range and multiple angles.

[0053] Figure 4This is a side view diagram of the scanning and mapping stage. Due to the limited vertical field of view of the laser radar and the different drafts of the bulk carrier being measured, the scanning range will also change. By starting the vertical base servo motor, the sensor integrated control box rotates around the horizontal axis to obtain point cloud information with a larger range and multiple angles.

[0054] Through the remote control device, the slide rail servo motor is started, and the slider base is controlled to drive the control box fixed base and the sensor integrated control box to move and scan to the other end on the horizontal moving slide rail. During this process, the vertical base servo motor is used to rotate the sensor integrated control box around the horizontal axis to ensure that the scanning range covers the entire hatch and avoids scanning blind spots.

[0055] Repeat after the scanning device reaches the other end of the horizontal slide rail Figure 3 and Figure 4 Operations in .

[0056] The control unit acquires and stores the posture state information and laser radar point cloud information, uses the multi-line laser SLAM algorithm to establish the scanned environment map, and can monitor and watch it in real time at a remote end. The specific method is: process each frame of laser scanning point cloud to obtain feature points, lines and surfaces, compare the coordinate changes of the same feature points of two adjacent frames of point cloud, and obtain the three-dimensional space relative displacement and relative three-dimensional attitude angle change between two adjacent frames of point cloud according to the laser scanning measurement principle; through the coordinate system conversion, the laser point cloud of the next frame can be registered with the laser point cloud of the previous frame, and through continuous registration processing, each frame of laser point cloud can be registered in the reference coordinate system of the starting point of the laser scanning measurement to obtain a three-dimensional topographic map of the environment.

[0057] Compared with vision, the feature points of LiDAR point cloud are sparse, so the adaptive voxel feature extraction is realized through the octree data structure, and the optimization objective function is constructed for the entire map in a recursive way. The BA cluster optimization algorithm is used to minimize the reprojection error between features, effectively optimize the consistency of LiDAR pose and map, and thus improve the accuracy and robustness of SLAM system mapping.

[0058] In this process, the pose estimation is corrected and constraints are added to the factor graph. Factor graph maintenance, failure detection, loop detection, etc. are used to ensure the robustness of the mapping process.

[0059] The operator observes the mapping effect of the bulk material in the cabin, controls the servo motor, and repeatedly and meticulously scans and maps the blind spots through the horizontal guide rail 205 and the double rotating pan-tilt heads.

[0060] You can also choose to use a handheld sensor scanning module to carry out 3D point cloud mapping.

[0061] The third stage: volume measurement stage.

[0062] Target balls are added to the corners of the hatch, and deep learning methods are used to train the recognition of targets of different shapes, so that the center of the target ball can be automatically identified, thereby fitting the reference plane on the hatch, which is used as the reference plane for calculating the volume of full and empty cabins.

[0063] Import point cloud data into professional processing software to optimize the point cloud. Use point cloud denoising function to effectively reduce noise points and improve data quality; at the same time, enhance the point cloud density in sparse areas to ensure the integrity and accuracy of the model.

[0064] The software can automatically align different point cloud models, or manually select point cloud feature points to accurately align different point cloud models and achieve data integration. Through point cloud translation and rotation operations, as well as automatic matching and merging based on overlapping areas, a unified point cloud model of bulk materials in the cabin is constructed.

[0065] The software's point cloud selection tool can be used to perform fine editing and data removal of point clouds. At the same time, the point cloud elevation changes can be used for gradient coloring to distinguish different point cloud data, automatically identifying and displaying the height differences of bulk material stacking in the cabin.

[0066] Based on the established three-dimensional point cloud map of bulk materials and the reference plane of the cabin hatch obtained, the volume of bulk materials can be accurately estimated and the volume measurement of cargo capacity can be achieved.

[0067] The above embodiment of the present application adopts a structure that integrates the laser radar sensor scanning and measuring module with the horizontal guide rail and the dual rotating pan-tilt platform. The scanning module can be disassembled and installed separately. The device structure is simple and installed on the transfer ship truss. It can provide a large range and multi-angle scanning space, solving the problem of small range and dead angle when directly using the laser radar to extract the three-dimensional point cloud. The multi-line laser SLAM algorithm is adopted, which is suitable for various complex environments, effectively and accurately detects the target information and its own posture, and solves the problem that the ship's motion posture cannot be compensated and the three-dimensional point cloud extraction accuracy is low. The target ball is arranged in the corner of the hatch, and the deep learning method is used to automatically identify the fitting hatch reference plane. The professional measurement software is used to analyze and manage the three-dimensional point cloud data, which solves the problem that the hatch position is difficult to determine and the cargo volume is difficult to measure. Based on the above dual pan-tilt laser scanning device, a set of working methods is provided, which can realize the efficient and stable operation of the system under different environmental conditions, and perform real-time detection. It is suitable for installation on a mobile slide rail, a rotating pan-tilt platform, remote control, hand-held and other measurement methods.

[0068] In several embodiments provided in the present application, the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and a part of a module, a program segment or a code contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0069] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0070] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

Claims

1. A method for measuring the volume of bulk cargo on a ship, characterized in that: The following steps are involved: Acquiring a hatch reference plane of the ship; Acquiring three-dimensional point cloud data of the bulk cargo, and optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; The volume of the bulk cargo is determined based on the hatch reference plane and the optimized three-dimensional point cloud data.

2. The method for measuring the volume of bulk cargo on a ship according to claim 1, characterized in that: The step of obtaining the hatch reference plane of the ship comprises: According to the target balls arranged at the hatch corners of the ship, a deep learning algorithm is used to perform recognition training on target balls of different shapes to realize automatic recognition of the target ball center; The hatch reference plane is obtained by fitting according to the target sphere center.

3. The method for measuring the volume of bulk cargo on a ship according to claim 1, characterized in that: The obtaining of the three-dimensional point cloud data of the bulk cargo comprises: Obtaining point cloud information of the bulk cargo by means of a dual-pan laser scanning device; Using a simultaneous localization and mapping SLAM algorithm, a three-dimensional topographic map of the environment is established based on the point cloud information; In the process of establishing the three-dimensional topographic map of the environment, the estimated value of the position and posture of the bulk cargo is corrected to obtain the three-dimensional point cloud data of the bulk cargo.

4. The method for measuring the volume of bulk cargo on a ship according to claim 3, characterized in that: The dual-pan laser scanning device includes a sensor scanning device and a movement control device; The sensor scanning device includes a sensor integrated control box and a laser radar, and the laser radar is installed inside the sensor integrated control box; The mobile control device comprises a control box fixed base, a slider base and a horizontal guide rail, the sensor integrated control box is mounted on the control box fixed base, the control box fixed base is mounted on the slider base, and the slider base is mounted on the horizontal guide rail; The slider base includes a slide rail servo motor, the end of the slide rail servo motor is provided with a gear, which is meshed with the rack on the horizontal guide rail, and the slide rail servo motor drives the gear to rotate on the rack, so that the slider base moves horizontally on the horizontal guide rail; The control box fixed base includes a horizontal base servo motor and a vertical base servo motor.

5. The method for measuring the volume of bulk cargo on a ship according to claim 4, characterized in that: The step of obtaining point cloud information of the bulk cargo by using a dual-pan laser scanning device includes: The sensor integrated control box is driven to rotate around the vertical axis by the horizontal base servo motor, so that the laser radar obtains point cloud information with an increased horizontal field of view angle; The vertical base servo motor drives the sensor integrated control box to rotate around the horizontal axis, so that the laser radar obtains point cloud information with an increased vertical field of view; The point cloud information after the horizontal field of view angle is increased and the point cloud information after the vertical field of view angle is increased are used as the point cloud information of the bulk cargo.

6. The method for measuring the volume of bulk cargo on a ship according to claim 3, characterized in that: The method of using the SLAM algorithm to establish a three-dimensional terrain map of the environment according to the point cloud information includes: Process each frame of point cloud information to obtain feature points, feature lines and feature surfaces; Compare the coordinate changes of the same feature points in the point cloud information of two adjacent frames, and obtain the three-dimensional spatial relative displacement and relative three-dimensional posture angle change between the point cloud information of two adjacent frames based on the principle of laser scanning vision measurement; Through coordinate system conversion, the point cloud information of the latter frame is registered with the point cloud information of the previous frame. Through continuous registration processing, the point cloud information of each frame can be registered in the reference coordinate system of the starting point of the laser scanning measurement to obtain a three-dimensional topographic map of the environment.

7. The method for measuring the volume of bulk cargo on a ship according to claim 3, characterized in that: In the process of establishing the three-dimensional topographic map of the environment, the estimated value of the position of the bulk cargo is corrected to obtain the three-dimensional point cloud data of the bulk cargo, including: Determining initial three-dimensional point cloud data of the bulk cargo according to the three-dimensional topographic map of the environment; According to the initial three-dimensional point cloud data of the bulk cargo, factor graph maintenance, laser radar failure detection and closed-loop detection are used to correct the estimated value of the posture of the bulk cargo to obtain the three-dimensional point cloud data of the bulk cargo.

8. The method for measuring the volume of bulk cargo on a ship according to claim 1, characterized in that: The step of optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: A denoising operation and a density increasing operation are performed on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

9. The method for measuring the volume of bulk cargo on a ship according to claim 1, characterized in that: The step of optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: The three-dimensional point cloud data is translated and rotated to obtain optimized three-dimensional point cloud data.

10. The method for measuring the volume of bulk cargo on a ship according to claim 1, characterized in that: The step of optimizing the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: Refined editing and data elimination operations are performed on the three-dimensional point cloud data to obtain optimized three-dimensional point cloud data.

11. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to execute the method for measuring the volume of bulk cargo of a ship as described in any one of claims 1-9.

Citation Information

Patent Citations

  • High-speed and high-precision tank volume measurement method and device based on three-dimensional laser scanning

    CN104180771A

  • Box body volume measurement method and device, computer equipment and storage medium

    CN113066117A

  • Sand carrier volume metering system and method based on three-dimensional laser and intelligent remote control

    CN117537708A

  • Multi-sensor fusion three-dimensional point cloud extraction device and method for bulk material volume measurement

    CN117870803A