Methods and apparatus for calibrating the visible area of ​​a ship unloader, and a computer device.

By acquiring 3D point cloud data and 2D image data of the ship unloader, and calculating the transformation matrix using the hinge component of the grab bucket, the calibration problem of 3D point cloud data and 2D image data is solved, realizing effective information fusion and accuracy of automatic operation. It is suitable for targetless calibration and online calibration of ship unloaders.

CN123076891APending Publication Date: 2026-11-10SIEMENS AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202480090752.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-04-25
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In existing technologies, the 3D point cloud data and 2D image data of ship unloaders lack effective calibration, which makes it impossible to effectively integrate information from different dimensions, thus limiting the performance of perception algorithms and the operation of ship unloaders.

Method used

By acquiring the three-dimensional point cloud data and two-dimensional image data of the ship unloader, and using the hinge component of the grab bucket to calculate the transformation matrix, the calibration and fitting of the three-dimensional point cloud data and the two-dimensional image data are realized.

Benefits of technology

It achieves effective fusion of 3D point cloud data and 2D image data, improves information utilization and accuracy of automatic operation, supports targetless calibration and online calibration, and reduces computing costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123076891A_ABST
    Figure CN123076891A_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, computer device, and storage medium for calibrating the visible area of ​​a ship unloader. Specifically, the method includes: acquiring three-dimensional point cloud data of the grab bucket position and obtaining the three-dimensional center point of the grab bucket's hinge from the point cloud data; acquiring two-dimensional image data of the grab bucket position and obtaining the two-dimensional center point of the hinge from the two-dimensional image data; matching the three-dimensional center points of the hinge at multiple different positions moved in space with the two-dimensional center point of the hinge to calculate a transformation matrix; and fitting the three-dimensional point cloud data of the grab bucket position to the two-dimensional image data of the grab bucket position according to the transformation matrix. According to the above method, position calibration can be performed between a targetless laser sensor and an RGB camera. After calibration, the three-dimensional point cloud data of the visible area of ​​the ship unloader can be fitted to the two-dimensional image data, thereby promoting the fusion of information from different dimensions and supplementing the information content, and significantly facilitating the analysis of the area information of the ship unloader and its automated operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data calibration, and more specifically to methods and apparatus, computer devices, and storage media for calibrating the visible area of ​​a ship unloader. Background Technology

[0002] The grab position and hatch edge of a grab unloader (GSU) can be used to guide operators in loading and unloading cargo from docked cargo ships. Multimodal signals can provide rich information about the aforementioned target scenario and can adapt to various weather and lighting conditions. For perception-based automated cargo handling systems, the multimodal processor relies on precisely calibrated LiDAR sensors and red-green-blue (RGB) cameras. However, due to specific working scenarios and crane oscillations, non-interactive calibration is often required. Otherwise, calibration failures will limit the performance of subsequent perception algorithms and the operation and use of the unloader. Summary of the Invention

[0003] The content portion of this disclosure is provided in a simplified form to introduce some selected concepts, which are further described in the following detailed embodiments. This content portion is not intended to identify any key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0004] Based on this, this application discloses a method for calibrating the visible area of ​​a ship unloader, the method comprising: Obtain three-dimensional point cloud data of the visible area of ​​the ship unloader, and obtain the three-dimensional center point of the grab bucket's hinge from the three-dimensional point cloud data; Two-dimensional image data of the visible area of ​​the unloader is obtained, and the two-dimensional center point of the hinge of the grab bucket is obtained from the two-dimensional image data; The three-dimensional center point of the hinge at multiple different spatial locations is matched with the two-dimensional center point of the hinge to calculate the transformation matrix; and The three-dimensional point cloud data of the visible area of ​​the ship unloader is fitted into the two-dimensional image data according to the transformation matrix.

[0005] Based on the above method, position calibration can be performed between the targetless laser sensor and the RGB camera. After calibration, the three-dimensional point cloud data of the visible area of ​​the ship unloader can be fitted into the two-dimensional image data, thereby promoting the fusion of information from different dimensions and supplementing the information content, and significantly facilitating the analysis of the regional information of the ship unloader and its automated operation.

[0006] Further, acquiring the three-dimensional point cloud data of the visible area of ​​the unloader and obtaining the three-dimensional center point of the grab bucket's hinge from the three-dimensional point cloud data includes: The hinge position region is calculated based on the 3D point cloud data of the grab position obtained by segmentation through the object detector and the positional relationship between the hinge and the grab; and The three-dimensional center point of the hinge is calculated based on the three-dimensional point cloud data within the hinge location area.

[0007] Using the above method, the three-dimensional center point of the hinge can be accurately determined, and the three-dimensional center point can be easily matched with the corresponding center point in the subsequent two-dimensional data.

[0008] Further, acquiring two-dimensional image data of the visible area of ​​the unloader and obtaining the two-dimensional center point of the hinge of the grab bucket from the two-dimensional image data includes: Two-dimensional image data of the visible area of ​​the ship unloader is acquired from a top-down perspective, and the minimum bounding rectangle area of ​​the grab bucket is obtained from the two-dimensional image data from the top-down perspective; and Within the minimum bounding rectangle area, the two-dimensional center point of the hinge is obtained by scanning the window template.

[0009] The above method can be used to easily obtain the two-dimensional image data of the grab and the hinge to determine the two-dimensional center point of the hinge, thereby facilitating subsequent matching and calibration with the three-dimensional center point.

[0010] Further, the step of matching the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge to calculate the transformation matrix includes: The grab is moved to different positions, and the corresponding three-dimensional center point and two-dimensional center point of the hinge at those different positions are obtained; and The transformation matrix is ​​calculated by matching multiple three-dimensional center points with multiple two-dimensional center points.

[0011] By using the above method, and by collecting the three-dimensional center point and the two-dimensional center point of the hinge at multiple different locations, the calculation of the transformation matrix is ​​more accurate.

[0012] Furthermore, this application discloses a device for calibrating the visible area of ​​a ship unloader, the device comprising: A three-dimensional data module is configured to acquire three-dimensional point cloud data of the visible area of ​​the unloader and to acquire the three-dimensional center point of the grab's hinge from the three-dimensional point cloud data. A two-dimensional data module is configured to acquire two-dimensional image data of the visible area of ​​the unloader and to acquire the two-dimensional center point of the hinge of the grab bucket from the two-dimensional image data. A transformation matrix module, configured to match the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge to calculate a transformation matrix; and A data conversion module is configured to fit the three-dimensional point cloud data of the visible area of ​​the ship unloader to the two-dimensional image data according to the transformation matrix.

[0013] This application further provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the above-described method.

[0014] This application further provides a computer-readable storage medium storing a computer program, wherein the computer program performs the above-described method when executed by a processor.

[0015] This application further provides a computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions, wherein the computer-executable instructions, when executed, cause at least one processor to perform the above-described method. Attached Figure Description

[0016] Embodiments of this disclosure are shown in the accompanying drawings by way of example rather than limitation, and similar reference numerals in the drawings denote the same or similar components.

[0017] Figure 1 This is a schematic flowchart of a method for calibrating the visible area of ​​a ship unloader according to one embodiment of this application.

[0018] Figure 2 This is a schematic diagram of an apparatus for calibrating the visible area of ​​a ship unloader according to one embodiment of this application.

[0019] Figure 3 This is a schematic diagram of a computer device for calibrating the visible area of ​​a ship unloader according to one embodiment of this application.

[0020] Figure 4 This is a rendering of three-dimensional point cloud data and two-dimensional image data according to one embodiment of this application.

[0021] The attached figures are labeled as follows: Steps S101-S104 200: Equipment 201: Module 202: Module 203: Module 204: Module 300: Computer device 302: Processor 304: Memory Detailed Implementation

[0022] In the following description, numerous specific details are set forth for purposes of explanation. However, it should be understood that embodiments of this disclosure can be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to affect the understanding of the description.

[0023] Throughout this specification, references to “an embodiment,” “an embodiment,” “an exemplary embodiment,” “some embodiments,” “various embodiments,” etc., indicate that embodiments of this disclosure may include specific features, structures, or characteristics. However, this does not mean that every embodiment must include these specific features, structures, or characteristics. Furthermore, some embodiments may have some or all of the features described for other embodiments, or may not have the features described for other embodiments.

[0024] One implementation scenario of this application is as follows: For a grab bucket on a cargo ship, spatial three-dimensional point cloud data of the visible area of ​​the unloader can usually be obtained through radar, and planar two-dimensional image data of the visible area of ​​the unloader can also be obtained through a camera. Three-dimensional point cloud data is useful for presenting some spatial and depth information, and two-dimensional image data is useful for representing some length information, etc. However, there is a lack of a good calibration process between three-dimensional data and two-dimensional data. Therefore, for the same scene or object, three-dimensional data and two-dimensional data are not universally applicable and cannot be used to analyze the same target object. Therefore, this application utilizes the hinge assembly on the grab bucket to obtain a transformation matrix between the data by calibrating the three-dimensional center point and the two-dimensional center point of the hinge. Then, the transformation matrix is ​​used to correspondingly fit the remaining three-dimensional point cloud data of the visible area of ​​the unloader to the two-dimensional image data, thereby promoting the fusion of information from different dimensions and supplementing the information content, and significantly promoting the analysis of the regional information of the unloader and the automatic operation.

[0025] like Figure 1 As shown, this application discloses a method for calibrating the visible area of ​​a ship unloader, the method comprising the following: S101: Obtain three-dimensional point cloud data of the visible area of ​​the ship unloader, and obtain the three-dimensional center point of the grab bucket's hinge from the three-dimensional point cloud data.

[0026] Specifically, the visible area of ​​the unloader is the spatial area used for manual operation and cargo unloading. This area includes a grab bucket and a hinge, the grab bucket being used for cargo unloading and the hinge located on the grab bucket. The purpose of this step is to obtain the three-dimensional center point of the hinge from three-dimensional point cloud data obtained via LiDAR. To locate the hinge area, a coarse extent of the grab bucket is first extracted as base data. In some embodiments, this application uses multiple specially trained object detectors to segment the grab bucket area from a multi-axis aligned three-dimensional point cloud. Then, based on the grab bucket area, the hinge area is correspondingly determined according to the known positional relationship between the hinge and the grab bucket, such as the relative distance between the hinge and the grab bucket in the case of articulation. The three-dimensional center point of the hinge is then obtained by averaging all point clouds falling within the hinge area, and this three-dimensional center point can be represented as [x...]. L , y L , z L ].

[0027] Furthermore, as mentioned above, the steps of acquiring three-dimensional point cloud data of the visible area of ​​the ship unloader and obtaining the three-dimensional center point of the grab's hinge from the three-dimensional point cloud data include the following: The hinge position region is calculated based on the 3D point cloud data of the grab position obtained through object detector segmentation and the positional relationship between the hinge and the grab; and The three-dimensional center point of the hinge is calculated based on the three-dimensional point cloud data within the hinge location area.

[0028] Using the above method, the three-dimensional center point of the hinge can be accurately determined, and the three-dimensional center point can be easily matched with the corresponding center point in the subsequent two-dimensional data.

[0029] S102: Obtain two-dimensional image data of the visible area of ​​the ship unloader, and obtain the two-dimensional center point of the grab bucket's hinge from the two-dimensional image data.

[0030] Specifically, the purpose of this step is to locate the two-dimensional center point of the hinge based on an RGB image. Specifically, a specially trained grab detector is used to capture two-dimensional image data of the visible area of ​​the unloader from a top-down view. Then, the minimum bounding rectangle region of the grab is obtained from the two-dimensional image data of the visible area of ​​the unloader from a top-down view. This minimum bounding rectangle region is equivalent to the outer envelope of the grab's shape. Then, within this minimum bounding rectangle region, a template matching method based on a scanning window is used to extract a finer-grained or clearer hinge region from the image of the grab region. Subsequently, the two-dimensional center point of the hinge is located, and this two-dimensional center point can be labeled as [u]. R , v R The two-dimensional center point is used as the matching point for the three-dimensional center point of the hinge.

[0031] In some embodiments, the two-dimensional image data of the visible area of ​​the unloader obtained from a top-down view can be rotated to a two-dimensional image area obtained from an axis-aligned view, thereby further facilitating the acquisition of the minimum bounding rectangle area of ​​the grab and enabling subsequent scanning window matching.

[0032] As mentioned above, acquiring two-dimensional image data of the visible area of ​​the ship unloader and obtaining the two-dimensional center point of the grab's hinge from the two-dimensional image data includes the following: Two-dimensional image data of the visible area of ​​the ship unloader is acquired from a top-down view, and the minimum bounding rectangle area of ​​the grab bucket is obtained from the two-dimensional image data from the top-down view; and The two-dimensional center point of the hinge is obtained by scanning the window template within the smallest bounding rectangle area.

[0033] The above method can be used to easily obtain two-dimensional data of the grab and hinge to determine the two-dimensional center point of the hinge, thereby facilitating subsequent matching and calibration with the three-dimensional center point.

[0034] S103: Match the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge to calculate the transformation matrix.

[0035] In the aforementioned steps, solving for the transformation matrix can be understood as solving the parameters of a perspective-n-point (PnP) problem based on the gradient method. The transformation matrix is ​​calculated based on a set of two-dimensional and three-dimensional points. This application treats the solution process as a least-squares problem and uses a gradient-based correction method to solve for the parameters of the fitted model. Therefore, the transformation matrices R and t are further obtained.

[0036] Specifically, [x L , y L , z L ] and [u R , v R ] are considered as elements of the descent method used for the least squares problem and input into the fitting model, wherein the fitting model can be expressed as:

[0037] In addition, the calculation formula can be obtained:

[0038] in It is a transformation matrix. Representing three-dimensional data, and Represents two-dimensional data.

[0039] As mentioned above, the steps to match the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge in order to calculate the transformation matrix include the following: The grab bucket is moved to different positions, and the corresponding three-dimensional center point and two-dimensional center point of the hinge at those different positions are obtained; and Multiple 3D center points are matched with multiple 2D center points to calculate the transformation matrix.

[0040] Specifically, in some embodiments, the grab can be moved to different locations within the overlapping area of ​​the LiDAR and RGB cameras, such as multiple predetermined calibration candidate locations. Detected reference point pairs (i.e., the corresponding 3D and 2D center points obtained at the candidate locations) are recorded, and the transformation matrix is ​​calculated using these detected reference point pairs. It should be noted that the proposed method can also be used as an online calibration procedure due to its relatively low computational cost.

[0041] By using the above method, and by collecting three-dimensional and two-dimensional data of the grab at multiple different locations, the calculation of the transformation matrix is ​​more accurate.

[0042] S104: Fit the three-dimensional point cloud data of the visible area of ​​the ship unloader into the two-dimensional image data according to the transformation matrix.

[0043] After estimating the transformation matrices R and t, all sparse 3D points of the LiDAR sensor can be transferred to the RGB view as additional depth channels. For example... Figure 4 As shown, Figure 4 (a) represents the three-dimensional point cloud data of the grab bucket, where the white cross "+" represents the three-dimensional center point of the hinge. Figure 4 (b) represents the two-dimensional image data of the grab bucket, where the white cross "+" indicates the two-dimensional center point of the hinge. Figure 4 (c) represents a diagram showing the effect of fitting three-dimensional point cloud data to two-dimensional image data according to an embodiment of the present application, wherein gray dots represent three-dimensional point cloud data, which is fitted to two-dimensional image data as shown in the figure.

[0044] The above method allows for convenient calibration of 3D and 2D data, enabling the fitting of remaining 3D data into 2D image data, thereby improving data utilization and information complementarity. For objects within the same scene, multidimensional data can be referenced and used simultaneously. Alternatively, position calibration can be performed between a targetless laser sensor and an RGB camera. After calibration, the 3D point cloud data of the ship unloader's visible area can be fitted into the 2D image data, promoting the fusion of information from different dimensions, supplementing information content, and significantly facilitating the analysis of the ship unloader's regional information and automated operation.

[0045] In some embodiments, LiDAR-RGB calibration can be divided into two aspects: 1. Target-based calibration. This requires one or more specially designed calibration boards to retrieve corresponding matching points from overlapping fields of view. This significantly limits its application scenarios because it is difficult to cover large areas with calibration boards large enough. More importantly, in the real world (especially when using GPS), it is not possible to establish an online calibration system using specially designed markers.

[0046] 2. Target-free calibration. Besides specially designed markers, objects with the same semantic meaning can also be used as calibration reference points. An existing method employs general object detection or segmentation techniques to extract semantic objects from various modes of transportation scenarios. However, it is difficult to find natural objects such as pedestrians and walkers in the GSU environment. Furthermore, for target-free calibration methods based on surface segmentation, it is difficult to obtain representative points from the objects, resulting in low calibration accuracy and high computational cost.

[0047] To address the aforementioned issues, this application employs a targetless LiDAR-RGB calibration method based on hinge point positioning. A fully automated targetless calibration system is required. This application offers the following advantages: 1. It automatically locates reference point pairs using semantically meaningful positions, eliminating the need for manual operation throughout the calibration process. 2. No calibration board is required. 3. The computational cost is relatively low, allowing the automated GSU system to perform online calibration. The requirement for frequent changes in the positions of multiple sensors caused by crane oscillation can be met.

[0048] The method of this application can be applied to any automatic targetless LiDAR-RGB camera calibration system that uses a grab bucket as a reference object for automatic GSU scenes.

[0049] It should be understood that, despite Figure 1 Each step in the flowchart is shown sequentially according to the arrows, but these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise expressly stated in this application, there are no strict restrictions on the execution of these steps, and they can be performed in other orders. Furthermore, Figure 1 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least a portion of the steps or stages of other steps.

[0050] Figure 2 A device 200 is provided for calibrating the visible area of ​​a ship unloader. The device 200 includes: The three-dimensional data module 201 is configured to acquire three-dimensional point cloud data of the visible area of ​​the ship unloader and to acquire the three-dimensional center point of the hinge of the grab bucket from the three-dimensional point cloud data. Two-dimensional data module 202, the two-dimensional data module is configured to acquire two-dimensional image data of the visible area of ​​the ship unloader, and to acquire the two-dimensional center point of the hinge of the grab bucket from the two-dimensional image data; Transformation matrix module 203, configured to match the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge to calculate the transformation matrix; and The data conversion module 204 is configured to fit the three-dimensional point cloud data of the visible area of ​​the ship unloader into two-dimensional image data according to the transformation matrix.

[0051] Furthermore, the three-dimensional data module 201 is configured as follows: The hinge position region is calculated based on the 3D point cloud data of the grab position obtained by segmentation through an object detector and the positional relationship between the hinge and the grab; and The three-dimensional center point of the hinge is calculated based on the three-dimensional point cloud data within the hinge location area.

[0052] Furthermore, the two-dimensional data module 202 is configured as follows: Two-dimensional image data of the visible area of ​​the unloader is acquired from a top-down perspective, and the minimum bounding rectangle area of ​​the grab bucket is obtained from the two-dimensional image data from the top-down perspective; and Within the minimum bounding rectangle area, the two-dimensional center point of the hinge is obtained by scanning the window template.

[0053] Furthermore, the transformation matrix module 203 is configured as follows: The grab is moved to different positions, and the corresponding three-dimensional center point and two-dimensional center point of the hinge at each of these positions are obtained; and The transformation matrix is ​​calculated by matching multiple three-dimensional center points with multiple two-dimensional center points.

[0054] It should be noted that the device may include more or fewer modules to achieve the described functions. For example, Figure 2At least one module can be further divided into multiple different sub-modules, and each sub-module is configured to perform at least some of the operations described herein in conjunction with a corresponding module. Additionally, in some instances, device 200 may further include extra modules configured to perform other operations described in the specification. Furthermore, those skilled in the art will understand that exemplary device 200 can be implemented using software, hardware, firmware, or any combination thereof.

[0055] Figure 3 A computer device is provided. According to one embodiment, the computer device 300 may include a processor 302, and the processor 302 executes a computer program stored in a memory 304. The computer program performs the methods described above when executed by the processor.

[0056] Those skilled in the art should understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the solution in this application and does not constitute a limitation on the computer device to which this application is applied. Specifically, the computer device may include more or fewer components than those shown in the figure, or some components may be combined, or different components may be deployed.

[0057] Those skilled in the art will understand that all or some of the method processes in the foregoing embodiments can be implemented by a computer program instructing the relevant hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the process of implementing the foregoing method embodiments. Any reference to memory, storage device, database, or other media used in the various embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage devices. Non-volatile memory can include random access memory (RAM) or external cache memory. By way of example and not limitation, RAM can take many forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0058] This application further provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program performs the above-described steps.

[0059] This application further provides a computer program product. The computer program product is tangibly stored on a computer-readable medium and includes computer-executable instructions. When executed, the computer-executable instructions cause at least one processor to perform the methods described above.

[0060] Furthermore, the computer program can be stored and run on the cloud to execute the method. Additionally, program components can be deployed across multiple devices and the cloud. For example, corresponding steps can be deployed and run on a local or on-site computer, or on different cloud devices to transmit data via a communication connection; alternatively, they can be deployed and run on a local or on-site computer. This application does not limit the manner or method; the corresponding technologies can be flexibly deployed and implemented, and devices and technologies such as cloud computing, big data, and supercomputing capabilities can be fully utilized to achieve method execution and completion.

[0061] Some embodiments of this disclosure may include an article of manufacture. The article of manufacture may include a storage medium configured to store logic. Examples of storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of logic may include various software units, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application programming interfaces (APIs), instruction sets, computational code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. In some embodiments, for example, the article of manufacture may store executable computer program instructions. When executed by a processor, the executable computer program instructions cause the processor to perform the methods and / or operations described herein. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, and dynamic code. Executable computer program instructions can be implemented using a predetermined computer language, method, or syntax to instruct a computer to perform specific functions. Instructions can be implemented using any suitable high-level programming language, low-level programming language, object-oriented programming language, visual programming language, compiled programming language, and / or interpreted programming language.

[0062] The foregoing has described examples including the disclosed architecture. It is certainly impossible to describe every conceivable combination of components and / or methods, but those skilled in the art will understand that many other combinations and permutations are also possible. Therefore, the novel architecture is intended to cover all such alternatives, modifications, and variations falling within the spirit and scope of the appended claims.

Claims

1. A method for calibrating the visible area of ​​a ship unloader, the method comprising: Obtain three-dimensional point cloud data of the visible area of ​​the ship unloader, and obtain the three-dimensional center point of the grab bucket's hinge from the three-dimensional point cloud data; Two-dimensional image data of the visible area of ​​the unloader is obtained, and the two-dimensional center point of the hinge of the grab bucket is obtained from the two-dimensional image data; The three-dimensional center point of the hinge at multiple different spatial locations is matched with the two-dimensional center point of the hinge to calculate the transformation matrix; as well as The three-dimensional point cloud data of the visible area of ​​the ship unloader is fitted into the two-dimensional image data according to the transformation matrix.

2. The method according to claim 1, wherein obtaining three-dimensional point cloud data of the visible area of ​​the unloader and obtaining the three-dimensional center point of the grab's hinge from the three-dimensional point cloud data comprises: The hinge position region is calculated based on the 3D point cloud data of the grab position obtained by segmentation through the object detector and the positional relationship between the hinge and the grab; and The three-dimensional center point of the hinge is calculated based on the three-dimensional point cloud data within the hinge location area.

3. The method according to claim 1, wherein acquiring two-dimensional image data of the visible area of ​​the unloader and acquiring the two-dimensional center point of the hinge of the grab bucket from the two-dimensional image data comprises: Acquire two-dimensional image data of the visible area of ​​the unloader from a top-down perspective, and obtain the minimum bounding rectangle region of the grab bucket from the two-dimensional image data from the top-down perspective; and Within the minimum bounding rectangle area, the two-dimensional center point of the hinge is obtained by scanning the window template.

4. The method according to claim 1, wherein matching the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge to calculate the transformation matrix comprises: The grab is moved to different positions, and the corresponding three-dimensional center point and two-dimensional center point of the hinge at those different positions are obtained; and The transformation matrix is ​​calculated by matching multiple three-dimensional center points with multiple two-dimensional center points.

5. An apparatus (200) for calibrating the visible area of ​​a ship unloader, the apparatus comprising: A three-dimensional data module (201) is configured to acquire three-dimensional point cloud data of the visible area of ​​the unloader and to acquire the three-dimensional center point of the hinge of the grab bucket from the three-dimensional point cloud data. Two-dimensional data module (202), the two-dimensional data module is configured to acquire two-dimensional image data of the visible area of ​​the unloader, and acquire the two-dimensional center point of the hinge of the grab bucket from the two-dimensional image data; A transformation matrix module (203) is configured to match the three-dimensional center point of the hinge at multiple different spatial locations with the two-dimensional center point of the hinge in order to calculate the transformation matrix; as well as A data conversion module (204) is configured to fit the three-dimensional point cloud data of the visible area of ​​the ship unloader to the two-dimensional image data according to the transformation matrix.

6. The device (200) according to claim 5, wherein the three-dimensional data module (201) is configured to: The hinge position region is calculated based on the 3D point cloud data of the grab position obtained by segmentation through an object detector and the positional relationship between the hinge and the grab; and The three-dimensional center point of the hinge is calculated based on the three-dimensional point cloud data within the hinge location area.

7. The device (200) according to claim 5, wherein the two-dimensional data module (202) is configured to: Two-dimensional image data of the visible area of ​​the unloader is acquired from a top-down perspective, and the minimum bounding rectangle area of ​​the grab bucket is obtained from the two-dimensional image data from the top-down perspective; and Within the minimum bounding rectangle area, the two-dimensional center point of the hinge is obtained by scanning the window template.

8. The device (200) according to claim 5, wherein the transformation matrix module (203) is configured to: The grab is moved to different positions, and the corresponding three-dimensional center point and two-dimensional center point of the hinge at each of these positions are obtained; and The transformation matrix is ​​calculated by matching multiple three-dimensional center points with multiple two-dimensional center points.

9. A computer apparatus comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, performs the steps of the method according to any one of claims 1 to 4.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the method according to any one of claims 1 to 4.

11. A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions, wherein the computer-executable instructions, when executed, cause at least one processor to perform the method according to any one of claims 1 to 4.