Crane, information processing device, information processing method, and program

Abstract

This invention provides the optimal placement of sensors on a crane and a method for calibrating those sensors in the automation of crane operations. [Solution] The system includes one or more first sensors installed on the slewing part of a crane attached to a mobile body, one or more second sensors installed on a reference structure located separately from the crane on the mobile body, and a calibration unit that performs a first calibration for each of the first sensors using the reference structure and a second calibration for the second sensors using a target structure used in crane operations, which is separated from the mobile body and arranged around the mobile body.

Description

Technical Field

[0004] ,

[0002] , , , , , , , , , , , , ,

[0005] , , , , , , ,

[0003] <00000​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​To automate crane operations, the position of the crane and the positions of the target structures (objects) used in the crane operations, which are located around the crane and are separate from the crane itself, are necessary. However, if the object structures are large, a single sensor, as in Patent Document 1, cannot measure the entire object structure, and therefore cannot obtain the information necessary to automate crane operations.

[0006] One example of the purpose of this disclosure is to provide the locations for installing multiple sensors and the calibration method for each sensor in the automation of crane operations. [Means for solving the problem]

[0007] To achieve the above objective, the crane in one aspect of this disclosure is One or more first sensors installed on the slewing part of a crane installed on a mobile body, One or more second sensors installed on a reference structure located at a different location from the crane of the moving body, A calibration unit that performs a first calibration on each of the first sensors using the aforementioned reference structure, and performs a second calibration on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body, It is characterized by having the following features.

[0008] To achieve the above objective, the information processing device in one aspect of this disclosure is: An acquisition unit that acquires first measurement information measured by one or more first sensors installed on the slewing part of a crane installed on a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. A calibration unit that performs a first calibration on each of the first sensors using the aforementioned reference structure, and performs a second calibration on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body, It is characterized by having the following features.

[0009] Furthermore, in order to achieve the above objectives, the information processing method in one aspect of this disclosure is: Information processing device, The system acquires first measurement information measured by one or more first sensors installed on the slewing section of a crane attached to a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Using the aforementioned reference structure, a first calibration is performed on each of the first sensors, and a second calibration is performed on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body. It is characterized by the following:

[0010] Furthermore, in order to achieve the above objectives, the program in one aspect of this disclosure is On the computer, The system acquires first measurement information measured by one or more first sensors installed on the slewing section of a crane attached to a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Using the aforementioned reference structure, a first calibration is performed on each of the first sensors, and a second calibration is performed on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body. It is characterized by the following: [Effects of the Invention]

[0011] As described above, this disclosure provides a location for installing sensors on a crane and a method for calibrating sensors in the automation of crane operations. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a diagram illustrating an example of a mobile body equipped with a crane. [Figure 2]FIG. 2 is a diagram for explaining an example of a system having an information processing apparatus. [Figure 3] FIG. 3 is a diagram showing the positional relationship between a moving body and a target structure. [Figure 4] FIG. 4 is a diagram for explaining the relationship between a sensor and a turning angle. [Figure 5] FIG. 5 is a diagram for explaining an example of calibration of a sensor by an information processing apparatus. [Figure 6] FIG. 6 is a diagram for explaining an example of the first calibration. [Figure 7] FIG. 7 is a diagram for explaining an example of the second calibration. [Figure 8] FIG. 8 is a diagram for explaining an example of a computer that realizes the information processing apparatus in the embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0013] Hereinafter, embodiments will be described with reference to the drawings. In the drawings described below, elements having the same function or corresponding functions are denoted by the same reference numerals, and repeated descriptions thereof may be omitted.

[0014] [[ID=2�]] (Embodiment) Using FIG. 1, the configuration of a moving body equipped with a crane and an information processing apparatus that controls the crane in the embodiment will be described. FIG. 1 is a diagram for explaining an example of a moving body equipped with a crane.

[0015] [Device Configuration] The moving body 100 is, for example, a moving body such as a vehicle equipped with a crane 1, a ship, etc. As the vehicle, for example, a crane truck etc. can be considered. As the ship, for example, a dredger etc. can be considered. In the embodiment, for the sake of easy understanding of the explanation, the case where the moving body 100 is a dredger will be described. However, the moving body 100 is not limited to a dredger.

[0016] In the example shown in Figure 1, the mobile unit 100 comprises a crane 1, reference structures 2a and 2b, and an information processing device 10. The information processing device 10 is a device that controls the crane 1 (a crane control device or a crane operation automation device).

[0017] Crane 1 is equipped with a slewing section 1a. The slewing section 1a is equipped with a jib 1b. In the case of a dredger, for example, the jib 1b is fitted with wires and a grab bucket, which are not shown in Figure 1. The wires include support wires for supporting the grab bucket and opening / closing wires for opening and closing the grab bucket. The grab bucket is used to suspend the seabed using wires, excavate the seabed, and load the excavated soil onto a barge.

[0018] Reference structures 2a and 2b are fixed to the mobile body 100. Reference structures 2a and 2b are structures fixed to a location separate from the crane 1 of the mobile body 100. Furthermore, the point cloud information corresponding to reference structures 2a and 2b has a reference world coordinate system set. Reference structures 2a and 2b are, for example, spud devices installed behind the crane 1 that drive piles into the seabed to fix the position of the dredging vessel. However, reference structures 2a and 2b are not limited to spud devices.

[0019] Sensors (first sensors) S1, S2, S3, S4, S5, S6, and S7 are installed on the slewing section 1a of the crane 1 mounted on the mobile body 100. Sensors (second sensors) S8 and S9 are installed on reference structures 2a and 2b fixed to locations other than the crane 1 of the mobile body 100. However, the location and number of sensors are not limited to those shown in Figure 1.

[0020] The information processing device 10 performs the first calibration of sensors S1, S2, S3, S4, S5, S6, and S7 using reference structures 2a and 2b. The information processing device 10 is also separated from the mobile body 100 and performs the second calibration of sensors S8 and S9 using target structures (not shown in Figure 1) used in crane operations, which are placed around the mobile body 100.

[0021] The target structure, if the mobile body 100 is a dredger, may be, for example, a pollution control frame, a barge frame, or both.

[0022] Thus, in this embodiment, multiple sensors are installed at appropriate positions on the slewing section of the crane and the reference structure, so that even if the target structure is large, the entire target structure can be measured.

[0023] Furthermore, if the mobile body 100 and the target structure placed around the mobile body 100 are not fixed to each other, for example, if the mobile body 100 is a dredger and both the mobile body 100 and the target structure (pollution control frame, or barge frame, or both) are at sea, calibration becomes difficult because there is no reference point in the world coordinate system necessary for calibration. However, in this embodiment, calibration can be performed for each of the multiple sensors, thus enabling the automation of crane operations.

[0024] Furthermore, automating crane operations reduces the occurrence of accidents caused by operator errors, thereby ensuring worker safety. It also ensures the health of workers even when crane operations are performed in harsh environments. Moreover, automating crane operations can help address the shortage of workers in demanding work environments.

[0025] [System Configuration] Next, the configuration of the information processing device 10 in the embodiment will be described in more detail using Figures 1, 2, and 3. Figure 2 is a diagram illustrating an example of a system having an information processing device. Figure 3 is a diagram showing the positional relationship between the moving object and the target structure.

[0026] As shown in Figure 2, the system in the embodiment includes a crane 1, an information processing device 10, sensors (first sensors) S1, S2, S3, S4, S5, S6, S7, and sensors (second sensors) S8, S9.

[0027] Furthermore, as shown in Figure 3, target structures 31 (31a, 31b, 31c) are arranged around the mobile body 100 in the embodiment. For example, if the mobile body 100 is a dredger, target structure 31a is a pollution control frame, and target structures 31b and 31c are barge frames. The barge frames are, for example, mounted on a barge. Note that the barge frames may consist of only one of the target structures 31b or 31c.

[0028] ●The sensor will be explained in detail. Sensors S1, S2, S3, S4, S5, S6, S7, S8, and S9 each output point cloud information (measurement information) containing the distance and position from the sensor's location to the information processing device 10 via a network (wireless and / or wired).

[0029] The network is a wired or wireless network installed on the mobile device 100. The network is a general communication network constructed using communication lines such as LAN (Local Area Network), Bluetooth (registered trademark), and Wi-Fi (Wireless Fidelity) (registered trademark).

[0030] Furthermore, the point cloud information (measurement information) measured by each of the sensors S1, S2, S3, S4, S5, S6, S7, S8, and S9 is in the sensor's own local coordinate system (sensor coordinates). Note that sensors S1, S2, S3, S4, S5, S6, S7, S8, and S9 are sensors capable of measuring point cloud information such as LiDAR (Light Detection And Ranging).

[0031] Sensor S1 is a sensor for monitoring the condition of wires (support wires, opening / closing wires) attached to crane 1. Sensor S1 should be installed in a position where it can measure the condition of the wires. In other words, sensor S1 should be installed so that the wires are included within the measurable area (measurement area) of sensor S1.

[0032] In the example shown in Figure 1, sensor S1 is installed next to the jib 1b on the front side in the Y-axis direction (short side of the moving body 100) (on the side of sensors S2 and S4), and in the center in the Z-axis direction (height) of the swivel section 1a. Alternatively, sensor S1 may be installed on the side of sensors S3 and S5.

[0033] Sensor S2 is a sensor for monitoring the state of the target structures 31a and 31c. Sensor S2 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S2 should be installed so that the vertices t1 and t2 of the target structure 31a (pollution prevention frame) are included within the measurement area of ​​sensor S2.

[0034] In the example shown in Figure 1, sensor S2 is installed in front of the X-axis (longitudinal direction of the moving body 100) on the side (side A) of the slewing section 1a facing the reference structure 2a, and in the center of the Z-axis (height) direction of the slewing section 1a. Sensor S2 is used to measure the target structure 31a (pollution control frame) when the slewing angle of the slewing section 1a is 0 degrees (when the jib 1b is at 0 degrees (front) in Figure 3). Sensor S2 is also used to measure the target structure 31c (barge frame) when the slewing angle is around -90 degrees (when the jib 1b is at 270 degrees in Figure 3).

[0035] Sensor S3 is a sensor for monitoring the state of the target structures 31a and 31b. Sensor S3 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S3 should be installed so that the measurement area of ​​sensor S3 includes the vertices t3 and t4 of the target structure 31a (pollution prevention frame).

[0036] In the example shown in Figure 1, sensor S3 is installed in front of the X-axis direction on the side (side B) of the slewing section 1a on the side facing the reference structure 2b, and at the center of the Z-axis (height) direction of the slewing section 1a. Sensor S3 is used to measure the target structure 31a (pollution control frame) when the slewing angle of the slewing section 1a is 0 degrees. Sensor S3 is also used to measure the target structure 31b (barge frame) when the slewing angle is +90 degrees (when the jib 1b is at 90 degrees in Figure 3).

[0037] Sensor S4 is a sensor for monitoring the state of the target structures 31a and 31b. Sensor S4 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S4 should be installed so that the vertices p1 and p2 of the target structure 31b (barge frame) are included within the measurement area of ​​sensor S4.

[0038] In the example shown in Figure 1, sensor S4 is installed on the side of the swivel section 1a facing the reference structure 2a (side A) in the X-axis direction, in front of the swivel section 1a, and on the upper part in the Z-axis (height) direction. Sensor S4 is used to measure the target structure 31b (barge frame) when the swivel angle of the swivel section 1a is 0 degrees. When the swivel angle is 0 degrees, sensor S6 also measures the target structure 31b (barge frame). Sensor S4 is also used to measure the target structure 31a (pollution control frame) when the swivel angle is -90 (270) degrees. When the swivel angle is -90 degrees, sensor S6 also measures the target structure 31a (pollution control frame).

[0039] Sensor S5 is a sensor for monitoring the state of the target structures 31a and 31c. Sensor S5 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S5 should be installed so that the vertices q1 and q2 of the target structure 31c (barge frame) are included within the measurement area of ​​sensor S5.

[0040] In the example shown in Figure 1, sensor S5 is installed on the side of the swivel section 1a facing the reference structure 2b (side B), in the X-axis direction forward, and on the upper part of the swivel section 1a in the Z-axis direction (height). Sensor S5 is used to measure the target structure 31c (barge frame) when the swivel angle of the swivel section 1a is 0 degrees. When the swivel angle is 0 degrees, sensor S7 also measures the target structure 31c (barge frame). Sensor S5 is also used to measure the target structure 31a (pollution control frame) when the swivel angle is +90 degrees. When the swivel angle is +90 degrees, sensor S7 also measures the target structure 31a (pollution control frame).

[0041] Sensor S6 is a sensor for monitoring the state of the target structures 31a and 31b. Sensor S6 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S6 should be installed so that the vertices p3 and p4 of the target structure 31b (barge frame) are included within the measurement area of ​​sensor S6.

[0042] In the example shown in Figure 1, sensor S6 is installed on the rear side of the pivot section 1a on the side facing the reference structure 2a (side A) in the X-axis direction, and on the upper part of the pivot section 1a in the Z-axis (height) direction. Sensor S6 is used to measure the target structure 31b (barge frame) when the pivot angle of the pivot section 1a is 0 degrees. When the pivot angle is 0 degrees, sensor S4 also measures the target structure 31b (barge frame). Sensor S6 is also used to measure the target structure 31a (pollution control frame) when the pivot angle is -90 (270) degrees. When the pivot angle is -90 (270) degrees, sensor S4 also measures the target structure 31a (pollution control frame).

[0043] Sensor S7 is a sensor for monitoring the state of the target structures 31a and 31c. Sensor S7 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S7 should be installed so that the measurement area of ​​sensor S7 includes vertices q3 and q4 of the target structure 31c (barge frame).

[0044] In the example shown in Figure 1, sensor S7 is installed on the rear in the X-axis direction of the side (side B) of the swivel section 1a on the side facing the reference structure 2b, and on the upper part of the swivel section 1a in the Z-axis direction (height). Sensor S7 is used to measure the target structure 31c (barge frame) when the swivel angle of the swivel section 1a is 0 degrees. When the swivel angle is 0 degrees, sensor S5 also measures the target structure 31c (barge frame). Sensor S7 is also used to measure the target structure 31a (pollution control frame) when the swivel angle is +90 degrees. When the swivel angle is +90 degrees, sensor S5 also measures the target structure 31a (pollution control frame).

[0045] Sensor S8 is a sensor for monitoring the state of the target structure 31b. Sensor S8 should be installed in a position on the reference structure 2a where it can measure the state of the target structure 31b. In other words, sensor S8 should be installed so that the measurement area of ​​sensor S8 includes the vertices p2, p3, and p4 of the target structure 31b (barge frame).

[0046] In the example shown in Figure 1, the sensor S8 is installed at the rear in the X-axis direction of the side (side A) of the target structure 31a of the reference structure 2a, and at the top in the Z-axis (height) direction of the swivel section 1a.

[0047] Sensor S9 is a sensor for monitoring the state of the target structure 31c. Sensor S9 should be installed on the reference structure 2b at a position where it can measure the state of the target structure 31b. In other words, sensor S9 should be installed so that the measurement area of ​​sensor S9 includes the vertices q2, q3, and q4 of the target structure 31c (barge frame).

[0048] In the example shown in Figure 1, the sensor S9 is installed at the rear in the X-axis direction of the side (side B) of the target structure 31b of the reference structure 2b, and at the top in the Z-axis (height) direction of the swivel section 1a.

[0049] ●The information processing device will be explained in detail. The information processing device 10 is, for example, a CPU (Central Processing Unit), a programmable device such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), a circuit equipped with one or more of these, or a computer.

[0050] In Figures 1 and 3, the information processing device 10 is located inside the crane 1, but it may also be located outside the crane 1. For example, the information processing device 10 may be located on a part of the mobile body 100 other than the crane 1, or outside the mobile body 100.

[0051] When installed outside the crane 1 or mobile body 100, the information processing device 10 controls the crane 1 via a network. The network is a general network constructed using communication lines such as the internet, LAN, dedicated line, telephone line, corporate network, mobile communication network, Bluetooth, Wi-Fi, etc.

[0052] Furthermore, a storage device (not shown) is provided inside the information processing device 10. However, the storage device may be provided outside the information processing device 10. The storage device may be a database, a server computer, or a circuit with memory.

[0053] The information processing device 10 includes an acquisition unit 11, a calibration unit 12, a conversion unit 13, and a control unit 14.

[0054] The acquisition unit 11 acquires point cloud information (first measurement information) measured by one or more sensors (first sensors) S1, S2, S3, S4, S5, S6, S7 installed on the slewing section 1a of the crane 1 installed on the mobile body 100 at predetermined intervals (unit time) and stores it in the storage device. The acquisition unit 11 also acquires point cloud information (second measurement information) measured by one or more sensors (second sensors) S8, S9 installed on reference structures 2a, 2b located at locations other than the crane 1 of the mobile body 100 at predetermined intervals (unit time) and stores it in the storage device.

[0055] The calibration unit 12 performs calibration (first calibration) for each of the sensors S1 to S7 using the reference structures 2a and 2b, and further performs calibration (second calibration) for sensors S8 and S9 using the target structures 31a, 31b, and 31c used in crane operations, which are separated from the mobile body 100 and arranged around the mobile body 100.

[0056] ●The first proofreading process will be explained in detail. The calibration unit 12 first transmits instructions (rotation instruction information) to the control unit 14 to rotate the slewing section 1a of the crane 1 in the direction of the reference structures 2a and 2b, for which the world coordinate system is set, in order to perform calibration (first calibration) for each of the sensors S1 to S7. The control unit 14 then receives the rotation instruction information and rotates the slewing section 1a to the rotation angle indicated by the rotation instruction information.

[0057] When calibrating sensors S1 to S7, for each sensor S1 to S7, the swivel section 1a is rotated to a preset swivel angle γs1 (corresponding to S1), γs2 (corresponding to S2), γs3 (corresponding to S3), γs4 (corresponding to S4), γs5 (corresponding to S5), γs6 (corresponding to S6), and γs7 (corresponding to S7) so that the reference structure 2a, 2b, or both are included in the measurement area of ​​the sensor.

[0058] Next, after rotating the swivel unit 1a, the calibration unit 12 generates conversion information used to convert the point cloud information (first measurement information: local coordinate system) obtained by each of the sensors S1 to S7 measuring the reference structure 2a, 2b, or both, into the world coordinate system. Note that conversion information is generated for each of the sensors S1 to S7.

[0059] Transformation information includes, for example, a transformation matrix. Specifically, a transformation matrix is ​​calculated using three points from the pre-stored reference point cloud information (world coordinate system) of reference structures 2a and 2b that coincide with three points from the point cloud information (local coordinate system) measured for reference structures 2a and 2b (transformation matrix calculation process). The transformation matrix calculation process calculates the transformation matrix by, for example, using a process that uses at least three non-collinear (not collinear) points. For details on the transformation matrix calculation process, please refer to the following references.

[0060] References: Xu Gang, Tsuji Saburo, 3D Vision, Kyoritsu Shuppan, 1999, 1st edition, 2nd printing. Chapter 2.3 Calculating the transformation matrix using projection matrices and external variables.

[0061] However, the transformation matrix calculation process is not limited to the process described above; any process that changes the local coordinate system to the world coordinate system is acceptable. Furthermore, the relationship between the local coordinate system and the world coordinate system for each of the sensors S1 to S7 can be expressed as shown in Equation 1.

[0062]

number

[0063] Here, the coordinates of the point cloud obtained using the transformation matrix described above (world coordinate system) need to be corrected to the coordinates at the position of 0 degrees rotation angle, since the rotating unit 1a is rotated from rotation angle 0 degrees to rotation angle γ (γs1, γs2, γs3, γs4, γs5, γs6, γs7) for each of the sensors S1 to S7.

[0064] Figure 4 is a diagram illustrating the relationship between the sensor and the rotation angle. For example, as shown in Figure 4, the coordinates (world coordinate system) of the point cloud obtained at the position of sensor S6' after rotation are rotated by a rotation angle of γs6 degrees from the position of sensor S6 before rotation. Therefore, in order to correct this, it is necessary to rotate it in the opposite direction by a rotation angle of γs6.

[0065] Therefore, the calibration unit 12 generates correction information to correct the coordinates (world coordinate system) of the converted point clouds of each sensor S1 to S7 based on the rotation angle. The calibration unit 12 also associates sensor identification information for identifying the sensors, conversion information, and correction information and stores them in a memory device. The correction information is, for example, a correction matrix. Furthermore, the correction information for each sensor can be represented as shown in Equation 2.

[0066]

number

[0067] ●I will now explain the second proofreading process in detail. The calibration (second calibration) of sensors S8 and S9 is performed using the target structures 31b and 31c (barge frames) as a reference. Specifically, when calibrating sensor S8, the calibration (second calibration) is performed using the target structure 31b as a reference. Similarly, when calibrating sensor S9, the calibration (second calibration) is performed using the target structure 31c as a reference. The calibration (second calibration) of sensors S8 and S9 is performed using the calibration unit 12 and the conversion unit 13.

[0068] The conversion unit 13 converts the point cloud information (measurement information: local coordinate system) measured by sensors S1 to S9 (first sensor and second sensor) into a world coordinate system using conversion information (transformation matrix). The conversion unit 13 also corrects the point cloud information converted into a world coordinate system based on the rotation angle of the rotation unit 1a (using correction information (correction matrix)).

[0069] When calibrating sensor S8, first, the conversion unit 13 converts the point cloud information (measurement information: local coordinate system) of the target structure 31b (barge frame) measured by sensors S4 and S6 into a world coordinate system. Next, the conversion unit 13 corrects the world coordinate system of the converted target structure 31b.

[0070] Next, the calibration unit 12 generates transformation information (transformation matrix) to convert the point cloud information (measurement information: local coordinate system) measured by the sensor S8 of the target structure 31b to the world coordinate system, using the world coordinate system of the transformed target structure 31b as a reference. The calibration unit 12 also associates sensor identification information for identifying the sensor 8, correction information, and transformation information and stores them in a memory device.

[0071] Furthermore, when calibrating sensor S9, first, the conversion unit 13 converts the point cloud information (measurement information: local coordinate system) of the target structure 31c (barge frame) measured by sensors S5 and S7 into a world coordinate system. Next, the conversion unit 13 corrects the world coordinate system of the converted target structure 31c.

[0072] Next, the calibration unit 12 generates transformation information (transformation matrix) to convert the point cloud information (measurement information: local coordinate system) obtained by the sensor S9 from measuring the target structure 31c into a world coordinate system. The calibration unit 12 also associates sensor identification information for identifying the sensor 9 with the transformation information and stores it in a memory device.

[0073] In the first calibration, the control unit 14 controls the rotation of the slewing section 1a to a position where sensors S1 to S7 can each measure the reference structures 2a and 2b. Furthermore, during operation, the control unit 14 uses transformation information (transformation matrix) and correction information (correction matrix) to transform from the local coordinate system to the world coordinate system and controls the crane 1.

[0074] [Device operation] Next, the calibration of sensors S1 to S9 in the embodiment will be explained using Figures 5, 6, and 7. Figure 5 is a diagram illustrating an example of sensor calibration by the information processing device. Figure 6 is a diagram illustrating an example of the first calibration. Figure 7 is a diagram illustrating an example of the second calibration. In the following explanation, figures will be referred to as appropriate. In the embodiment, the information processing method is carried out by operating the information processing device. Therefore, the explanation of the information processing method in the embodiment will be replaced by the following explanation of the operation of the information processing device.

[0075] The information processing device 10 performs a first calibration and a second calibration. Specifically, as shown in Figure 5, the information processing device 10 first performs the first calibration (step A1), and then performs the second calibration (step A2).

[0076] ●The first proofreading step will be explained. As shown in Figure 6, the calibration unit 12 first selects the sensor S1 (step B1). Next, the calibration unit 12 transmits rotation instruction information to the control unit 14 in order to calibrate the sensor S1 (step B2).

[0077] Next, the control unit 14 receives the rotation instruction information and rotates the rotation section 1a of the crane 1 to the rotation angle γs1 indicated by the rotation instruction information of the sensor S1 (step B3).

[0078] Next, the calibration unit 12 acquires calibration point cloud information (first measurement information: local coordinate system) that the sensor S1 has measured, which includes the reference structure 2a, 2b, or both (step B4).

[0079] Next, the calibration unit 12 generates transformation information (transformation matrix) by performing a transformation matrix calculation process using the calibration point cloud information (local coordinate system) measured by the sensor S1 for the reference structures 2a and 2b, and the reference point cloud information (world coordinate system) of the reference structures 2a and 2b that has been stored in advance (step B5).

[0080] Next, the calibration unit 12 generates correction information (correction matrix) to correct the coordinates (world coordinate system) of the converted point cloud from sensor S1 based on the rotation angle (step B6).

[0081] Next, the calibration unit 12 associates sensor identification information for identifying the sensor, conversion information, and correction information, and stores them in a memory device (step B7).

[0082] Next, the calibration unit 12 determines whether or not the calibration of sensor S7 is complete (step B8). If the calibration of sensor S7 is not complete in step B8 (step B8: No), it selects the next sensor S2 after sensor S1 (step B9). In this way, sensors S1 through S7 are selected, and the processes described in steps B2 through B7 above are executed for each selected sensor.

[0083] Then, in step B8, if the calibration of sensor S7 is completed (step B8: Yes), the first calibration is terminated.

[0084] ●The second proofreading step will be explained. As shown in Figure 7, the calibration unit 12 first selects sensor S8 (step C1). Next, the conversion unit 13 converts the point cloud information (measurement information: local coordinate system) of the target structure 31b to the world coordinate system (step C2). In the case of sensor S8, the conversion unit 13 converts the point cloud information (measurement information: local coordinate system) of the target structure 31b (barge frame) measured by sensors S4 and S6 to the world coordinate system. Next, the conversion unit 13 corrects the converted world coordinate system of the target structure 31b (step C3).

[0085] Next, the calibration unit 12 generates transformation information (transformation matrix) to convert the point cloud information (measurement information: local coordinate system) measured by the sensor S8 of the target structure 31b to the world coordinate system, using the world coordinate system of the target structure 31b that has been transformed and corrected as a reference (step C4). Next, the calibration unit 12 associates the sensor identification information for identifying the sensor 8 with the transformation information and stores it in the memory device (step C5).

[0086] Next, the calibration unit 12 determines whether or not the calibration of sensor S9 has been completed (step C6). If the calibration of sensor S9 is not completed in step C6 (step C6: No), it selects the next sensor S9 after sensor S8 (step C7). In this way, sensors S8 through S9 are selected, and the processes shown in steps C2 to C5 above are executed for each selected sensor.

[0087] If sensor S9 is selected, in step C2, the conversion unit 13 converts the point cloud information (measurement information: local coordinate system) of the target structure 31c (barge frame) measured by sensors S5 and S7 into a world coordinate system. Next, in step C3, the conversion unit 13 corrects the world coordinate system of the converted target structure 31c.

[0088] Next, in step C4, the calibration unit 12 generates transformation information (transformation matrix) to be used to convert the calibration point cloud information (first measurement information for calibration: local coordinate system) obtained by the sensor S9 measuring the target structure 31c into a world coordinate system. Next, in step C5, the calibration unit 12 associates sensor identification information for identifying the sensor 9 with the transformation information and stores it in a memory device.

[0089] Then, in step C6, if the calibration of sensor S9 is completed (step B8: Yes), the second calibration is completed.

[0090] [Effects of the Embodiment] As described above, according to this embodiment, multiple sensors are installed at appropriate positions on the slewing section of the crane and the reference structure, so that even if the target structure is large, the entire target structure can be measured.

[0091] Furthermore, if the mobile body 100 and the target structure placed around the mobile body 100 are not fixed to each other, for example, if the mobile body 100 is a dredger and both the mobile body 100 and the target structure (pollution control frame, or barge frame, or both) are at sea, calibration becomes difficult because there is no reference point in the world coordinate system necessary for calibration. However, in this embodiment, calibration can be performed for each of the multiple sensors, thus enabling the automation of crane operations.

[0092] Furthermore, automating crane operations reduces the occurrence of accidents caused by operator errors, thereby ensuring worker safety. It also ensures the health of workers even when crane operations are performed in harsh environments. Moreover, automating crane operations can help address the shortage of workers in demanding work environments.

[0093] In the case of a dredger, the sensor may be installed in a location on the slewing section 1a of the crane 1, or on the reference structures 2a and 2b, for example, in a location that allows for power supply and avoids the influence of seawater.

[0094] Furthermore, the position of the sensors changes even while crane 1 is being calibrated. Therefore, during the calibration of crane 1, the rotation angle may be acquired at predetermined intervals (unit time) and the coordinates of each sensor being calibrated may be corrected. Specifically, the rotation angle γ in the correction information is replaced with (-γ+θ). Here, θ is the angle representing the change in the position of the sensor.

[0095] [program] The program in the embodiment can be any program that causes a computer to execute steps A1 to A2, B1 to B9, and C1 to C7 shown in Figures 5, 6, and 7. By installing and executing this program on a computer, the information processing apparatus and information processing method in the embodiment can be realized. In this case, the computer's processor functions as an acquisition unit 11, a calibration unit 12, a conversion unit 13, and a control unit 14, and performs the processing.

[0096] Furthermore, the program in the embodiment may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the acquisition unit 11, calibration unit 12, conversion unit 13, or control unit 14.

[0097] [Physical configuration] Here, we will describe a computer that implements an information processing device by executing the program in the embodiment, using Figure 8. Figure 8 is a diagram illustrating an example of a computer that implements an information processing device in the embodiment.

[0098] As shown in Figure 8, the computer 110 comprises a CPU (Central Processing Unit) 111, main memory 112, storage device 113, input interface 114, display controller 115, data reader / writer 116, and communication interface 117. These components are connected to each other via a bus 121, enabling data communication. In addition to the CPU 111, or in place of the CPU 111, the computer 110 may also include a GPU or FPGA.

[0099] The CPU 111 loads the program in the embodiment, which consists of a set of codes stored in the storage device 113, into the main memory 112, and performs various calculations by executing each code in a predetermined order. The main memory 112 is typically a volatile storage device such as DRAM (Dynamic Random Access Memory).

[0100] Furthermore, the program in this embodiment is provided stored on a computer-readable recording medium 120. The program in this embodiment may also be distributed over the Internet via a communication interface 117.

[0101] Specific examples of the storage device 113 include hard disk drives and semiconductor storage devices such as flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

[0102] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0103] Specific examples of the recording medium 120 include general-purpose semiconductor memory devices such as CF (Compact Flash®) and SD (Secure Digital), magnetic recording media such as Flexible Disks, and optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0104] Furthermore, the information processing device 10 in this embodiment can be implemented not by a computer on which a program is installed, but by using hardware corresponding to each part, such as electronic circuits. Moreover, the information processing device 10 may be partially implemented by a program and the remaining part by hardware. In this embodiment, the computer is not limited to the computer shown in Figure 8.

[0105] [Note] The following additional notes are disclosed regarding the embodiments described above. Some or all of the embodiments described above can be expressed by (Note 1) to (Note 20) below, but are not limited to the following descriptions.

[0106] (Note 1) One or more first sensors installed on the slewing part of a crane installed on a mobile body, One or more second sensors installed on a reference structure located at a different location from the crane of the moving body, A calibration unit that performs a first calibration on each of the first sensors using the aforementioned reference structure, and performs a second calibration on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body, A crane.

[0107] (Note 2) The crane is mounted on a dredger, and the target structure is a pollution control frame, a barge frame, or both. The crane described in Appendix 1.

[0108] (Note 3) An acquisition unit that acquires first measurement information measured by one or more first sensors installed on the slewing part of a crane installed on a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. A calibration unit that performs a first calibration on each of the first sensors using the aforementioned reference structure, and performs a second calibration on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body, An information processing device having

[0109] (Note 4) The system includes a transformation unit that converts the local coordinate systems of the first sensor and the second sensor to the world coordinate system. The information processing device described in Appendix 3.

[0110] (Note 5) The conversion unit acquires the rotation angle of the rotating part at predetermined intervals and corrects the world coordinate system of the first sensor based on the rotation angle. The information processing device described in Appendix 4.

[0111] (Note 6) In the first calibration described above, the control unit has a control unit that controls the rotation of the rotating part to a position where the first sensor can measure the reference structure. The information processing device described in Appendix 3.

[0112] (Note 7) The calibration unit is Using the first measurement information measured by the first sensor after the first calibration has been completed, the world coordinate system of the target structure is determined. The second calibration is performed using the world coordinate system of the target structure as the reference. The information processing device described in Appendix 4.

[0113] (Note 8) The crane is mounted on a dredger, and the target structure is a pollution control frame, a barge frame, or both. The information processing device described in Appendix 3.

[0114] (Note 9) Information processing device, The system acquires first measurement information measured by one or more first sensors installed on the slewing section of a crane attached to a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Using the aforementioned reference structure, a first calibration is performed on each of the first sensors, and a second calibration is performed on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body. Information processing methods.

[0115] (Note 10) The aforementioned information processing device The local coordinate systems of the first sensor and the second sensor are converted to the world coordinate system. The information processing method described in Appendix 9.

[0116] (Note 11) The aforementioned information processing device The rotation angle of the rotating part is acquired at predetermined intervals, and the world coordinate system of the first sensor is corrected based on the rotation angle. The information processing method described in Appendix 10.

[0117] (Note 12) The aforementioned information processing device In the first calibration described above, the control is performed to rotate the rotating part to a position where the first sensor can measure the reference structure. The information processing method described in Appendix 9.

[0118] (Note 13) The aforementioned information processing device Using the first measurement information measured by the first sensor after the first calibration has been completed, the world coordinate system of the target structure is determined. The second calibration is performed using the world coordinate system of the target structure as the reference. The information processing method described in Appendix 10.

[0119] (Note 14) The crane is mounted on a dredger, and the target structure is a pollution control frame, a barge frame, or both. The information processing method described in Appendix 9.

[0120] (Note 15) On the computer, The system acquires first measurement information measured by one or more first sensors installed on the slewing section of a crane attached to a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Using the aforementioned reference structure, a first calibration is performed on each of the first sensors, and a second calibration is performed on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body. program.

[0121] (Note 16) To the aforementioned computer, The local coordinate systems of the first sensor and the second sensor are converted to the world coordinate system. The program described in Appendix 15.

[0122] (Note 17) To the aforementioned computer, The rotation angle of the rotating part is acquired at predetermined intervals, and the world coordinate system of the first sensor is corrected based on the rotation angle. The program described in Appendix 16.

[0123] (Note 18) To the aforementioned computer, In the first calibration described above, the control is made to rotate the pivoting part to a position where the first sensor can measure the reference structure. The program described in Appendix 15.

[0124] (Note 19) To the aforementioned computer, Using the first measurement information measured by the first sensor after the first calibration has been completed, the world coordinate system of the target structure is determined. The second calibration is performed using the world coordinate system of the target structure as the reference. The program described in Appendix 16.

[0125] (Note 20) The crane is mounted on a dredger, and the target structure is a pollution control frame, a barge frame, or both. The program described in Appendix 15.

[0126] Although the invention has been described above with reference to embodiments, the invention is not limited to the embodiments described above. Various modifications to the structure and details of the invention can be made that will be understood by those skilled in the art within the scope of the invention. [Industrial applicability]

[0127] As described above, this invention provides a method for automating crane operations, including the placement of sensors on the crane and a method for calibrating the sensors. It is also useful in fields where sensor calibration is required. [Explanation of Symbols]

[0128] 1 Crane 1a Swivel section 1b Jib 2a, 2b Reference structure 10 Information Processing Devices 11 Acquisition Department 12. Proofreading Department 13 Conversion section 14 Control Unit S1, S2, S3, S4, S5, S6, S7, S8, S9 sensors 100 Mobile Units 110 Computer 111 CPU 112 Main Memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Readers / Writers 117 Communication Interface 118 Input devices 119 Display device 120 recording media 121 Bus

Claims

1. One or more first sensors installed on the slewing part of a crane installed on a mobile body, One or more second sensors installed on a reference structure located at a different location from the crane of the moving body, Calibration means for performing a first calibration on each of the first sensors using the aforementioned reference structure, and a second calibration on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body; A crane.

2. The crane is mounted on a dredger, and the target structure is a pollution control frame, a barge frame, or both. The crane according to claim 1.

3. An acquisition means for acquiring first measurement information measured by one or more first sensors installed on the slewing section of a crane installed on a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Calibration means for performing a first calibration on each of the first sensors using the aforementioned reference structure, and a second calibration on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body; An information processing device having

4. The system includes a transformation means for converting the local coordinate systems of the first sensor and the second sensor to the world coordinate system. The information processing apparatus according to claim 3.

5. The conversion means acquires the rotation angle of the rotating part at predetermined intervals and corrects the world coordinate system of the first sensor based on the rotation angle. The information processing apparatus according to claim 4.

6. In the first calibration described above, the control means is provided to control the rotation of the rotating part to a position where the first sensor can measure the reference structure. The information processing apparatus according to claim 3.

7. The calibration means is Using the first measurement information measured by the first sensor after the first calibration has been completed, the world coordinate system of the target structure is determined. The second calibration is performed using the world coordinate system of the target structure as the reference. The information processing apparatus according to claim 4.

8. The crane is mounted on a dredger, and the target structure is a pollution control frame, a barge frame, or both. The information processing apparatus according to claim 3.

9. Information processing device, The system acquires first measurement information measured by one or more first sensors installed on the slewing section of a crane attached to a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Using the aforementioned reference structure, a first calibration is performed on each of the first sensors, and a second calibration is performed on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body. Information processing methods.

10. On the computer, The system acquires first measurement information measured by one or more first sensors installed on the slewing section of a crane attached to a mobile body, and second measurement information measured by one or more second sensors installed on a reference structure located at a different location from the crane on the mobile body. Using the aforementioned reference structure, a first calibration is performed on each of the first sensors, and a second calibration is performed on the second sensor using a target structure used in crane operations, which is separated from the mobile body and positioned around the mobile body. program.

Images