Information processing device, information processing method, and program

Abstract

The objective is to estimate the position coordinates of the corners on the upper surface of a hollow structure. [Solution] The information processing device includes: a classification unit that generates classes by classifying point cloud data of hollow structures located around the crane, separated from the crane and measured using sensors installed on the crane, according to similar features; a vertical surface estimation unit that estimates vertical surfaces using the point cloud data included in each class, and defines the surface whose inclination with respect to the horizontal plane falls within a preset vertical inclination range as the vertical surface; and a corner estimation unit that, if there are multiple estimated vertical surfaces, estimates the position coordinates of the corners of the upper surface of the structure using the multiple vertical surfaces.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program for estimating the position of a structure.

Background Art

[0002] As a related technique, Patent Document 1 discloses a point cloud data processing apparatus that extracts features from point cloud data of a measurement object and automatically generates data related to the contour of the object in a short time. According to the point cloud data processing apparatus of Patent Document 1, from the point cloud data in which a secondary image of a measurement object is associated with the three-dimensional coordinate data of a plurality of points constituting this two-dimensional image, point cloud data related to a non-planar region with a large calculation burden is removed, and after the data of the non-planar region is removed, a label for designating a plane is assigned to the point cloud data, and based on a local region continuous from the plane to which the label is assigned and using a local plane, the contour line of the object is calculated.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the point cloud data processing apparatus of Patent Document 1 does not estimate the position coordinates of the corner of the upper surface of a structure with a hollow inside. Specifically, it does not estimate the position coordinates of the corner of the upper surface of a structure with a hollow inside, which is arranged around a crane separated from the crane.

[0005] An example of the object of the present disclosure is to estimate the position coordinates of the corner of the upper surface of a structure with a hollow inside.

Means for Solving the Problems

[0006] To achieve the above objective, the information processing device in one aspect of this disclosure is: A classification unit generates classes by classifying point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, according to similar features. A vertical surface estimation unit estimates a surface using the point cloud data included for each class, and defines a surface as a vertical surface if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range. If there are multiple estimated vertical surfaces, a corner estimation unit estimates the position coordinates of the corners of the upper surface of the structure using the multiple vertical surfaces, It is characterized by having the following features.

[0007] Furthermore, in order to achieve the above objectives, the information processing method in one aspect of this disclosure is: Information processing device, Point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, are classified according to similar features to generate classes. Using the point cloud data included in each class, a surface is estimated, and if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, that surface is defined as a vertical surface. If there are multiple estimated vertical surfaces, the position coordinates of the corners on the upper surface of the structure are estimated using the multiple vertical surfaces. It is characterized by the following:

[0008] Furthermore, in order to achieve the above objectives, the program in one aspect of this disclosure is On the computer, Point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, are classified according to similar features to generate classes. Using the point cloud data included in each class, a surface is estimated, and if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, that surface is defined as a vertical surface. When there are a plurality of the estimated vertical planes, the position coordinates of the corners of the upper surface of the structure are estimated using the plurality of the vertical planes. characterized by causing the processing to be executed.

Advantages of the Invention

[0009] As described above, according to the present disclosure, the position coordinates of the corners of the upper surface of a structure having a hollow inside can be estimated.

Brief Description of the Drawings

[0010] [Figure 1] FIG. 1 is a diagram for explaining an example of an information processing apparatus. [Figure 2] FIG. 2 is a diagram for explaining an example of a moving body having an information processing apparatus. [Figure 3] FIG. 3 is a diagram showing the positional relationship between the moving body and the target structure. [Figure 4] FIG. 4 is a diagram for explaining an example of a system having an information processing apparatus. [Figure 5] FIG. 5 is a diagram for explaining classification. [Figure 6] FIG. 6 is a diagram for explaining vertical plane estimation. [Figure 7] FIG. 7 is a diagram for explaining corner estimation. [Figure 8] FIG. 8 is a diagram for explaining an example of the operation of the information processing apparatus. [Figure 9] FIG. 9 is a diagram for explaining an example of a computer that realizes the information processing apparatus in the embodiment.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described with reference to the drawings. In the drawings described below, elements having the same function or corresponding functions are denoted by the same reference numerals, and repeated descriptions thereof may be omitted.

[0012] (Embodiment) Using FIG. 1, the configuration of the information processing apparatus in the embodiment will be described. FIG. 1 is a diagram for explaining an example of the information processing apparatus.

[0013] [Device Configuration] The information processing apparatus shown in FIG. 1 is an apparatus (structure position coordinate estimation apparatus) for estimating the position coordinates of the corners on the upper surface of a structure (target structure) with a hollow inside. As shown in FIG. 1, the information processing apparatus 10 includes a classification unit 11, a vertical plane estimation unit 12, and a corner estimation unit 13.

[0014] The classification unit 11 classifies the point cloud data of the structures with hollow inside arranged around the crane, which is measured using the sensors installed on the crane and separated from the crane, into classes by similar features.

[0015] The vertical plane estimation unit 12 estimates a plane using the point cloud data included for each class, and sets a plane whose inclination with respect to the horizontal plane of the estimated plane is within a preset vertical inclination range as a vertical plane.

[0016] When there are a plurality of estimated vertical planes, the corner estimation unit 13 estimates the position coordinates of the corners on the upper surface of the structure using the plurality of vertical planes.

[0017] In this way, in the embodiment, the point cloud data of the structure with a hollow inside can be classified to generate classes, a vertical plane can be estimated using the point cloud data for each class, and the position coordinates of the corners on the upper surface of the structure can be estimated using the plurality of vertical planes.

[0018] [System Configuration] Subsequently, the configuration of the information processing apparatus 10 in the embodiment will be described in more detail using FIGS. 2, 3, and 4. FIG. 2 is a diagram for explaining an example of a moving body having the information processing apparatus. FIG. 3 is a diagram showing the positional relationship between the moving body and the target structure. FIG. 4 is a diagram for explaining an example of a system having the information processing apparatus.

[0019] ● The moving body will be described in detail. As shown in Figure 2, the mobile body 100 comprises a crane 1, reference structures 2a and 2b, and an information processing device 10. The mobile body 100 is also a mobile body such as a vehicle or ship equipped with the crane 1. A vehicle could be, for example, a crane truck. A ship could be, for example, a dredger. In this embodiment, for the sake of clarity, the case where the mobile body 100 is a dredger will be described. However, the mobile body 100 is not limited to a dredger.

[0020] Crane 1 is equipped with a slewing section 1a. Crane 1 is, for example, a truck crane, a rough terrain crane, an all-terrain crane, a crawler crane, a gantry crane, an unloader crane, a jib crane, an overhead crane, a cable crane, a stacker crane, etc.

[0021] The slewing section 1a is equipped with a jib 1b. In the case of a dredger, for example, the jib 1b is fitted with wires and a grab bucket, which are not shown in Figure 2. The wires include support wires for supporting the grab bucket and opening / closing wires for opening and closing the grab bucket. The grab bucket is used to excavate the seabed by being suspended using wires, and to load the excavated soil onto a barge.

[0022] Reference structures 2a and 2b are fixed to the mobile body 100. Reference structures 2a and 2b are structures fixed to a location separate from the crane 1 of the mobile body 100. Furthermore, the point cloud information corresponding to reference structures 2a and 2b has a reference world coordinate system set. Reference structures 2a and 2b are, for example, spud devices installed behind the crane 1 that drive piles into the seabed to fix the position of the dredging vessel. However, reference structures 2a and 2b are not limited to spud devices.

[0023] Sensors S1, S2, S3, S4, S5, S6, and S7 are installed on the slewing section 1a of the crane 1 mounted on the mobile body 100. Sensors S8 and S9 are installed on reference structures 2a and 2b fixed to locations other than the crane 1 of the mobile body 100. However, the location and number of sensors are not limited to those shown in Figure 2. ●The target structure will be explained in detail. As shown in Figure 3, if the mobile body 100 is a dredger, for example, structure 31a is a pollution control frame, and structures 31b and 31c are barge frames. The barge frame is installed on a barge, for example. Note that the barge frame may consist of only one of the structures 31b or 31c.

[0024] ●The sensor will be explained in detail. Sensors S1, S2, S3, S4, S5, S6, S7, S8, and S9 each output point cloud information (measurement information) containing the distance and position from the sensor's location to the information processing device 10 via a network (wireless and / or wired).

[0025] The network is a wired or wireless network installed on the mobile device 100. The network is a general communication network constructed using communication lines such as LAN (Local Area Network), Bluetooth (registered trademark), and Wi-Fi (Wireless Fidelity) (registered trademark).

[0026] Furthermore, the point cloud information (measurement information) measured by each of the sensors S1, S2, S3, S4, S5, S6, S7, S8, and S9 is in the sensor's own local coordinate system (sensor coordinates). Note that sensors S1, S2, S3, S4, S5, S6, S7, S8, and S9 are sensors capable of measuring point cloud information such as LiDAR (Light Detection And Ranging).

[0027] Sensor S1 is a sensor for monitoring the condition of wires (support wires, opening / closing wires) attached to crane 1. Sensor S1 should be installed in a position where it can measure the condition of the wires. In other words, sensor S1 should be installed so that the wires are included within the measurable area (measurement area) of sensor S1.

[0028] In the examples shown in Figures 2 and 3, sensor S1 is installed next to the jib 1b on the front side in the Y-axis direction (short side of the moving body 100) (on the side of sensors S2 and S4), and in the center in the Z-axis direction (height) of the swivel section 1a. Alternatively, sensor S1 may be installed on the side of sensors S3 and S5.

[0029] Sensor S2 is a sensor for monitoring the state of the target structures 31a and 31c. Sensor S2 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S2 should be installed so that the vertices t1 and t2 of the target structure 31a (pollution prevention frame) are included within the measurement area of ​​sensor S2.

[0030] In the examples shown in Figures 2 and 3, the sensor S2 is installed in front of the X-axis (longitudinal direction of the moving body 100) on the side (side A) of the slewing section 1a facing the reference structure 2a, and in the center of the Z-axis (height) direction of the slewing section 1a. The sensor S2 is used to measure the target structure 31a (pollution control frame) when the slewing angle of the slewing section 1a is 0 degrees (when the jib 1b is at 0 degrees (front) in Figure 3). The sensor S2 is also used to measure the target structure 31c (barge frame) when the slewing angle is around -90 degrees (when the jib 1b is at 270 degrees in Figure 3).

[0031] Sensor S3 is a sensor for monitoring the state of the target structures 31a and 31b. Sensor S3 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S3 should be installed so that the measurement area of ​​sensor S3 includes the vertices t3 and t4 of the target structure 31a (pollution prevention frame).

[0032] In the examples shown in Figures 2 and 3, the sensor S3 is installed in front of the X-axis direction on the side (side B) of the slewing section 1a on the side facing the reference structure 2b, and at the center of the Z-axis (height) direction of the slewing section 1a. The sensor S3 is used to measure the target structure 31a (pollution control frame) when the slewing angle of the slewing section 1a is 0 degrees. The sensor S3 is also used to measure the target structure 31b (barge frame) when the slewing angle is +90 degrees (when the jib 1b is at 90 degrees in Figure 3).

[0033] Sensor S4 is a sensor for monitoring the state of the target structures 31a and 31b. Sensor S4 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S4 should be installed so that the vertices p1 and p2 of the target structure 31b (barge frame) are included within the measurement area of ​​sensor S4.

[0034] In the examples shown in Figures 2 and 3, sensor S4 is installed on the side of the swivel section 1a facing the reference structure 2a (side A) in the X-axis direction, in front of the swivel section 1a, and on the upper part in the Z-axis (height) direction. Sensor S4 is used to measure the target structure 31b (barge frame) when the swivel angle of the swivel section 1a is 0 degrees. When the swivel angle is 0 degrees, sensor S6 also measures the target structure 31b (barge frame). Sensor S4 is also used to measure the target structure 31a (pollution control frame) when the swivel angle is -90 (270) degrees. When the swivel angle is -90 degrees, sensor S6 also measures the target structure 31a (pollution control frame).

[0035] Sensor S5 is a sensor for monitoring the state of the target structures 31a and 31c. Sensor S5 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S5 should be installed so that the vertices q1 and q2 of the target structure 31c (barge frame) are included within the measurement area of ​​sensor S5.

[0036] In the examples shown in Figures 2 and 3, sensor S5 is installed on the side of the swivel section 1a facing the reference structure 2b (side B), in the X-axis direction forward, and on the upper part of the swivel section 1a in the Z-axis (height) direction. Sensor S5 is used to measure the target structure 31c (barge frame) when the swivel angle of the swivel section 1a is 0 degrees. When the swivel angle is 0 degrees, sensor S7 also measures the target structure 31c (barge frame). Sensor S5 is also used to measure the target structure 31a (pollution control frame) when the swivel angle is +90 degrees. When the swivel angle is +90 degrees, sensor S7 also measures the target structure 31a (pollution control frame).

[0037] Sensor S6 is a sensor for monitoring the state of the target structures 31a and 31b. Sensor S6 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S6 should be installed so that the vertices p3 and p4 of the target structure 31b (barge frame) are included within the measurement area of ​​sensor S6.

[0038] In the examples shown in Figures 2 and 3, sensor S6 is installed on the rear side of the pivot section 1a on the side facing the reference structure 2a (side A) in the X-axis direction, and on the upper part of the pivot section 1a in the Z-axis (height) direction. Sensor S6 is used to measure the target structure 31b (barge frame) when the pivot angle of the pivot section 1a is 0 degrees. When the pivot angle is 0 degrees, sensor S4 also measures the target structure 31b (barge frame). Sensor S6 is also used to measure the target structure 31a (pollution control frame) when the pivot angle is -90 (270) degrees. When the pivot angle is -90 (270) degrees, sensor S4 also measures the target structure 31a (pollution control frame).

[0039] Sensor S7 is a sensor for monitoring the state of the target structures 31a and 31c. Sensor S7 should be installed in a position where it can measure the state of the target structure 31. That is, sensor S7 should be installed so that the measurement area of ​​sensor S7 includes vertices q3 and q4 of the target structure 31c (barge frame).

[0040] In the examples shown in Figures 2 and 3, sensor S7 is installed on the rear in the X-axis direction of the side (side B) of the swivel section 1a on the side facing the reference structure 2b, and on the upper part of the swivel section 1a in the Z-axis (height) direction. Sensor S7 is used to measure the target structure 31c (barge frame) when the swivel angle of the swivel section 1a is 0 degrees. When the swivel angle is 0 degrees, sensor S5 also measures the target structure 31c (barge frame). Sensor S7 is also used to measure the target structure 31a (pollution control frame) when the swivel angle is +90 degrees. When the swivel angle is +90 degrees, sensor S5 also measures the target structure 31a (pollution control frame).

[0041] Sensor S8 is a sensor for monitoring the state of the target structure 31b. Sensor S8 should be installed in a position on the reference structure 2a where it can measure the state of the target structure 31b. In other words, sensor S8 should be installed so that the measurement area of ​​sensor S8 includes the vertices p2, p3, and p4 of the target structure 31b (barge frame).

[0042] In the examples shown in Figures 2 and 3, the sensor S8 is installed at the rear in the X-axis direction of the side (side A) of the target structure 31a of the reference structure 2a, and at the top in the Z-axis (height) direction of the swivel section 1a.

[0043] Sensor S9 is a sensor for monitoring the state of the target structure 31c. Sensor S9 should be installed on the reference structure 2b at a position where it can measure the state of the target structure 31b. In other words, sensor S9 should be installed so that the measurement area of ​​sensor S9 includes the vertices q2, q3, and q4 of the target structure 31c (barge frame).

[0044] In the examples shown in Figures 2 and 3, the sensor S9 is installed at the rear in the X-axis direction of the side (side B) of the target structure 31b of the reference structure 2b, and at the top in the Z-axis (height) direction of the swivel section 1a.

[0045] ●The information processing device will be explained in detail. The information processing device 10 is, for example, a programmable device such as a CPU (Central Processing Unit) or FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, or a computer. The information processing device 10 is also a device that controls the crane 1 (crane control device or crane operation automation device).

[0046] In the example shown in Figure 4, the information processing device 10 is located inside the crane 1, but it may also be located outside the crane 1. For example, the information processing device 10 may be located on a part of the mobile body 100 other than the crane 1, or outside the mobile body 100.

[0047] When installed outside the crane 1 or mobile body 100, the information processing device 10 controls the crane 1 via a network. The network is a general network constructed using communication lines such as the internet, LAN, dedicated line, telephone line, corporate network, mobile communication network, Bluetooth, Wi-Fi, etc.

[0048] Furthermore, a storage device (not shown) is provided inside the information processing device 10. However, the storage device may be provided outside the information processing device 10. The storage device may be a database, a server computer, or a circuit with memory.

[0049] The information processing device 10 includes a conversion unit 14, a preprocessing unit 15, a classification unit 11, a vertical plane estimation unit 12, a corner estimation unit 13, and a control unit 16.

[0050] The transformation unit 14 transforms the local coordinate system of the target structure 31 (31a, 31b, 31c) measured by sensors S1 to S9 into a world coordinate system (coordinate transformation processing). The coordinate transformation processing uses general transformation processing. In the coordinate transformation processing, for example, a transformation matrix is ​​used to transform the local coordinate system into a world coordinate system. Specifically, the parameters obtained in the preliminary preparation are used to transform the LiDAR coordinates into world coordinates.

[0051] The preprocessing unit 15 uses the point cloud data of the target structure 31 to perform processing to reduce the processing time of the classification unit 11, the vertical plane estimation unit 12, and the corner estimation unit 13, as well as processing to remove noise. The preprocessing unit 15 includes a region designation unit 15a, a noise reduction unit 15b, and a voxel unit 15c.

[0052] Specifically, the area designation unit 15a uses the point cloud data of the target structure 31 to specify the estimation area for estimating the target structure 31. Specifically, the estimation area is specified using the X, Y, and Z axes of the world coordinate system. The estimation area can be specified automatically or manually. By specifying the estimation area in this way, the estimation processing time can be reduced by performing the estimation process using only the point cloud data of the estimation area.

[0053] The noise reduction unit 15b sets one or more noise determination areas in the specified estimation area, and if the number of point cloud data points included in the noise determination area is less than or equal to a preset reference value, it determines that the point cloud data points in the noise determination area are noise (outliers). Subsequently, it removes the point cloud data points determined to be noise.

[0054] The voxel section 15c performs voxel processing on the point cloud data of the specified estimation area. By applying voxel processing, the number of points in the point cloud can be further reduced, thereby reducing the estimation processing time. In addition, voxel processing has the effect of smoothing the point cloud density and eliminating bias to certain parts during estimation, thereby improving the accuracy of surface estimation.

[0055] The classification unit 11 uses sensors S1 to S9 installed on the crane 1 and the mobile body 100 to measure point cloud data of the hollow target structures 31 (31a, 31b, 31c) located around the crane 1, which are separated from the crane 1. The classification unit 11 uses clustering processing based on normal segmentation to classify the data according to similar features and generate classes.

[0056] Figure 5 is a diagram illustrating the classification. As shown in the example in Figure 5, the point cloud data of the target structure 31 is classified as indicated by the dashed lines. Note that although four classes are shown in the example in Figure 5 for convenience, in reality, all point cloud data is classified.

[0057] The vertical surface estimation unit 12 first estimates a surface using the point cloud data included in each class. Specifically, surface estimation involves selecting arbitrary points and fitting the surface using the least squares method. The distance between the unselected points and the surface is calculated, and points whose distance is greater than a predetermined value are excluded as outliers. The surface is then fitted again using the point cloud data excluding the outliers. If the proportion of non-outlier points exceeds a certain threshold, that surface is adopted as the final estimation result.

[0058] Specifically, the vertical plane estimation unit 12 determines a plane as a vertical plane if its inclination relative to the horizontal plane falls within a predetermined vertical inclination range. Next, the vertical plane estimation unit 12 uses the vertical plane to determine the combination of opposite sides (long side and short side).

[0059] Figure 6 is a diagram illustrating vertical plane estimation. In example A of Figure 6, four black-painted areas 61, 62, 63, and 64 are shown as vertical planes of the target structure 31. In example B of Figure 6, vertical planes 61 and 63 are shown as opposite sides (short sides), and vertical planes 62 and 64 are shown as opposite sides (long sides).

[0060] Furthermore, even a single surface may be classified into multiple classes by the classification unit 11. Therefore, similar classes are grouped together based on the inclination and position of the surface.

[0061] Furthermore, depending on the shape of the target structure 31, the surfaces may not be perpendicular, or they may be tilted due to the influence of the surrounding environment (e.g., waves), so perpendicular surfaces are extracted using likelihood.

[0062] Note that the vertical plane does not need to be four faces. This is because, depending on the positional relationship between the target structure 31 and sensors S1 to S9, there will be faces that are not visible (hidden). However, a minimum of three faces is required. In the case of two faces, only one intersection point can be found, making it impossible to determine the direction of interpolation.

[0063] If there are multiple estimated vertical planes, the corner estimation unit 13 uses the multiple vertical planes to estimate the position coordinates of the corners on the upper surface of the target structure 31 (31a, 31b, 31c). Specifically, the corner estimation unit 13 estimates the intersection lines of the vertical planes of the target structure 31 and the upper surface, estimates the intersection points (candidate corner points) of the intersection lines and the upper surface, and estimates the optimal vertex position coordinates from among the candidate corner points that match the specified size of the target structure 31.

[0064] Figure 7 is a diagram illustrating corner estimation. The corner estimation unit 13 first estimates the intersection lines 71a of vertical planes 61 and 62, 71b of vertical planes 62 and 63, 71c of vertical planes 63 and 64, and 71d of vertical planes 64 and 61, as shown in Figure 7.

[0065] Next, the corner estimation unit 13 estimates the upper surface 73 (the surface composed of line segments 72a, 72b, 72c, and 72d in Figure 7) which is formed by the vertical surfaces 61 to 64. Next, the corner estimation unit 13 estimates the position coordinates of the intersection points (corner candidate points) 74a, 74b, 74c, 74d, and 74e of the intersection lines 71a to 71d and the upper surface 73.

[0066] Next, the corner estimation unit 13 uses the position coordinates of the candidate corner points 74a, 74b, 74c, 74d, and 74e to determine whether they are included in the pre-set range of position coordinates for the vertices of the upper surface 73 of the target structure 31. Subsequently, the corner estimation unit 13 sets the position coordinates of the corners of the upper surface 73 of the target structure 31 (31a, 31b, 31c) to the candidate corner points 74a, 74b, 74c, and 74d that are included in the range of position coordinates for the vertices of the upper surface 73.

[0067] Furthermore, if the number of candidate corner points is less than the number of vertices on the upper surface 73 of the target structure 31 (31a, 31b, 31c) that has been set in advance (i.e., the vertical surface is not visible due to occlusion), the corner estimation unit 13 interpolates to increase the number of candidate corner points based on the position coordinate range.

[0068] Furthermore, if the number of candidate corner points is greater than or equal to the number of vertices on the upper surface 73 of the target structure 31 (31a, 31b, 31c) that has been set in advance, the corner estimation unit 13 reduces the number of candidate corner points based on the position coordinate range.

[0069] During operation, the control unit 16 acquires control information (including position coordinates of the corners on the upper surface of the target structure 31) for controlling the crane 1, and controls the crane 1 based on the control information.

[0070] [Device operation] Next, the operation of the information processing device in the embodiment will be described using Figure 8. Figure 8 is a diagram illustrating an example of the operation of the information processing device. In the following description, the diagram will be referred to as appropriate. In this embodiment, the information processing method is implemented by operating the information processing device. Therefore, the explanation of the information processing method in this embodiment will be replaced by the following explanation of the operation of the information processing device.

[0071] As shown in Figure 8, first, the conversion unit 14 converts the local coordinate system of the target structure 31 (31a, 31b, 31c) measured by sensors S1 to S9 into the world coordinate system (coordinate transformation process) (step A1).

[0072] Next, the area designation unit 15a uses the point cloud data of the target structure 31 to designate an estimation area for estimating the target structure 31 (step A2).

[0073] Next, the noise reduction unit 15b sets one or more noise determination areas in the specified estimation area. If the number of point cloud data points included in a noise determination area is less than or equal to a preset reference value, it determines the point cloud data points in the noise determination area as noise (abnormal values) and removes the point cloud data points determined to be noise (step A3).

[0074] Next, the voxel unit 15c performs voxel processing on the point cloud data of the specified estimation region (step A4).

[0075] Next, the classification unit 11 uses sensors S1 to S9 installed on the crane 1 and the mobile body 100 to measure point cloud data of the hollow target structures 31 (31a, 31b, 31c) located around the crane 1 and separated from the crane 1. This data is then classified according to similar features using clustering processing based on normal segmentation to generate classes (Step A5).

[0076] Next, the vertical plane estimation unit 12 estimates the vertical planes using the point cloud data included for each class (step A6). Specifically, in step A6, the vertical plane estimation unit 12 defines a plane as a vertical plane if the inclination of the estimated plane with respect to the horizontal plane falls within a preset vertical inclination range. Next, in step A6, the vertical plane estimation unit 12 uses the vertical planes to determine the combination of opposite sides (long side and short side).

[0077] Next, if there are multiple estimated vertical surfaces, the corner estimation unit 13 uses the multiple vertical surfaces to estimate the position coordinates of the corners on the upper surface of the target structure 31 (31a, 31b, 31c) (step A7).

[0078] Specifically, in step A7, the corner estimation unit 13 first estimates the intersection lines 71a of vertical planes 61 and 62, 71b of vertical planes 62 and 63, 71c of vertical planes 63 and 64, and 71d of vertical planes 64 and 61, as shown in Figure 7.

[0079] Next, the corner estimation unit 13 estimates the upper surface 73 (the surface composed of line segments 72a, 72b, 72c, and 72d in Figure 7) which is formed by the vertical surfaces 61 to 64. Next, the corner estimation unit 13 estimates the position coordinates of the intersection points (corner candidate points) 74a, 74b, 74c, 74d, and 74e of the intersection lines 71a to 71d and the upper surface 73.

[0080] Next, the corner estimation unit 13 uses the position coordinates of the candidate corner points 74a, 74b, 74c, 74d, and 74e to determine whether they are included in the pre-set range of position coordinates for the vertices of the upper surface 73 of the target structure 31. Subsequently, the corner estimation unit 13 sets the position coordinates of the corners of the upper surface 73 of the target structure 31 (31a, 31b, 31c) to the candidate corner points 74a, 74b, 74c, and 74d that are included in the range of position coordinates for the vertices of the upper surface 73.

[0081] Furthermore, if the number of candidate corner points is less than the number of vertices on the upper surface 73 of the target structure 31 (31a, 31b, 31c) that has been set in advance (i.e., the vertical surface is not visible due to occlusion), the corner estimation unit 13 interpolates to increase the number of candidate corner points based on the position coordinate range.

[0082] Furthermore, if the number of candidate corner points is greater than or equal to the number of vertices on the upper surface 73 of the target structure 31 (31a, 31b, 31c) that has been set in advance, the corner estimation unit 13 reduces the number of candidate corner points based on the position coordinate range.

[0083] During operation, the control unit 16 acquires control information (including information such as the position coordinates of the corners on the upper surface of the target structure 31) for controlling the crane 1, and controls the crane 1 based on the control information (step A8).

[0084] [Effects of the Embodiment] As described above, according to the embodiment, point cloud data of a hollow structure can be classified to generate classes, vertical planes can be estimated using the point cloud data for each class, and the position coordinates of the corners of the upper surface of the structure can be estimated using multiple vertical planes.

[0085] [program] The program in this embodiment can be any program that causes a computer to execute steps A1 to A8 shown in Figure 8. By installing and running this program on a computer, the information processing device and information processing method in this embodiment can be realized. In this case, the computer's processor functions as a conversion unit 14, a preprocessing unit 15, a classification unit 11, a vertical plane estimation unit 12, a corner estimation unit 13, and a control unit 16, and performs the processing.

[0086] Furthermore, the program in the embodiment may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the following: the conversion unit 14, the preprocessing unit 15, the classification unit 11, the vertical plane estimation unit 12, the corner estimation unit 13, or the control unit 16.

[0087] [Physical configuration] Here, a computer that implements an information processing device by executing the program in the embodiment will be described using Figure 9. Figure 9 is a diagram illustrating an example of a computer that implements an information processing device in the embodiment.

[0088] As shown in Figure 9, the computer 110 comprises a CPU (Central Processing Unit) 111, main memory 112, storage device 113, input interface 114, display controller 115, data reader / writer 116, and communication interface 117. These components are connected to each other via a bus 121, enabling data communication. In addition to the CPU 111, or in place of the CPU 111, the computer 110 may also include a GPU or FPGA.

[0089] The CPU 111 loads the program in the embodiment, which consists of a set of codes stored in the storage device 113, into the main memory 112, and performs various calculations by executing each code in a predetermined order. The main memory 112 is typically a volatile storage device such as DRAM (Dynamic Random Access Memory).

[0090] Furthermore, the program in this embodiment is provided stored on a computer-readable recording medium 120. The program in this embodiment may also be distributed over the Internet via a communication interface 117.

[0091] Specific examples of the storage device 113 include hard disk drives and semiconductor storage devices such as flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

[0092] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0093] Specific examples of the recording medium 120 include general-purpose semiconductor memory devices such as CF (Compact Flash®) and SD (Secure Digital), magnetic recording media such as Flexible Disks, and optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0094] Furthermore, the information processing device 10 in this embodiment can be implemented not by a computer on which a program is installed, but by using hardware corresponding to each part, such as electronic circuits. Moreover, the information processing device 10 may be partially implemented by a program and the remaining part by hardware. In this embodiment, the computer is not limited to the computer shown in Figure 9.

[0095] [Note] The following additional notes are disclosed regarding the embodiments described above. Some or all of the embodiments described above can be expressed by (Note 1) to (Note 21) below, but are not limited to the following descriptions.

[0096] (Note 1) A classification unit generates classes by classifying point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, according to similar features. A vertical surface estimation unit estimates a surface using the point cloud data included for each class, and defines a surface as a vertical surface if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range. If there are multiple estimated vertical surfaces, a corner estimation unit estimates the position coordinates of the corners of the upper surface of the structure using the multiple vertical surfaces, An information processing device having

[0097] (Note 2) The system includes a conversion unit that converts the local coordinate system of the sensor to the world coordinate system of the crane. The information processing device described in Appendix 1.

[0098] (Note 3) The system has a region designation unit that specifies the region in which to estimate the structure using the point cloud data. The information processing device described in Appendix 2.

[0099] (Note 4) A noise determination unit sets up multiple noise determination areas in the specified area, determines the point cloud data in the noise determination area as noise if the number of point cloud data included in the noise determination area is less than or equal to a preset reference value, and removes the point cloud data determined to be noise. A voxel unit that performs voxel processing on point cloud data of the specified region, The information processing device described in Appendix 3.

[0100] (Note 5) The corner estimation unit is, The intersection line of the vertical planes and the upper surface of the vertical planes are estimated, the position coordinates of the corner candidate point at the intersection of the intersection line and the upper surface are estimated, and it is determined whether or not the position coordinates of the corner candidate point are included in a predetermined range of position coordinates for the vertices of the upper surface of the structure. If it is included, the position coordinates of the corner candidate point are set as the position coordinates of the corner of the upper surface of the structure. The information processing device described in Appendix 1.

[0101] (Note 6) The corner estimation means is, If the number of candidate corner points is less than the number of vertices on the upper surface of the structure that has been set in advance, interpolation is performed to increase the number of candidate corner points based on the position coordinate range. If the number of candidate corner points is greater than or equal to the number of vertices on the upper surface of the structure that have been set in advance, the number of candidate corner points is reduced based on the position coordinate range. The information processing device described in Appendix 5.

[0102] (Note 7) The crane is mounted on a dredger, and the structure is a pollution control frame, a barge frame, or both. The information processing device described in Appendix 1.

[0103] (Note 8) Information processing device, Point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, are classified according to similar features to generate classes. Using the point cloud data included in each class, a surface is estimated, and if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, that surface is defined as a vertical surface. If there are multiple estimated vertical surfaces, the position coordinates of the corners on the upper surface of the structure are estimated using the multiple vertical surfaces. Information processing methods.

[0104] (Note 9) The aforementioned information processing device The local coordinate system of the sensor is converted to the world coordinate system of the crane. The information processing method described in Appendix 8.

[0105] (Note 10) The aforementioned information processing device Using the aforementioned point cloud data, specify the region in which to estimate the structure. The information processing method described in Appendix 9.

[0106] (Note 11) The aforementioned information processing device In the specified region, multiple noise determination regions are set, and if the number of point cloud data included in the noise determination region is less than or equal to a preset reference value, the point cloud data in the noise determination region is determined to be noise, and the point cloud data determined to be noise is removed. Voxel processing is performed on the point cloud data of the specified region. The information processing method described in Appendix 10.

[0107] (Note 12) The aforementioned information processing device The intersection line of the vertical planes and the upper surface of the vertical planes are estimated, the position coordinates of the corner candidate point at the intersection of the intersection line and the upper surface are estimated, and it is determined whether or not the position coordinates of the corner candidate point are included in a predetermined range of position coordinates for the vertices of the upper surface of the structure. If it is included, the position coordinates of the corner candidate point are set as the position coordinates of the corner of the upper surface of the structure. The information processing method described in Appendix 8.

[0108] (Note 13) The aforementioned information processing device If the number of candidate corner points is less than the number of vertices on the upper surface of the structure that has been set in advance, interpolation is performed to increase the number of candidate corner points based on the position coordinate range. If the number of candidate corner points is greater than or equal to the number of vertices on the upper surface of the structure that have been set in advance, the number of candidate corner points is reduced based on the position coordinate range. The information processing method described in Appendix 12.

[0109] (Note 14) The crane is mounted on a dredger, and the structure is a pollution control frame, a barge frame, or both. The information processing device described in Appendix 8.

[0110] (Note 15) On the computer, Point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, are classified according to similar features to generate classes. Using the point cloud data included in each class, a surface is estimated, and if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, that surface is defined as a vertical surface. If there are multiple estimated vertical surfaces, the position coordinates of the corners on the upper surface of the structure are estimated using the multiple vertical surfaces. A program that executes a process.

[0111] (Note 16) To the aforementioned computer, The local coordinate system of the sensor is converted to the world coordinate system of the crane. The program described in Appendix 15 that executes the process.

[0112] (Note 17) To the aforementioned computer, Using the aforementioned point cloud data, specify the region in which to estimate the structure. The program described in Appendix 16 that executes the process.

[0113] (Note 18) To the aforementioned computer, In the specified region, multiple noise determination regions are set, and if the number of point cloud data included in the noise determination region is less than or equal to a preset reference value, the point cloud data in the noise determination region is determined to be noise, and the point cloud data determined to be noise is removed. Voxel processing is performed on the point cloud data of the specified region. The program described in Appendix 17 that executes the process.

[0114] (Note 19) To the aforementioned computer, The intersection line of the vertical planes and the upper surface of the vertical planes are estimated, the position coordinates of the corner candidate point at the intersection of the intersection line and the upper surface are estimated, and it is determined whether or not the position coordinates of the corner candidate point are included in a predetermined range of position coordinates for the vertices of the upper surface of the structure. If it is included, the position coordinates of the corner candidate point are set as the position coordinates of the corner of the upper surface of the structure. The program described in Appendix 15 that executes the process.

[0115] (Note 20) To the aforementioned computer, If the number of candidate corner points is less than the number of vertices on the upper surface of the structure that has been set in advance, interpolation is performed to increase the number of candidate corner points based on the position coordinate range. If the number of candidate corner points is greater than or equal to the number of vertices on the upper surface of the structure that have been set in advance, the number of candidate corner points is reduced based on the position coordinate range. The program described in Appendix 19 that executes the process.

[0116] (Note 21) The crane is mounted on a dredger, and the structure is a pollution control frame, a barge frame, or both. The program described in Appendix 15.

[0117] Although the invention has been described above with reference to embodiments, the invention is not limited to the embodiments described above. Various modifications to the structure and details of the invention can be made that will be understood by those skilled in the art within the scope of the invention. [Industrial applicability]

[0118] According to the above description, it is possible to estimate the position coordinates of the corners on the upper surface of a hollow structure. Furthermore, it is useful in fields where it is necessary to estimate the position coordinates of the corners on the upper surface of a hollow structure. [Explanation of Symbols]

[0119] 1. Claim 1a Swivel section 1b Jib S1, S2, S3, S4, S5, S6, S7, S8, S9 sensors 10 Information Processing Devices 11 Classification section 12 Vertical plane estimation part 13 Corner Estimation Section 14 Conversion section 15 Pre-processing section 15a Area specification part 15b Noise reduction section 15c voxel section 16 Control Unit 31, 31a, 31b, 31c Target structures 100 Mobile Units 110 Computer 111 CPU 112 Main Memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Readers / Writers 117 Communication Interface 118 Input devices 119 Display device 120 recording media 121 Bus

Claims

1. A classification means that generates classes by classifying point cloud data of hollow structures located around the crane and separated from the crane, measured using sensors installed on the crane, according to similar features, A vertical surface estimation means that estimates a surface using the point cloud data included in each of the aforementioned classes, and defines a surface as a vertical surface if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, If there are multiple estimated vertical surfaces, a corner estimation means for estimating the position coordinates of the corners of the upper surface of the structure using the multiple vertical surfaces, An information processing device having

2. The system has a transformation means for converting the local coordinate system of the sensor to the world coordinate system of the crane. The information processing apparatus according to claim 1.

3. The system has a region designation means for designating a region in which the structure is estimated using the point cloud data. The information processing apparatus according to claim 2.

4. A noise determination means sets up multiple noise determination areas in the specified area, determines the point cloud data in the noise determination area as noise if the number of point cloud data included in the noise determination area is less than or equal to a preset reference value, and removes the point cloud data determined to be noise. A voxel means for performing voxel processing on point cloud data of the specified region, The information processing apparatus according to claim 3, having the following features.

5. The corner estimation means is, The intersection line of the vertical planes and the upper surface of the vertical planes are estimated, the position coordinates of the corner candidate point at the intersection of the intersection line and the upper surface are estimated, and it is determined whether or not the position coordinates of the corner candidate point are included in a predetermined range of position coordinates for the vertices of the upper surface of the structure. If it is included, the position coordinates of the corner candidate point are set as the position coordinates of the corner of the upper surface of the structure. The information processing apparatus according to claim 1.

6. The corner estimation means is, If the number of candidate corner points is less than the number of vertices on the upper surface of the structure that has been set in advance, interpolation is performed to increase the number of candidate corner points based on the position coordinate range. If the number of candidate corner points is greater than or equal to the number of vertices on the upper surface of the structure that have been set in advance, the number of candidate corner points is reduced based on the position coordinate range. The information processing apparatus according to claim 5.

7. The crane is mounted on a dredger, and the structure is a pollution control frame, a barge frame, or both. The information processing apparatus according to claim 1.

8. Information processing device, Point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, are classified according to similar features to generate classes. Using the point cloud data included in each class, a surface is estimated, and if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, that surface is defined as a vertical surface. If there are multiple estimated vertical surfaces, the position coordinates of the corners on the upper surface of the structure are estimated using the multiple vertical surfaces. Information processing methods.

9. On the computer, Point cloud data of hollow structures located around the crane and the crane separated from the crane, measured using sensors installed on the crane, are classified according to similar features to generate classes. Using the point cloud data included in each class, a surface is estimated, and if the inclination of the estimated surface with respect to the horizontal plane falls within a predetermined vertical inclination range, that surface is defined as a vertical surface. If there are multiple estimated vertical surfaces, the position coordinates of the corners on the upper surface of the structure are estimated using the multiple vertical surfaces. A program that executes a process.

Images