Information processing device, control method, program, and storage medium

Abstract

To provide an information processor which can accurately estimate the attitude of a measurement device.SOLUTION: A controller 13 of an information processor 1 functions as acquisition means, normal line calculation means, and attitude estimation means. The acquisition means acquires measurement data of an object measured by a measurement device provided on a marine vessel. The normal line calculation means calculates the normal line of the object, on the basis of the measurement data. The attitude estimation means estimates the attitude of the measurement device, on the basis of the normal line of the object.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] This disclosure relates to the procedures for when a vessel is docked. [Background technology]

[0002] Technologies for assisting with ship docking (berthing) have been known for some time. For example, Patent Document 1 describes a method for controlling the attitude of a ship in an automatic docking device that performs automatic ship docking, such that light emitted from a lidar is reflected by objects around the docking position and received by the lidar. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-59403 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] When using measuring devices such as lidars to assist with docking, it is conceivable to install the measuring device sideways near the side of the vessel so that the quay can be captured within the field of view. However, since the point cloud data obtained by the measuring device is in a coordinate system based on the measuring device, it is necessary to perform a coordinate transformation to the vessel coordinate system based on the position of the measuring device from the reference point of the vessel's coordinate system and the attitude of the measuring device relative to the vessel. If the aforementioned position and attitude required for the coordinate transformation are not accurate, accurate point cloud data will not be generated. Furthermore, since the crew and cargo are expected to differ from one operation to the next, the vessel's attitude may change due to these factors. However, measuring devices that do not have built-in sensors capable of measuring acceleration and angular velocity cannot detect changes in attitude, so even if the vessel's attitude changes, an incorrect coordinate transformation will be performed based on the vessel's coordinate system before the attitude change. In this case as well, inaccurate point cloud data will be generated.

[0005] The present disclosure has been made to solve the above problems, and a main object thereof is to provide an information processing apparatus capable of accurately estimating the attitude of a measuring device.

Means for Solving the Problems

[0006] The invention according to the claim is acquisition means for acquiring measurement data of quay measured by a measuring device provided on a ship; normal line calculation means for calculating the normal line of the quay based on the measurement data; attitude estimation means for estimating the attitude of the measuring device based on the normal line; and an information processing apparatus having the above.

[0007] Also, the invention according to the claim is a control method executed by a computer, the method comprising: acquiring measurement data of quay measured by a measuring device provided on a ship; calculating the normal line of the quay based on the measurement data; estimating the attitude of the measuring device based on the normal line. This is a control method.

[0008] Also, the invention according to the claim is a program for causing a computer to execute a process of acquiring measurement data of quay measured by a measuring device provided on a ship, calculating the normal line of the quay based on the measurement data, and estimating the attitude of the measuring device based on the normal line.

Brief Description of the Drawings

[0009] [Figure 1] It is a schematic configuration diagram of an operation support system. [Figure 2]This is a block diagram showing the hardware configuration of an information processing device. [Figure 3] (A) An overhead view showing the lidar mounted laterally on the port side of the target vessel, with the quay located on the port side of the target vessel. (B) A front view showing the lidar mounted laterally on the port side of the target vessel, with the quay located on the port side of the target vessel. [Figure 4] (A) Shows the distribution of the quay point cloud when the point cloud data generated by the lidar is treated directly as ship coordinate system data without coordinate transformation. (B) Shows the distribution of the quay point cloud when appropriate coordinate transformation is performed. [Figure 5] (A) A front view of the vessel and the quay when the vessel is kept horizontal. (B) A front view of the vessel and the quay when the vessel changes direction in the rolling direction. [Figure 6] (A) A side view of the target vessel and the quay when the target vessel is kept horizontal. (B) A side view of the target vessel and the quay when the target vessel changes direction in the pitch direction. [Figure 7] This is an example of a controller function block related to coordinate transformation of point cloud data. [Figure 8] (A) Shows the quay point cloud in the ship's coordinate system without performing coordinate transformation on the point cloud data obtained when the target vessel's attitude was not horizontal. (B) Shows the quay point cloud in the ship's coordinate system after performing coordinate transformation based on this embodiment on the point cloud data obtained when the target vessel's attitude was not horizontal. [Figure 9] (A) Shows the normal to the top of the quay in the ship's coordinate system when the target vessel is kept level. (B) Shows the normal to the top of the quay in the ship's coordinate system when the target vessel is tilted. [Figure 10] (A) to (D) show the flow of the normal vector calculation process. [Figure 11] (A) to (C) show the process for calculating the normal vector for each measurement point in the quay wall point cloud. [Figure 12] (A) This figure shows the change in pitch angle in the ship's coordinate system. (B) This figure shows the change in roll angle in the ship's coordinate system. [Figure 13] This is an example of a flowchart illustrating the overview of point cloud data processing. [Figure 14] This is a flowchart for calculating the normal vector of the quay wall. [Figure 15] This is a flowchart for the process of estimating the amount of change in posture. [Figure 16] This is a flowchart of the coordinate transformation process. [Modes for carrying out the invention]

[0010] According to a preferred embodiment of this disclosure, the information processing device includes acquisition means for acquiring measurement data of an object measured by a measuring device installed on a ship, normal calculation means for calculating the normal of the object based on the measurement data, and attitude estimation means for estimating the attitude of the measuring device based on the normal. According to this embodiment, the information processing device can accurately estimate the attitude of the measuring device.

[0011] In one embodiment of the above-described information processing device, the information processing device further includes coordinate transformation means that perform a coordinate transformation to convert the measurement data into data in a ship coordinate system based on the ship, based on the attitude estimation result. In this embodiment, the information processing device can accurately perform the coordinate transformation of the measurement data.

[0012] In another embodiment of the information processing apparatus described above, the coordinate transformation means performs the coordinate transformation based on a plurality of attitude estimation results obtained within a predetermined period. In this embodiment, the information processing apparatus can suitably perform the coordinate transformation based on attitude estimation results in which the influence of disturbances has been reduced.

[0013] In another embodiment of the information processing device described above, the normal calculation means calculates the normal to a predetermined surface formed on the object. In this embodiment, the information processing device can suitably calculate a normal that serves as a reference in estimating the attitude of a measuring device.

[0014] In another embodiment of the information processing device described above, the measurement data is point cloud data representing a plurality of points to be measured, and the normal calculation means calculates the normal based on the points to be measured on the surface represented by the point cloud data. In this embodiment, the information processing device can suitably calculate the normal of an object to a predetermined surface.

[0015] In another embodiment of the information processing device described above, the normal calculation means calculates a normal for each measured point of the object using surrounding measured points, and extracts the measured points of the surface based on the vector components of the normal. In this embodiment, the information processing device can suitably extract the measured points of a predetermined surface of the object from point cloud data.

[0016] In a preferred example, the object is a quay, and the normal calculation means calculates the normal to the upper surface of the quay. In another preferred example, the attitude estimation means estimates at least one of the pitch angle and roll angle of the measuring device based on the normal.

[0017] According to another preferred embodiment of the present disclosure, a control method performed by a computer includes acquiring measurement data of an object measured by a measuring device installed on a ship, calculating the normal vector of the object based on the measurement data, and estimating the attitude of the measuring device based on the normal vector. By performing this control method, the computer can accurately estimate the attitude of the measuring device.

[0018] According to another preferred embodiment of the present disclosure, the program causes a computer to perform the following processes: acquire measurement data of an object measured by a measuring device installed on a ship, calculate the normal vector of the object based on the measurement data, and estimate the attitude of the measuring device based on the normal vector. By executing this program, the computer can accurately estimate the attitude of the measuring device. Preferably, the program is stored on a storage medium. [Examples]

[0019] Preferred embodiments of the present invention will be described below with reference to the drawings.

[0020] (1) Overview of the flight support system Figure 1 shows a schematic configuration of the navigation support system according to this embodiment. The navigation support system comprises an information processing device 1 that moves together with the vessel, which is a moving object, and a group of sensors 2 mounted on the vessel. Hereafter, the vessel on which the navigation support system is installed will also be referred to as the "target vessel".

[0021] The information processing device 1 is electrically connected to the sensor group 2 and provides operational support for the target vessel based on the outputs of the various sensors included in the sensor group 2. Operational support may include docking support such as automatic docking. In this embodiment, the information processing device 1 estimates the attitude of the lidar 3 based on the point cloud data of the quay measured by the lidar 3 included in the sensor group 2, and converts the point cloud data into a coordinate system based on the target vessel (also called the "ship coordinate system"). Hereafter, as an example, an example of estimating the attitude of the lidar 3 based on data measured on the quay will be shown, but it is not limited to a quay; any object that can be measured by the lidar 3 (an object with a surface oriented in a predetermined direction) is acceptable.

[0022] The information processing device 1 may be a navigation device installed on a ship, or it may be an electronic control device built into a ship.

[0023] Sensor group 2 includes various external and internal sensors installed on the ship. In this embodiment, sensor group 2 includes, for example, a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3.

[0024] LIDA 3 is an external sensor that discretely measures the distance to an object in the external environment by emitting a pulsed laser within a predetermined angular range in the horizontal and vertical directions, and generates three-dimensional point cloud data indicating the position of the object. Specifically, LIDA 3 has an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data based on the received signal output by the light receiving unit. The data measured for each direction of laser light irradiation (scanning position) is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light specified based on the received signal described above. Hereafter, the point measured by the irradiation of laser light within the measurement range of LIDA 3, or the measured data thereof, will also be called the "measured point". Note that the point cloud data can be considered as an image (frame) in which each measurement direction is a pixel, and the measured distance and reflectance value in each measurement direction are used as pixel values. In this case, the direction of laser beam emission (i.e., measurement direction) differs depending on the elevation angle in the vertical arrangement of pixels, and the direction of laser beam emission differs depending on the horizontal angle in the horizontal arrangement of pixels.

[0025] Furthermore, LIDA 3 is not limited to the scanning type LIDA described above, but may also be a flash type LIDA that generates 3D data by diffusing laser light into the field of view of a 2D array sensor. LIDA 3 is an example of a "measuring device" in the present invention.

[0026] (2) Configuration of an information processing device Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly consists of an interface 11, a memory 12, and a controller 13. Each of these elements is interconnected via a bus line.

[0027] Interface 11 performs interface operations related to the exchange of data between the information processing device 1 and external devices. In this embodiment, interface 11 acquires output data from each sensor in the sensor group 2 and supplies it to the controller 13. Interface 11 also supplies signals related to the control of the target vessel, generated by the controller 13, to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel includes a drive source such as an engine or electric motor, a propeller that generates thrust in the direction of travel based on the driving force of the drive source, a thruster that generates lateral thrust based on the driving force of the drive source, and a rudder, etc., which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation such as automatic docking, interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, interface 11 supplies control signals generated by the controller 13 to the electronic control device. Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices by cables, etc. Furthermore, interface 11 may perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0028] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0029] Furthermore, memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, memory 12 may store map data including information about the location of the docking place. In another example, memory 12 stores information about the downsampling size when downsampling is performed on the point cloud data obtained when the lidar 3 performs one cycle of scanning.

[0030] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to the operation support of the target vessel by executing a program stored in memory 12, etc. The controller 13 functions as an "acquisition means," a "normal vector calculation means," an "attitude estimation means," and a computer that executes the program. The detailed block configuration of the controller 13 will be described later.

[0031] Furthermore, the processing performed by the controller 13 is not limited to being implemented by software through a program, but may also be implemented by a combination of hardware, firmware, and software. Additionally, the processing performed by the controller 13 may be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program executed by the controller 13 in this embodiment may be implemented using this integrated circuit.

[0032] (3) Overview of point cloud data processing Next, we will explain the processing of point cloud data. When handling point cloud data in the ship's coordinate system, the point cloud data generated by LIDA 3 is in a coordinate system based on LIDA 3 (also called the "LIDA coordinate system"). Therefore, the controller 13 needs to perform a coordinate transformation of the point cloud data using the position and attitude of LIDA 3 relative to the target ship.

[0033] FIG. 3(A) is an overhead view showing the state where the lidar 3 is mounted horizontally on the left side of the target ship and a quay wall is located on the left side of the target ship, and FIG. 3(B) is a front view thereof. Here, the ship coordinate system is (X S , Y S , Z S ), and the lidar coordinate system is (X L , Y L , Z L ). Also, the X S axis coincides with the bow direction, the Y S axis coincides with the left side direction, and the Z S axis is an axis perpendicular to the X S axis and the Y S axis. Also, the X L axis coincides with the front direction of the lidar 3, the Y L axis coincides with the left direction of the lidar 3, and the Z L axis is an axis perpendicular to the X L axis and the Y L axis. As shown in FIGS. 3(A) and 3(B), since the X-axis and the Y-axis are basically different between the lidar coordinate system and the ship coordinate system, it is necessary to perform coordinate conversion in order to appropriately handle the point cloud data in the ship coordinate system.

[0034] FIG. 4(A) shows the distribution of the measured points of the quay wall (also referred to as "quay wall point cloud") when the point cloud data generated by the lidar 3 is directly treated as data in the ship coordinate system without coordinate conversion, and FIG. 4(B) shows the distribution of the quay wall point cloud when appropriate coordinate conversion is performed. The solid circles in the figure represent the measured points.

[0035] As shown in FIG. 4(A), if coordinate conversion is not performed, the lidar coordinate system will overlap the ship coordinate system, and the quay wall point cloud shown by the point cloud data will deviate from the originally measured quay wall position. On the other hand, by performing coordinate conversion of the point cloud data using the position and orientation of the lidar 3 with respect to the target ship, as shown in FIG. 4(B), it is possible to obtain point cloud data in the ship coordinate system that accurately represents the originally measured quay wall position.

[0036] Here, we will provide a supplementary explanation regarding the problems related to coordinate transformations due to changes in the attitude of the target vessel.

[0037] Generally, the crew and cargo may differ from one operation to the next, which means the attitude of the vessel may change. While ballast water generally helps maintain a horizontal position, even with ballast, a perfectly horizontal position may not always be achievable. Furthermore, many small vessels lack ballast.

[0038] Figure 5(A) shows a front view of the target vessel and the quay when the target vessel is kept horizontal, and Figure 5(B) shows a front view of the target vessel and the quay when the target vessel changes direction in the roll direction. Figure 6(A) shows a side view of the target vessel and the quay when the target vessel is kept horizontal, and Figure 6(B) shows a side view of the target vessel and the quay when the target vessel changes direction in the pitch direction. In each figure, the quay point cloud represented by the coordinate transformation of the point cloud data after the coordinate transformation is performed without considering the change in the attitude of the target vessel (i.e., based on the position and attitude of LIDA 3 when horizontal) is indicated by a filled circle.

[0039] Point cloud data acquired when the attitude of the target vessel has changed needs to be transformed based on the position and attitude of the LIDA 3 relative to the changed attitude of the target vessel. However, since the LIDA 3 does not have a built-in attitude detection sensor such as an IMU, it cannot detect the change in attitude. Therefore, the point cloud data after the change in the attitude of the target vessel is transformed based on the position and attitude of the vessel when it is horizontal. As a result, point cloud data that differs from the actual position is output as point cloud data after coordinate transformation. Taking the above into consideration, in this embodiment, the controller 13 accurately detects the amount of change in the attitude of the LIDA 3 in accordance with the change in the attitude of the target vessel and performs coordinate transformation of the point cloud data according to the detection result.

[0040] (4) Functional Blocks Figure 7 shows an example of the functional blocks of the controller 13 related to the coordinate transformation of point cloud data. Functionally, the controller 13 includes a quay point cloud acquisition unit 14, a normal vector calculation unit 15, an attitude change amount estimation unit 16, and a coordinate transformation unit 17. Furthermore, the data output by the normal vector calculation unit 15, the attitude change amount estimation unit 16, and the coordinate transformation unit 17 are outlined in callouts 90-92. In Figure 7, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to this. The same applies to the diagrams of other functional blocks described later.

[0041] The quay wall point cloud acquisition unit 14 acquires the point cloud data generated by the lidar 3 at each frame period and extracts the quay wall point cloud from the point cloud data. In this case, the quay wall point cloud acquisition unit 14 extracts the quay wall point cloud based on any clustering method such as Euclidean clustering. The quay wall point cloud acquisition unit 14 also removes data located below the water surface position from the point cloud data generated by the lidar 3 as water surface measurement data (i.e., false detection data). The quay wall point cloud acquisition unit 14 estimates the water surface position based, for example, on the average value in the height direction of the point cloud data generated by the lidar 3 when there are no objects other than the water surface in the surrounding area. The quay wall point cloud acquisition unit 14 may also perform downsampling on the point cloud data after removing the water surface reflection data, which is a process that integrates the measured points into a grid space of a predetermined size. Downsampling may be performed before the removal of false detection data.

[0042] The normal vector calculation unit 15 extracts the quay point cloud representing the upper surface of the quay (also called the "upper quay point cloud") from the quay point cloud acquired by the quay point cloud acquisition unit 14, and calculates the normal vector of the upper quay point cloud. Note that the callout 90 shows the upper quay point cloud indicated by the point cloud data that has undergone coordinate transformation without considering the change in the attitude of the target vessel, and this upper quay point cloud is shifted from the actual upper surface of the quay by the amount of the change in the attitude of the target vessel.

[0043] The attitude change estimation unit 16 estimates the attitude change of the lidar 3 based on the normal vector of the upper quay wall point cloud. In this case, the attitude change estimation unit 16 calculates the change in the pitch angle of the lidar 3 (also called "pitch angle Δθ") and the change in the roll angle of the lidar 3 (also called "roll angle Δφ") as the attitude change of the lidar 3. Callout 91 shows an overview of the calculation method for the pitch angle Δθ and roll angle Δφ in the ship coordinate system.

[0044] The coordinate transformation unit 17 performs coordinate transformation of the point cloud data based on the change in the attitude of the target vessel. In this case, as indicated by the callout 92, the coordinate transformation of the point cloud data is performed so that the normal of the upper quay point cloud shown by the transformed point cloud data matches the normal of the actual quay. Processes for converting point cloud data in a coordinate system based on a lidar installed on a moving body (in this embodiment, the vessel) to the coordinate system of the moving body, and processes for converting from the coordinate system of the moving body to the world coordinate system, etc., are disclosed, for example, in International Publication WO2019 / 188745.

[0045] Figure 8(A) shows the quay point cloud when the point cloud data obtained when the target vessel's attitude is not horizontal is treated directly as point cloud data in the ship's coordinate system, while Figure 8(B) shows the quay point cloud when the point cloud data obtained when the target vessel's attitude is not horizontal is converted to the ship's coordinate system based on this embodiment. As shown in Figure 8(A), if no coordinate transformation is performed according to the target vessel's attitude, point cloud data will be generated that shows a quay point cloud that is shifted from the actual position of the quay. On the other hand, as shown in Figure 8(B), if a coordinate transformation is performed according to the target vessel's attitude, point cloud data will be generated that shows a quay point cloud that is in line with the actual position of the quay.

[0046] (5) Normal vector of the point group on the upper quay wall Next, we will explain a specific example of a method for calculating the normal vector of the point cloud on the upper quay wall.

[0047] Figure 9(A) shows the normal vector "N0" of the top surface of the quay in the ship coordinate system when the target vessel is kept horizontal, and Figure 9(B) shows the normal vector "N1" of the top surface of the quay in the ship coordinate system when the target vessel is tilted (i.e., when its attitude changes). As shown in Figure 9(A), the normal vector N0 of the top surface of the quay when the attitude of the target vessel is horizontal is the Z coordinate system of the ship coordinate system. S The axis and direction become equal. On the other hand, as shown in Figure 9(B), when the attitude of the target vessel is not horizontal, the normal N1 to the top surface of the quay is the Z coordinate system of the vessel. S The axis and orientation are different. Below, we will explain a specific example of how to calculate the normal to the top surface of the quay wall.

[0048] Figures 10(A) to 10(D) show the flow of the calculation process for the normal of the upper surface of the quay wall. First, as shown in Figure 10(A), the quay wall point cloud acquisition unit 14 extracts the quay wall point cloud based on an arbitrary clustering method such as Euclidean clustering.

[0049] Next, the normal calculation unit 15 calculates the normal vector for each measured point in the quay wall point cloud, as shown in Figure 10(B). For example, for each measured point, the normal calculation unit 15 performs principal component analysis using other measured points located within a predetermined distance from the target measured point, and calculates the third principal component vector obtained by the principal component analysis as the normal vector for the target measured point. The normal calculation unit 15 then extracts the measured points whose Z-axis component of the normal vector is above a predetermined threshold as the upper quay wall point cloud. The normal calculation unit 15 then performs principal component analysis on the point cloud data of the upper quay wall point cloud, as shown in Figure 10(C), and calculates the third principal component vector obtained by the principal component analysis as the normal vector for the upper quay wall. Figure 10(D) shows the normal vector N1 for the upper quay wall calculated when the attitude of the target vessel is not horizontal.

[0050] Here, we will provide a supplementary explanation of a specific example of how to calculate the normal vectors for each measured point in the quay wall point cluster shown in Figure 10(B). Figures 11(A) to 11(C) show the flow of the normal vector calculation process for each measured point in the quay wall point cluster. Here, the measured points for which normal vectors are to be calculated are called "target points".

[0051] Generally, calculating normals requires estimating a plane, thus requiring at least three points. Therefore, a radius R1 is set such that at least two points are included in the space centered on the target point. The radius R1 is pre-stored, for example, in memory 12. Note that if the radius R1 is too small, the direction of the normal will not be stable, and if it is too large, the change will be small, so it needs to be set appropriately depending on the target point cloud.

[0052] First, as shown in Figure 11(A), the normal calculation unit 15 recognizes the point to be measured that is within a radius R1 from the target point. Then, as shown in Figure 11(B), the normal calculation unit 15 performs principal component analysis on the recognized point to be measured and recognizes the first principal component axis, the second principal component axis, and the third principal component axis. Then, as shown in Figure 11(C), it recognizes the third principal component axis as the normal to the target point.

[0053] (6) Calculation of posture change Next, we will specifically explain how to calculate the attitude change amount of Lida 3 based on the normal to the top surface of the quay.

[0054] The normal vector N1 = [N 1x ,N 1y ,N 1z ] T The normal vector N0 = [N] when the ship's attitude is kept horizontal is given by [N]. 0x ,N 0y ,N 0z ] T It can be obtained by rotating the coordinate axes and is expressed by the following equation (1).

[0055]

number

[0056]

number

[0057]

number

[0058]

number

[0059] Furthermore, the roll angle Δφ is calculated based on equation (2). First, based on equation (2), the following equation (5) is derived.

[0060]

number

[0061]

number

[0062]

number

[0063] As described above, based on the vector component of the normal to the upper surface of the quay wall, the attitude change amount estimation unit 16 can suitably calculate the pitch angle Δθ and roll angle Δφ corresponding to the attitude deformation amount using equations (4) and (7).

[0064] Preferably, the attitude change amount estimation unit 16 may average the attitude change amount estimation results from multiple times, taking into consideration that the estimation results of the attitude change amount, which are performed for each frame period, will fluctuate due to the rocking of the ship caused by waves and wind.

[0065] Generally, in order to determine the constant change in attitude, it is necessary to exclude as much as possible the temporary change in attitude caused by environmental factors. Furthermore, the change in attitude of a ship increases or decreases due to rocking caused by waves and wind. Specifically, if the change in attitude of the ship is α, the change in attitude due to waves and wind is β, and the estimated change in attitude is A, then the following equation holds. A = α + β

[0066] Therefore, the attitude change amount estimation unit 16 determines the average α (see equation (8) below) obtained when the attitude change amount estimation is performed n times within the preceding fixed period as the attitude change amount to be used in the coordinate transformation.

[0067]

number

[0068] (7) Coordinate transformation Here, I will provide some additional explanation regarding coordinate transformation of point cloud data.

[0069] Generally, point cloud data is a collection of 3D coordinate values ​​and can therefore be represented as a matrix. The point cloud data output by LIDA3 is "P0=[P0 x ,P0 y ,P0 z ] T Assuming that the attitude of the target vessel is kept horizontal, the point cloud data obtained by transforming P0 to the ship's coordinate system is "P1=[P1 x ,P1 y ,P1 z ] TFurthermore, for point cloud data when the attitude of the target vessel is no longer horizontal, the point cloud data obtained by transforming the coordinates to the ship's coordinate system assuming that the attitude of the target vessel is kept horizontal is defined as "P2=[P2 x ,P2 y ,P2 z ] T ", the point cloud data corrected for P2 by the amount of attitude change is "P3=[P3 x ,P3 y ,P3 z ] T "

[0070] In this case, the simultaneous transformation matrix used to generate point cloud data P1 by transforming the coordinates of point cloud data P0 is given by equation (9) below.

[0071]

number

[0072] The attitude of the target vessel is roll angle Δφ S and pitch angle Δθ S Assuming a change in , the point cloud data P2 can be expressed by the following equation (10) using the point cloud data P1. Note that the change in yaw angle ψ is considered to be negligibly small.

[0073]

number

[0074] Based on the above, the simultaneous transformation matrix for correcting based on the roll angle Δφ and pitch angle Δθ obtained by attitude change estimation is expressed by the following equation (11) using point cloud data P2 and point cloud data P3.

[0075]

number

[0076] (8) Processing flow Figure 13 is an example of a flowchart showing an overview of the processing of point cloud data (point cloud data processing) in this embodiment. The information processing device 1 repeatedly executes the processing shown in the flowchart of Figure 13 according to the frame period.

[0077] First, the information processing device 1 acquires the point cloud data generated by the lidar 3 (step S11). In this case, the point cloud data is represented in the lidar coordinate system. Then, the information processing device 1 executes the quay wall normal calculation process, which is the process of calculating the normal to the upper surface of the quay wall (step S12). Details of step S12 will be explained with reference to Figure 14.

[0078] Next, the information processing device 1 performs an attitude change estimation process, which is a process to estimate the amount of attitude change of the lidar 3 (step S13). Details of step S13 will be explained with reference to Figure 15. Furthermore, based on the amount of attitude change estimated in step S13, the information processing device 1 performs a coordinate transformation process, which is a process to convert the point cloud data obtained in step S11 into data in the ship coordinate system (step S14). Details of step S14 will be explained with reference to Figure 16.

[0079] Figure 14 is a flowchart of the quay normal calculation process performed in step S12 of Figure 13.

[0080] First, the information processing device 1 extracts the quay wall point cloud from the point cloud data by clustering (step S21). Next, the information processing device 1 calculates the normal vector of each measured point in the quay wall point cloud and extracts the measured points whose Z-axis component is greater than or equal to a predetermined value as the upper quay wall point cloud (step S22).

[0081] Next, the information processing device 1 determines whether or not it was able to extract the point cloud of the quay surface (step S23). In this case, for example, if the number of points in the point cloud of the quay surface is greater than or equal to a predetermined number, the information processing device 1 determines that the point cloud of the quay surface was extracted, and if the number of points is less than the predetermined number, it determines that the point cloud of the quay surface was not extracted.

[0082] Then, if the information processing device 1 determines that it has been able to extract the point cloud of the quay surface (Step S23; Yes), it performs principal component analysis on the point cloud of the quay surface and determines the vector representing the third principal component axis as the normal to the quay surface (Step S24). On the other hand, if the information processing device 1 determines that it has not been able to extract the point cloud of the quay surface (Step S23; No), it determines that the process has failed, issues a predetermined error notification, and terminates the flowchart process without performing any further processing (Step S25).

[0083] Figure 15 is a flowchart of the attitude change amount estimation process performed in step S13 of Figure 13.

[0084] First, the information processing device 1 calculates the pitch angle Δθ from the normal to the top surface of the quay after the change in the attitude of the target vessel (i.e., the normal calculated in step S24 of Figure 14) based on equation (4) (step S31). Then, the information processing device 1 calculates the roll angle Δφ from the normal to the top surface of the quay after the change in the attitude of the target vessel (i.e., the normal calculated in step S24 of Figure 14) based on equation (7) (step S32).

[0085] Figure 16 is a flowchart of the coordinate transformation process performed in step S14 of Figure 13.

[0086] First, the information processing device 1 obtains the absolute values ​​of the roll angle Δφ and pitch angle Δθ, which correspond to the attitude change amounts in the roll direction and pitch direction calculated in the attitude change amount estimation process in step S13 (step S41). Then, the information processing device 1 determines whether or not both the obtained absolute value of the roll angle Δφ and the absolute value of the pitch angle Δθ are below a threshold (step S42). If at least one of the above absolute values ​​is greater than the threshold (step S42; No), the information processing device 1 performs a process to notify that the attitude change amount is large (step S45). In this case, for example, the information processing device 1 may consider that an abnormality or error has occurred and output a warning.

[0087] On the other hand, if all of the absolute positions mentioned above are below a threshold (step S42; Yes), the information processing device 1 averages the estimation results over a predetermined period (step S43). In this case, the roll angle Δφ and pitch angle Δθ are averaged together with a predetermined number of estimation results calculated at past processing times. Then, the information processing device 1 uses the time-averaged roll angle Δφ and pitch angle Δθ to perform a coordinate transformation on the point cloud data acquired in step S11 (step S44).

[0088] As described above, the controller 13 of the information processing device 1 functions as an acquisition means, a normal vector calculation means, and an attitude estimation means. The acquisition means acquires measurement data of an object measured by a measuring device installed on the ship. The normal vector calculation means calculates the normal vector of the object based on the measurement data. The attitude estimation means estimates the attitude of the measuring device based on the normal vector of the object. In this configuration, the information processing device 1 can accurately estimate the attitude of the measuring device and accurately perform coordinate transformation of the measurement data generated by the measuring device.

[0089] In the above-described embodiment, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a controller. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic storage medium (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage medium (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0090] The present invention has been described above with reference to the examples, but the present invention is not limited to the above examples. Various modifications to the structure and details of the present invention can be made that can be understood by a person skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents, etc., that has been cited is incorporated herein by reference. [Explanation of Symbols]

[0091] 1. Information Processing Device 2 Sensor Groups 3 Riders

Claims

1. An acquisition means for acquiring measurement data of the quay measured by a measuring device installed on a ship, A normal calculation means for calculating the normal of the quay wall based on the aforementioned measurement data, An attitude estimation means for estimating the attitude of the measuring device based on the aforementioned normal, An information processing device having

2. The information processing apparatus according to claim 1, further comprising coordinate transformation means for performing a coordinate transformation to convert the measurement data into data in a ship coordinate system based on the ship, based on the estimation result of the attitude.

3. The information processing apparatus according to claim 2, wherein the coordinate transformation means performs the coordinate transformation based on a plurality of estimated attitudes obtained within a predetermined period.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the normal calculation means calculates the normal to a predetermined surface formed on the quay wall.

5. The aforementioned measurement data is point cloud data representing multiple measurement points, The information processing apparatus according to claim 4, wherein the normal calculation means calculates the normal based on the measured points on the surface represented by the point cloud data.

6. The information processing apparatus according to claim 5, wherein the normal calculation means calculates a normal for each measurement point on the quay wall using surrounding measurement points, and extracts measurement points on the surface based on the vector components of the normal.

7. The information processing apparatus according to any one of claims 4 to 6, wherein the normal calculation means calculates the normal to the upper surface of the quay wall.

8. The information processing apparatus according to any one of claims 1 to 7, wherein the attitude estimation means estimates at least one of the pitch angle and roll angle of the measuring device based on the normal.

9. A control method performed by a computer, By acquiring measurement data of the quay using measuring devices installed on the ship, Based on the aforementioned measurement data, the normal of the quay wall is calculated, Based on the aforementioned normal, the orientation of the measuring device is estimated. Control method.

10. By acquiring measurement data of the quay using measuring devices installed on the ship, Based on the aforementioned measurement data, the normal of the quay wall is calculated, A program that causes a computer to perform a process to estimate the attitude of the measuring device based on the aforementioned normal vector.

11. A storage medium storing the program described in claim 10.

Images