Information processing device, control method, program, and storage medium

Abstract

To provide an information processing device capable of appropriately acquiring accurate information on docking.SOLUTION: A controller 13 of an information processing device 1 disclosed herein acquires first measurement data generated by a first LiDAR 31 installed on a target ship and second measurement data generated by a second LiDAR 32 installed on the target ship, the second LiDAR having a measurement range different from that of the first LIDAR 31. The controller 13 then identifies a first nearest point, a point nearest to a docking point for the first LIDAR 31, using the first measurement data, and a second nearest point, a point nearest to the docking point for the second LiDAR 32, using the second measurement data. The controller 13 then derives a ship-to-shore distance, which is a distance between the target ship and the docking point, on the basis of the first nearest point and the second nearest point.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to ship docking procedures. [Background technology]

[0002] Conventionally, there have been known technologies for providing support for docking (berthing) of ships. For example, Patent Document 1 describes a method for controlling an automatic docking device that automatically docks a ship by changing the attitude of the ship so that light emitted from a lidar is reflected by objects around the docking position and can be received by the lidar. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-59403 Summary of the Invention [Problem to be solved by the invention]

[0004] When providing advanced berthing assistance such as automatic berthing of a ship at a berthing location, it is necessary to accurately grasp the distance to the berthing location. However, Patent Document 1 does not disclose a method for calculating the accurate distance to the berthing location.

[0005] The present disclosure has been made to solve the above-mentioned problems, and has as its main object to provide an information processing device that can accurately calculate the distance to the shore from a ship. [Means for solving the problem]

[0006] The claimed invention is a measurement data acquisition means for acquiring first measurement data generated by a first measurement device provided on the vessel and second measurement data generated by a second measurement device provided on the vessel and having a measurement range different from that of the first measurement device; a nearest point specifying means for specifying a first nearest point, which is the nearest point to the docking location with respect to the first measuring device, based on the first measurement data, and for specifying a second nearest point, which is the nearest point to the docking location with respect to the second measuring device, based on the second measurement data; a shore distance calculation means for calculating a shore distance, which is a distance between the ship and the berthing location, based on the first nearest point and the second nearest point; The information processing device has the following.

[0007] The claimed invention also includes: A computer-implemented control method comprising: Acquire first measurement data generated by a first measuring device installed on a vessel and second measurement data generated by a second measuring device installed on the vessel and having a measurement range different from that of the first measuring device; identifying a first nearest point that is the nearest point of the docking location relative to the first measurement device based on the first measurement data, and identifying a second nearest point that is the nearest point of the docking location relative to the second measurement device based on the second measurement data; Calculating a shore distance, which is a distance between the ship and the docking location, based on the first nearest point and the second nearest point. It is a control method.

[0008] The claimed invention also includes: Acquire first measurement data generated by a first measuring device installed on a vessel and second measurement data generated by a second measuring device installed on the vessel and having a measurement range different from that of the first measuring device; identifying a first nearest point that is the nearest point of the docking location relative to the first measurement device based on the first measurement data, and identifying a second nearest point that is the nearest point of the docking location relative to the second measurement device based on the second measurement data; The program causes a computer to execute a process of calculating a shore distance, which is the distance between the ship and the docking location, based on the first nearest point and the second nearest point. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic configuration diagram of a driving assistance system. [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Figure 3] FIG. 2 is a functional block diagram relating to a docking assistance process. [Figure 4] (A) An example of a hull coordinate system based on the hull of the target ship is shown. (B) A perspective view of a structure with the normal vector clearly indicated. [Figure 5] This is a diagram showing a docking situation in which both the top and side surfaces of the structure at the docking location are included as the inner field of view and detection surfaces. [Figure 6] (A) A diagram showing a docking situation where the inner field of view is the top and side surfaces, but the detection surface is only the side surfaces. (B) An example of the data array generated by the lidar in one scanning cycle. (C) A diagram showing a docking situation where the inner field of view is the top and side surfaces, but the detection surface is only the top surface. [Figure 7] (A) A diagram showing a docking situation where the inner field of view and the detection surface are both only the side, and (B) A diagram showing a docking situation where the inner field of view and the detection surface are both only the top. [Figure 8] FIG. 1 is a perspective view of a structure showing neighboring points and nearest neighboring points. [Figure 9] This is a top view of the target ship and the structure where it will be docked. [Figure 10] FIG. 10 is a diagram showing an outline of a method for calculating a berthing side straight line. [Figure 11] FIG. 10 is a diagram showing an outline of a method for calculating the distance to the opposite bank. [Figure 12] Shown is an overhead view of the target vessel equipped with three lidars. [Figure 13] This is a diagram clearly showing the nearest neighbor point and the berthing side line Ls when noise occurs in the nearest neighbor point search. [Figure 14] FIG. 10 is a diagram showing an outline of a method for calculating a docking speed. [Figure 15] FIG. 10 is a diagram illustrating an outline of a method for calculating an approach angle. [Figure 16]10 shows an example of a data structure of reliability information. [Figure 17] 10 is an example of a flowchart illustrating an outline of a docking assistance process. DETAILED DESCRIPTION OF THE INVENTION

[0010] According to a preferred embodiment of the present disclosure, an information processing device includes: measurement data acquisition means for acquiring first measurement data generated by a first measuring device mounted on a ship and second measurement data generated by a second measuring device mounted on the ship and having a measurement range different from that of the first measuring device; nearest point identification means for identifying a first nearest point that is the point closest to the docking location relative to the first measuring device based on the first measurement data and for identifying a second nearest point that is the point closest to the docking location relative to the second measuring device based on the second measurement data; and shore distance calculation means for calculating the shore distance, which is the distance between the ship and the docking location, based on the first nearest point and the second nearest point. According to this aspect, the information processing device can accurately calculate the shore distance from the ship based on measurement data output by multiple measuring devices mounted on the ship.

[0011] In one aspect of the information processing device, the information processing device further includes a straight-line equation generating means for generating a straight-line equation based on the first nearest point and the second nearest point, and the opposite-shore distance calculating means calculates the opposite-shore distance based on the length of a perpendicular line cast from at least one of the first measuring device or the second measuring device to the straight line represented by the straight-line equation. With this aspect, the information processing device can geometrically calculate the opposite-shore distance using the nearest point identified for each measuring device.

[0012] In one aspect of the information processing device, the linear equation generating means calculates the linear equation by converting the coordinates of the first nearest point and the coordinates of the second nearest point into a common coordinate system based on coordinate system information that indicates the relationship between the coordinate system of the first measurement data and the coordinate system of the second measurement data. This aspect allows the information processing device to accurately generate a linear equation required to calculate the opposite shore distance.

[0013] In one aspect of the information processing device, the measurement data acquisition means acquires third measurement data generated by a third measurement device provided on the ship and having a measurement range different from those of the first measurement device and the second measurement device, the nearest point identification means further identifies a third nearest point that is the point nearest to the docking location with respect to the third measurement device based on the third measurement data, and the straight-line equation generation means calculates the straight-line equation based on the first nearest point, the second nearest point, and the third nearest point. According to this aspect, even when three or more measurement devices are present, the information processing device can accurately generate the straight-line equation required to calculate the distance to the shore based on the measurement data of these measurement devices.

[0014] In one aspect of the information processing device, the information processing device further includes a neighboring point searching means that searches for a predetermined number of first neighboring points of the docking location for the first measurement device based on the first measurement data and searches for a predetermined number of second neighboring points of the docking location for the second measurement device based on the second measurement data, and the nearest point identifying means selects the first nearest point from the first nearest points based on the distance between the first nearest points and selects the second nearest point from the second nearest points based on the distance between the second nearest points. According to this aspect, the information processing device can accurately determine the nearest points without being affected by noise by taking the distance between the nearest points into consideration.

[0015] In another aspect of the information processing device, the nearest point identification means excludes a nearest point between the first nearest points or the second nearest points whose distance to another nearest point is equal to or greater than a predetermined threshold from candidates for the first nearest point and the second nearest point. This aspect enables the information processing device to prevent a nearest point that is far from other nearest points from being selected as the nearest point, and to accurately determine the nearest point without being affected by noise.

[0016] In another aspect of the information processing device, the measurement data acquisition means acquires, as the first measurement data and the second measurement data, data generated by the first measurement device and the second measurement device based on the water surface position, by at least one of removing water surface reflection data and reducing the number of data points by downsampling. With this aspect, the information processing device can preferably acquire measurement data from which noise has been removed or adjusted so that the amount of data is not excessive, and calculate the distance to the opposite shore.

[0017] According to another preferred embodiment of the present disclosure, there is provided a control method executed by a computer, which acquires first measurement data generated by a first measuring device provided on a ship and second measurement data generated by a second measuring device provided on the ship and having a measurement range different from that of the first measuring device, identifies a first nearest point that is the point closest to the docking location relative to the first measuring device based on the first measurement data, identifies a second nearest point that is the point closest to the docking location relative to the second measuring device based on the second measurement data, and calculates a shore distance that is the distance between the ship and the docking location based on the first nearest point and the second nearest point. By executing this control method, the computer can accurately calculate the shore distance from the ship based on measurement data output by multiple measuring devices provided on the ship.

[0018] According to another preferred embodiment of the present disclosure, a program causes a computer to execute the following processes: acquire first measurement data generated by a first measuring device installed on a ship and second measurement data generated by a second measuring device installed on the ship and having a measurement range different from that of the first measuring device; identify a first nearest point that is the point closest to the docking location relative to the first measuring device based on the first measurement data; identify a second nearest point that is the point closest to the docking location relative to the second measuring device based on the second measurement data; and calculate a shore distance, which is the distance between the ship and the docking location, based on the first nearest point and the second nearest point. By executing this program, the computer can accurately calculate the shore distance from the ship based on measurement data output by multiple measuring devices installed on the ship. Preferably, the program is stored in a storage medium. [Example]

[0019] Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. For convenience, the letter "A" with the symbol "x" attached thereto will be represented as "Ax" in this specification.

[0020] (1) Overview of the driving assistance system Fig. 1 shows a schematic configuration of a driving assistance system according to an embodiment. The driving assistance system includes an information processing device 1 that moves together with a ship, which is a moving body, and a sensor group 2 mounted on the ship. Hereinafter, the ship that moves together with the information processing device 1 will also be referred to as the "target ship."

[0021] The information processing device 1 is electrically connected to the sensor group 2, and performs operational support such as automatic operation control of the target ship on which the information processing device 1 is installed, based on the outputs of various sensors included in the sensor group 2. The operational support also includes berthing support such as automatic berthing (docking). Here, "docking" includes not only docking the target ship at a quay, but also docking the target ship at a structure such as a pier. In addition, hereinafter, "docking location" is a general term for structures such as quays and piers that are the target for docking. The information processing device 1 may be a navigation device installed on the target ship, or an electronic control device built into the ship.

[0022] The sensor group 2 includes various external and internal sensors provided on the target ship. In this embodiment, the sensor group 2 includes at least a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3.

[0023] The LIDAR 3 emits a pulsed laser beam over a predetermined angular range in the horizontal and vertical directions to discretely measure the distance to an object in the external world and generate three-dimensional point cloud data indicating the position of the object. In this case, the LIDAR 3 includes an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data based on the light receiving signal output by the light receiving unit. Data (also referred to as "measurement points") measured for each direction (scanning position) of laser light irradiation is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light identified based on the above-mentioned light receiving signal. Note that the LIDAR 3 is not limited to the above-mentioned scan-type LIDAR, but may also be a flash-type LIDAR that generates three-dimensional data by irradiating a diffused laser beam within the field of view of a two-dimensional array sensor. The LIDAR 3 is an example of a "measurement device" in the present invention.

[0024] (2) Configuration of information processing device 2 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly includes an interface 11, a memory 12, and a controller 13. These elements are connected to each other via a bus line.

[0025] The interface 11 performs interface operations related to the exchange of data between the information processing device 1 and an external device. In this embodiment, the interface 11 acquires output data from each sensor in the sensor group 2 and supplies it to the controller 13. The interface 11 also supplies, for example, signals related to the control of the target vessel generated by the controller 13 to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel may include a drive source such as an engine or an electric motor, a screw that generates a forward thrust based on the drive force of the drive source, a thruster that generates a lateral thrust based on the drive force of the drive source, and a rudder, which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation such as automatic docking, the interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, the interface 11 supplies the control signal generated by the controller 13 to the electronic control device. The interface 11 may be a wireless interface such as a network adapter for wireless communication, or a hardware interface for connecting to an external device via a cable or the like. The interface 11 may also perform interface operations with various peripheral devices such as an input device, a display device, and a sound output device.

[0026] The memory 12 is configured by various types of volatile and non-volatile memory, such as a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk drive, and a flash memory. The memory 12 stores programs for the controller 13 to execute predetermined processes. The programs executed by the controller 13 may be stored in a storage medium other than the memory 12.

[0027] The memory 12 also stores information necessary for the processing executed by the information processing device 1 in this embodiment. For example, the memory 12 may store map data including information about the position of a docking location. In another example, the memory 12 stores information about the downsampling size when downsampling is performed on point cloud data obtained when the LIDAR 3 performs one scanning cycle.

[0028] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 executes programs stored in the memory 12, etc., to perform processing related to operational support for the target ship, etc.

[0029] The controller 13 also functionally includes a berthing location detection unit 15 and a berthing parameter calculation unit 16. The berthing location detection unit 15 performs processing related to the detection of a berthing location based on the point cloud data output by the LIDAR 3. The berthing parameter calculation unit 16 calculates parameters (also referred to as "berthing parameters") required for berthing at the berthing location. The berthing parameters include the distance to the berthing location (near-berthing distance), the approach angle to the berthing location, and the speed at which the vehicle approaches the berthing location (near-berthing speed). The berthing parameter calculation unit 16 also calculates information (also referred to as "reliability information") indicating the reliability of berthing at the berthing location based on the processing results of the berthing location detection unit 15 and the berthing parameters. The controller 13 functions as a "measurement data acquisition means," a "neighborhood point search means," a "nearest neighbor point identification means," a "neighborhood distance calculation means," a "straight-line equation generation means," a computer that executes programs, etc.

[0030] The processes executed by the controller 13 are not limited to being realized by software programs, but may be realized by any combination of hardware, firmware, and software. Furthermore, the processes executed by the controller 13 may be realized by using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcomputer. In this case, the programs executed by the controller 13 in this embodiment may be realized by using this integrated circuit.

[0031] (3) Berthing support processing Next, we will explain the docking assistance process executed by the information processing device 1. In summary, the information processing device 1 generates a linear equation along the side of the docking location based on point cloud data of the lidar 3 measured in the direction of the docking location, and calculates docking parameters such as the distance to the opposite shore based on the linear equation.

[0032] (3-1) Functional Blocks 3 is a functional block diagram of the docking location detection unit 15 and the docking parameter calculation unit 16 related to the docking assistance process. The docking location detection unit 15 functionally comprises a normal vector calculation block 20, a field of view / detection plane specification block 21, a normal number specification block 22, a mean / variance calculation block 23, and a docking situation determination block 24. The docking parameter calculation unit 16 functionally comprises a neighboring point search block 25, a nearest neighbor determination block 26, a straight line equation generation block 27, a docking distance calculation block 28, an approach angle calculation block 29, a docking speed calculation block 30, and a reliability information generation block 40.

[0033] The normal vector calculation block 20 calculates the normal vector of the plane formed by the docking location (also called the "docking surface") based on point cloud data generated by the LIDAR 3 in the direction in which the docking location is located. In this case, the normal vector calculation block 20 calculates the above-mentioned normal vector based on point cloud data generated by the LIDAR 3, for example, whose measurement range includes the docking side of the target ship. Information regarding the measurement range of the LIDAR 3 and the direction in which the docking location is located may be registered in advance in the memory 12, for example.

[0034] In this case, the normal vector calculation block 20 preferably performs downsampling of the point cloud data and removal of data obtained by the laser light reflecting off the water surface (also called "water surface reflection data").

[0035] In this case, the normal vector calculation block 20 first removes data that exists below the water surface position from the point cloud data generated by the LIDAR 3 as water surface reflection data (i.e., false detection data). Note that the normal vector calculation block 20 estimates the water surface position based on, for example, the average value in the height direction of the point cloud data generated by the LIDAR 3 when there are no objects other than the water surface in the vicinity. Then, the normal vector calculation block 20 performs downsampling on the point cloud data after the water surface reflection data has been removed, which is a process of integrating measurement points for each grid space of a predetermined size. Then, for each measurement point indicated by the point cloud data after downsampling, the normal vector calculation block 20 calculates a normal vector using multiple surrounding measurement points. Note that downsampling may be performed before removing the data reflected by the water surface.

[0036] The field of view / detection surface identification block 21 identifies the surface of the docking location that exists within the field of view angle of the lidar 3 (also referred to as the "inner field of view") and the surface of the docking location that is detected based on the normal vector calculated by the normal vector calculation block 20 (also referred to as the "detection surface"). In this case, the field of view / detection surface identification block 21 identifies whether the top surface and / or side surface of the docking location are included as the inner field of view and the detection surface. A specific identification method will be described with reference to Figures 5 to 7.

[0037] The normal number specification block 22 extracts the vertical normal vectors and the normal vectors in the direction perpendicular to the vertical normal vectors (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation block 20, and calculates the number of vertical normal vectors and the number of horizontal normal vectors. Here, the normal number specification block 22 regards the vertical normal vectors as normals to the measurement points on the top surface of the docking location, and the horizontal normal vectors as normals to the measurement points on the side surfaces of the docking location, and calculates the respective numbers as an index of the reliability of the docking location.

[0038] The mean / variance calculation block 23 extracts the vertical normal vectors and the normal vectors perpendicular to the vertical normal vectors (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation block 20, and calculates the mean and variance of the vertical normal vectors and the mean and variance of the horizontal normal vectors.

[0039] The docking situation determination block 24 acquires the processing results of the field of view / detection plane determination block 21, the number of normals determination block 22, and the mean / variance calculation block 23, which are specified or calculated based on the same point cloud data, as a determination result representing the detection situation of the docking location at the time of generating the point cloud data.Then, the docking situation determination block 24 supplies the processing results of the field of view / detection plane determination block 21, the number of normals determination block 22, and the mean / variance calculation block 23 to the docking parameter calculation unit 16 as a determination result of the detection situation of the docking location.

[0040] The neighboring point search block 25 detects a predetermined number of neighboring points of the docking location for the target ship from the measurement points that make up the point cloud data by searching for multiple points with short distances. The nearest neighbor determination block 26 performs processing to determine the nearest point from the predetermined number of neighboring points detected by the neighboring point search block 25.

[0041] The straight line equation generation block 27 generates a straight line (also called the "berthing side line Ls") along the side of the docking location based on the nearest point determined by the nearest neighbor determination block 26. The shore distance calculation block 28 calculates the shore distance corresponding to the shortest distance between the target ship and the docking location based on the shore side line Ls generated by the straight line equation generation block 27. In this case, the shore distance calculation block 28 calculates the shortest distance between the shore side line Ls and each rider 3 as the shore distance. Note that instead of considering the shortest distance for each rider 3 as the shore distance, the shore distance calculation block 28 may determine the shortest distance among the shortest distances for each rider 3 as the shore distance, or may determine the average of these shortest distances as the shore distance.

[0042] The approach angle calculation block 29 calculates the approach angle of the target vessel with respect to the berthing location based on the berthing side line Ls generated by the straight line equation generation block 27. The berthing speed calculation block 30 calculates the berthing speed, which is the speed at which the target vessel approaches the berthing location, based on the berthing distance calculated by the berthing distance calculation block 28.

[0043] The reliability information generation block 40 generates reliability information based on the processing results of the docking situation determination block 24, the neighboring point search block 25, the nearest neighbor determination block 26, the distance to the shore calculation block 28, and the approach angle calculation block 29. The reliability information will be described in detail later.

[0044] (3-2) Operations on normal vectors Next, a specific example of the processing of the normal vector calculation block 20, the normal number specification block 22, and the mean / variance calculation block 23 will be described with reference to FIG.

[0045] FIG. 4(A) shows an example of a hull coordinate system based on the hull of the target ship. As shown in FIG. 4(A), the front (forward) direction of the target ship is the "x" coordinate, the lateral direction of the target ship is the "y" coordinate, and the height direction of the target ship is the "z" coordinate. The measurement data measured by the LIDAR 3 in the coordinate system based on the LIDAR 3 is converted into the hull coordinate system shown in FIG. 4(A). Note that the process of converting point cloud data in a coordinate system based on a LIDAR installed on a moving body into the coordinate system of the moving body is disclosed, for example, in International Publication WO2019 / 188745.

[0046] 4(B) is a perspective view of the structure 50, which is the docking location, clearly showing measurement points that indicate measurement positions measured by the lidar 3 and normal vectors calculated based on the measurement points. In FIG. 4(B), the measurement points are indicated by circles, and the normal vectors are indicated by arrows. This shows an example in which both the top and side surfaces of the structure 50 were measured by the lidar 3.

[0047] As shown in FIG. 4B, the normal vector calculation block 20 calculates normal vectors for the measurement points on the side and top surfaces of the structure 50. Because normal vectors are vectors perpendicular to the target plane or curved surface, they are calculated using multiple measurement points that can be configured as a surface. Therefore, a grid of predetermined length and width or a circle of predetermined radius is set, and calculations are performed using measurement points within the grid. In this case, the normal vector calculation block 20 may calculate normal vectors for each measurement point or at predetermined intervals. The normal number determination block 22 then determines that a normal vector whose z-component is greater than a predetermined threshold is a normal vector pointing in the vertical direction. Note that the normal vectors are assumed to be unit vectors. Furthermore, a normal vector whose z-component is less than a predetermined threshold is a normal vector pointing in the horizontal direction. The normal number determination block 22 then determines the number of vertical normal vectors (here, five) and the number of horizontal normal vectors (here, four). Furthermore, the mean / variance calculation block 23 calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction. Note that the measurement points around the edge portion are on the top surface or the side surface, so the direction is oblique.

[0048] (3-3) Identifying the inner field of view and detection surface The process of specifying the inner field of view and the detection surface will be explained below by dividing the process into docking situations A to E, which represent specific docking situations.

[0049] (3-3-1) Berthing situation A: Both the side and top of the berthing location are within the field of view and the detection surface. Fig. 5 is a diagram showing a docking situation in which both the top and side surfaces of a structure 50, which is the docking location, are included as the inner field of view and detection surface. In Fig. 5, a dashed line 51 indicates the estimated water surface position. Circles below the dashed line 51 indicate measurement points that have been removed as measurement points below the water surface position. Circles on the top and side surfaces of the structure 50 indicate measurement points corresponding to the normal vectors calculated by the normal vector calculation block 20.

[0050] In this case, the field of view and detection surface identification block 21 detects both the top and side surfaces of the structure 50 as detection surfaces based on the normal vectors calculated by the normal vector calculation block 20. For example, the field of view and detection surface identification block 21 determines that the top surface of the structure 50 has been detected if the number of vertical normal vectors is equal to or greater than a predetermined number and the variance calculated by the mean and variance calculation block 23 is within a threshold value. Also, the field of view and detection surface identification block 21 determines that the side surface of the structure 50 has been detected if the number of horizontal normal vectors is equal to or greater than a predetermined number and the variance calculated by the mean and variance calculation block 23 is within a threshold value. The predetermined number and threshold value are stored in advance in, for example, the memory 12.

[0051] In addition, in the example of Figure 5, the field of view / detection surface identification block 21 determines that both the top and side surfaces of the structure 50 are within the field of view angle of the lidar 3, since the top and side surfaces of the structure 50 are detected as detection surfaces.

[0052] In docking situation A shown in Figure 5, the inner field of view and the detection surface include both the top and side surfaces of structure 50, which is the docking location, so it is estimated that the docking location has been detected accurately and that the reliability of the detection of the docking location is high.

[0053] (3-3-2) Berthing situation B: Both sides of the berthing location are inside the field of view and only the sides are detected. FIG. 6(A) is a diagram showing a docking situation in which the inner surface of the field of view is the top surface and the side surface, while the detection surface is only the side surface.

[0054] In the case of the docking situation shown in FIG. 6(A), the field of view / detection plane identification block 21 detects the side of the structure 50 as the detection plane based on the normal vectors calculated by the normal vector calculation block 20. For example, the field of view / detection plane identification block 21 detects the side of the structure 50 as the detection plane because the number of horizontal normal vectors is equal to or greater than a predetermined number and the number of vertical normal vectors is less than a predetermined number. On the other hand, the field of view / detection plane identification block 21 determines that the inner field of view includes both the top and side surfaces because there is a scanning position that can measure an area above the scanning position that detected the side of the structure 50 (in other words, the highest vertical number (described later in FIG. 6(B)) of the data that detected the side of the structure 50 is not the number at the top of the vertical field of view). FIG. 6(B) shows an example of an array of data generated by the lidar 3 in one scanning cycle. Each piece of data generated by the lidar 3 is identified by a combination of a vertical number (here, 1 to m) and a horizontal number (here, 1 to n) according to the emission direction of the laser light. In the example of FIG. 6(B), the data with vertical number 1 is the number at the top of the vertical field of view. In this case, in the example of FIG. 6(A), if the highest vertical number in the data detecting the side of the structure 50 is not 1, the field of view / detection surface identification block 21 determines that the inner field of view includes both the top surface and the side surface.

[0055] In the docking situation shown in Figure 6(A), although the inner field of view includes both the top and side surfaces, only the side surfaces of the docking location can be detected, so it is estimated that the reliability of detecting the docking location is lower than in the case of Figure 5.

[0056] (3-3-3) Berthing situation C: Both sides of the berthing location are inside the field of view and only the top surface is detected. FIG. 6(C) is a diagram showing a docking situation in which the inner surface of the field of view is the top and side surfaces, while the detection surface is only the top surface.

[0057] In the case of the docking situation shown in Fig. 6(C), the field of view and detection plane identification block 21 detects the side of the structure 50 as the top surface based on the normal vector calculated by the normal vector calculation block 20. On the other hand, the field of view and detection plane identification block 21 determines that the inner surface of the field of view includes both the top surface and the side surface because there is a scanning position that can measure below the scanning position that detected the top surface of the structure 50 (in other words, the lowest vertical number of the data that detected the top surface of the structure 50 is not the number at the bottom of the vertical field of view (number m in Fig. 6(B))).

[0058] In the docking situation shown in Figure 6(C), although the inner field of view includes both the top and side surfaces, only the top surface of the docking location can be detected, so it is estimated that the reliability of detecting the docking location is lower than in the cases of Figures 5 and 6(A).

[0059] (3-3-4) Berthing situation D: The inner field of view and the detection surface are both side-on FIG. 7(A) is a diagram showing a docking situation in which the inner field of view and the detection surface are both side surfaces.

[0060] In the case of the docking situation shown in FIG. 7(A), the field of view and detection plane identification block 21 detects part of the side of the structure 50 as the detection plane based on the normal vector calculated by the normal vector calculation block 20. Furthermore, the field of view and detection plane identification block 21 determines that the inner field of view includes only part of the side because there is no scanning position capable of measuring above the scanning position where the side of the structure 50 was detected (in other words, the highest vertical number of the data detecting the side of the structure 50 is the number at the top of the vertical field of view (number 1 in FIG. 6(B))). Furthermore, in the docking situation shown in FIG. 7(A), only part of the side of the docking location has been detected, so it is estimated that the reliability of the detection of the docking location is lower than in the case of FIG. 5, etc.

[0061] (3-3-5) Berthing situation E: The inner field of view and the detection surface are both on the upper surface only. FIG. 7(B) is a diagram showing a docking situation in which the inner field of view and the detection surface are both only the upper surface.

[0062] In the docking situation shown in FIG. 7(B), the field of view / detection plane identification block 21 detects only the top surface of the structure 50 as the detection plane based on the normal vector calculated by the normal vector calculation block 20. Furthermore, the field of view / detection plane identification block 21 determines that the inner field of view includes only the top surface because there is no scanning position capable of measuring below the scanning position where the side of the structure 50 was detected (in other words, the lowest vertical number of the data detecting the top surface of the structure 50 is the number at the bottom of the vertical field of view (number m in FIG. 6(B))). Furthermore, in the docking situation shown in FIG. 7(B), only a portion of the top surface of the structure 50 can be detected, and it is estimated that the reliability of detecting the docking location is lower than in the case of FIG. 5, etc. Furthermore, in this case, the nearest point does not become the reference position of the structure 50 when calculating the distance to the opposite bank, and therefore the distance to the opposite bank cannot be calculated accurately.

[0063] (3-4) Nearest point search Next, a specific example of the processing of the neighbor point search block 25 and the nearest neighbor determination block 26 will be described with reference to Fig. 8(A) and Fig. 8(B). Fig. 8(A) is a perspective view of a structure 50, clearly indicating the measurement points of the structure 50 measured by the LIDAR 3 and the nearest points therein. Fig. 8(B) is a perspective view of a structure 50, clearly indicating the measurement points of the structure 50 measured by the LIDAR 3 and the nearest points therein.

[0064] The neighboring point search block 25 searches for a predetermined number of neighboring points based on distance information of the point cloud data acquired from the LIDAR 3. In the example of Fig. 8(A), the neighboring point search block 25 finds seven neighboring points. In this case, it is preferable that the neighboring point search block 25 removes data from the point cloud data that represents a position below a height that can be estimated as the water surface position, as water surface reflection data (i.e., false detection data) obtained by the laser light reflecting off the water surface.

[0065] The nearest neighbor determination block 26 determines the nearest neighbor among the neighbor points found by the neighbor point search block 25 as a nearest neighbor candidate and calculates the positional difference between the candidate and the other neighbor points (i.e., the distance between the neighbor points). In this case, the nearest neighbor determination block 26 calculates a representative value, such as the average or median, of the distances to the other neighbor points as the positional difference between the nearest neighbor candidate and the other neighbor points. If the positional difference is less than a predetermined threshold, the nearest neighbor determination block 26 considers the candidate as the nearest neighbor. Near a quay, strong waves or winds can accidentally capture sea spray or floating objects. Therefore, the threshold is used to determine whether the measurement point is noise or an object other than a dock location, and is stored in advance in the memory 12, for example. On the other hand, if the difference is equal to or greater than the predetermined threshold, the nearest neighbor determination block 26 determines that the candidate is likely to be noise or a measurement point of another object, and does not consider the candidate as the nearest neighbor. In this case, the nearest neighbor determination block 26 regards the neighbor point next closest to the above candidate as a new candidate, calculates the difference as described above, compares it with the threshold value, and determines whether it is the nearest neighbor point.

[0066] For example, in the example of FIG. 8(A), nearest neighbor determination block 26 first extracts neighbor point 59 as a candidate for the nearest neighbor point. Then, because the difference between neighbor point 59 and the other neighbor points is equal to or greater than a predetermined threshold, nearest neighbor determination block 26 considers neighbor point 59 to be noise or the like and removes it. Next, nearest neighbor determination block 26 extracts neighbor point 60, which is the next closest to neighbor point 59, as a candidate for the next nearest neighbor point, and because the difference between neighbor point 60 and the other neighbor points is less than a predetermined threshold, nearest neighbor determination block 26 determines that neighbor point 60 is the nearest neighbor point, as shown in FIG. 8(B).

[0067] The nearest neighbor determination block 26 determines the nearest neighbor point by the above-described process, thereby enabling it to determine the nearest neighbor point robustly against noise. Furthermore, by determining the nearest neighbor point robustly against noise, the calculation of the berthing side line Ls by the line equation generation block 27 and the calculation of the opposite bank distance by the opposite bank distance calculation block 28, which will be described later, can also be performed accurately. The effect of this will be described in detail in the section "(3-5-5) Influence of noise when searching for the nearest neighbor point."

[0068] (3-5) Calculation of the berthing side line Ls and the distance to the opposite shore Next, a detailed description will be given of a method for calculating the berthing side straight line Ls and the distance to the shore based on the berthing side straight line Ls. As a representative example, a case where the lidars 3 are installed at two locations, one in front and one in rear of the target ship, will be described below.

[0069] (3-5-1) Overview 9(A) and 9(B) are diagrams showing the target ship and the structure 50, which is the docking location, as viewed from above. Here, in Fig. 9(A) and 9(B), a line segment "L1" connecting the first LIDAR 31, which is the front LIDAR 3, to the nearest point (also referred to as the "first nearest point") obtained based on the point cloud data of the first LIDAR 31, and a line segment "L2" connecting the second LIDAR 32, which is the rear LIDAR 3, to the nearest point (also referred to as the "second nearest point") obtained based on the point cloud data of the second LIDAR 32, are clearly shown.

[0070] 9(A), the orientation of the target vessel is nearly parallel to the side of the structure 50 where the vessel is docked, and the angle is small, so the lengths of the line segments L1 and L2 are equal to the shortest distance (i.e., the distance across the shore) between each rider 3 and the structure 50. On the other hand, in the example of FIG. 9(B), the orientation of the target vessel is at a not small angle to the side of the structure 50 where the vessel is docked, and so the lengths of the line segments L1 and L2 are not equal to the shortest distance (i.e., the distance across the shore) between each rider 3 and the structure 50.

[0071] Taking the above into consideration, the information processing device 1 generates a berthing side line Ls connecting the first nearest point and the second nearest point, finds the feet of perpendicular lines from the first rider 31 and the second rider 32 to the berthing side line Ls, and calculates this distance as the berthing distance. In this way, the information processing device 1 calculates the berthing distances, which are the shortest distances to the berthing locations ahead and behind the target ship.

[0072] (3-5-2) Calculation of the berthing side straight line Ls Fig. 10(A) is a diagram showing an outline of a method for calculating the berthing side straight line Ls. Hereinafter, a reference point (referred to as "first reference point O1") in a coordinate system (referred to as "first coordinate system") used in the point cloud data of the first lidar 31 is taken as the installation position of the first lidar 31, and a reference point (referred to as "second reference point O2") in a coordinate system (referred to as "second coordinate system") used in the point cloud data of the second lidar 32 is taken as the installation position of the second lidar 32. Then, the coordinates of the first nearest point in the first coordinate system based on the first reference point O1 are defined as "p1 = [px1, py1, pz1] T " and the coordinates of the second nearest point in the second coordinate system based on the second reference point O2 are "p2 = [px2,py2,pz2] T "

[0073] Figure 10(B) shows the relationship between coordinates p1 and p2 when the first reference point O1 and the second reference point O2 are in the same position. As can be seen from this figure, if the first reference point O1 and the second reference point O2 remain in the same position, the coordinate p1 of the first nearest point and the coordinate p2 of the second nearest point will be close to each other, and it will be impossible to calculate the berthing side line Ls. Therefore, in this case, the line equation generation block 27 first uniformly expresses the coordinates of the first nearest point and the second nearest point using a first coordinate system based on the first reference point O1.

[0074] The coordinates of the second reference point O2 as seen from the first reference point O1 (in other words, the coordinates of the second reference point O2 in the first coordinate system) are expressed as "[Ox0, Oy0, Oz0] T ", the coordinate p2 of the second nearest point is converted to the following coordinate "p2'". p2'=[Ox0+px2,Oy0+py2,Oz0+pz2] T

[0075] The coordinates of the second reference point O2 as seen from the first reference point O1 are [Ox0, Oy0, Oz0] T The information about the coordinates [Ox0, Oy0, Oz0] is stored in advance in the memory 12, for example. This information indicates the relationship between the first coordinate system used in the point cloud data of the first lidar 31 and the second coordinate system used in the point cloud data of the second lidar 32, and is an example of "coordinate system information" in the present disclosure. T By using this, it becomes possible to treat the first nearest point and the second nearest point in the same coordinate system. Fig. 10(C) shows the positional relationship between the first reference point O1, the second reference point O2, the coordinate p1, and the coordinate p2' in the first coordinate system based on the first reference point O1.

[0076] Here, the vector "u" (see FIG. 10C) from the coordinate p2' of the second nearest neighbor point to the coordinate p1 of the first nearest neighbor point is expressed by the following formula (1).

[0077]

number

[0078]

number

[0079]

number

[0080] (3-5-3) Calculating the distance to the opposite bank Fig. 11(A) is a diagram showing an outline of calculating the shortest distance "d1(k)" from the first reference point O1 to the structure 50 at time k and the shortest distance "d2(k)" from the second reference point O2 to the structure 50 at time k. Fig. 11(B) is an enlarged view of a portion related to the calculation of the shortest distance d1(k).

[0081] First, the shore distance calculation block 28 calculates the coordinate "H1" of the foot of the perpendicular line from the first reference point O1 to the berthing side line Ls. Here, if the vector from the coordinate p1 of the first nearest point to the coordinate H1 is "b1" (=H1-p1), and the vector from the coordinate p1 to the first reference point O1 is "a1" (=O1-p1), the dot product of vector a1 and vector u' is expressed by the following equation (4).

[0082]

number

[0083]

number

[0084]

number

[0085] Next, we will consider how to calculate the distance between the first reference point O1 and the coordinate H1. Generally, the distance from the ship to the docking location requires a value on the horizontal plane, so the shore distance calculation block 28 calculates the above-mentioned distance while ignoring the z component, which is the component in the height direction. Therefore, in this case, the shore distance calculation block 28 calculates only the x and y components to find the distance d1(k) using the following equation (7):

[0086]

number

[0087]

number

[0088] As described above, the shore distance calculation block 28 can suitably calculate the distance d1(k) and the distance d2(k) corresponding to the shore distance at time k based on the berthing side straight line Ls. Note that the calculation of the shore distance using this method is valid even in the case of Figure 9(A), so there is no need to distinguish between the two cases.

[0089] Note that instead of calculating the distance from each RIDER 3 to the docking location as the docking distance (shortest distance), the docking distance calculation block 28 may calculate the distance from a predetermined reference position (for example, the center position of the ship) of the target ship to the docking location as the docking distance (shortest distance). In this case, information about the reference position is stored in advance in the memory 12 as coordinate values ​​in the hull coordinate system or a coordinate system (first coordinate system, second coordinate system) based on the RIDER 3. Also, the docking distance calculation block 28 may set multiple positions of the outer plating (outer edge) of the target ship as multiple candidates for the reference position, calculate the shortest distance from each of these positions of the outer plating to the docking location, and calculate the shortest shortest distance as the shortest distance to the target ship (docking distance).

[0090] (3-5-4) When there are three or more riders When there are three or more LIDARs 3, the shore distance calculation block 28 can calculate the shore side line Ls by principal component analysis, regression analysis such as the least squares method, etc. The procedure for finding the direction vector u of the shore side line Ls using principal component analysis is described below.

[0091] 12 shows an overhead view of a target ship equipped with three LIDARs 3 (a first LIDAR 31, a second LIDAR 32, and a third LIDAR 33). Here, the coordinate of the nearest point (first nearest point) calculated based on the point cloud data of the first LIDAR 31 is defined as "p1 = (x1, y1, z1)," the coordinate of the nearest point (second nearest point) calculated based on the point cloud data of the second LIDAR 32 is defined as "p2 = (x2, y2, z2)," and the coordinate of the nearest point (third nearest point) calculated based on the point cloud data of the third LIDAR 33 is defined as "p3 = (x3, y3, z3)." Coordinates p1 to p3 represent coordinates in a common coordinate system (for example, a first coordinate system based on the first reference point O1).

[0092] In this case, the opposite shore distance calculation block 28 first calculates the covariance matrix "C" shown in the following equation (9) using coordinates p1 to p3. Note that here, since there are three nearest neighbor points, n=3, "μx" is the average value of x1, x2, and x3, "μy" is the average value of y1, y2, and y3, and "μz" is the average value of z1, z2, and z3.

[0093]

number

[0094]

number

[0095]

number

[0096] (3-5-5) Noise effects during nearest neighbor search Next, we will provide a supplementary explanation of the effect on the calculated berthing side line Ls when noise occurs in the nearest neighbor point search. Figures 13(A) and 13(B) are diagrams clearly showing the nearest neighbor point and the berthing side line Ls when noise occurs in the nearest neighbor point search. Figure 13(A) corresponds to an example where an error due to noise occurs in the first nearest neighbor point, and Figure 13(B) corresponds to an example where an error due to noise occurs in the second nearest neighbor point.

[0097] As shown in Figures 13(A) and 13(B), if noise occurs in the nearest point search, the shortest distance will be inaccurate not only on the side where the noise is reflected (the first LIDAR 31 side in Figure 13(A) and the second LIDAR 32 side in Figure 13(B)), but also on the side where the nearest point was correctly extracted (the second LIDAR 32 side in Figure 13(A) and the first LIDAR 31 side in Figure 13(B)). In addition, the angle from the berthing location (approach angle) will also be inaccurate. As a result, the calculated shortest distance (distance to the opposite shore) may be longer than the actual distance, or the approach angle from the berthing location may be smaller. These may result in a value that is deviated from the actual value in a safer direction in high-risk situations such as when the distance between the target ship and the berthing location is close or the approach angle is large, which may reduce safety.

[0098] Taking the above into consideration, the nearest neighbor determination block 26 according to this embodiment performs a process of determining the nearest neighbor by taking into consideration the distance between the neighboring points. As a result, even if a neighboring point generated by noise exists, it is possible to suitably suppress the neighboring point from being determined as the nearest neighbor, and to suitably maintain the accuracy of calculations of the berthing side line Ls, the distance to the opposite shore, the approach angle, etc.

[0099] (3-6) Calculation of berthing speed Next, a method for calculating the approaching speed will be described. The approaching speed calculation block 30 calculates the time change in the distance to the shore (shortest distance) obtained by each of the first rider 31 and the second rider 32 as the approaching speed.

[0100] 14 is a diagram showing an outline of a method for calculating the docking speed, where "d1" represents the distance to the shore (shortest distance) of the first rider 31, "d2" represents the distance to the shore (shortest distance) of the second rider 32, "k" represents the current processing time, "k-1" represents the immediately preceding processing time, and "Δt" represents the time interval from the immediately preceding calculation time of the distance to the shore.

[0101] Here, the docking speed calculation block 30 inserts a filter to deal with an increase in noise when dividing the difference in the distance to the shore between the previous time and the current time by the time interval. Specifically, the docking speed calculation block 30 uses the time constant "τ" and the Laplace operator "s" to calculate the docking speed "v1" for the front rider 3 and the docking speed "v2" for the rear rider 3 according to the following equations.

[0102]

number

[0103] This allows the docking speed calculation block 30 to appropriately calculate the docking speed taking into account the influence of noise.

[0104] (3-7) Calculating the approach angle Next, we will explain how to calculate the approach angle. The approach angle calculation block 29 calculates the approach angle using "atan2," a function that determines the arc tangent from two arguments that define the tangent. Specifically, the approach angle calculation block 29 calculates the approach angle by calculating the function atan2 from the direction vector u' of the berthing side straight line Ls.

[0105] 15 is a diagram showing an outline of a method for calculating an approach angle. In this case, the approach angle calculation block 29 calculates the approach angle "Ψ(k)" at time k based on the following equation, in which the x-component "ux'" and the y-component "uy'" of the direction vector u' shown in equation (2) are respectively arguments of the function atan2.

[0106]

number

[0107] (3-8) Generation of reliability information The reliability information generation block 40 generates a flag for each element such as the field of view when detecting the docking location, the surface detection of the docking location, the number and variance of normal vectors, etc., and generates a vector of the generated flag as reliability information. Hereinafter, a flag of "1" indicates that the reliability of the corresponding element is high, and a flag of "0" indicates that the reliability of the corresponding element is low.

[0108] 16 shows an example of the data structure of the reliability information generated by the reliability information generation block 40. As shown in FIG. 16, the reliability information has the following items: "Top surface," "Side surface," "Neighboring points," "Distance," and "Angle." The item "Top surface" has the sub-items "Viewing angle," "Detection," "Number of normals," and "Variance," and the item "Side surface" has the sub-items "Viewing angle," "Detection," "Number of normals," and "Variance." The item "Neighboring points" has the sub-item "Variance," the item "Distance" has the sub-items "Amount of change" and "Rate of change," and the item "Angle" has the sub-item "Amount of change."

[0109] Here, the reliability information generation block 40 registers a flag in the sub-item "viewing angle" of the item "upper surface" that is "1" if the upper surface of the docking location is within the range of the viewing angle, and "0" if the upper surface is outside the viewing angle. Also, the reliability information generation block 40 registers a flag in the sub-item "viewing angle" of the item "upper surface" that is "1" if the upper surface of the docking location is the detection surface, and "0" if the upper surface is not the detection surface. Also, the reliability information generation block 40 registers a flag in the sub-item "number of normals" of the item "upper surface" that is "1" if the number of normal vectors to the upper surface of the docking location is equal to or greater than a predetermined threshold (e.g., 10), and "0" if the number is less than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "variance" sub-item of the "top surface" item that is set to "1" if the variances of the x, y, and z components of the normal vector relative to the top surface of the docking location are all less than a predetermined threshold (e.g., 1.0), and is set to "0" if any of the variances is equal to or greater than the threshold. The reliability information generation block 40 also registers flags in each sub-item of the "side surface" item that are determined according to the same rules as for each sub-item of the "top surface" item.

[0110] The reliability information generation block 40 also registers a flag in the "variance" sub-item of the "neighborhood point" item that is set to "1" if the variances of the x, y, and z components of the neighboring points searched for by the neighboring point search block 25 are all less than a threshold (e.g., 1.0), and a flag that is set to "0" if any of the variances is equal to or greater than the threshold. The reliability information generation block 40 also registers a flag in the "change amount" sub-item of the "distance" item that is set to "1" if the change amount in the opposite bank distance calculated by the opposite bank distance calculation block 28 from one time before is less than a predetermined threshold (e.g., 1.0 m), and a flag that is set to "0" if the change amount is equal to or greater than the threshold. The reliability information generation block 40 also registers a flag in the "change rate" sub-item of the "distance" item that is set to "1" if the change rate in the opposite bank distance calculated by the opposite bank distance calculation block 28 from one time before is less than a predetermined threshold (e.g., ±10%), and a flag that is set to "0" if the change rate is equal to or greater than the threshold. In addition, the reliability information generation block 40 registers a flag in the sub-item "change amount" of the item "angle" that is set to "1" if the change amount of the approach angle calculated by the approach angle calculation block 29 from one time before is less than a predetermined threshold value (e.g., 1.0 degree), and to "0" if the change amount is equal to or greater than the threshold value.

[0111] The above-mentioned threshold values ​​are set to suitable values ​​stored in advance in the memory 12, for example.

[0112] Using reliability information with such a data structure, it is possible to grasp the reliability of the calculated distance to berth, berthing speed, and approach angle. Note that when each sub-item of the reliability information is "1," the reliability is highest. The information processing device 1 then adjusts the output of the drive source when berthing based on this reliability information. For example, the information processing device 1 may determine the upper limit of the target ship's speed when berthing according to the total value of each sub-item indicated by the reliability information. In this case, the information processing device 1 determines that the smaller the total value, the lower the reliability of the information regarding the berthing location and the more careful berthing is required, and reduces the upper limit of the target ship's speed when berthing.

[0113] (3-9) Processing flow 17 is an example of a flowchart showing an outline of the docking support process in this embodiment. The information processing device 1 repeatedly executes the process of the flowchart in FIG.

[0114] First, the information processing device 1 acquires point cloud data in the direction of the docking location (step S11). In this case, the information processing device 1 acquires point cloud data generated by, for example, a lidar 3 of the target ship whose measurement range includes the docking side. The information processing device 1 may further downsample the acquired point cloud data and remove data reflected on the water surface.

[0115] Next, the docking location detection unit 15 of the information processing device 1 calculates normal vectors based on the point cloud data acquired in step S11 (step S12). Furthermore, in step S12, the docking location detection unit 15 calculates the number of normal vectors, the variance of the normal vectors, etc. Furthermore, based on the processing result of step S12, the docking location detection unit 15 identifies the inner field of view and the detection plane (step S13).

[0116] Next, the berthing parameter calculation unit 16 performs a neighboring point search based on the point cloud data acquired in step S11 and the information on the normal vectors calculated in step S12, and determines the nearest point by nearest point determination (step S14).

[0117] Next, the docking parameter calculation unit 16 calculates the docking side line Ls using the nearest point obtained in step S14 (step S15).

[0118] Next, the docking parameter calculation unit 16 calculates the docking parameters, ie, the distance to the opposite shore, the approach angle, and the docking speed, using the docking side straight line Ls calculated in step S15 (step S16).

[0119] Then, the berthing parameter calculation unit 16 generates reliability information based on the results of identifying the inner field of view and the detection surface in step S13 and the results of calculating the berthing parameters in step S16 (step S17). Thereafter, the information processing device 1 controls the ship based on the reliability information (step S18). This allows the information processing device 1 to accurately control the ship regarding berthing based on the reliability that accurately reflects the berthing situation.

[0120] The information processing device 1 then determines whether the target ship has come alongside (docking) (step S19). In this case, the information processing device 1 determines whether the target ship has come alongside (docking) based on, for example, the output signal of the sensor group 2 or user input via the interface 11. If the information processing device 1 determines that the target ship has come alongside (step S19; Yes), it terminates the processing of the flowchart. On the other hand, if the target ship has not come alongside (step S19; No), the information processing device 1 returns the processing to step S11.

[0121] As described above, the controller 13 of the information processing device 1 according to the first embodiment acquires first measurement data generated by the first lidar 31 provided on the target vessel and second measurement data generated by the second lidar 32 provided on the target vessel and having a measurement range different from that of the first lidar 31. The controller 13 then identifies a first nearest point that is the point closest to the docking location relative to the first lidar 31 based on the first measurement data, and identifies a second nearest point that is the point closest to the docking location relative to the second lidar 32 based on the second measurement data. The controller 13 then calculates the shore distance, which is the distance between the target vessel and the docking location, based on the first nearest point and the second nearest point. According to this aspect, the information processing device 1 can accurately calculate the shore distance, which is one of the important parameters when docking at a docking location.

[0122] In the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a controller or the like that is a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0123] Although the present invention has been described above with reference to the examples, the present invention is not limited to the above examples. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art in accordance with the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference. [Explanation of symbols]

[0124] 1. Information processing equipment 2 Sensor group 3 (31, 32, 33) Rider

Claims

[Claim 1] a measurement data acquisition means for acquiring first measurement data generated by a first measurement device provided on the vessel and second measurement data generated by a second measurement device provided on the vessel and having a measurement range different from that of the first measurement device; a nearest point specifying means for specifying a first nearest point, which is the point nearest to the docking location with respect to the first measurement device, based on the first measurement data, and for specifying a second nearest point, which is the point nearest to the docking location with respect to the second measurement device, based on the second measurement data; a shore distance calculation means for calculating a shore distance, which is a distance between the ship and the berthing location, based on the first nearest point and the second nearest point; An information processing device having the above.

Images