Information processing device, control method, program, and storage medium

Abstract

A controller 13 of an information processing device 1 has at least an acquisition means, a straight line calculation means, and a distance calculation means. The acquisition means acquires measurement data, which is a set of data representing a plurality of measured points measured by a measurement device provided in a ship. The straight line calculation means calculates, on the basis of the measurement data, a straight line extending along a berthing location at which the ship is to berth. The distance calculation means calculates an opposite shore distance between the ship and the berthing location on the basis of distances between the straight line and the individual measured points.

Description

[Technical Field]

[0001] This disclosure relates to the procedures for when a vessel is docked. [Background technology]

[0002] Technologies for assisting with ship docking (berthing) have been known for some time. For example, Patent Document 1 describes a method for controlling the attitude of a ship in an automatic docking device that performs automatic ship docking, such that light emitted from a lidar is reflected by objects around the docking position and received by the lidar. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-59403 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] For ships, safe and smooth docking at a designated berthing location is crucial, and the realization of docking support systems for ship handling assistance and automated navigation is particularly desired. Therefore, it is necessary to calculate the distance to the intended berthing location with high accuracy.

[0005] This disclosure is made to solve the above-mentioned problems, and its main purpose is to provide an information processing device that can suitably calculate the distance from a vessel to a berthing location. [Means for solving the problem]

[0006] The invention described in the claims is, Measuring devices installed on ships distance This is a collection of data representing multiple measured points. As point cloud data A means of acquiring measurement data, A straight-line calculation means for calculating a straight line along the shore landing location where the ship lands based on the measurement data; A distance calculation means for calculating the distance between the ship and the shore landing location based on the distances between the straight line and each of the measured points; It is an information processing device having the above. Furthermore, the invention described in the claims is, An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, A nearest neighbor search means for searching for the nearest neighbor point to the vessel from the point to be measured, based on the distance between the line and each of the points to be measured. A distance calculation means for calculating the distance between the vessel and the berthing location based on the nearest neighbor point, It is an information processing device. Furthermore, the invention described in the claims is, An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, The system includes a distance calculation means that calculates the distance between the vessel and the berthing location based on the distance between the straight line and each of the points to be measured, The aforementioned straight line calculation means is an information processing device that calculates the straight line based on the nearest neighbor points searched from the measured point based on the straight line calculated by the straight line calculation means before the current processing time, when an index representing the variation of the measured point with respect to the straight line is greater than or equal to a threshold. Furthermore, the invention described in the claims is, An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, The system includes a distance calculation means that calculates the distance between the vessel and the berthing location based on the distance between the straight line and each of the points to be measured, The aforementioned straight line calculation means is an information processing device that calculates the straight line based on the nearest neighbor point searched from the measured point based on a straight line parallel to the orientation of the vessel, when an index representing the variation of the measured point relative to the straight line is greater than or equal to a threshold, and no straight line has been calculated within a predetermined time from the current processing time. Furthermore, the invention described in the claims is, An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, A distance calculation means for calculating the distance between the vessel and the berthing location based on the distance between the aforementioned straight line and each of the aforementioned measurement points, A means for calculating the shortest distance to the opposite shore, which extracts the minimum length of the perpendicular line drawn from the contour point of the vessel to the straight line, and calculates the shortest distance from the vessel's hull to the docking location. Proximity part identification means for identifying the part of the vessel closest to the berthing location, It is an information processing device.

[0007] Also, the invention according to the claims is A control method executed by a computer, Measurement data that is a set of data representing a plurality of measured points measured by a measuring device provided on a ship is acquired, distance Based on the measurement data, a straight line along the shore landing location where the ship lands is calculated, As point cloud data and based on the distances between the straight line and each of the measured points, the distance between the ship and the shore landing location is calculated. Based on the measurement data, a straight line along the shore landing location where the ship lands is calculated, Based on the distance between the aforementioned straight line and each of the measurement points, the distance between the vessel and the berthing location is calculated. This is a control method.

[0008] Furthermore, the invention described in the claims is, Measuring devices installed on ships distance This is a collection of data representing multiple measured points. As point cloud data Acquire measurement data, Based on the aforementioned measurement data, a straight line along the berthing location where the vessel will dock is calculated. This program causes a computer to perform a process to calculate the distance between the vessel and the berthing location based on the distance between the aforementioned straight line and each of the measurement points. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of the driver assistance system. [Figure 2] This is a block diagram showing the hardware configuration of an information processing device. [Figure 3] This is a functional block diagram related to berthing support processing. [Figure 4] (A) An example of a hull coordinate system based on the hull of the target vessel. (B) A view of the lidar capturing the quay where the vessel is docked. (C) A perspective view of the structure with the normal vectors clearly indicated. [Figure 5] (A) to (C) are diagrams showing the processing flow of the first generation method. [Figure 6] (A) A top view showing the first shoreline line L1 and the nearest neighbor point p1. (B) A diagram showing the unit vector u used to search for the nearest neighbor point p1. [Figure 7] (A) to (C) are diagrams showing the process flow for calculating a provisional straight line L′ based on the nearest neighbor point p1 and the nearest neighbor point p2, and for calculating the distance to the opposite bank based on the provisional straight line L′. [Figure 8] This shows the process flow for calculating the updated provisional line L′′. [Figure 9](A) A diagram clearly showing the nearest neighbor points to be extracted. (B) An example showing the data of the point measured with the shortest measurement distance by the lidar extracted as the nearest neighbor. (C) An example showing the value of only the y coordinate in the ship coordinate system being compared and the point closest to the ship's origin extracted as the nearest neighbor. [Figure 10] This figure shows the measurement points of the side point cloud data of the first shoreline L1 and the forward lighter. [Figure 11] (A) to (C) are diagrams showing the processing flow when a straight line L on the shore side has been generated within a predetermined time period in the past. [Figure 12] (A) to (C) are diagrams showing the processing flow when there is no docking side straight line L generated within a predetermined time period in the past. [Figure 13] An example of a data structure for confidence level information is shown. [Figure 14] This is an example of a flowchart illustrating the overview of the docking support process in the embodiment. [Figure 15] This is an example of a flowchart for calculating the shoreline straightness based on the first generation method. [Figure 16] This is an example of a flowchart for calculating the straight line of the shoreline based on the second generation method. [Figure 17] This is an overhead view of the target vessel, with its outline points clearly indicated. [Figure 18] This diagram clearly shows the distance from the contour point to the straight line of the shoreline using arrows. [Figure 19] This diagram clearly shows the nearest parts and the shortest distance. [Figure 20] This section shows the indicators and confidence levels included in the confidence information related to the modified example. [Figure 21] Figure 20 is an overhead view of the target vessel and berthing location, clearly indicating the indicators shown. [Modes for carrying out the invention]

[0010] According to a preferred embodiment of this disclosure, the information processing device includes: acquisition means for acquiring measurement data, which is a set of data representing a plurality of points to be measured by a measuring device installed on a ship; straight line calculation means for calculating a straight line along the berthing location where the ship docks, based on the measurement data; and distance calculation means for calculating the distance between the ship and the berthing location based on the distance between the straight line and each of the points to be measured. According to this embodiment, the information processing device can suitably calculate the distance between the ship and the berthing location based on the measurement data generated by the measuring device.

[0011] In one embodiment of the above-described information processing device, the information processing device further includes nearest neighbor search means for searching for the nearest neighbor point to the vessel from the point to be measured based on the straight line, and the distance calculation means calculates the distance based on the nearest neighbor point. In this embodiment, the information processing device can suitably calculate the distance between the vessel and the berthing location using the nearest neighbor point calculated based on the straight line.

[0012] In another embodiment of the information processing device described above, the measuring device includes a first measuring device and a second measuring device, wherein the straight line calculation means calculates a second straight line connecting a first nearest neighbor point to the vessel searched based on the straight line from the point of measurement of the first measurement data measured by the first measuring device, and a second nearest neighbor point to the vessel searched based on the straight line from the point of measurement of the second measurement data measured by the second measuring device, and the distance calculation means calculates the distance based on the second straight line. In this embodiment, the information processing device can accurately calculate a straight line along the berthing location.

[0013] In another embodiment of the information processing device described above, if the second straight line and the straight line calculated before the second straight line are not similar, the straight line calculation means re-searches for the first nearest neighbor point and the second nearest neighbor point based on the second straight line, calculates a third straight line connecting the re-searched first nearest neighbor point and the second nearest neighbor point, and the distance calculation means calculates the distance based on the third straight line. In this embodiment, the information processing device can more accurately calculate a straight line along the shoreing location.

[0014] In another embodiment of the information processing device described above, the line calculation means repeats the re-search for the first nearest neighbor point and the second nearest neighbor point and the calculation of the line until the calculated line converges. In this embodiment, the information processing device can suitably calculate a line that is suitable as a line along the shoreing location.

[0015] In another embodiment of the information processing device described above, if the index representing the variation of the measured point relative to the straight line is greater than or equal to a threshold, the straight line calculation means calculates the straight line based on the nearest neighbor point searched from the measured point based on the straight line calculated by the straight line calculation means before the current processing time. In this embodiment, the information processing device can accurately calculate a straight line along the docking location even when there is variation in the measurement data relative to the straight line.

[0016] In another embodiment of the information processing device described above, if an index representing the variability of the measured points relative to the straight line is greater than or equal to a threshold, and no straight line has been calculated within a predetermined time from the current processing time, the straight line calculation means calculates the straight line based on the nearest neighbor point searched from the measured point based on a straight line parallel to the direction of the vessel. In this embodiment, the information processing device can accurately calculate a straight line along the berthing location even when there is variability in the measurement data relative to the straight line and no recently calculated straight line exists.

[0017] In another embodiment of the above-described information processing device, the information processing device further includes a shortest distance calculation means for extracting the minimum length of the perpendiculars drawn from the contour points of the vessel to the straight line and calculating the shortest distance from the vessel's hull to the berthing location, and a proximity location identification means for identifying the part of the vessel that is closest to the berthing location. In this embodiment, the information processing device can suitably calculate the shortest distance from the vessel's hull to the berthing location and identify the part of the vessel that is closest to the berthing location.

[0018] According to another preferred embodiment of the present disclosure, a control method performed by a computer is provided, comprising: acquiring measurement data which is a set of data representing a plurality of points to be measured by a measuring device installed on a ship; calculating a straight line along the berthing location where the ship docks based on the measurement data; and calculating the distance between the ship and the berthing location based on the distance between the straight line and each of the points to be measured. By performing this control method, the computer can suitably calculate the distance between the ship and the berthing location.

[0019] In another preferred embodiment of the present disclosure, the program causes a computer to perform the following processes: acquire measurement data, which is a set of data representing a plurality of points to be measured by a measuring device installed on a ship; calculate a straight line along the berthing location where the ship docks based on the measurement data; and calculate the distance between the ship and the berthing location based on the distance between the straight line and each of the points to be measured. By executing this program, the computer can suitably calculate the distance between the ship and the berthing location. Preferably, the program is stored in a storage medium. [Examples]

[0020] Preferred embodiments of the present invention will be described below with reference to the drawings.

[0021] (1) Overview of the driver assistance system Figures 1(A) to 1(C) show the schematic configuration of the navigation support system according to this embodiment. Specifically, Figure 1(A) shows a block diagram of the navigation support system, Figure 1(B) is a top view illustrating the field of view (also called the "measurement range" or "distance-measurable range") 90 of the vessel and the lidar 3 described later, which are included in the navigation support system, and Figure 1(C) is a rear view showing the field of view 90 of the vessel and the lidar 3. The navigation support system comprises an information processing device 1 that moves together with the vessel, which is a moving object, and a group of sensors 2 mounted on the vessel. Hereafter, the vessel on which the navigation support system is installed will also be called the "target vessel".

[0022] The information processing device 1 is electrically connected to the sensor group 2 and provides operational support for the target vessel based on the outputs of the various sensors included in the sensor group 2. Operational support includes berthing support such as automatic docking. Here, "berthing" includes not only docking the target vessel at a quay but also docking it at a structure such as a pier. Furthermore, hereafter, "berthing location" refers to the general term for structures such as quays and piers that are the target of berthing. The information processing device 1 may be a navigation device installed on the vessel or an electronic control device built into the vessel.

[0023] Sensor group 2 includes various external and internal sensors installed on the ship. In this embodiment, sensor group 2 includes, for example, a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3.

[0024] The LIDA3 is an external sensor that discretely measures the distance to an object in the external environment by emitting a pulsed laser within a predetermined angular range in the horizontal direction (see Figure 1(B)) and a predetermined angular range in the vertical direction (see Figure 1(C)), and generates three-dimensional point cloud data indicating the position of the object.

[0025] LIDA 3 comprises an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data based on the received signal output by the light receiving unit. The data measured for each direction of laser light irradiation (scanning position) is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light specified based on the received signal described above. Hereafter, the point measured by the irradiation of laser light within the measurement range of LIDA 3, or the data thereof, will also be referred to as the "measured point".

[0026] In the examples shown in Figures 1(B) and 1(C), the target vessel is equipped with four lidars 3: one pointed forward on the port side, one pointed aft on the port side, one pointed forward on the starboard side, and one pointed aft on the starboard side. When the target vessel approaches the berthing location, the forward and aft lidars 3, located on the side of the vessel that is docking alongside the berthing location, generate point cloud data measuring the berthing location. Hereafter, the forward lidar 3 that measures the berthing location will be referred to as the "forward lidar," and the aft lidar 3 that measures the berthing location will be referred to as the "backward lidar." The forward and aft measuring lidars are examples of the "first measuring device" and "second measuring device." Note that the arrangement of lidars 3 is not limited to the examples shown in Figures 1(B) and 1(C).

[0027] LiDAR 3 is not limited to the scanning type LiDAR described above, but may also be a flash type LiDAR that generates 3D data by diffusing laser light into the field of view of a 2D array sensor. LiDAR 3 is an example of a "measuring device" in the present invention.

[0028] (2) Configuration of an information processing device Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly consists of an interface 11, a memory 12, and a controller 13. Each of these elements is interconnected via a bus line.

[0029] Interface 11 performs interface operations related to the exchange of data between the information processing device 1 and external devices. In this embodiment, interface 11 acquires output data from each sensor in the sensor group 2 and supplies it to the controller 13. Interface 11 also supplies signals related to the control of the target vessel, generated by the controller 13, to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel includes a drive source such as an engine or electric motor, a propeller that generates thrust in the direction of travel based on the driving force of the drive source, a thruster that generates thrust in the lateral direction based on the driving force of the drive source, and a rudder, etc., which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation such as automatic docking, interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, interface 11 supplies control signals generated by the controller 13 to the electronic control device. Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices by cables, etc. Furthermore, interface 11 may perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0030] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0031] Furthermore, memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, memory 12 may store map data including information about the location of the docking place. In another example, memory 12 stores information about the downsampling size when downsampling is performed on the point cloud data obtained when the lidar 3 performs one cycle of scanning.

[0032] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to the operation support of the target vessel by executing programs stored in memory 12, etc.

[0033] Furthermore, the controller 13 functionally includes a docking location detection unit 15 and a docking parameter calculation unit 16. The docking location detection unit 15 performs processing related to the detection of a docking location based on the point cloud data output by the lidar 3. The docking parameter calculation unit 16 calculates the parameters necessary for docking at the docking location (also called "docking parameters"). Here, the docking parameters include the distance from the target vessel to the docking location (distance to the opposite shore), the approach angle of the target vessel to the docking location, and the speed at which the target vessel approaches the docking location (docking speed). The docking parameter calculation unit 16 also calculates information representing the reliability of docking at the docking location (also called "reliability information") based on the processing results of the docking location detection unit 15 and the docking parameters. The controller 13 functions as an "acquisition means," a "straight line calculation means," a "nearest neighbor search means," a "distance calculation means," and a computer that executes programs.

[0034] Furthermore, the processing performed by the controller 13 is not limited to being implemented by software through a program, but may also be implemented by a combination of hardware, firmware, and software. Additionally, the processing performed by the controller 13 may be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program executed by the controller 13 in this embodiment may be implemented using this integrated circuit.

[0035] (3) Overview of docking support procedures Next, we will explain the overview of the docking support process performed by the information processing device 1. Based on the point cloud data generated by the forward lidar and the point cloud data generated by the rear lidar, the information processing device 1 generates a straight line along the side of the docking location (also called the "docking side line L"). Then, based on the docking side line L, the information processing device 1 calculates docking parameters such as the distance to the opposite shore.

[0036] Figure 3 is a functional block diagram of the berthing location detection unit 15 and the berthing parameter calculation unit 16 related to the berthing support process. Functionally, the berthing location detection unit 15 includes a normal vector calculation block 20, a field of view / detection surface identification block 21, a normal number identification block 22, a mean / variance calculation block 23, and a berthing status determination block 24. Functionally, the berthing parameter calculation unit 16 includes a straight line calculation block 27, a distance calculation block 28, an entry angle calculation block 29, a berthing speed calculation block 30, and a reliability information generation block 40.

[0037] The normal vector calculation block 20 calculates the normal vector of the surface formed by the berthing location (also called the "berthing surface") based on the point cloud data generated by the lidar 3 in the direction in which the berthing location exists. In this case, the normal vector calculation block 20 calculates the normal vector based on, for example, the point cloud data generated by the forward lidar and the rear lidar, which each include the berthing side in their measurement ranges for the target vessel. Information regarding the measurement range of the lidar 3 and the direction of the berthing location may be pre-registered in, for example, memory 12.

[0038] In this case, the normal vector calculation block 20 preferably performs downsampling of the point cloud data and removal of data obtained by the reflection of laser light from the water surface (also called "water surface reflection data").

[0039] In this case, first, the normal vector calculation block 20 removes data located below the water surface position from the point cloud data generated by the lidar 3, treating it as water surface reflection data (i.e., false detection data). The normal vector calculation block 20 estimates the water surface position based, for example, on the average value in the height direction of the point cloud data generated by the lidar 3 when there are no objects other than the water surface in the surrounding area. Then, the normal vector calculation block 20 performs downsampling on the point cloud data after the water surface reflection data has been removed, which is a process that integrates the measured points for each grid space of a predetermined size. Finally, the normal vector calculation block 20 calculates a normal vector for each measured point indicated by the downsampled point cloud data using multiple surrounding measured points. Note that downsampling may be performed before removing the data reflected from the water surface.

[0040] The field of view / detection surface identification block 21 identifies the surface of the docking location that is within the field of view angle of the Lidar 3 (also called the "inner surface of the field of view") and the surface of the docking location detected based on the normal vector calculated by the normal vector calculation block 20 (also called the "detection surface"). In this case, the field of view / detection surface identification block 21 determines whether the upper surface and / or the side surface of the docking location are included in the inner surface of the field of view and the detection surface.

[0041] The normal vector number identification block 22 extracts the vertical normal vector and the normal vector perpendicular to it (i.e., the horizontal direction) from the normal vector calculated by the normal vector calculation block 20, and calculates the number of vertical normal vectors and the number of horizontal normal vectors. Here, the normal vector number identification block 22 considers the vertical normal vector to represent the normal to the measured point on the upper surface of the docking location, and the horizontal normal vector to represent the normal to the measured point on the side of the docking location, and calculates the number of each as an indicator of the reliability of the docking location.

[0042] The mean and variance calculation block 23 extracts the vertical normal vector and the normal vector perpendicular to it (i.e., horizontal) from the normal vectors calculated by the normal vector calculation block 20, and calculates the mean and variance of the vertical normal vector and the mean and variance of the horizontal normal vector.

[0043] The docking status determination block 24 obtains the processing results of the field of view / detection surface identification block 21, the normal number identification block 22, and the mean / variance calculation block 23, which are identified or calculated based on the same point cloud data, as determination results representing the detection status of the docking location at the time the point cloud data was generated. The docking status determination block 24 then supplies the processing results of the field of view / detection surface identification block 21, the normal number identification block 22, and the mean / variance calculation block 23 as determination results of the detection status of the docking location to the docking parameter calculation unit 16.

[0044] Based on the processing results of the normal vector calculation block 20, the straight line calculation block 27 extracts point cloud data (also called "side point cloud data") that constitutes the side of the docking location from the point cloud data generated by the forward lidar and the point cloud data generated by the rear lidar. For example, the straight line calculation block 27 extracts the data of the measured point, which will be the normal vector oriented in the horizontal direction, as side point cloud data. Then, the straight line calculation block 27 generates a docking side straight line L (more specifically, an equation representing the straight line) based on the nearest neighbor point found from the side point cloud data of the forward lidar and the nearest neighbor point found from the side point cloud data of the rear lidar. Details of the calculation method for the docking side straight line L will be described later.

[0045] The opposite shore distance calculation block 28 calculates the opposite shore distance, which corresponds to the shortest distance between the target vessel and the berthing location, based on the berthing side line L generated by the straight line generation block 27. Here, the opposite shore distance calculation block 28 calculates the opposite shore distance as, for example, the shortest distance between the berthing side line L and each rider 3 (i.e., the forward rider and the aft rider). Alternatively, the opposite shore distance may be calculated as the shortest distance from a reference point such as the vessel's center position to the berthing side line L. Note that instead of considering the shortest distance for each rider 3 as the opposite shore distance, the opposite shore distance calculation block 28 may define the opposite shore distance as the shorter of the shortest distances for each rider 3, or it may define the opposite shore distance as the average of these shortest distances.

[0046] The approach angle calculation block 29 calculates the approach angle of the target vessel to the berthing location based on the berthing side line L generated by the straight line generation block 27. Specifically, the approach angle calculation block 29 calculates the approach angle using the function "atan2", which finds the arctangent from two arguments that define the tangent. More specifically, the approach angle calculation block 29 calculates the approach angle from the direction vector of the berthing side line L by calculating the function atan2.

[0047] The berthing speed calculation block 30 calculates the berthing speed, which is the speed at which the target vessel approaches the berthing location, based on the distance to the opposite shore calculated by the distance to the opposite shore calculation block 28. For example, the berthing speed calculation block 30 calculates the berthing speed as the change in the distance to the opposite shore (shortest distance) over time.

[0048] The reliability information generation block 40 generates reliability information based on the processing results of the docking status determination block 24, the straight line calculation block 27, the opposite bank distance calculation block 28, and the approach angle calculation block 29. Details of the reliability information will be described later.

[0049] Next, specific examples of the processing of the normal vector calculation block 20, the normal number determination block 22, and the mean / variance calculation block 23 will be explained with reference to Figure 4.

[0050] Figure 4(A) shows an example of a hull coordinate system based on the hull of the target vessel. As shown in Figure 4(A), the forward direction of the target vessel is the "x" coordinate, the side direction of the target vessel is the "y" coordinate, and the height direction of the target vessel is the "z" coordinate. The measurement data from the coordinate system based on LIDA 3, measured by LIDA 3, is then converted to the hull coordinate system shown in Figure 4(A). The process of converting point cloud data from a coordinate system based on a LIDA installed on a moving object to the coordinate system of the moving object is disclosed, for example, in International Publication WO2019 / 188745.

[0051] Figure 4(B) shows the lidar 3 capturing the quay where the ship docks. Figure 4(C) is a perspective view of the docking location (the quay) with respect to the quay, clearly showing the measured points representing the measurement positions measured by the lidar 3, and the normal vectors calculated based on those measured points. In Figure 4(C), the measured points are indicated by circles, and the normal vectors are indicated by arrows. This example shows that both the top and side surfaces of the quay were measured by the lidar 3.

[0052] As shown in Figure 4(C), the normal vector calculation block 20 calculates normal vectors for the measurement points on the side and top surfaces of the quay wall. Since normal vectors are vectors perpendicular to the target plane or curved surface, they are calculated using multiple measurement points that can form a surface. Therefore, a grid with predetermined lengths for length and width, or a circle with a predetermined radius, is set up, and the calculation is performed using the measurement points located within it. In this case, the normal vector calculation block 20 may calculate a normal vector for each measurement point, or it may calculate normal vectors at predetermined intervals. The normal number identification block 22 then determines that normal vectors whose z component is greater than a predetermined threshold are normal vectors oriented in the vertical direction. It is assumed that the normal vectors are unit vectors. Furthermore, normal vectors whose z component is less than a predetermined threshold are determined to be normal vectors oriented in the horizontal direction. The normal number identification block 22 then identifies the number of vertical normal vectors (5 in this case) and the number of horizontal normal vectors (4 in this case). Furthermore, the mean / variance calculation block 23 calculates the mean and variance of the vertical normal vector and the mean and variance of the horizontal normal vector. Note that for edge areas, the measurement direction will be diagonal because the surrounding measurement points are on the top or side.

[0053] (4) Details of the method for generating the straight line along the shoreline. Next, we will explain the methods for generating the shoreline straight line L (first generation method and second generation method).

[0054] (4-1) First generation method In the first generation method, the docking parameter calculation unit 16 first generates a provisional docking side line L (also called the "first docking side line L1") based on the side point cloud data of the forward lighter, and generates a provisional docking side line L (also called the "second docking side line L2") based on the side point cloud data of the rear lighter. Then, the docking parameter calculation unit 16 searches for the nearest neighbor points from the side point cloud data of the forward lighter based on the first docking side line L1, and searches for the nearest neighbor points from the side point cloud data of the rear lighter based on the second docking side line L2. After that, the docking parameter calculation unit 16 generates a line connecting these nearest neighbor points, and using the generated line, searches for the nearest neighbor points again using the same procedure, and generates a line connecting the searched nearest neighbor points. Then, when the docking parameter calculation unit 16 determines that the equation of the line to be generated has not changed and has converged, it sets the converged line as the docking side line L. Through this process, the docking parameter calculation unit 16 extracts a point at the tip of the docking location as the nearest neighbor point and suitably determines a straight line passing through the tip of the docking location as the docking side line L. As will be described later, the process of repeatedly calculating the straight line should be executed and continued only when there is sufficient time until the next point cloud data output time (i.e., the next processing time based on the frame period of the point cloud data).

[0055] Figures 5(A) to 5(C) illustrate the processing flow of the first generation method. First, as shown in Figure 5(A), the berthing parameter calculation unit 16 generates a first berthing side line L1 from the side point cloud data of the forward lighter, and uses this line to extract the nearest neighbor point "p1" closest to the target vessel from the measured points in the side point cloud data of the forward lighter. Similarly, the berthing parameter calculation unit 16 generates a second berthing side line L2 from the side point cloud data of the rear lighter, and uses this line to extract the nearest neighbor point "p2" closest to the target vessel from the measured points in the side point cloud data of the rear lighter.

[0056] Next, as shown in Figure 5(B), the docking parameter calculation unit 16 generates a provisional straight line "L'" connecting the nearest neighbor points p1 and p2. Using the provisional straight line L', it extracts the nearest neighbor point "p'1" from the measured points in the side point cloud data of the forward lidar, and extracts the nearest neighbor point "p'2" from the measured points in the side point cloud data of the rear lidar. Then, as shown in Figure 5(C), the docking parameter calculation unit 16 generates a straight line "L''" connecting the nearest neighbor points p'1 and p'2. Since the provisional straight line L' and the provisional straight line L'' are similar, the last calculated provisional straight line L'' is set as the docking side straight line L. In determining the approximation between the provisional straight line L' and the provisional straight line L'', for example, the docking parameter calculation unit 16 takes the difference between the x, y, and z elements of each unit vector, and determines that they are not similar if the maximum value is greater than or equal to a threshold, and determines that they are similar if it is less than the threshold. Subsequently, the docking parameter calculation unit 16 calculates the distance to the opposite shore based on the docking side line L. Provisional line L′ is an example of a "first line," and provisional line L′′ is an example of a "second line." Also, nearest neighbor points p1 and p′1 are examples of "first nearest neighbor points," and nearest neighbor points p2 and p′2 are examples of "second nearest neighbor points."

[0057] Here, the details of the method for calculating the nearest neighbor point p1 shown in Figure 5(A) will be explained with reference to Figures 6(A) and 6(B). Figure 6(A) is a top view clearly showing the first shoreline L1 and the nearest neighbor point p1, and Figure 6(B) is a diagram clearly showing the unit vector "u" used to search for the nearest neighbor point p1.

[0058] First, the docking parameter calculation unit 16 applies principal component analysis and least squares method to the lateral point cloud data of the forward lidar to generate the first docking lateral line L1 shown in equation (1) below.

[0059]

number

[0060] The docking parameter calculation unit 16 then calculates a unit vector u that is perpendicular to the first docking side line L1 in a two-dimensional plane, as shown in equation (2) below.

[0061]

number

[0062]

number

[0063]

number

[0064] Next, we will provide a supplementary explanation regarding the provisional line L' based on the nearest neighbor points p1 and p2. Figures 7(A) to 7(C) show the process flow for calculating the provisional line L' based on the nearest neighbor points p1 and p2, and for calculating the distance to the opposite bank based on the provisional line L'. Figure 7(A) shows the nearest neighbor point p1 found based on the first shoreline line L1, and the nearest neighbor point p2 found based on the second shoreline line L2. Figure 7(B) shows the provisional line L' connecting the nearest neighbor points p1 and p2. Figure 7(C) shows the process of calculating the distance to the opposite bank by calculating the length of the perpendicular from the position of the forward lighter to the provisional line L', and the length of the perpendicular from the position of the rear lighter to the provisional line L', respectively.

[0065] Here, as shown in Figures 7(A) to 7(C), the provisional line L′ connecting the nearest neighbor point p1 found from the side point cloud data of the forward lidar and the nearest neighbor point p2 found from the side point cloud data of the rear lidar is approximately parallel to the side of the docking location. Therefore, by using such a provisional line L′ as the docking side line L, the information processing device 1 can accurately calculate various docking parameters such as the distance to the opposite shore. To determine the length of the perpendiculars from the forward and rear reference points to the docking side line L, point p in Figure 6(B) can be set to either the nearest neighbor point p1 or the nearest neighbor point p2, which are points on the docking side line L, and the calculation can be performed using equation (4).

[0066] Preferably, the shoreing parameter calculation unit 16 updates the provisional straight line L' to make the shoreing side straight line L more accurate if there is sufficient time until the next point cloud data output time. In this case, for example, the shoreing parameter calculation unit 16 determines that there is sufficient time until the next point cloud data output time if the time length from the current time to the next point cloud data output time is greater than or equal to a predetermined threshold.

[0067] Figures 8(A) to 8(C) show the process flow for calculating the updated provisional line L′′. Figure 8(A) shows the nearest neighbor point p1 found based on the first shoreline L1 and the nearest neighbor point p2 found based on the second shoreline L2. Figure 8(B) shows the provisional line L′ connecting nearest neighbor points p1 and p2, the nearest neighbor point p′1 found from the lateral point cloud data of the forward lidar based on the provisional line L′, and the nearest neighbor point p′2 found from the lateral point cloud data of the rear lidar based on the provisional line L′. Figure 8(C) shows the provisional line L′′ connecting nearest neighbor points p′1 and p′2.

[0068] In this case, the docking parameter calculation unit 16 compares the direction vectors of the first docking side line L1 and the second docking side line L2 with the direction vector of the provisional line L'. If the difference between them is greater than or equal to a predetermined threshold, it determines that the process has not converged and that the provisional line L' needs to be updated. Therefore, in this case, as shown in Figure 8(B), the docking parameter calculation unit 16 uses the provisional line L' to search for the nearest neighbor point p'1 from the side point cloud data of the forward lidar and to search for the nearest neighbor point p'2 from the side point cloud data of the rear lidar. Then, as shown in Figure 8(C), the docking parameter calculation unit 16 calculates a provisional line L'' connecting the nearest neighbor points p'1 and p'2, and considers the provisional line L'' as the docking side line L to calculate the distance to the opposite shore, etc. It can be seen that the two arrows indicating the distance to the opposite shore in Figure 8(C) are more accurate than those in Figure 7(C). By doing so, the shoreing parameter calculation unit 16 can more accurately calculate the shoreing side line L and accurately calculate various shoreing parameters such as the distance to the opposite shore.

[0069] For a detailed explanation of the processing flow of the first generation method, please refer to Figure 15 and see the following description.

[0070] Here, we will provide a supplementary explanation regarding the effectiveness of the first generation method. Generally, in order to determine the distance to the opposite shore, it is necessary to extract the point closest to the tip of the shore (i.e., the nearest neighbor) from the point cloud data projected onto the shore in a two-dimensional plane. On the other hand, depending on the measurement direction of the lidar 3 and the orientation of the ship relative to the shore, it may not be possible to accurately extract the nearest neighbor by simply comparing the distance from the lidar 3 to each point.

[0071] Figure 9(A) is a diagram that clearly shows the nearest neighbor points to be extracted. Figure 9(B) shows an example in which the data of the point to be measured with the shortest measurement distance of the LIDA3 is extracted as the nearest neighbor point. Figure 9(C) shows an example in which only the y-coordinate values ​​of the ship's coordinate system are compared and the point closest to the ship's origin is extracted as the nearest neighbor point. In the method of extracting the data of the point to be measured with the shortest measurement distance of the LIDA3 as the nearest neighbor point (see Figure 9(B)) and the method of comparing only the y-coordinate values ​​of the ship's coordinate system and extracting the point closest to the ship's origin as the nearest neighbor point (see Figure 9(C)), the point located closest to the tip of the quay (see Figure 9(A)) is not accurately extracted as the nearest neighbor point. Taking the above into consideration, in the first generation method, the docking parameter calculation unit 16 searches for the nearest neighbor point p1 using the first docking side line L1 based on the point cloud data of the forward lidar, and the nearest neighbor point p2 using the second docking side line L2 based on the point cloud data of the rear lidar, and calculates a line connecting them. This makes it possible to generate a docking side line L that passes through the tip of the docking location, thus enabling accurate calculation of various docking parameters such as the distance to the opposite shore. Furthermore, even if the docking location is a quay with poor flatness, a docking side line L that passes through the tip of the docking location can be generated by repeatedly performing the process of updating the line.

[0072] (4-2) Second generation method In the second generation method, after calculating the first shore side line L1 and the second shore side line L2, the shore parameter calculation unit 16 searches for the nearest neighbor point based on the shore side line L calculated in the past, if the variance of the side point cloud data for each line is greater than or equal to a predetermined threshold. Note that "variance" is just one example of an "indicator representing variability," and in the following, standard deviation may be used instead of variance.

[0073] Generally, when a quay captured by LIDA3 is a docking location with poor flatness, the variability of the lateral point cloud data increases, resulting in an inaccurate docking lateral line L. Furthermore, when generating a docking lateral line L based on lateral point cloud data for a quay with poor flatness, the variance of the measured points represented by the lateral point cloud data relative to the docking lateral line L becomes large.

[0074] Taking the above into consideration, in the second generation method, the docking parameter calculation unit 16 does not calculate the docking side line L based on the side point cloud data if the standard deviation of each point in the side point cloud data obtained at the current processing time with respect to the first docking side line L1 and the second docking side line L2 is greater than or equal to a predetermined threshold (for example, σ > 0.5). Figure 10 shows the measured points of the side point cloud data of the forward lidar and the first docking side line L1. In Figure 10, "σ" represents the variance of the measured points of the side point cloud data of the forward lidar with respect to the first docking side line L1. 2 This is explicitly stated.

[0075] If the berthing parameter calculation unit 16 has generated a berthing side line L within a predetermined time period in the past (for example, within 1 second), it searches for the nearest point at the current processing time and calculates the berthing side line L based on that berthing side line L. On the other hand, if the berthing parameter calculation unit 16 has not generated a berthing side line L within a predetermined time period in the past (for example, within 1 second), it searches for the nearest point at the current processing time and calculates the berthing side line L based on a line parallel to the orientation of the target vessel.

[0076] Figures 11(A) to 11(C) show the processing flow when a shoreline straight line L has been generated within a predetermined time period in the past.

[0077] First, if a past shore lateral line L has been generated within a predetermined time, the shore parameter calculation unit 16 searches for the nearest neighbor points p1 and p2 from the lateral point cloud data of the forward lidar and the lateral point cloud data of the rear lidar, respectively, based on the past shore lateral line L, as shown in Figure 11(A). In this case, the shore parameter calculation unit 16 determines the nearest neighbor points p1 and p2 by calculating the distance to each measurement point using a vector perpendicular to the direction vector of the past shore lateral line L.

[0078] Next, as shown in Figure 11(B), the docking parameter calculation unit 16 generates a provisional straight line L' connecting the nearest neighbor points p1 and p2, and searches for the nearest neighbor points p'1 and p'2 from the side point cloud data of the forward lighter and the side point cloud data of the rear lighter, respectively, based on the provisional straight line L'. Furthermore, as shown in Figure 11(C), the docking parameter calculation unit 16 generates a provisional straight line L'' passing through the nearest neighbor points p'1 and p'2. Subsequently, if the provisional straight line L' and the provisional straight line L'' are not approximate, the docking parameter calculation unit 16 searches for the nearest neighbor points again using the provisional straight line L'' in the same procedure and repeats the process of generating a straight line connecting the searched nearest neighbor points. On the other hand, if the docking parameter calculation unit 16 determines that the equation of the generated straight line has not changed and has converged (i.e., the straight line before and after the update is approximate), it sets the converged straight line as the docking side straight line L and performs calculations such as the distance to the opposite shore. In the examples in Figures 11(A) to 11(C), it can be seen that the nearest neighbor point gradually approaches the tip of the quay, and the straight line approaches one that passes through the tip of the quay.

[0079] Figures 12(A) to 12(C) show the processing flow when there is no shoreline straight line L generated within a predetermined time period in the past.

[0080] In this case, as shown in Figure 12(A), the berthing parameter calculation unit 16 sets straight lines parallel to the orientation of the target vessel as the first berthing side line L1 and the second berthing side line L2, respectively, and searches for the nearest neighbor points p1 and p2 based on the first berthing side line L1 and the second berthing side line L2. Next, as shown in Figure 12(B), the berthing parameter calculation unit 16 generates a provisional straight line L' connecting the nearest neighbor points p1 and p2, and searches for the nearest neighbor points p'1 and p'2 from the side point cloud data of the forward rider and the side point cloud data of the rear rider, respectively, based on the provisional straight line L'. Furthermore, as shown in Figure 12(C), the berthing parameter calculation unit 16 generates a provisional straight line L'' passing through the nearest neighbor points p'1 and p'2. Subsequently, if the provisional line L′ and the provisional line L′′ are not approximate, the docking parameter calculation unit 16 searches for the nearest neighbor point again using the provisional line L′′ and repeats the process of generating a line connecting the searched nearest neighbor points. On the other hand, if the docking parameter calculation unit 16 determines that the equation of the generated line has not changed and has converged (i.e., the line before and after the update is approximate), it sets the converged line as the docking side line L and performs calculations such as the distance to the opposite shore. In the examples in Figures 12(A) to 12(C), it can be seen that the nearest neighbor point gradually approaches the tip of the quay, and the line approaches one that passes through the tip of the quay.

[0081] Furthermore, if no berthing side line L has been generated within the past predetermined time, and the angle between the target vessel and the side of the quay is large, the initial line will differ significantly from the berthing side line L that should have been calculated, which tends to increase the number of iterations required for convergence. Therefore, preferably, if there is insufficient time before the next output of the rider 3, the berthing parameter calculation unit 16 will terminate the line update midway and use the last calculated line as the berthing side line L. Even in this case, the line that is approaching convergence will be used as the initial value for the next time, so by continuing this process, convergence will be completed within the time limit.

[0082] The detailed processing flow of the second generation method will be described later, referring to Figure 16.

[0083] (5) Generating confidence information Next, a specific example of generating reliability information will be explained. The reliability information generation block 40 generates flags for each element, such as the field of view at the time of detection of the docking location, the surface detection of the docking location, the number of normal vectors, and the variance, and generates a vector of the generated flags as reliability information. Hereafter, a flag of "1" will indicate that the reliability of the corresponding element is high, and a flag of "0" will indicate that the reliability of the corresponding element is low.

[0084] Figure 13 shows an example of the data structure of confidence information generated by the confidence information generation block 40. As shown in Figure 13, the confidence information has the following items: "Top view", "Side view", "Linear view", "Distance", and "Angle". The item "Top view" has the sub-items "Field of view angle", "Detection", "Normal number", and "Variance", the item "Side view" has the sub-items "Field of view angle", "Detection", "Normal number", and "Variance", the item "Linear view" has the sub-item "Variance", the item "Distance" has the sub-items "Amount of change" and "Rate of change", and the item "Angle" has the sub-item "Amount of change".

[0085] Here, the reliability information generation block 40 registers a flag in the "Field of View Angle" sub-item of the "Top Surface" item, which is set to "1" if the top surface of the docking location is within the field of view, and to "0" if the top surface is outside the field of view. The reliability information generation block 40 also registers a flag in the "Detection" sub-item of the "Top Surface" item, which is set to "1" if the top surface of the docking location is a detection surface, and to "0" if the top surface is not a detection surface. The reliability information generation block 40 also registers a flag in the "Number of Normals" sub-item of the "Top Surface" item, which is set to "1" if the number of normal vectors to the top surface of the docking location is equal to or greater than a predetermined threshold (e.g., 10), and to "0" if the number is less than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "Variance" sub-item of the "Upper Surface" item, where "1" is set if the variance of the x, y, and z components of the normal vector to the upper surface of the docking location is all less than a predetermined threshold (e.g., 1.0), and "0" is set if any of the variances is equal to or greater than the threshold. The reliability information generation block 40 also registers flags in each sub-item of the "Side" item, which are determined by the same rules as for each sub-item of the "Upper Surface" item.

[0086] Furthermore, the reliability information generation block 40 registers a flag in the "Variance" sub-item of the "Linear Line" item that represents the reliability of the shoreline line L calculated by either the first generation method or the second generation method. For example, the reliability information generation block 40 registers a flag that is "1" if the variance of the shoreline point cloud data with respect to the shoreline line L is less than a predetermined threshold, and "0" if the variance is equal to or greater than the threshold.

[0087] Furthermore, the reliability information generation block 40 registers a flag in the "Change Amount" sub-item of the "Distance" item, which is set to "1" if the change amount of the distance to the opposite bank calculated by the opposite bank distance calculation block 28 from one time point in the previous period is less than a predetermined threshold (e.g., 1.0 m), and to "0" if the change amount is equal to or greater than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "Rate of Change" sub-item of the "Distance" item, which is set to "1" if the rate of change of the distance to the opposite bank calculated by the opposite bank distance calculation block 28 from one time point in the previous period is less than a predetermined threshold (e.g., ±10%), and to "0" if the rate of change is equal to or greater than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "Change Amount" sub-item of the "Angle" item, which is set to "1" if the change amount of the approach angle calculated by the approach angle calculation block 29 from one time point in the previous period is less than a predetermined threshold (e.g., 1.0 degrees), and to "0" if the change amount is equal to or greater than the threshold.

[0088] The thresholds mentioned above are set to conforming values ​​pre-stored in, for example, memory 12. Furthermore, confidence information may be generated for each Writer 3.

[0089] With confidence information having such a data structure, it is possible to understand the reliability of the calculated distance to the opposite shore, docking speed, and approach angle. A value of "1" for each sub-item of the confidence information indicates the highest level of reliability. The information processing device 1 then adjusts the output of the drive source during docking based on this confidence information. For example, the information processing device 1 may determine the upper limit of the target vessel's speed during docking based on the sum of the sub-items indicated by the confidence information. In this case, the information processing device 1 might determine that the smaller the sum, the lower the reliability of the information regarding the docking location, and therefore requires careful docking, thus reducing the upper limit of the target vessel's speed during docking.

[0090] (6) Processing flow (6-1) Processing Overview Figure 14 is an example of a flowchart illustrating the overview of the docking support process in this embodiment. The information processing device 1 repeatedly executes the processes shown in the flowchart of Figure 14.

[0091] First, the information processing device 1 acquires point cloud data in the direction of the berthing location (step S11). In this case, the information processing device 1 acquires, for example, point cloud data generated by a lidar 3 that includes the berthing side of the target vessel within its measurement range. The information processing device 1 may also further downsample the acquired point cloud data and remove data reflected from the water surface.

[0092] Next, the docking location detection unit 15 of the information processing device 1 calculates normal vectors based on the point cloud data acquired in step S11 (step S12). Furthermore, in step S12, the docking location detection unit 15 calculates the number of normal vectors and the variance of the normal vectors. Also, based on the processing results of step S12, the docking location detection unit 15 identifies the inner surface of the field of view and the detection surface (step S13).

[0093] Next, the shoreing parameter calculation unit 16 executes a shoreing side line calculation process, which is the process of calculating the shoreing side line L (step S14). The shoreing side line calculation process based on the first generation method will be described later with reference to Figure 15, and the shoreing side line calculation process based on the second generation method will be described later with reference to Figure 16.

[0094] Next, the docking parameter calculation unit 16 calculates the docking parameters, namely the distance to the opposite shore, the approach angle, and the docking speed, using the docking side line L calculated in step S15 (step S15).

[0095] Then, the berthing parameter calculation unit 16 generates confidence information based on the processing results from steps S13 to S15 (step S16). Subsequently, the information processing device 1 controls the vessel based on the confidence information (step S17). As a result, the information processing device 1 can accurately perform vessel control related to berthing based on a confidence level that accurately reflects the berthing situation.

[0096] The information processing device 1 then determines whether or not the target vessel has docked (step S18). In this case, the information processing device 1 determines whether or not the target vessel has docked based, for example, on the output signals of the sensor group 2 or user input via the interface 11. If the information processing device 1 determines that the target vessel has docked (step S18; Yes), it terminates the flowchart process. On the other hand, if the information processing device 1 determines that the target vessel has not docked (step S18; No), it returns to step S11.

[0097] (6-2) Processing for calculating the straight line of the shoreline based on the first generation method Figure 15 is an example of a flowchart for calculating the shoreline side straightness based on the first generation method. The shoreline parameter calculation unit 16 executes the process shown in the flowchart of Figure 15 in step S14 of Figure 14.

[0098] First, the docking parameter calculation unit 16 extracts lateral point cloud data from the point cloud data generated by the forward lighter and the rear lighter, respectively (step S21). Then, the docking parameter calculation unit 16 calculates the first docking lateral line L1 based on the lateral point cloud data of the forward lighter and calculates the second docking lateral line L2 based on the lateral point cloud data of the rear lighter (step S22).

[0099] The docking parameter calculation unit 16 then searches for the nearest neighbor points p1 and p2 corresponding to the first docking side line L1 and the second docking side line L2, respectively (step S23). In this case, the docking parameter calculation unit 16 determines the nearest neighbor point p1 based on the dot product of a vector perpendicular to the first docking side line L1 and each data point in the side point cloud data of the forward lidar (position vector of the point being measured), and determines the nearest neighbor point p2 based on the dot product of a vector perpendicular to the second docking side line L2 and each data point in the side point cloud data of the rear lidar.

[0100] The docking parameter calculation unit 16 then calculates a provisional straight line L' connecting the nearest neighbor points p1 and p2 (step S24). The docking parameter calculation unit 16 then determines whether the provisional straight line L' approximates both the first docking side line L1 and the second docking side line L2 (step S25). For example, the docking parameter calculation unit 16 determines that the provisional straight line L' approximates the first docking side line L1 and the second docking side line L2 if the maximum difference between the elements of the unit vectors of the provisional straight line L' and the first docking side line L1 is less than or equal to a predetermined value, and the maximum difference between the elements of the unit vectors of the provisional straight line L' and the second docking side line L2 is less than or equal to a predetermined value. If the above conditions are not met, the unit determines that the provisional straight line L' does not approximate the first docking side line L1 and the second docking side line L2.

[0101] Then, if the provisional line L′ does not approximate at least one of the first shoreline line L1 and the second shoreline line L2 (Step S25; No), the shoreline parameter calculation unit 16 determines whether there is sufficient time until the output time of the next point cloud data for lidar 3 (Step S26). Then, if the provisional line L′ approximates both the first shoreline line L1 and the second shoreline line L2 (Step S25; Yes), or if there is not sufficient time until the output time of the next point cloud data for lidar 3 (Step S26; No), the shoreline parameter calculation unit 16 sets the provisional line to the shoreline line L (Step S30).

[0102] On the other hand, if there is sufficient time (Step S26; Yes), the docking parameter calculation unit 16 searches for the nearest neighbors (nearest neighbors p1 and p2) to the provisional straight line (Step S27). Then, the docking parameter calculation unit 16 calculates a new provisional straight line connecting the nearest neighbors (Step S28). Then, the docking parameter calculation unit 16 determines whether the newly calculated provisional straight line is similar to the previous provisional straight line (Step S29). If the newly calculated provisional straight line is similar to the previous provisional straight line (Step S29; Yes), the docking parameter calculation unit 16 sets the provisional straight line to the docking side straight line L (Step S30). On the other hand, if the newly calculated provisional straight line is not similar to the previous provisional straight line (Step S29; No), the docking parameter calculation unit 16 returns to Step S26.

[0103] (6-3) Calculation process for the straight line of the shoreline based on the second generation method Figure 16 is an example of a flowchart for calculating the shoreline side straightness based on the second generation method. The shoreline parameter calculation unit 16 executes the process shown in the flowchart of Figure 16 in step S14 of Figure 14.

[0104] First, the docking parameter calculation unit 16 extracts lateral point cloud data from the point cloud data generated by the forward lighter and the rear lighter, respectively (step S31). Then, the docking parameter calculation unit 16 calculates the first docking lateral line L1 based on the lateral point cloud data of the forward lighter and calculates the second docking lateral line L2 based on the lateral point cloud data of the rear lighter (step S32).

[0105] The docking parameter calculation unit 16 then calculates the variance of the forward lighter's side point cloud data with respect to the first docking side line L1 and the variance of the forward lighter's side point cloud data with respect to the second docking side line L2 (step S33). The docking parameter calculation unit 16 then determines whether either of the calculated variances is greater than or equal to a predetermined value (step S34).

[0106] The berthing parameter calculation unit 16 then determines that the first berthing side line L1 and the second berthing side line L2 should not be used if any of the above-mentioned variances are greater than or equal to a predetermined value (step S34; Yes), and proceeds to step S42. The berthing parameter calculation unit 16 then determines whether or not a previous berthing side line L generated within a predetermined time period in the past exists (step S42). If a previous berthing side line L generated within a predetermined time period in the past exists (step S42; Yes), that berthing side line L is designated as a provisional line (step S44), and the process then proceeds to step S39. On the other hand, if no berthing side line L generated within a predetermined time period in the past exists (step S42; No), the berthing parameter calculation unit 16 generates lines parallel to the orientation of the target vessel as the first berthing side line L1 and the second berthing side line L2 (step S43).

[0107] On the other hand, if any of the above-mentioned variances are less than a predetermined value (step S34; No), or after step S43 is performed, the shoreing parameter calculation unit 16 searches for the nearest neighbor points corresponding to the first shoreing side line L1 and the second shoreing side line L2, respectively (step S35). Then, after step S35, the shoreing parameter calculation unit 16 calculates a provisional straight line connecting the nearest neighbor points (step S36).

[0108] The docking parameter calculation unit 16 then determines whether the provisional line approximates both the first docking lateral line L1 and the second docking lateral line L2 (step S37). If the provisional line does not approximate at least one of the first docking lateral line L1 and the second docking lateral line L2 (step S37; No), the docking parameter calculation unit 16 determines whether there is sufficient time until the output time of the next point cloud data for lidar 3 (step S38). If the provisional line L' approximates both the first docking lateral line L1 and the second docking lateral line L2 (step S37; Yes), or if there is not sufficient time until the output time of the next point cloud data for lidar 3 (step S38; No), the docking parameter calculation unit 16 sets the provisional line to docking lateral line L (step S45).

[0109] On the other hand, if there is sufficient time (Step S38; Yes), the docking parameter calculation unit 16 searches for the nearest neighbors (nearest neighbors p1 and p2) to the provisional straight line (Step S39). Then, the docking parameter calculation unit 16 calculates a new provisional straight line connecting the nearest neighbors (Step S40). Then, the docking parameter calculation unit 16 determines whether the newly calculated provisional straight line is similar to the previous provisional straight line (Step S41). If the newly calculated provisional straight line is similar to the previous provisional straight line (Step S41; Yes), the docking parameter calculation unit 16 sets the provisional straight line to the docking side straight line L (Step S45). On the other hand, if the newly calculated provisional straight line is not similar to the previous provisional straight line (Step S41; No), the docking parameter calculation unit 16 returns to Step S38.

[0110] (7) Variation (Variation 1) The information processing device 1 may determine the part of the ship's hull closest to the quay (proximity part) and the shortest distance from that proximity part to the quay, and use this information to assist in maneuvering the ship when docking or undocking.

[0111] In this case, the information processing device 1 determines the aforementioned adjacent area and shortest distance by following steps (1) to (3). (Step 1) Calculate the distance from multiple points representing the outline of your ship (also called "outline points Po") to the straight line L on the side of the docking area. (Step 2) Extract the minimum value from the distance from the calculated contour point Po to the straight line L on the side of the shore. (Step 3) The extracted minimum value is determined to be the shortest distance to the quay, and the contour point Po that corresponds to the minimum is determined to be the area close to the quay.

[0112] Figure 17 is an overhead view of the target vessel with its contour points Po clearly indicated. For example, memory 12 stores contour data, which is position data indicating the contour position of the target vessel. The contour data is data in which multiple (24 in this case) contour points Po representing the contour of the target vessel are recorded as coordinates in the vessel coordinate system. Here, the forward (forward) direction of the target vessel is "X b "Coordinates, the side direction of the target vessel is "Y" b The coordinates, the vertical direction of the target vessel is "Z b The coordinate system is defined as follows. Then, the measurement data of the coordinate system based on Lida 3, measured by Lida 3, is converted to the ship's coordinate system. The process of converting point cloud data of a coordinate system based on a Lida installed on a moving object to the coordinate system of the moving object is disclosed, for example, in International Publication WO2019 / 188745.

[0113] Figure 18 is a diagram that clearly shows the distance from contour point Po to the straight line L of the shoreline using arrows. These distances are calculated by determining the length of the perpendicular from each contour point Po to the straight line L of the shoreline, as in the method described above. Here, arrows indicating the aforementioned distances for 24 contour points Po are clearly shown. Figure 19 is a diagram that clearly shows the nearest location and the shortest distance. Here, the contour point Po corresponding to the minimum value extracted in (procedure 2) is highlighted as the nearest location by a frame 90, and the shortest distance from the nearest location to the quay is indicated by an arrow 91.

[0114] Therefore, according to Modification 1, the part of the vessel closest to the target quay and the distance to the quay can be determined, which helps to support safer and smoother ship operation.

[0115] (Modification 2) Figure 20 shows the indicators and confidence levels included in the confidence information related to the modified example. Figure 21 is an overhead view of the target vessel and berthing location, clearly indicating the indicators shown in Figure 20. Here, the measurement point at the berthing location is also referred to as the "target point."

[0116] The index "c3" is an index based on the number of points of the target point, and here it is represented as a linear function with the variable x as the number of points of the target point as an example. The index "c2" is an index based on the standard deviation of the target point, and here it is represented as a linear function with the variable x as the standard deviation of the target point as an example. The index "c1" is an index indicating whether both the front and rear shore walls could be measured by the two lidars 3. Here, as an example, when both can be measured, it is set to "1.0", and when only one can be measured, it is set to "0.0". The index "c0" is an index based on the distance between both ends of the target point (both ends in the direction along the shore contact side straight line L), and it is represented as a linear function with the variable x as the distance between the above-mentioned both ends. Also, the indexes c0 to c3 are calculated so as to be limited within the range of 0 to 1.

[0117] The calculation reliability "c" is the reliability based on each of the above-mentioned indexes c0 to c3. Here, it is the weighted average value of the indexes c0 to c3 using the weight coefficients "w0" to "w3" corresponding to the importance of the indexes c0 to c3. Also, an example of the set values of the weight coefficients w0 to w3 is shown in the figure. Since each of the indexes c0 to c3 is within the range of 0 to 1, the calculation reliability c, which is their weighted average value, is also calculated as a numerical value within the range of 0 to 1.

[0118] The overall reliability "r" is the reliability regarding the detection of the side surface of the shore wall, the side surface detection reliability "q s ", the reliability regarding the detection of the upper surface of the shore wall, the upper surface detection reliability "q u ", and the reliability based on the calculation reliability c. Here, weighted average values of the reliabilities q s , q u , m0, m1, and c are used with the weight coefficients "w qs ", "w qu ", "w c ". Also, an example of the set values of the weight coefficients w s , q u , and c. Also, an example of the set values of the weight coefficients w qs , w qu , w m0 , w m1 , w c is shown in the figure. Note that the information processing device 1, for example, the side surface detection reliability qs This is calculated based on the "Side" item of the confidence information shown in Figure 14, and the top surface detection confidence q u This may be calculated based on the "Top View" item in the confidence information of the same figure. Then, the side detection confidence q s and top surface detection confidence q u All of these values ​​are calculated to be in the range of 0 to 1. Furthermore, if both the forward and aft quays are detected, the side detection confidence q of the forward quay is calculated. s0 , the confidence level of detection on the side of the rear quay q s1 , detection confidence q on the upper surface of the forward quay u0 , detection confidence q on the upper surface of the rear quay u1 It is calculated as follows: Side detection confidence q s and top surface detection confidence q u Since both the calculation confidence level c and the calculation confidence level c are in the range of 0 to 1, the overall confidence level r, which is their weighted average, is also calculated as a value in the range of 0 to 1. Therefore, the closer the overall confidence level r is to 1, the more reliable the calculated docking parameters are, and the closer the overall confidence level r is to 0, the less reliable the calculated docking parameters are.

[0119] The information processing device 1 can suitably adjust the output of the drive source when docking, etc., based on reliability information including the indicators and reliability of the modified model.

[0120] As described above, the controller 13 of the information processing device 1 has at least an acquisition means, a straight line calculation means, and a distance calculation means. The acquisition means acquires measurement data, which is a set of data representing multiple points to be measured by a measuring device installed on the ship. The straight line calculation means calculates a straight line along the berthing location where the ship docks, based on the measurement data. The distance calculation means calculates the distance from the ship to the berthing location based on the distance between the straight line and each of the points to be measured. By performing this process, the controller 13 can accurately calculate the distance from the ship to the berthing location, with the target measuring device as the reference point.

[0121] In the above-described embodiment, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a controller. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic storage medium (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage medium (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0122] The present invention has been described above with reference to the examples, but the present invention is not limited to the above examples. Various modifications to the structure and details of the present invention can be made that can be understood by a person skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents, etc., that has been cited is incorporated herein by reference. [Explanation of symbols]

[0123] 1. Information Processing Device 2 Sensor Groups 3 Riders

Claims

1. An acquisition means that acquires measurement data as point cloud data, which is a collection of data representing multiple points on which distances have been measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, A distance calculation means for calculating the distance between the vessel and the berthing location based on the distance between the aforementioned straight line and each of the aforementioned measurement points, An information processing device having

2. An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, A nearest neighbor search means for searching for the nearest neighbor point to the vessel from the point to be measured, based on the distance between the line and each of the points to be measured. A distance calculation means for calculating the distance between the vessel and the berthing location based on the nearest neighbor point, An information processing device having

3. The measuring device includes a first measuring device and a second measuring device. The straight line calculation means calculates a second straight line connecting a first nearest neighbor point to the vessel, which is searched based on the straight line from the point of measurement of the first measurement data measured by the first measuring device, and a second nearest neighbor point to the vessel, which is searched based on the straight line from the point of measurement of the second measurement data measured by the second measuring device. The distance calculation means calculates the distance based on the second straight line, as described in claim 2.

4. If the second line and the line calculated before the second line are not similar, the line calculation means re-searches for the first nearest neighbor point and the second nearest neighbor point based on the second line, and calculates a third line connecting the re-searched first nearest neighbor point and the second nearest neighbor point. The distance calculation means calculates the distance based on the third straight line, as described in claim 3.

5. The information processing apparatus according to claim 4, wherein the line calculation means repeats the re-search for the first nearest neighbor point and the second nearest neighbor point and the calculation of the line until the calculated line converges.

6. An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, The system includes a distance calculation means that calculates the distance between the vessel and the berthing location based on the distance between the straight line and each of the points to be measured, The line calculation means is an information processing device that calculates the line based on the nearest neighbor points searched from the measured point based on the line calculated by the line calculation means before the current processing time, when an index representing the variation of the measured point relative to the line is greater than or equal to a threshold.

7. An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, The system includes a distance calculation means that calculates the distance between the vessel and the berthing location based on the distance between the straight line and each of the points to be measured, The aforementioned straight line calculation means is an information processing device that calculates the straight line based on the nearest neighbor point searched from the measured point based on a straight line parallel to the orientation of the vessel, when an index representing the variation of the measured point relative to the straight line is greater than or equal to a threshold, and no straight line has been calculated within a predetermined time from the current processing time.

8. An acquisition means for acquiring measurement data as point cloud data, which is a collection of data representing multiple measured points measured by a measuring device installed on a ship, A straight line calculation means that calculates a straight line along the berthing location where the vessel docks, based on the aforementioned measurement data, A distance calculation means for calculating the distance between the vessel and the berthing location based on the distance between the aforementioned straight line and each of the aforementioned measurement points, A means for calculating the shortest distance to the opposite shore, which extracts the minimum length of the perpendicular line drawn from the contour point of the vessel to the straight line, and calculates the shortest distance from the vessel's hull to the docking location. Proximity part identification means for identifying the part of the vessel closest to the berthing location, An information processing device having

9. A control method performed by a computer, Measurement data is acquired as point cloud data, which is a collection of data representing multiple points on which distances have been measured by measuring devices installed on the ship. Based on the aforementioned measurement data, a straight line along the berthing location where the vessel will dock is calculated. Based on the distance between the aforementioned straight line and each of the measurement points, the distance between the vessel and the berthing location is calculated. Control method.

10. Measurement data is acquired as point cloud data, which is a collection of data representing multiple points on which distances have been measured by measuring devices installed on the ship. Based on the aforementioned measurement data, a straight line along the berthing location where the vessel will dock is calculated. A program that causes a computer to perform a process to calculate the distance between the vessel and the berthing location based on the distance between the aforementioned straight line and each of the aforementioned measurement points.

11. A storage medium storing the program described in claim 10.

Images