Information processing device, control method, program, and storage medium

Abstract

The present invention provides an information processing device, etc., capable of suitably generating straight lines along the side of a docking area. [Solution] The information processing device comprises an acquisition means, a cluster generation means, and a line generation means. The acquisition means acquires measurement data, which is a set of data representing multiple measured points at a berthing location measured by a measuring device installed on the ship. The cluster generation means generates one or more clusters by dividing the multiple measured points by clustering. The line generation means generates a line along the berthing location where the ship docks, based on one or more clusters.

Description

[Technical Field]

[0001] This disclosure relates to the handling of vessels when they are docking. [Background technology]

[0002] Technologies for assisting with ship docking (berthing) have been known for some time. For example, Patent Document 1 describes a method for controlling the attitude of a ship in an automatic docking device that performs automatic ship docking, such that light emitted from a lidar is reflected by objects around the docking position and received by the lidar. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-59403 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] For ships, safe and smooth docking at a berthing location is crucial, and the realization of docking support systems for ship handling assistance and automated navigation is particularly desired. Therefore, docking support systems require high-precision calculation of parameters such as distance, speed, and angle relative to the docking location. Furthermore, calculating these parameters requires the accurate generation of straight lines along the sides of the docking location.

[0005] This disclosure was made to solve the above-mentioned problems, and its main objective is to provide an information processing device capable of suitably generating straight lines along the side of a docking location. [Means for solving the problem]

[0006] The invention described in the claim is an information processing device comprising: acquisition means for acquiring measurement data which is a set of data representing a plurality of measured points of a berthing location measured by a measuring device installed on a ship; cluster generation means for generating one or more clusters by dividing the plurality of measured points by clustering; and line generation means for generating a straight line along the berthing location where the ship is berthing, based on the one or more clusters.

[0007] Furthermore, the invention described in the claims is a control method performed by a computer, which acquires measurement data which is a set of data representing multiple points to be measured at a berthing location measured by a measuring device installed on a ship, generates one or more clusters by dividing the multiple points to be measured by clustering, and generates a straight line along the berthing location where the ship is berthing based on the one or more clusters.

[0008] Furthermore, the invention described in the claim is a program which causes a computer to perform the following processes: acquire measurement data which is a set of data representing multiple points to be measured at a berthing location measured by a measuring device installed on a ship; generate one or more clusters by dividing the multiple points to be measured by clustering; and generate a straight line along the berthing location where the ship is berthing based on the one or more clusters. [Brief explanation of the drawing]

[0009] [Figure 1A] Block diagram of the flight support system. [Figure 1B] A top view illustrating the field of view of the ship and lidar included in the navigation support system. [Figure 1C] A diagram showing the field of view of a ship and a rider from the rear. [Figure 2] A block diagram showing an example of the hardware configuration of an information processing device. [Figure 3] Functional block diagram related to berthing support processing. [Figure 4A] A diagram showing a rider capturing the quay as it docks. [Figure 4B]Perspective view of the quay wall showing the measured points and the nearest points for each vertical line at the quay wall approach location measured by the rider. [Figure 5A] Diagram showing an example of the data structure of reliability information. [Figure 5B] Diagram showing an example of the indicators and reliability included in the reliability information. [Figure 5C] Overhead view of the target ship and the approach location showing the indicators shown in FIG. 5B. [Figure 6A] Diagram showing an example of a hull coordinate system based on the hull of the target ship. [Figure 6B] Perspective view of the structure showing the normal vector. [Figure 7A] Diagram showing the situation of approaching the quay wall where a pile fender or the like is installed on the side of the quay wall. [Figure 7B] Perspective view showing the nearest points corresponding to the approach location in FIG. 7A. [Figure 7C] Diagram showing an example of the straight line Lx generated according to the nearest points in FIG. 7B. [Figure 7D] Diagram showing an example of the straight line Lx generated according to the nearest points in FIG. 7B. [Figure 7E] Diagram showing an example of the case where the opposite shore distance is calculated according to the straight line Lx in FIG. 7D. [Figure 8A] Diagram showing an example of the center of gravity position GP, the reference position RP, and the distance Dg. [Figure 8B] Diagram for explaining the parameters related to the calculation of the grouping threshold. [Figure 8C] Diagram showing an example of the process related to clustering. [Figure 8D] Diagram showing an example of the process related to clustering. [Figure 9] Diagram showing an example of the case where clustering is performed on a plurality of measured points included in the approach location edge point group in FIG. 8A. [Figure 10] Diagram showing an example of the distances Dg1 and Dg2 calculated when clustering is performed as shown in FIG. 9. [Figure 11A] Diagram showing an example of the straight line Lc1 generated as the approach side straight line Ls by the first generation method. [Figure 11B] This figure shows an example of generating a straight line Ls on the quay side where pie-shaped fenders or similar are installed on the side of the quay. [Figure 12A] A diagram illustrating an example of the clustering process. [Figure 12B] A diagram illustrating an example of the clustering process. [Figure 12C] Figure 12B shows an example of a straight line generated when clustering is performed. [Figure 12D] A diagram illustrating the variance for the linear Lck. [Figure 12E] This figure shows an example of a straight line Ls used on the shore side instead of a straight line Lck. [Figure 13] A diagram showing an example of an approximate straight line that roughly represents the edge portion at a docking location. [Figure 14] Figure 13 shows an example of clustering performed using an approximate straight line on multiple measurement points included in the docking location edge point cloud. [Figure 15] Figure 14 shows an example of the distances Dh1 and Dh2 calculated when clustering is performed. [Figure 16] This figure shows an example of a straight line Ld1 generated as a straight line Ls on the shoreline by the second generation method. [Figure 17A] This figure shows an example of an approximate straight line generated when the lateral portion of the fender is also acquired as a measurement point. [Figure 17B] Figure 17A shows an example of clustering performed using the approximate line. [Figure 17C] Figure 17B shows the distance from the reference position to the centroid position for each cluster. [Figure 17D] This figure shows an example of the shoreline straight line Ls generated according to Figures 17B and 17C. [Figure 17E] This diagram shows an example of a case where a straight line of the shoreline is generated corresponding to each of the quay walls in multiple directions. [Figure 18A] A flowchart illustrating the overview of the docking support process. [Figure 18B]A flowchart showing an example of the process related to the first method for generating a straight line along the shoreline. [Figure 18C] A flowchart showing an example of the process related to the second method for generating the straight line along the shoreline. [Modes for carrying out the invention]

[0010] In one preferred embodiment of the present invention, the information processing device includes: acquisition means for acquiring measurement data which is a set of data representing multiple points to be measured at a berthing location measured by a measuring device installed on a ship; cluster generation means for generating one or more clusters by dividing the multiple points to be measured by clustering; and line generation means for generating a straight line along the berthing location where the ship is berthing, based on the one or more clusters.

[0011] The above-described information processing device includes an acquisition means, a cluster generation means, and a straight line generation means. The acquisition means acquires measurement data, which is a set of data representing multiple measured points at a berthing location measured by a measuring device installed on the ship. The cluster generation means generates one or more clusters by dividing the multiple measured points by clustering. The straight line generation means generates a straight line along the berthing location where the ship docks, based on the one or more clusters. This makes it possible to suitably generate a straight line along the side of the berthing location.

[0012] In one embodiment of the information processing device described above, the cluster is the cluster that is closest to the ship.

[0013] In one embodiment of the information processing device described above, the cluster generation means performs clustering based on the resolution of the measuring device and the average distance between the reference position on the ship and the plurality of points to be measured.

[0014] In one embodiment of the above-described information processing device, the cluster generation means performs clustering based on the distance between one or more approximate straight lines that approximately represent the edge portion at the docking location and the plurality of points to be measured.

[0015] In one embodiment of the information processing device described above, the line generation means calculates a plurality of centroid positions corresponding to each of the plurality of clusters, and generates the line based on the cluster in which the distance between the plurality of centroid positions and the reference position on the ship is smallest.

[0016] In one embodiment of the above-described information processing device, the line generation means estimates that there are multiple quay walls at the berthing location when the difference in the direction vectors of the multiple approximate lines is large and the number of data belonging to the multiple clusters is greater than or equal to a predetermined value, and generates the line in multiple directions corresponding to each of the multiple quay walls.

[0017] In one embodiment of the above-described information processing device, the acquisition means acquires data as the measurement data obtained by removing water surface reflection data generated by the reflection of light emitted by the measuring device from the water surface, or by reducing the number of data points through downsampling, from the data generated by the measuring device.

[0018] In another embodiment of the present invention, a control method executed by a computer acquires measurement data, which is a set of data representing multiple points to be measured at a berthing location measured by a measuring device installed on the ship; generates one or more clusters by dividing the multiple points to be measured by clustering; and generates a straight line along the berthing location where the ship docks, based on the cluster that has the shortest distance from the ship. This makes it possible to suitably generate a straight line along the side of the berthing location.

[0019] In yet another embodiment of the present invention, the program acquires measurement data, which is a set of data representing multiple points to be measured at a berthing location measured by a measuring device installed on a ship; generates one or more clusters by dividing the multiple points to be measured by clustering; and causes a computer to execute a process to generate a straight line along the berthing location where the ship docks, based on the cluster that has the shortest distance from the ship. By executing this program on a computer, the above information processing device can be realized. This program can be stored on a storage medium and used. [Examples]

[0020] Preferred embodiments of the present invention will be described below with reference to the drawings.

[0021] [Overview of the driver assistance system] Figures 1A to 1C show the schematic configuration of the navigation support system according to this embodiment. Specifically, Figure 1A shows a block diagram of the navigation support system, Figure 1B is a top view illustrating the field of view (also called the "measurement range" or "distance-measurable range") 90 of the vessel and the lidar 3 described later, which are included in the navigation support system, and Figure 1C is a rear view showing the field of view 90 of the vessel and the lidar 3. The navigation support system comprises an information processing device 1 that moves together with the vessel, which is a moving object, and a group of sensors 2 mounted on the vessel. Hereafter, the vessel on which the navigation support system is installed will also be called the "target vessel".

[0022] The information processing device 1 is electrically connected to the sensor group 2 and provides operational support for the target vessel based on the outputs of the various sensors included in the sensor group 2. Operational support includes berthing support such as automatic docking. Here, "berthing" includes not only docking the target vessel at a quay but also docking it at a structure such as a pier. Furthermore, hereafter, "berthing location" refers to the general term for structures such as quays and piers that are the target of berthing. The information processing device 1 may be a navigation device installed on the vessel or an electronic control device built into the vessel.

[0023] Sensor group 2 includes various external and internal sensors installed on the ship. In this embodiment, sensor group 2 includes, for example, a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3.

[0024] LIDA 3 is an external sensor that discretely measures the distance to an object in the outside world by emitting a pulsed laser within a predetermined angular range in the horizontal direction (see Figure 1B) and a predetermined angular range in the vertical direction (see Figure 1C), and generates three-dimensional point cloud data indicating the position of the object. In the examples in Figures 1B and 1C, LIDA 3 is provided on the ship, with one LIDA pointed towards the port side and another towards the starboard side. Note that the arrangement of LIDA 3 is not limited to the examples in Figures 1B and 1C. For example, the target ship may have multiple LIDA 3 (for example, LIDAs provided at the front and rear of the target ship) that measure in the same lateral direction so that multiple measurement data of the berthing location can be obtained simultaneously when docking. Also, the number of LIDA 3 installed on the target ship is not limited to two; it may be one or three or more.

[0025] LIDA 3 comprises an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data based on the received signal output by the light receiving unit. The data measured for each direction of laser light irradiation (scanning position) is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light specified based on the received signal described above. Hereafter, a point measured by irradiation with laser light within the measurement range of LIDA 3, or the measurement data thereof, will also be referred to as the "measured point".

[0026] Here, point cloud data can be considered as an image (frame) where each measurement direction is represented by a pixel, and the measured distance and reflectance value for each measurement direction are represented by the pixel value. In this case, the direction of laser beam emission (i.e., measurement direction) differs in elevation and depression angles in the vertical arrangement of pixels, and the direction of laser beam emission differs in horizontal angles in the horizontal arrangement of pixels. Hereafter, when the point cloud data is considered as an image, the measured points corresponding to the rows of pixels (i.e., vertical columns) whose horizontal index positions coincide will also be called "vertical lines." Furthermore, when the point cloud data is considered as an image, the horizontal index will be called the "horizontal number," and the vertical index will be called the "vertical number."

[0027] Furthermore, LIDA 3 is not limited to the scanning type LIDA described above, but may also be a flash type LIDA that generates 3D data by diffusing laser light into the field of view of a 2D array sensor. LIDA 3 is an example of a "measuring device" in the present invention.

[0028] [Configuration of the information processing device] Figure 2 is a block diagram showing an example of the hardware configuration of an information processing device. The information processing device 1 mainly consists of an interface 11, a memory 12, and a controller 13. Each of these elements is interconnected via a bus line.

[0029] Interface 11 performs interface operations related to the exchange of data between the information processing device 1 and external devices. In this embodiment, interface 11 acquires output data from each sensor in the sensor group 2 and supplies it to the controller 13. Interface 11 also supplies signals related to the control of the target vessel, generated by the controller 13, to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel includes a drive source such as an engine or electric motor, a propeller that generates thrust in the direction of travel based on the driving force of the drive source, a thruster that generates thrust in the lateral direction based on the driving force of the drive source, and a rudder, etc., which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation such as automatic docking, interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, interface 11 supplies control signals generated by the controller 13 to the electronic control device. Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices by cables, etc. Furthermore, interface 11 may perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0030] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0031] Furthermore, memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, memory 12 may store map data including information about the location of the docking place. In another example, memory 12 stores information about the downsampling size when downsampling is performed on the point cloud data obtained when the lidar 3 performs one cycle of scanning.

[0032] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to the operation support of the target vessel by executing programs stored in memory 12, etc.

[0033] Furthermore, the controller 13 functionally includes a docking location detection unit 15 and a docking parameter calculation unit 16. The docking location detection unit 15 performs processing related to the detection of docking locations based on the point cloud data output by the lidar 3. The docking parameter calculation unit 16 calculates the parameters necessary for docking at the docking location (also called "docking parameters"). Here, the docking parameters include the distance between the target vessel and the docking location (distance to the opposite shore), the approach angle of the target vessel to the docking location, and the speed at which the target vessel approaches the docking location (docking speed). The docking parameter calculation unit 16 also calculates information representing the reliability of docking at the docking location (also called "reliability information") based on the processing results of the docking location detection unit 15 and the docking parameters. The controller 13 functions as an "acquisition means," a "cluster generation means," a "straight line generation means," and a computer that executes programs.

[0034] Furthermore, the processing performed by the controller 13 is not limited to being implemented by software through a program, but may also be implemented by a combination of hardware, firmware, and software. Additionally, the processing performed by the controller 13 may be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program executed by the controller 13 in this embodiment may be implemented using this integrated circuit.

[0035] [Overview of docking support procedures] Next, an overview of the docking support process performed by the information processing device 1 will be described. Based on the point cloud data of the lidar 3 measured in the direction in which the docking location exists, the information processing device 1 generates a straight line along the side of the docking location (also called the "docking side line Ls"). In other words, the docking side line Ls is a straight line along the side of the quay wall of the docking location. Then, based on the docking side line Ls, the information processing device 1 calculates docking parameters such as the distance to the opposite shore.

[0036] Figure 3 is a functional block diagram of the docking location detection unit 15 and the docking parameter calculation unit 16 related to docking support processing. Functionally, the docking location detection unit 15 includes a normal vector calculation block 20, a field of view / detection surface identification block 21, a normal number identification block 22, a mean / variance calculation block 23, and a docking status determination block 24. Functionally, the docking parameter calculation unit 16 includes a nearest point search block 26, a straight line generation block 27, a distance calculation block 28, an entry angle calculation block 29, a docking speed calculation block 30, and a reliability information generation block 40.

[0037] The normal vector calculation block 20 calculates the normal vector of the surface formed by the berthing location (also called the "berthing surface") based on the point cloud data generated by the lidar 3 in the direction in which the berthing location exists. In this case, the normal vector calculation block 20 calculates the normal vector based on, for example, the point cloud data generated by the lidar 3, which includes the berthing side of the target vessel in its measurement range. Information regarding the measurement range of the lidar 3 and the direction of the berthing location may be pre-registered in, for example, memory 12.

[0038] In this case, the normal vector calculation block 20 preferably performs downsampling of the point cloud data and removal of data obtained by the reflection of laser light from the water surface (also called "water surface reflection data").

[0039] In this case, first, the normal vector calculation block 20 removes data located below the water surface position from the point cloud data generated by the lidar 3, treating it as water surface reflection data (i.e., false detection data). The normal vector calculation block 20 estimates the water surface position based, for example, on the average value in the height direction of the point cloud data generated by the lidar 3 when there are no objects other than the water surface in the surrounding area. Then, the normal vector calculation block 20 performs downsampling on the point cloud data after the water surface reflection data has been removed, which is a process that integrates the measured points for each grid space of a predetermined size. Finally, the normal vector calculation block 20 calculates a normal vector for each measured point indicated by the downsampled point cloud data using multiple surrounding measured points. Note that downsampling may be performed before removing the data reflected from the water surface.

[0040] The field of view / detection surface identification block 21 identifies the surface of the docking location that is within the field of view angle of the Lidar 3 (also called the "inner surface of the field of view") and the surface of the docking location detected based on the normal vector calculated by the normal vector calculation block 20 (also called the "detection surface"). In this case, the field of view / detection surface identification block 21 determines whether the upper surface and / or the side surface of the docking location are included in the inner surface of the field of view and the detection surface.

[0041] The normal vector number identification block 22 extracts the vertical normal vector and the normal vector perpendicular to it (i.e., the horizontal direction) from the normal vector calculated by the normal vector calculation block 20, and calculates the number of vertical normal vectors and the number of horizontal normal vectors. Here, the normal vector number identification block 22 considers the vertical normal vector to represent the normal to the measured point on the upper surface of the docking location, and the horizontal normal vector to represent the normal to the measured point on the side of the docking location, and calculates the number of each as an indicator of the reliability of the docking location.

[0042] The mean and variance calculation block 23 extracts the vertical normal vector and the normal vector perpendicular to it (i.e., horizontal) from the normal vectors calculated by the normal vector calculation block 20, and calculates the mean and variance of the vertical normal vector and the mean and variance of the horizontal normal vector.

[0043] The docking status determination block 24 obtains the processing results of the field of view / detection surface identification block 21, the normal number identification block 22, and the mean / variance calculation block 23, which are identified or calculated based on the same point cloud data, as determination results representing the detection status of the docking location at the time the point cloud data was generated. The docking status determination block 24 then supplies the processing results of the field of view / detection surface identification block 21, the normal number identification block 22, and the mean / variance calculation block 23 as determination results of the detection status of the docking location to the docking parameter calculation unit 16.

[0044] The nearest neighbor search block 26 searches for the nearest neighbor point closest to the target vessel for each vertical line from the measured points that make up the point cloud data. For example, as shown in Figure 4A, when the lidar 3 has captured a quay where the vessel is docking, the nearest neighbor point, which is the point closest to the vessel, will be the edge between the top and side surfaces of the quay. Through this process, the nearest neighbor search block 26 can obtain nearest neighbor points such as those shown in Figure 4B. Figure 4B is a perspective view of a quay, clearly showing the measured points of the quay, which is the docking location, and the nearest neighbor points for each vertical line, as measured by the lidar.

[0045] The straight line generation block 27 generates a docking side line Ls, which is a straight line along the side of the docking location, based on the nearest neighbor determined by the nearest neighbor search block 26.

[0046] The opposite shore distance calculation block 28 calculates the opposite shore distance, which corresponds to the shortest distance between the target vessel and the berthing location, based on the berthing side line Ls generated by the line generation block 27. Here, if there are multiple lidars 3 that can measure the berthing location, the opposite shore distance calculation block 28 generates a berthing side line Ls by combining the point cloud data of multiple lidars 3 and calculates the opposite shore distance as the shortest distance to each lidar 3. Alternatively, the opposite shore distance calculation block 28 may generate a berthing side line Ls for each point cloud data of lidar 3 and calculate the opposite shore distance as the shortest distance between each berthing side line Ls and each lidar 3. Furthermore, the opposite shore distance calculation block 28 may calculate the opposite shore distance as the shortest distance from a reference point such as the vessel's center position to the berthing side line Ls. Furthermore, instead of considering the shortest distance for each rider 3 as the distance to the opposite shore in the opposite shore calculation block 28, the distance to the opposite shore may be determined as the shorter of the shortest distances for each rider 3, or the average of these shortest distances may be determined as the distance to the opposite shore.

[0047] The approach angle calculation block 29 calculates the approach angle of the target vessel to the berthing location based on the berthing side line Ls generated by the line generation block 27. Specifically, the approach angle calculation block 29 calculates the approach angle using the function "atan2", which finds the arctangent from two arguments that define the tangent. More specifically, the approach angle calculation block 29 calculates the approach angle from the direction vector of the berthing side line Ls by calculating the function atan2.

[0048] The berthing speed calculation block 30 calculates the berthing speed, which is the speed at which the target vessel approaches the berthing location, based on the distance to the opposite shore calculated by the distance to the opposite shore calculation block 28. For example, the berthing speed calculation block 30 calculates the berthing speed as the change in the distance to the opposite shore (shortest distance) over time.

[0049] The reliability information generation block 40 generates reliability information based on the processing results of the docking status determination block 24, the straight line generation block 27, the opposite bank distance calculation block 28, and the approach angle calculation block 29.

[0050] Here, we will explain the first specific example of generating confidence information. The confidence information generation block 40 generates flags for each element such as the field of view at the time of detection of the docking location, the surface detection of the docking location, the number of normal vectors, and the variance, and generates confidence information from the vector of the generated flags. Hereafter, a flag of "1" will indicate that the confidence level of the corresponding element is high, and a flag of "0" will indicate that the confidence level of the corresponding element is low.

[0051] Figure 5 shows an example of the data structure of confidence information generated by the confidence information generation block 40. As shown in Figure 5, the confidence information has the following items: "Top view", "Side view", "Linear view", "Distance", and "Angle". The item "Top view" has the sub-items "Field of view angle", "Detection", "Normal number", and "Variance", and the item "Side view" has the sub-items "Field of view angle", "Detection", "Normal number", and "Variance". The item "Linear view" has the sub-item "Absolute value", the item "Distance" has the sub-items "Amount of change" and "Rate of change", and the item "Angle" has the sub-item "Amount of change".

[0052] Here, the reliability information generation block 40 registers a flag in the "Field of View Angle" sub-item of the "Top Surface" item, which is set to "1" if the top surface of the docking location is within the field of view, and to "0" if the top surface is outside the field of view. The reliability information generation block 40 also registers a flag in the "Detection" sub-item of the "Top Surface" item, which is set to "1" if the top surface of the docking location is a detection surface, and to "0" if the top surface is not a detection surface. The reliability information generation block 40 also registers a flag in the "Number of Normals" sub-item of the "Top Surface" item, which is set to "1" if the number of normal vectors to the top surface of the docking location is equal to or greater than a predetermined threshold (e.g., 10), and to "0" if the number is less than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "Variance" sub-item of the "Upper Surface" item, where "1" is set if the variance of the x, y, and z components of the normal vector to the upper surface of the docking location is all less than a predetermined threshold (e.g., 1.0), and "0" is set if any of the variances is equal to or greater than the threshold. The reliability information generation block 40 also registers flags in each sub-item of the "Side" item, which are determined by the same rules as for each sub-item of the "Upper Surface" item.

[0053] Furthermore, the reliability information generation block 40 registers a flag in the "Absolute Value" sub-item of the "Line" item that represents the reliability of the berthing side line Ls generated by either the first generation method or the second generation method described later. For example, if there are riders 3 at the front and rear of the target vessel, the reliability information generation block 40 registers a flag that is "1" if the difference for each component between the direction vector of the berthing side line Ls generated by either the first generation method or the second generation method and the direction vector of the line connecting the nearest points of the riders 3 at the front and rear of the target vessel is all less than a threshold, and "0" if any of the differences are greater than or equal to the threshold.

[0054] Furthermore, the reliability information generation block 40 registers a flag in the "Change Amount" sub-item of the "Distance" item, which is set to "1" if the change amount of the distance to the opposite bank calculated by the opposite bank distance calculation block 28 from one time point in the previous period is less than a predetermined threshold (e.g., 1.0 m), and to "0" if the change amount is equal to or greater than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "Rate of Change" sub-item of the "Distance" item, which is set to "1" if the rate of change of the distance to the opposite bank calculated by the opposite bank distance calculation block 28 from one time point in the previous period is less than a predetermined threshold (e.g., ±10%), and to "0" if the rate of change is equal to or greater than the threshold. Furthermore, the reliability information generation block 40 registers a flag in the "Change Amount" sub-item of the "Angle" item, which is set to "1" if the change amount of the approach angle calculated by the approach angle calculation block 29 from one time point in the previous period is less than a predetermined threshold (e.g., 1.0 degrees), and to "0" if the change amount is equal to or greater than the threshold.

[0055] The thresholds mentioned above are set to conforming values ​​pre-stored in, for example, memory 12. Furthermore, confidence information may be generated for each Writer 3.

[0056] Next, we will explain a second specific example related to the generation of confidence information. Figure 5B is a diagram showing an example of the indicators and confidence levels included in the confidence information. Figure 5C is an overhead view of the target vessel and berthing location, clearly showing the indicators shown in Figure 5B. Here, the measurement points near the edge of the berthing location are also referred to as "target points."

[0057] The index "c3" is an index based on the score of the target point, and here it is expressed as a linear function with the variable x as the score of the target point as an example. The index "c2" is an index based on the standard deviation of the target point, and here it is expressed as a linear function with the variable x as the standard deviation of the target point as an example. The index "c1" is an index indicating whether both the front and rear quay walls could be measured by two lidars 3. Here, as an example, when both could be measured, it is set to "1.0", and when only one could be measured, it is set to "0.0". The index "c0" is an index based on the distance between both ends of the target point (both ends in the direction along the shore side straight line L), and it is expressed as a linear function with the variable x as the distance between the above-mentioned both ends. Also, the indices c0 to c3 are calculated so as to be restricted to the range of 0 to 1.

[0058] The calculation reliability "c" is a reliability based on each of the above-mentioned indices c0 to c3. Here, it is the weighted average value of the indices c0 to c3 using the weight coefficients "w0" to "w3" according to the importance of the indices c0 to c3. Also, an example of the set values of the weight coefficients w0 to w3 is shown. Since each of the indices c0 to c3 is in the range of 0 to 1, the calculation reliability c, which is their weighted average value, is also calculated as a numerical value in the range of 0 to 1.

[0059] The overall reliability "r" is the side detection reliability "q s ", which is the reliability regarding the detection of the side surface of the quay wall, and the top surface detection reliability "q u ", which is the reliability regarding the detection of the top surface of the quay wall, and the calculation reliability c. Here, for each reliability q s 、q u 、m0, m1, and c, it is the weighted average value of the reliability q qs 、q qu 、c using the weight coefficients "w c ", "w s ", "w u " according to their importance. Also, an example of the set values of the weight coefficients w qs 、w qu 、w m0 、w m1 、w c is shown. Note that the information processing device 1, for example, the side detection reliability qs This is calculated based on the "Side" item of the reliability information shown in Figure 5A, and the top surface detection reliability q u This may be calculated based on the "Top View" item in the confidence information of the same figure. Then, the side detection confidence q s and top surface detection confidence q u All of these values ​​are calculated to be in the range of 0 to 1. Furthermore, if both the forward and aft quays are detected, the side detection confidence q of the forward quay is calculated. s0 , the confidence level of detection on the side of the rear quay q s1 , detection confidence q on the upper surface of the forward quay u0 , detection confidence q on the upper surface of the rear quay u1 It is calculated as follows: Side detection confidence q s and top surface detection confidence q u Since both the calculation confidence level c and the calculation confidence level c are in the range of 0 to 1, the overall confidence level r, which is their weighted average, is also calculated as a value in the range of 0 to 1. Therefore, the closer the overall confidence level r is to 1, the more reliable the calculated docking parameters are, and the closer the overall confidence level r is to 0, the less reliable the calculated docking parameters are.

[0060] In other words, the confidence information generated by the second specific example includes each indicator and confidence level calculated using the method described above.

[0061] Next, specific examples of the processing of the normal vector calculation block 20, the normal number determination block 22, and the mean / variance calculation block 23 will be explained with reference to Figures 6A and 6B.

[0062] Figure 6A shows an example of a hull coordinate system based on the hull of the target vessel. As shown in Figure 6A, the forward direction of the target vessel is the "x" coordinate, the side direction of the target vessel is the "y" coordinate, and the height direction of the target vessel is the "z" coordinate. The measurement data from the coordinate system based on LIDA 3, measured by LIDA 3, is converted to the hull coordinate system shown in Figure 6A. The process of converting point cloud data from a coordinate system based on a LIDA installed on a moving object to the coordinate system of the moving object is disclosed, for example, in International Publication WO2019 / 188745.

[0063] Figure 6B is a perspective view of the quay where the vessel docks, clearly showing the measurement points (represented by the LIDA 3) and the normal vectors calculated based on those measurement points. In Figure 6B, the measurement points are indicated by circles, and the normal vectors are indicated by arrows. This example shows that both the top and side surfaces of the quay were measured by the LIDA 3.

[0064] As shown in Figure 6B, the normal vector calculation block 20 calculates normal vectors for the measurement points on the side and top surfaces of the quay wall. Since normal vectors are vectors perpendicular to the target plane or curved surface, they are calculated using multiple measurement points that can form a surface. Therefore, a grid with predetermined lengths for length and width, or a circle with a predetermined radius, is set up, and the calculation is performed using the measurement points located within it. In this case, the normal vector calculation block 20 may calculate a normal vector for each measurement point, or it may calculate normal vectors at predetermined intervals. The normal number identification block 22 then determines that normal vectors whose z component is greater than a predetermined threshold are normal vectors oriented in the vertical direction. It is assumed that the normal vectors are unit vectors. Furthermore, normal vectors whose z component is less than a predetermined threshold are determined to be normal vectors oriented in the horizontal direction. The normal number identification block 22 then identifies the number of vertical normal vectors (5 in this case) and the number of horizontal normal vectors (4 in this case). Furthermore, the mean / variance calculation block 23 calculates the mean and variance of the vertical normal vector and the mean and variance of the horizontal normal vector. Note that for edge areas, the measurement direction will be diagonal because the surrounding measurement points are on the top or side.

[0065] [Details on how to generate the straight line along the shoreline] Figure 7A shows a situation where a ship is docking at a location where pie-shaped fenders or similar are installed on the side of a quay. In this case, the distance to the opposite shore needs to be calculated as the distance to the fender. Note that the fenders will also be referred to as the quay. As mentioned above, the docking side line Ls can be found by considering the set of nearest neighbors for each vertical line, but because there are parts with and without fenders, an inaccurate line Lx may be generated, as shown in Figures 7B to 7D. If an inaccurate line Lx like the one in Figure 7D is generated, the distance to the opposite shore, which is the shortest distance to that line Lx, will also be inaccurate (see Figure 7E). Therefore, it is necessary to extract only the nearest neighbors of the fender sections and generate the docking side line Ls. Accordingly, the methods for generating the docking side line Ls (first generation method and second generation method) will be explained.

[0066] (1st generation method) The first generation method is to determine the quay side line Ls by performing clustering processing based on the distance between points. Figure 8A shows a portion of the quay where lidar 3 detected areas with and without fenders.

[0067] The berthing parameter calculation unit 16 calculates the centroid position GP in the point cloud data (hereinafter also referred to as the "berthing location edge point cloud") which includes multiple measured points that constitute the edge of the berthing location (more specifically, the boundary of the top surface and side surface). The berthing parameter calculation unit 16 also calculates the distance Dg between the reference position RP on the target vessel and the centroid position GP. That is, the distance Dg can be treated as the average distance between the reference position RP and the multiple measured points included in the berthing location edge point cloud. The reference position RP may be a predetermined position set on the target vessel, or it may be the position on the target vessel where the lidar 3 is installed. Furthermore, the centroid position GP, ​​reference position RP, and distance Dg can be represented, for example, as shown in Figure 8A.

[0068] The docking parameter calculation unit 16 calculates the grouping threshold Gth by applying the coefficient k, the resolution θr of the lidar 3, and the distance Dg to the following formula (1). The resolution θr is a value expressed as the angle (in radians) between two adjacent measurement points. The grouping threshold Gth calculated by the following formula (1) is calculated as a value in meters and is a value used in the clustering described later.

[0069]

number

[0070] Here, the value obtained by multiplying the resolution θr and the distance Dg is approximately equal to the average distance between two adjacent measurement points, as shown in Figure 8B. Therefore, for example, if the coefficient k in the above formula (1) is set to 5, the grouping threshold Gth is calculated as approximately 5 times the average distance between two adjacent measurement points. Figure 8B is a diagram illustrating the parameters involved in calculating the grouping threshold.

[0071] Furthermore, if, for example, the rider 3 measures an object on the water surface in front of the docking location, the position of the center of gravity GP may shift towards the vessel, potentially preventing the calculation of an appropriate grouping threshold Gth. This is because if the distance Dg becomes shorter than it should be, the grouping threshold Gth becomes smaller, and the clustering conditions become stricter. Therefore, in that case, there is a risk that clusters will not be generated. For this reason, according to this embodiment, the docking parameter calculation unit 16 sets the grouping threshold Gth to a predetermined value (e.g., 1) if, for example, the sum of the flags registered in each sub-item other than the sub-item "absolute value" of the confidence information generated in the previous time processing according to the first specific example described above is less than or equal to a predetermined threshold. Alternatively, according to this embodiment, the docking parameter calculation unit 16 sets the grouping threshold Gth to a predetermined value (e.g., 1) if, for example, the value of one or more flags registered in each sub-item other than the sub-item "absolute value" of the confidence information generated in the previous time processing according to the first specific example described above is less than or equal to a predetermined threshold. Alternatively, the docking parameter calculation unit 16 sets the grouping threshold Gth to a predetermined value (e.g., 1) if, for example, the overall confidence level r included in the confidence information generated in the processing of the previous time according to the second specific example described above is 0.3 or less. Therefore, if the confidence level of the previous time is high, it is determined that the current distance Dg is also highly reliable and an appropriate grouping threshold Gth is generated according to the distance Dg. If the confidence level of the previous time is low, it is determined that the current distance Dg is not highly reliable and the grouping threshold Gth is set to a larger fixed value. This ensures both the prevention of the risk of clusters not being generated and the generation of appropriate clusters.

[0072] Furthermore, if the confidence level of the previous time is low, instead of setting the grouping threshold Gth to a predetermined value, the coefficient k used in formula (1) may be increased to a larger value so that a larger value can be calculated as the grouping threshold Gth.

[0073] The docking parameter calculation unit 16 uses a grouping threshold Gth to perform clustering on multiple measurement points included in the docking location edge point cloud.

[0074] Specifically, for example, the berthing parameter calculation unit 16 classifies two adjacent measurement points into the same cluster if the distance between them is less than or equal to the grouping threshold Gth (see Figures 8C and 8D). Also, for example, the berthing parameter calculation unit 16 classifies two adjacent measurement points into different clusters if the distance between them is greater than the grouping threshold Gth (see Figures 8C and 8D). Figures 8C and 8D show examples of the clustering process. Through this process, for example, multiple measurement points included in the berthing location edge point cloud shown in Figure 8A are classified into clusters C1 and C2 as shown in Figure 9. Furthermore, through the process described above, for example, measurement points in the part of the berthing location where fenders are installed and measurement points in the part of the berthing location where fenders are not installed can be classified into different clusters. Figure 9 shows an example of when clustering is performed on multiple measurement points included in the berthing location edge point cloud of Figure 8A.

[0075] The docking parameter calculation unit 16 calculates the centroid positions GP1, GP2, ..., GPn in the point cloud data containing multiple measurement points for each of the n clusters obtained by clustering. The docking parameter calculation unit 16 also calculates the distance between the centroid positions GP1, GP2, ..., GPn and the reference position RP. Through this process, for example, as shown in Figure 10, the distance Dg1 between the centroid position GP1 of cluster C1 and the reference position RP is calculated. Similarly, through the same process, for example, as shown in Figure 10, the distance Dg2 (>Dg1) between the centroid position GP2 of cluster C2 and the reference position RP is calculated. Figure 10 shows examples of the distances Dg1 and Dg2 calculated when clustering is performed as shown in Figure 9.

[0076] The docking parameter calculation unit 16 detects the smallest distance Dgk (1 ≤ k ≤ n) from among the distances Dg1, Dg2, ..., Dgn calculated for each of the n clusters. The docking parameter calculation unit 16 also selects multiple measurement points included in cluster Ck, for which distance Dgk has been calculated, as point cloud data TDA for generating the docking side line Ls.

[0077] The docking parameter calculation unit 16 calculates the vector of the first principal component (the vector of the principal component axis with the largest eigenvalue) by performing principal component analysis on multiple measurement points included in the point cloud data TDA. Then, based on the aforementioned vector of the first principal component and the centroid position GPk of cluster Ck, the docking parameter calculation unit 16 generates a straight line Lck corresponding to multiple measurement points included in cluster Ck, and acquires the generated straight line Lck as the docking side straight line Ls. Through this process, for example, as shown in Figure 11A, a straight line Lc1 corresponding to multiple measurement points included in cluster C1 in Figure 10 can be generated as the docking side straight line Ls. Figure 11A is a diagram showing an example of a straight line Lc1 generated as the docking side straight line Ls by the first generation method.

[0078] Furthermore, if the shoreing parameter calculation unit 16 obtains one cluster through clustering (i.e., if multiple clusters are not obtained), it should treat that one cluster as the cluster with the smallest distance from the reference position RP, and generate the shoreing side line Ls using the multiple measurement points included in that one cluster.

[0079] According to the process described above, the berthing parameter calculation unit 16 can generate one or more clusters by clustering multiple measurement points included in the berthing location edge point cloud. Furthermore, according to the process described above, the berthing parameter calculation unit 16 can generate a straight line along the berthing location where the vessel will berth, based on the cluster with the shortest distance to the vessel. In addition, the berthing parameter calculation unit 16 can generate an appropriate berthing side straight line Ls according to the berthing location, even when the vessel berths at a location where pie-shaped fenders or the like are installed on the side of the quay, as shown in Figure 11B. As a result, the distance to the opposite shore can be accurately calculated by determining the length of the perpendicular to the berthing side straight line Ls. Figure 11B is a diagram showing an example of generating a berthing side straight line Ls at a location where pie-shaped fenders or the like are installed on the side of the quay.

[0080] However, according to the processing described above, for example, if LIDA 3 also acquires measurement points on the side of the fender, it may be possible to generate an inappropriate straight line as the berthing side straight line Ls due to the inability to properly cluster multiple measurement points included in the berthing location edge point cloud. As shown in Figure 12A, if measurement points are also acquired from the side of the fender, and the point interval in that area is shorter than the grouping threshold, it will not be divided into two clusters, but will become a single cluster as shown in Figure 12B. As a result, the generated straight line will be inaccurate as shown in Figure 12C. Therefore, in the first generation method, if the above situation occurs, the newly generated straight line is discarded, and the previously generated straight line is acquired as the berthing side straight line Ls. A specific example of this processing will be explained below.

[0081] The docking parameter calculation unit 16 calculates the variance or standard deviation for multiple measurement points included in the straight line Lck generated as described above. The variance for the straight line Lck is calculated as shown in Figure 12D, for example. If the variance or standard deviation calculated as described above is below a predetermined threshold, the docking parameter calculation unit 16 acquires the newly generated straight line Lck as the docking side straight line Ls and stores information indicating the straight line Lck in the memory 12. If the variance or standard deviation calculated as described above is greater than the predetermined threshold, the docking parameter calculation unit 16 discards the newly generated straight line Lck, reads the docking side straight line Ls stored in the memory 12, and uses it as the current docking side straight line Ls (see Figure 12E).

[0082] (Second generation method) The second generation method involves performing a line search using the Hough transform and then performing clustering based on the generated approximate line to obtain the shoreline line Ls. Figure 13 shows an example of an approximate straight line that roughly represents the edge portion at the docking site.

[0083] The docking parameter calculation unit 16 calculates an approximate straight line La that approximately represents the edge portion at the docking location by performing a line search using Hough transform on multiple measurement points included in the docking location edge point cloud. Through this process, for example, approximate straight lines La1 and La2 as shown in Figure 13 can be calculated.

[0084] The docking parameter calculation unit 16 performs clustering on multiple measured points included in the docking location edge point cloud based on the distance between the approximate straight line La and the multiple measured points. Specifically, the docking parameter calculation unit 16 classifies each measured point whose distance from the point to the approximate straight line is less than or equal to a predetermined threshold Dt into the same cluster. With this clustering, the multiple measured points included in the docking location edge point cloud are classified to belong to one of the same number of clusters as the number of approximate straight lines La. Furthermore, according to the aforementioned clustering, for example, as shown in Figure 14, each measured point whose distance from the approximate straight line La1 is less than or equal to a predetermined threshold Dt is classified into cluster E1. Furthermore, according to the aforementioned clustering, for example, as shown in Figure 14, each measured point whose distance from the approximate straight line La2 is less than or equal to a predetermined threshold Dt is classified into cluster E2. Figure 14 shows an example of clustering performed on multiple measured points included in the docking location edge point cloud of Figure 13 using an approximate straight line calculated by the Hough transform.

[0085] The docking parameter calculation unit 16 calculates the centroid positions GQ1, GQ2, ..., GQn in the point cloud data containing multiple measurement points for each of the n clusters obtained by clustering using n approximate straight lines La obtained by the Hough transform. The docking parameter calculation unit 16 also calculates the distance between the centroid positions GQ1, GQ2, ..., GQn and the reference position RP. Through this process, for example, as shown in Figure 15, the distance Dh1 between the centroid position GQ1 of cluster E1 and the reference position RP is calculated. Similarly, through the same process, for example, as shown in Figure 15, the distance Dh2 (>Dh1) between the centroid position GQ2 of cluster E2 and the reference position RP is calculated. Figure 15 shows examples of the distances Dh1 and Dh2 calculated when clustering is performed as shown in Figure 14.

[0086] The docking parameter calculation unit 16 detects the smallest distance Dhm (1 ≤ m ≤ n) from among the distances Dh1, Dh2, ..., Dhn calculated for each of the n clusters. The docking parameter calculation unit 16 also selects multiple measurement points included in cluster Em, for which the distance Dhm has been calculated, as point cloud data TDB for generating the docking side line Ls.

[0087] The docking parameter calculation unit 16 calculates the vector of the first principal component (the vector of the principal component axis with the largest variance) by performing principal component analysis on multiple measurement points included in the point cloud data TDB. Then, based on the aforementioned vector of the first principal component and the centroid position GQm of cluster Em, the docking parameter calculation unit 16 generates a straight line Ldm corresponding to the multiple measurement points included in cluster Em, and acquires the generated straight line Ldm as the docking side straight line Ls. Through this process, for example, as shown in Figure 16, a straight line Ld1 corresponding to the multiple measurement points included in cluster E1 in Figure 15 can be generated as the docking side straight line Ls. Figure 16 is a diagram showing an example of a straight line Ld1 generated as the docking side straight line Ls by the second generation method. Generally, the straight line obtained by the Hough transform is an approximate straight line, but if the required accuracy of the docking parameter calculation value is not high, the approximate straight line La may be used as the docking side straight line Ls when Dhm is detected.

[0088] Furthermore, if the shoreing parameter calculation unit 16 obtains one cluster through clustering (i.e., if multiple clusters are not obtained), it should treat that one cluster as the cluster with the smallest distance from the reference position RP, and generate the shoreing side line Ls using the multiple measurement points included in that one cluster.

[0089] According to the process described above, for example, if LIDA 3 also acquires the lateral portion of the fender as a measurement point, an approximate straight line is generated that approximately represents the edge portion of the lateral portion of the fender (see Figure 17A). However, according to the process described above, the shoreline straight line Ls can be generated using multiple measurement points included in the cluster with the smallest distance between the centroid position of each cluster obtained by clustering and the reference position RP. As shown in Figure 17B, when three approximate straight lines are generated by line search using the Hough transform, clustering is performed based on the distance from each approximate straight line. Therefore, as shown in Figure 17C, by comparing the distance to the centroid of each cluster, the cluster of the fender portion that protrudes the furthest towards the sea is extracted. As a result, the shoreline straight line Ls is generated as shown in Figure 17D. In other words, according to the second generation method, an appropriate shoreline straight line Ls can be generated even when LIDA 3 also acquires the lateral portion of the fender as a measurement point. Furthermore, according to the second generation method, even if the lidar 3 measures quays in multiple directions, it is possible to generate appropriate shoreline straight lines Ls corresponding to each of the quays in those multiple directions, such as the straight lines L1 and L2 in Figure 17E. If there are multiple approximate straight lines generated by Hough change, and the difference in their direction vectors is large, and the number of points (data) belonging to each cluster is greater than or equal to a predetermined value, it is determined that there are quays in multiple directions, as shown in Figure 17E, and shoreline straight lines Ls for each direction are generated, and the distance to each quay is calculated. In Figure 17E, distance d1 represents the distance from the origin O to the opposite shore from line L1, and distance d2 represents the distance from the origin O to the opposite shore from line L2. Also, in Figure 17C, distances r1, r2, and r3 represent the distance from the reference position RP to the centroid position.

[0090] (Processing flow) Figure 18A is a flowchart illustrating the overview of the docking support process in this embodiment. The information processing device 1 repeatedly executes the processes shown in the flowchart of Figure 18A.

[0091] First, the information processing device 1 acquires point cloud data in the direction of the berthing location (step S11). In this case, the information processing device 1 acquires, for example, point cloud data generated by a lidar 3 that includes the berthing side of the target vessel within its measurement range. The information processing device 1 may also further downsample the acquired point cloud data and remove data reflected from the water surface.

[0092] Next, the docking location detection unit 15 of the information processing device 1 calculates normal vectors based on the point cloud data acquired in step S11 (step S12). Furthermore, in step S12, the docking location detection unit 15 calculates the number of normal vectors and the variance of the normal vectors. Also, based on the processing results of step S12, the docking location detection unit 15 identifies the inner surface of the field of view and the detection surface (step S13).

[0093] Next, the docking parameter calculation unit 16 performs a nearest neighbor search for each vertical line based on the point cloud data acquired in step S11 to determine the nearest neighbor point for each vertical line (step S14).

[0094] Next, the shoreing parameter calculation unit 16 generates the shoreing side line Ls using the nearest neighbor points for each vertical line obtained in step S14 (step S15). In this case, the shoreing parameter calculation unit 16 generates the shoreing side line Ls based on the first generation method or the second generation method described above.

[0095] Here, the process related to the first method for generating the shoreline straight line, which is performed in step S15, will be explained with reference to Figure 18B. Figure 18B is a flowchart showing an example of the process related to the first method for generating the shoreline straight line.

[0096] The docking parameter calculation unit 16 obtains confidence information generated in the previous processing time from the memory 12 (step S31), and then determines whether the confidence included in the confidence information is greater than a predetermined threshold (step S32).

[0097] If the confidence level is below a predetermined threshold (step S32: No), the shoreing parameter calculation unit 16 sets the grouping threshold Gth to a predetermined value (step S33) and then performs the process described in step S36 below. If the confidence level is greater than a predetermined threshold (step S32: Yes), the shoreing parameter calculation unit 16 calculates the distance Dg from the reference position RP to the center of gravity position GP (step S34), calculates the grouping threshold Gth based on the distance Dg (step S35), and then performs the process described in step S36 below.

[0098] The docking parameter calculation unit 16 performs clustering on the docking location edge point cloud using the grouping threshold Gth (step S36).

[0099] Next, the docking parameter calculation unit 16 calculates the distance from the reference position RP to the centroid position of each cluster for the clusters obtained in step S36 (step S37).

[0100] Next, the docking parameter calculation unit 16 obtains the point cloud contained in the cluster with the smallest distance from the reference position RP to the center of gravity position, based on the processing result of step S37 (step S38).

[0101] Next, the docking parameter calculation unit 16 generates a straight line using the point cloud acquired in step S38 (step S39), and then determines whether the variance of the point cloud with respect to the straight line is below a threshold (step S40).

[0102] If the variance is greater than the threshold (step S40: No), the shoreing parameter calculation unit 16 discards the straight line generated in step S39, obtains the shoreing side straight line Ls from memory 112 (step S41), and then continues the processing from step S16 onwards. If the variance is less than or equal to the threshold (step S40: Yes), the shoreing parameter calculation unit 16 obtains the straight line generated in step S39 as the shoreing side straight line Ls (step S42), stores the shoreing side straight line Ls in memory 12 (step S43), and then continues the processing from step S16 onwards.

[0103] Next, the process related to the second method for generating the shoreline straight line, which is performed in step S15, will be explained with reference to Figure 18C. Figure 18C is a flowchart showing an example of the process related to the second method for generating the shoreline straight line.

[0104] The docking parameter calculation unit 16 generates an approximate straight line that approximately represents the edge portion at the docking location by performing a line search using the Hough transform on the docking location edge point cloud (step S51).

[0105] Next, the shoreing parameter calculation unit 16 performs clustering using the approximate lines generated in step S51 (step S52). Specifically, the shoreing parameter calculation unit 16 performs clustering by extracting points whose distance from each approximate line generated in step S51 is within a predetermined threshold Dt.

[0106] The docking parameter calculation unit 16 determines whether the conditions are met for the approximate lines generated in step S51, specifically whether there are multiple approximate lines with large differences in direction vectors, and whether the number of points (data) belonging to each cluster is greater than or equal to a predetermined value (step S53).

[0107] If the approximate straight line generated in step S51 does not meet the aforementioned conditions (step S53: No), the berthing parameter calculation unit 16 estimates that there is one quay wall surface at the berthing location and sets the direction for generating the berthing side straight line to one direction corresponding to that one quay wall surface (step S54), and then performs the process described in step S56 below. If the approximate straight line generated in step S51 meets the aforementioned conditions (step S53: Yes), the berthing parameter calculation unit 16 estimates that there are multiple quay wall surfaces at the berthing location and sets the direction for generating the berthing side straight line to multiple directions corresponding to each of the multiple quay wall surfaces (step S55), and then performs the process described in step S56 below.

[0108] The shoreing parameter calculation unit 16 selects the initial direction for generating the shoreing side line (step S56), and then calculates the distance from the reference position RP to the centroid position of each cluster for the cluster in that initial direction (step S57).

[0109] Next, the docking parameter calculation unit 16 obtains the point cloud contained in the cluster with the smallest distance from the reference position RP to the center of gravity position, based on the processing result of step S57 (step S58).

[0110] Next, the docking parameter calculation unit 16 generates a straight line using the point cloud acquired in step S58 (step S59), acquires the generated straight line as the docking side line Ls (step S60), and then determines whether there are any other directions for which the docking side line has not yet been acquired (step S61).

[0111] If there are other directions for which the shore side line Ls has not been obtained (step S61: Yes), the shore parameter calculation unit 16 selects the next direction for generating the shore side line (step S62) and then continues the process in step S57. If there are no other directions for which the shore side line Ls has not been obtained (step S61: No), the shore parameter calculation unit 16 continues the process from step S16 onward.

[0112] The docking parameter calculation unit 16 uses the docking side line Ls calculated in step S15 to calculate the docking parameters, which are the distance to the opposite shore, the approach angle, and the docking speed (step S16).

[0113] Then, the berthing parameter calculation unit 16 generates confidence information based on the results of identifying the inner surface of the field of view and the detection surface in step S13 and the calculation results of the berthing parameters in step S16 (step S17). Subsequently, the information processing device 1 controls the vessel based on the confidence information (step S18). As a result, the information processing device 1 can accurately control the vessel regarding berthing based on a confidence level that accurately reflects the berthing situation.

[0114] The information processing device 1 then determines whether or not the target vessel has docked (step S19). In this case, the information processing device 1 determines whether or not the target vessel has docked based on, for example, the output signal of the sensor group 2 or user input via the interface 11. If the information processing device 1 determines that the target vessel has docked (step S19; Yes), it terminates the flowchart process. On the other hand, if the information processing device 1 determines that the target vessel has not docked (step S19; No), it returns to step S11.

[0115] According to the process described above, the acquisition means acquires measurement data, which is a collection of data representing multiple measured points at the berthing location measured by a measuring device installed on the vessel. According to the process described above, the cluster generation means generates one or more clusters by dividing the multiple measured points through clustering. According to the process described above, the straight line generation means generates a straight line along the berthing location where the vessel is docked, based on the cluster with the shortest distance to the vessel.

[0116] As described above, according to this embodiment, a straight line Ls along the berthing side can be generated using each measured point in the cluster closest to the reference position RP, from among one or more clusters obtained by clustering multiple measurement points included in the berthing location edge point cloud. Therefore, according to this embodiment, even if, for example, buffer materials such as fenders are installed on the side of the berthing location, a straight line along the edge of the berthing location can be suitably generated.

[0117] In the embodiments described above, the program can be stored using various types of non-transitory computer-readable media and supplied to a control unit, which is a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (Random Access Memory)).

[0118] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the structure and details of the present invention can be made that are understandable to those skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that those skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents and other references is incorporated herein by reference. [Explanation of Symbols]

[0119] 1. Information Processing Device 2 Sensor Groups 3 Riders

Claims

[Claim 1] An acquisition means for acquiring measurement data, which is a set of data representing multiple measurement points at a docking location measured by a measuring device installed on a ship, Cluster generation means for generating one or more clusters by dividing the aforementioned multiple measurement points by clustering, A straight line generation means that generates a straight line along the berthing location where the vessel docks, based on one or more of the aforementioned clusters, An information processing device having

Images