Information processing device, control method, program, and storage medium

Abstract

To provide an information processor which can suitably calculate a normal line of a measured point measured by a measurement device.SOLUTION: A controller 13 of an information processor 1 mainly includes acquisition means, normal line calculation range setting means, and normal line calculation means. The acquisition means acquires measurement data which is a set of data indicating a plurality of measured points measured by a measurement device. The normal line calculation range setting means sets a normal line calculation range for generating a point set of the measurement points used in calculation of the respective normal lines of the measured points, on the basis of a measurement distance indicated by the measurement data. The normal line calculation means calculates the respective normal lines of the measured points, on the basis of the normal line calculation range.SELECTED DRAWING: Figure 18

Description

[Technical Field]

[0001] This disclosure relates to the processing of measurement data. [Background technology]

[0002] Technologies for assisting with ship docking (berthing) have been known for some time. For example, Patent Document 1 describes a method for controlling the attitude of a ship in an automatic docking device that performs automatic ship docking, such that light emitted from a lidar is reflected by objects around the docking position and received by the lidar. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-59403 [Overview of the project] [Problems that the invention aims to solve]

[0004] Generally, a quay consists of a flat ground portion (top surface) and a vertical wall portion (side) that separates the ground from the sea. In this case, it is desirable to extract the surface based on normal information calculated from measured point cloud data. On the other hand, in the case of a lidar, for example, the measured point cloud becomes sparse at long distances and dense at close distances, so the accuracy of normal calculation may deteriorate or become impossible depending on the distance to the point being measured.

[0005] This disclosure is made to solve the above-mentioned problems, and its main purpose is to provide an information processing device that can suitably calculate the normal vector of a point measured by a measuring device. [Means for solving the problem]

[0006] The invention described in the claims is, Installed on a shipAn acquisition means for acquiring measurement data, which is a set of data representing multiple points to be measured by a measuring device, A normal calculation range setting means sets a normal calculation range for generating a set of points of the measured points to be used to calculate the normal of each of the measured points, based on the measured distance indicated by the measurement data. A normal calculation means that calculates the normal of each of the points to be measured based on the normal calculation range, A quay wall determination means for determining the point to be measured that represents the quay wall based on the aforementioned normal, It is an information processing device.

[0007] Furthermore, the invention described in the claims is, A control method performed by a computer, Installed on a ship The measurement data, which is a collection of data representing multiple points measured by the measuring device, is acquired. Based on the measurement distance indicated by the measurement data, a normal calculation range is set for generating a set of points of the measured points to be used for calculating the normal of each of the measured points. Based on the normal calculation range, the normal of each of the points to be measured is calculated. death, Based on the aforementioned normal, the measurement point representing the quay wall is determined. This is a control method.

[0008] Furthermore, the invention described in the claims is, Installed on a ship The measurement data, which is a collection of data representing multiple points measured by the measuring device, is acquired. Based on the measurement distance indicated by the measurement data, a normal calculation range is set for generating a set of points of the measured points to be used for calculating the normal of each of the measured points. Based on the normal calculation range, the normal of each of the points to be measured is calculated. death, Based on the aforementioned normal, the measurement point representing the quay wall is determined. It is a program that instructs a computer to perform a process. [Brief explanation of the drawing]

[0009] [Figure 1]It is a schematic configuration diagram of an operation support system. [Figure 2] It is a block diagram showing the hardware configuration of an information processing device. [Figure 3] It is an example of a flowchart showing the outline of the processing in an embodiment. [Figure 4] It is a diagram showing the outline of the calculation process of the approach parameters. [Figure 5] (A) shows an example of a ship coordinate system based on the hull of the target ship. (B) is a perspective view of a structure with a normal vector indicated. [Figure 6] (A) is a front view of the target ship and the shore wall when the shore wall to be detected is relatively far away. (B) is a front view of the target ship and the shore wall when the shore wall to be detected is relatively close. [Figure 7] (A) shows the arrangement of the measured points within a 1m square showing the relationship between the distance between points and the normal calculation radius at the 30m point. (B) represents a linear equation according to the measured distance x of the point of interest when the normal calculation radius is "y". [Figure 8] (A) is a diagram showing the normal to each measured point on the side of the shore wall with high flatness by an arrow. (B) is a diagram showing the normal to each measured point on the side of the shore wall with irregularities by an arrow. [Figure 9] (A) to (C) are diagrams showing the side points of the shore wall provided with fenders. [Figure 10] It is a diagram showing the outline of the distance between surfaces used for the same surface integration process. [Figure 11] It is a diagram showing the outline of the area non-wall surface removal process. [Figure 12] (A) shows the outline of the first pseudo-area calculation method. (B) shows the outline of the second pseudo-area calculation method. [Figure 13] (A) shows a perspective view of the shore wall when there are 3 wall surface candidate clusters on the side surface and 3 wall surface candidate clusters on the upper surface respectively. (B) shows a perspective view of the shore wall with the wall surface candidates corresponding to the upper surface and the side surface finally indicated. [Figure 14] It is a cross-sectional view of the shore wall with the upper surface existence range indicated. [Figure 15](A)(B) This figure shows the procedure (b) to (d) for selecting candidate wall clusters for the top and side surfaces, respectively. [Figure 16] (A) This figure shows the classification results of the measurement points on the quay. (B) This figure shows the classification results of the measurement points after boundary point determination processing. [Figure 17] (A)(B) This figure shows an overview of the boundary point determination process. [Figure 18] This is an example of a flowchart for normal vector calculation. [Figure 19] This is an example of a flowchart for determining the orientation of a wall. [Figure 20] This is an example of a flowchart for distributed non-wall surface removal and same-surface integration processes. [Figure 21] This is an example of a flowchart for area-based non-wall surface removal processing. [Figure 22] This is an example of a flowchart for the optimal wall selection process. [Figure 23] This is an example of a flowchart for boundary point detection processing. [Figure 24] This is a diagram illustrating the overview of the first application example. [Figure 25] This is a diagram illustrating the overview of the first application example. [Figure 26] This figure shows an overview of the second application example. [Figure 27] This figure shows an overview of the second application example. [Figure 28] This is a diagram illustrating the overview of the third application example. [Figure 29] This is a diagram illustrating the overview of the third application example. [Figure 30] This table shows an example of how to calculate the confidence level for detecting a quay wall. [Modes for carrying out the invention]

[0010] According to a preferred embodiment of this disclosure, the information processing device includes: acquisition means for acquiring measurement data, which is a set of data representing a plurality of points to be measured measured by a measuring device; normal calculation range setting means for setting a normal calculation range for generating a set of points of the points to be measured to be used for calculating the normal of each of the points to be measured based on the measurement distance indicated by the measurement data; and normal calculation means for calculating the normal of each of the points to be measured based on the normal calculation range. According to this embodiment, the information processing device can suitably calculate the normal of each of the points to be measured measured by the measuring device.

[0011] In one embodiment of the above-described information processing device, the normal calculation range setting means increases the normal calculation range as the measurement distance increases. In this embodiment, the information processing device can suitably calculate the normal of each of the points to be measured by the measuring device.

[0012] In another embodiment of the above-described information processing device, the information processing device includes a clustering means for clustering the points to be measured based on the normal vectors, and a determination means for determining whether each of the clusters of points to be measured formed by the clustering is a candidate for a quay wall, based on the degree of variation of the normal vectors within the cluster. In this embodiment, the information processing device can suitably detect points to be measured on the wall surface of a quay wall with high flatness as candidates for a quay wall.

[0013] In another embodiment of the above-described information processing device, the information processing device includes a clustering means that performs clustering of the points to be measured based on the normal vector, and a determination means that estimates a plane approximating each of the clusters of the points to be measured formed by the clustering, and determines whether or not the clusters need to be merged based on the distance between the planes. In this embodiment, the information processing device can suitably determine whether or not clusters separated by obstacles, etc., need to be merged.

[0014] In another embodiment of the above-described information processing device, the information processing device includes a clustering means for clustering the points to be measured based on the normal vector, and a determination means for determining whether each of the clusters of points to be measured formed by the clustering is a candidate for a quay, based on the number of points to be measured forming the cluster and the area of ​​the cluster. In this embodiment, the information processing device can accurately determine clusters that are candidates for quays.

[0015] In another embodiment of the above-described information processing device, the information processing device includes a clustering means for clustering the points to be measured based on the normal vector, and a determination means for determining that the cluster of points to be measured that is located within a predetermined range and at the lowest position among the clusters of points to be measured that are candidates for the upper surface of the quay formed by the clustering is the cluster representing the upper surface. In this embodiment, the information processing device can accurately determine the cluster of points to be measured that represents the upper surface of the quay.

[0016] In another embodiment of the information processing device described above, the determination means determines that the cluster of measurement points that are candidates for the side of the quay and formed by the clustering, which is located lower than the cluster representing the top surface and is closest to the measurement device, is the cluster representing the side. In this embodiment, the information processing device can accurately determine the cluster of measurement points that represents the side of the quay.

[0017] In another embodiment of the above-described information processing device, the information processing device has a classification means that classifies each of the measured points based on the normal vector whether it is a wall point where the quay wall is measured or a non-wall point where something other than the quay wall is measured. The classification means reclassifies the non-wall points that lie between the wall points as wall points based on the shortest distance between the non-wall points and the wall points. In this embodiment, the information processing device can accurately classify measured points near the boundary between the top and side surfaces of the quay wall, which are difficult to classify accurately based solely on the orientation of the normal vector.

[0018] According to another preferred embodiment of the present disclosure, a control method performed by a computer is provided, comprising: acquiring measurement data which is a set of data representing a plurality of points to be measured measured by a measuring device; setting a normal calculation range for generating a set of points to be measured to be used for calculating the normal of each of the points to be measured based on the measurement distance indicated by the measurement data; and calculating the normal of each of the points to be measured based on the normal calculation range. By performing this control method, the computer can suitably calculate the normal of each of the points to be measured measured by the measuring device.

[0019] In another preferred embodiment of the present disclosure, the program is a program that causes a computer to perform the following processes: acquire measurement data, which is a set of data representing a plurality of points to be measured measured by a measuring device; set a normal calculation range for generating a set of points to be measured to be used for calculating the normal of each of the points to be measured, based on the measurement distance indicated by the measurement data; and calculate the normal of each of the points to be measured based on the normal calculation range. By executing this program, the computer can suitably calculate the normal of each of the points to be measured measured by the measuring device. Preferably, the program is stored in a storage medium. [Examples]

[0020] Preferred embodiments of the present invention will be described below with reference to the drawings.

[0021] (1) Overview of the flight support system Figures 1(A) to 1(C) show the schematic configuration of the navigation support system according to this embodiment. Specifically, Figure 1(A) shows a block diagram of the navigation support system, Figure 1(B) is a top view illustrating the field of view (also called the "measurement range" or "distance-measurable range") 90 of the vessel and the lidar 3 described later, which are included in the navigation support system, and Figure 1(C) is a rear view showing the field of view 90 of the vessel and the lidar 3. The navigation support system comprises an information processing device 1 that moves together with the vessel, which is a moving object, and a group of sensors 2 mounted on the vessel. Hereafter, the vessel on which the navigation support system is installed will also be called the "target vessel".

[0022] The information processing device 1 is electrically connected to the sensor group 2 and provides operational support for the target vessel based on the outputs of the various sensors included in the sensor group 2. Operational support includes berthing support such as automatic docking. In this embodiment, accurate operational support is achieved by accurately measuring the quay where the vessel will dock. The information processing device 1 may be a navigation device installed on the vessel, or it may be an electronic control device built into the vessel.

[0023] Sensor group 2 includes various external and internal sensors installed on the ship. In this embodiment, sensor group 2 includes, for example, a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3.

[0024] LIDA 3 is an external sensor that discretely measures the distance to an object in the outside world by emitting a pulsed laser within a predetermined angular range in the horizontal direction (see Figure 1(B)) and a predetermined angular range in the vertical direction (see Figure 1(C)), and generates three-dimensional point cloud data indicating the position of the object. In the examples of Figures 1(B) and 1(C), LIDA 3 is provided on the ship, with one LIDA directed toward the left side of the ship and another directed toward the right side of the ship. Note that the arrangement of LIDA 3 is not limited to the examples in Figures 1(B) and 1(C). For example, the target ship may have multiple LIDA 3 (for example, LIDAs provided in front of and behind the target ship) that measure in the same lateral direction so that multiple measurement data of the quay can be obtained simultaneously when docking. Also, the number of LIDA 3 installed on the target ship is not limited to two, but may be one or three or more.

[0025] LIDA 3 comprises an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data based on the received signal output by the light receiving unit. The data measured for each direction of laser light irradiation (scanning position) is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light specified based on the received signal described above. Hereafter, a point measured by irradiation with laser light within the measurement range of LIDA 3, or the measurement data thereof, will also be referred to as the "measured point".

[0026] Here, point cloud data can be considered as an image (frame) where each measurement direction is represented by a pixel, and the measured distance and reflectance value for each measurement direction are represented by the pixel values. In this case, the direction of laser beam emission (i.e., measurement direction) differs in elevation and depression angles in the vertical arrangement of pixels, and the direction of laser beam emission in horizontal angles differs in the horizontal arrangement of pixels. Hereafter, when the point cloud data is considered as an image, the measured points corresponding to the rows of pixels (i.e., vertical columns) whose horizontal index positions coincide will also be called "vertical lines".

[0027] Furthermore, LIDA 3 is not limited to the scanning type LIDA described above, but may also be a flash type LIDA that generates 3D data by diffusing laser light into the field of view of a 2D array sensor. LIDA 3 is an example of a "measuring device" in the present invention.

[0028] (2) Configuration of an information processing device Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly consists of an interface 11, a memory 12, and a controller 13. Each of these elements is interconnected via a bus line.

[0029] Interface 11 performs interface operations related to the exchange of data between the information processing device 1 and external devices. In this embodiment, interface 11 acquires output data from each sensor in the sensor group 2 and supplies it to the controller 13. Interface 11 also supplies signals related to the control of the target vessel, generated by the controller 13, to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel includes a drive source such as an engine or electric motor, a propeller that generates thrust in the direction of travel based on the driving force of the drive source, a thruster that generates lateral thrust based on the driving force of the drive source, and a rudder, etc., which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation such as automatic docking, interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, interface 11 supplies control signals generated by the controller 13 to the electronic control device. Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices by cables, etc. Furthermore, interface 11 may perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0030] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0031] Furthermore, memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, memory 12 may store map data including information about the location of the quay. In another example, memory 12 stores information about the downsampling size when downsampling is performed on the point cloud data obtained when the lidar 3 performs one cycle of scanning.

[0032] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to operational support for the target vessel by executing programs stored in memory 12, etc.

[0033] Furthermore, the controller 13 functionally includes a quay detection unit 15 and a docking parameter calculation unit 16. The quay detection unit 15 performs processing related to quay detection based on the point cloud data output by the lidar 3. The docking parameter calculation unit 16 calculates the parameters necessary for docking at the quay (also called "docking parameters"). Here, the docking parameters include the distance from the target vessel to the quay (distance to the opposite shore), the angle at which the target vessel approaches the quay, and the speed at which the target vessel approaches the quay (docking speed). The docking parameter calculation unit 16 may also calculate information representing the reliability of docking at the quay (also called "reliability information") based on the processing results of the quay detection unit 15 and the docking parameters. The controller 13 functions as an "acquisition means," a "normal calculation range setting means," a "normal calculation means," a "clustering means," a "determination means," a "classification means," and a computer that executes programs.

[0034] Furthermore, the processing performed by the controller 13 is not limited to being implemented by software through a program, but may also be implemented by a combination of hardware, firmware, and software. Additionally, the processing performed by the controller 13 may be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program executed by the controller 13 in this embodiment may be implemented using this integrated circuit.

[0035] (2) Processing Overview Figure 3 is an example of a flowchart showing an overview of the processes performed by the quay wall detection unit 15 in this embodiment.

[0036] First, the quay wall detection unit 15 acquires point cloud data generated by the lidar 3 at each frame period (step S1). The quay wall detection unit 15 may remove data located below the water surface position from the point cloud data generated by the lidar 3 as water surface measurement data (i.e., false detection data). The quay wall detection unit 15 estimates the water surface position based, for example, on the average value in the height direction of the point cloud data generated by the lidar 3 when there are no objects other than the water surface in the surrounding area. The quay wall detection unit 15 may also perform downsampling on the point cloud data after removing the water surface reflection data, which is a process that integrates the measured points into grid spaces of a predetermined size. Downsampling may be performed before removing the false detection data.

[0037] Next, the quay wall detection unit 15 performs a normal vector calculation process, which is the process of calculating the normal vector of each point to be measured represented by the point cloud data (step S2). Then, the quay wall detection unit 15 performs a wall orientation determination process based on the direction of the normal vector calculated in step S2 (step S3). In the wall orientation determination process, the quay wall detection unit 15 classifies (labels) each point to be measured as either a point to be measured on the top surface of the quay wall (also called a "top surface point"), a point to be measured on the side of the quay wall (also called a "side surface point"), or a point to be measured that is not on the quay wall (also called a "non-wall point").

[0038] Next, the quay detection unit 15 performs a dispersed non-wall removal process, which removes non-wall points that have been incorrectly labeled as top surface points or side surface points (also called "wall points") based on the dispersion of normals (step S4). Furthermore, the quay detection unit 15 performs a same-surface integration process, which integrates clusters of wall points in order to treat wall surfaces separated by fenders and the like as a single surface (step S5). Finally, the quay detection unit 15 performs an area non-wall removal process, which removes noise points and measurement points of small objects generated by noise based on the number of points and area of ​​the cluster (step S6).

[0039] Next, the quay detection unit 15 performs an optimal wall selection process, which is the process of selecting one set of clusters of top points and one set of clusters of side points, when there are multiple clusters of top points and multiple clusters of side points (step S7). Next, the quay detection unit 15 performs a boundary point determination process to determine whether a non-wall point (also called a "boundary point") existing between the top points and side points is a wall point located at the wall boundary (boundary between the top and side) (step S8). Then, the quay detection unit 15 outputs a quay point cloud, which is point cloud data representing the points to be measured that have been determined to be wall points corresponding to either top points or side points (step S9). After that, the docking parameter calculation unit 16 calculates docking parameters (distance from the opposite shore, docking speed, approach angle) based on the quay point cloud.

[0040] Figure 4 shows an overview of the berthing parameter calculation process by the berthing parameter calculation unit 16. The berthing parameter calculation unit 16 generates a straight line (also called the "berthing edge line") that is closest to the target vessel, using the quay point cloud (more specifically, side points). The berthing parameter calculation unit 16 then calculates the shortest distance to that line as the distance to the opposite shore. The berthing parameter calculation unit 16 also calculates the berthing speed by the time derivative of the distance to the opposite shore and calculates the approach angle, which represents the direction of the bow, from the angle with respect to the quay edge line. Here, if non-wall points such as small objects that are not quays or noise are not removed from the point cloud of measured points measured by the lidar 3, the berthing parameters cannot be calculated accurately. Taking the above into consideration, this embodiment will describe an accurate method for extracting the quay point cloud by the quay detection unit 15.

[0041] (3) Normal vector calculation process In the normal vector calculation process, the quay detection unit 15 calculates the normal vector for each measured point represented by the point cloud data acquired by the lidar 3. In this case, the quay detection unit 15 sequentially selects each measured point as a point to be processed (also called a "point of interest") and calculates the normal vector of the surface approximated by the group of measured points that exist within a certain distance from the point of interest. Then, the quay detection unit 15 classifies (labels) each measured point as a side point, top point, or non-wall point based on the angle of the normal vector (normal angle) in a coordinate system based on the hull of the target vessel (also called a "ship coordinate system").

[0042] Figure 5(A) shows an example of a ship coordinate system based on the hull of the target vessel. As shown in Figure 5(A), the forward direction of the target vessel is defined as the "X" coordinate, the side direction of the target vessel as the "Y" coordinate, and the height direction of the target vessel as the "Z" coordinate. The measurement data from the coordinate system based on LIDA 3, measured by LIDA 3, is then converted to the ship coordinate system shown in Figure 5(A). The process of converting point cloud data from a coordinate system based on a LIDA installed on a moving object to the coordinate system of the moving object is disclosed, for example, in International Publication WO2019 / 188745.

[0043] Figure 5(B) is a perspective view of the quay wall, clearly showing the measurement points and normal vectors calculated based on the measurement points, which were measured by the LIDA3. In Figure 5(B), the measurement points are indicated by circles, and the normal vectors are indicated by arrows. This example shows that both the top and side surfaces of the quay wall were measured by the LIDA3.

[0044] As shown in Figure 5(B), the quay wall detection unit 15 calculates normal vectors for the measurement points on the side and top surfaces of the quay wall. Since the normal vector is a vector perpendicular to the target plane or curved surface, it is calculated using multiple measurement points that can be formed as a surface. Therefore, the quay wall detection unit 15 sets a circle with a predetermined radius for the selected point of interest and calculates the normal vector using the measurement points located inside it. Hereafter, the radius mentioned above will be called the "normal calculation radius," and the circle (or sphere) mentioned above will also be called the "normal calculation range."

[0045] Here, the quay detection unit 15 determines the normal calculation radius according to the measurement distance of the point of interest. A specific example of this will be explained with reference to Figures 6(A) and 6(B). Figure 6(A) is a front view of the target vessel and quay when the quay to be detected is relatively far away, and Figure 6(B) is a front view of the target vessel and quay when the quay to be detected is relatively close. In Figures 6(A) and 6(B), the measurement points on the quay are indicated by circles, and furthermore, the normal calculation range for a certain measurement point is indicated by a solid line frame, and the normal is indicated by an arrow.

[0046] As shown in Figure 6(A), when the quay wall is relatively far away, the quay wall detection unit 15 takes into consideration that the number of points in the point cloud data is small (i.e., the points to be measured are sparsely distributed on the quay wall), making normal calculation difficult, and sets a large normal calculation radius. This increases the normal calculation range and allows for a suitable number of points to be measured used for normal calculation.

[0047] On the other hand, as shown in Figure 6(B), the quay detection unit 15 sets a smaller normal calculation radius when the quay is at close range. This is because if the same normal calculation radius as when it is at long range is used when it is at close range, the number of measurement points used for normal calculation will become too large, leading to problems such as increased computational cost and decreased accuracy of direction estimation.

[0048] Figure 7(A) shows the arrangement of points to be measured within a 1m square, illustrating the relationship between the distance between points and the normal calculation radius at a point 30m away. Here, "φ" represents the resolution of the lidar 3, and "x" represents the measured distance of the point of interest (the central point of measurement in Figure 7(A)). In this case, the distance between adjacent points of interest is represented by "φx". In this embodiment, the normal calculation radius is set for each point of interest and is represented by a linear equation that increases proportionally to the measured distance of the point of interest. In this case, if "a" is the slope of the linear equation (point-to-point distance multiplier parameter) and "b" is the intercept of the linear equation, the normal calculation radius is represented by the distance between points as "aφx+b". Figure 7(B) shows a linear equation corresponding to the measured distance x of the point of interest when the normal calculation radius is "y". Here, the coefficients a and b are set in advance based on experiments, etc., so that at least 5 points are included in the normal calculation range, and are stored in memory 12, etc.

[0049] In this way, the quay wall detection unit 15 adaptively sets the normal calculation radius according to the measurement distance of the point of interest, making it possible to stably calculate the normal for each point under measurement using a fixed number of points regardless of distance.

[0050] Subsequently, the quay detection unit 15, in the wall orientation determination process, classifies each measured point as either a top point, a side point, or a non-wall point based on the normal angle, which is the angle of the normal to the XY plane in the ship's coordinate system. In this case, the quay detection unit 15 classifies measured points whose normal angle is equal to or greater than a predetermined top threshold as top points, measured points whose normal angle is less than a predetermined side threshold (a threshold lower than the top threshold) as side points, and all other measured points as non-wall points.

[0051] (4) Distributed non-wall removal process Since the target quay does not necessarily have sides perpendicular to the ground, it is necessary to allow for a certain degree of leeway in the range of normal angles used to identify a wall surface in order to accommodate various types of quays. On the other hand, if the range of normal angles used to identify a wall surface is made too large, there is a risk of incorrectly identifying uneven surfaces as walls. Thus, it is necessary to accurately identify measurement points on the quay surface as wall points, excluding areas with significant irregularities due to fenders, etc.

[0052] Taking the above into consideration, the quay wall detection unit 15, in the dispersed non-wall surface removal process, removes clusters of top and side points determined in the wall surface orientation determination process that have a large dispersion of normals, as clusters that form non-wall points. The method for generating clusters of top and side points (clustering method) is any clustering method, such as distance-based Euclidean clustering.

[0053] Figure 8(A) shows the normals to each measurement point on a highly flat quay wall side indicated by arrows. Figure 8(B) shows the normals to each measurement point on an uneven quay wall side indicated by arrows. Here, "θ" represents the normal angle to a certain measurement point. In the example of Figure 8(A), the directions of the normals to each measurement point on the quay wall side are aligned, and the dispersion is small. Therefore, in this case, the quay wall detection unit 15 determines each measurement point on the quay wall side to be a wall surface point. Note that in Figure 8(A), the side is tilted at an angle, but by making the allowable angle of the normal angle θ (i.e., the side threshold mentioned above) somewhat large, it is possible to accurately determine that measurement points on such a side are also wall surface points.

[0054] On the other hand, in the example shown in Figure 8(B), there is variation in the direction of the normal to each measurement point on the side of the quay wall, resulting in a large variance. Therefore, in this case, the quay wall detection unit 15 determines each measurement point on the side of the quay wall as a non-wall point.

[0055] Here, we will explain a specific example of how to calculate the variance for each cluster. First, the quay wall detection unit 15 calculates the covariance matrix of each element in the XYZ directions of the normal group of the points to be measured for each cluster. Since the diagonal elements of the covariance matrix represent the variance values ​​in each axis direction (X axis, Y axis, Z axis), the quay wall detection unit 15 determines that the target cluster is a cluster of non-wall points if the maximum value of the diagonal elements of the covariance matrix is ​​greater than or equal to a predetermined threshold.

[0056] According to the dispersed non-wall removal process described above, even if the range of normal angles used to determine a wall is made somewhat large, measurement points on surfaces with low planarity (surfaces where normals are scattered and dispersion is large) can be accurately identified as non-wall points. Furthermore, because the range of normal angles used to determine a wall can be made somewhat large, measurement points on slanted sides, as shown in Figure 8(A), can be accurately identified as wall points, and wall point determination can be made robustly even with changes in the roll angle and pitch angle of the target vessel. Dispersion is one example of an index that represents the "degree of variation".

[0057] (5) Same-sided integration processing When detecting quay walls, if spatially close points are separated into clusters using methods such as Euclidean clustering, fenders and obstacles on the quay wall may cause clusters of candidate walls, which should originally lie on the same plane, to be divided into multiple clusters. If only one of these candidate wall clusters is adopted to represent a wall, the number of wall points used by the quay parameter calculation unit 16 to calculate the distance to the opposite shore will decrease, potentially leading to a deterioration in the accuracy of the opposite shore distance calculation.

[0058] Taking the above into consideration, in the same-face integration process, clusters of side points or top points (also called "wall candidate clusters") are integrated based on the direction of the normal and the inter-face distance between wall candidate clusters. Figure 9(A) is a diagram showing the side points of a quay with fenders installed. In Figure 9(A), the side points of the quay are divided into two due to the presence of the fenders, resulting in the formation of a first wall candidate cluster and a second wall candidate cluster (both of side point clusters in this case). Figure 9(B) shows the second wall candidate cluster adopted as the cluster representing the wall, and the first wall candidate cluster represented as the cluster representing the non-wall. This illustrates the state. In this example, the measured points that form the first wall candidate cluster are classified as non-wall points, resulting in a reduced number of side point points. Figure 9(C) shows the integrated wall candidate cluster obtained by integrating the first and second wall candidate clusters through same-face integration processing. In this example, the measured points belonging to the first and second wall candidate clusters are treated as side point points, thus suppressing the reduction in the number of side point points. Furthermore, the generated integrated wall candidate cluster has the advantage of being less likely to be excluded as non-walls in the area non-wall removal processing described later.

[0059] Figure 10 is a diagram illustrating the inter-face distance used in the same-face integration process. In Figure 10, "G1(x1,y1,z1)" represents the representative point (representative position) of the first wall candidate cluster, and "G2(x2,y2,z2)" represents the representative position of the second wall candidate cluster. Also, "n=(n x ,n y ,n z ) T Let "n" be the normal vector of the representative of the first candidate wall cluster (which is the same as the normal vector of the representative of the second candidate wall cluster). Also, "H" represents the intersection point between the plane approximating the second candidate wall cluster (extended plane) and the normal vector n.

[0060] As shown in Figure 10, the quay detection unit 15 calculates the inter-face distance as the dot product (|(G2-G1)·n|) of a vector connecting representative points such as the centroid and geometric center of the candidate wall surface and the normal vector. Since this inter-face distance can be calculated without changing even if the roll and pitch angles of the target vessel change, the process of integrating identical surfaces can be accurately performed regardless of the attitude of the target vessel.

[0061] Next, the integration conditions will be explained. The first condition of the quay wall detection unit 15 is that the normals of the candidate wall clusters to be compared are approximate. For example, in the case of side view candidate clusters, the normals are determined to be approximate if the angle difference on the XY plane is within 10 degrees. Next, the quay wall detection unit 15 sets the second condition that the distance between surfaces is less than a predetermined threshold. Furthermore, the quay wall detection unit 15 sets the third condition that the distance between representative points of the candidate wall clusters to be compared is less than a predetermined threshold. Note that the third condition does not have to be a mandatory condition. In this case, the first and second conditions may be necessary and sufficient conditions, or the first to third conditions may be necessary and sufficient conditions.

[0062] Here, we will provide a supplementary explanation of how to integrate three or more candidate wall clusters. In the first example, the quay wall detection unit 15 integrates sets of candidate wall clusters that satisfy the above conditions, recalculates the representative normal and representative point, and then sequentially compares and integrates with the next candidate. In the second example, the quay wall detection unit 15 makes integration decisions for candidate wall clusters that are close in distance, and finally integrates all candidate wall clusters that are determined to be merged.

[0063] (6) Area-based non-wall surface removal treatment The quay wall detection unit 15 removes candidate wall clusters corresponding to noise points and small objects (such as mooring ropes) in the area-based non-wall surface removal process, based on the number of points and area.

[0064] First, the quay wall detection unit 15 excludes outliers such as noise by treating small wall surface candidate clusters with a score less than a predetermined threshold as non-wall point clusters. Next, in addition to the area occupied by the wall surface candidate cluster in the space as a determination criterion, the quay wall detection unit 15 excludes small wall surface candidate clusters that do not satisfy a predetermined area as non-wall point clusters.

[0065] FIG. 11 is a diagram showing an overview of the area non-wall surface removal process. In FIG. 11, the measured points P1 to P3 are clusters each consisting of one point, and since the quay wall detection unit 15 determines that the scores of these clusters are less than the threshold, it regards these as non-wall points. Also, although the cluster Cw4 has a score equal to or greater than the threshold, since it does not satisfy the area threshold, the quay wall detection unit 15 regards the cluster Cw4 as a non-wall point cluster. On the other hand, the quay wall detection unit 15 retains the clusters Cw5 and Cw6 that satisfy both the score and area conditions as wall surface candidate clusters.

[0066] Here, the method for calculating the area of the wall surface candidate cluster will be described. The quay wall detection unit 15 performs principal component analysis for each wall surface candidate cluster and calculates the pseudo area occupied by each cluster. FIG. 12(A) shows an overview of the first pseudo area calculation method, and FIG. 12(B) shows an overview of the second pseudo area calculation method. In FIG. 12(A), "PC1" represents the first principal component axis of the cluster Cw5, "PC2" represents the second principal component axis of the cluster Cw5, "PC3" represents the first principal component axis of the cluster Cw6, and "PC4" represents the second principal component axis of the cluster Cw6. Also, in FIG. 12(B), "sd1" represents the standard deviation of the first principal component axis of the cluster Cw5, "sd2" represents the standard deviation of the second principal component axis of the cluster Cw5, "sd3" represents the standard deviation of the first principal component axis of the cluster Cw6, and "sd4" represents the standard deviation of the second principal component axis of the cluster Cw6.

[0067] In the first pseudo area calculation method, the quay wall detection unit 15 calculates the area when the clusters Cw5 and Cw6 are regarded as rectangles, respectively, with the lengths in the first and second principal component axis directions of the principal component analysis as the vertical and horizontal lengths. In this case, using a coefficient k (0 < k < 1), the quay wall detection unit 15 calculates the area S1 of the cluster Cw5 based on the first pseudo area calculation method as S1 = k × w s × h s and set the area S1 of the cluster Cw6 as S1 = k × w u × h u In this case, the lengths (w s、 h s、 w u、 h u ) are determined by the differences between the maximum and minimum values of the measured points within the cluster in the directions of the first and second principal component axes. In the first pseudo - area calculation method, when the cluster is close to a rectangle, a value close to the actual area can be obtained.

[0068] In the second pseudo - area calculation method, the quay - wall detection unit 15 calculates the standard deviation from the eigenvalues of the first principal component and the second principal component, and uses the product as the size (i.e., the degree of variation) of the cluster to calculate the area. In this case, the quay - wall detection unit 15 calculates the area S2 of the cluster Cw5 based on the second pseudo - area calculation method as S2 = k × sd1 × sd2 = k√(λ1λ2) and set the area S2 of the cluster Cw6 as S2 = k × sd3 × sd4 = k√(λ3λ4) where "λ1" represents the eigenvalue of the first principal component of the cluster Cw5, "λ2" represents the eigenvalue of the second principal component of the cluster Cw5, "λ3" represents the eigenvalue of the first principal component of the cluster Cw6, and "λ4" represents the eigenvalue of the second principal component of the cluster Cw6. In the second pseudo - area calculation method, the sizes of various shapes can be generally evaluated.

[0069] When the target quay - wall has a complex shape, there are multiple wall - surface candidate clusters. Therefore, in the optimal wall - surface selection process, the quay - wall detection unit 15 selects the wall - surface candidate cluster that is most likely to be the quay - wall.

[0070] (7) Optimal wall selection process

[0071] ​In this case, the quay wall detection unit 15 calculates a representative position for each candidate wall cluster and selects the optimal surface based on their relative positions. Finally, the quay wall detection unit 15 narrows down the candidate wall clusters corresponding to the top and sides to one each, and considers the unselected candidate wall clusters as clusters of non-wall points such as obstacles. Figure 13(A) shows a perspective view of the quay wall when there are three candidate wall clusters each for the sides and the top. Figure 13(B) shows a perspective view of the quay wall with the final candidate wall clusters corresponding to the top and sides clearly indicated.

[0072] The details of the optimal wall selection process will now be explained. First, the quay wall detection unit 15 pre-sets a range of possible heights for the upper surface of the quay wall (also called the "upper surface existence range") and excludes candidate wall clusters on the upper surface that fall outside that range (for example, clusters such as sea surface reflection points and building roofs).

[0073] Figure 14 is a cross-sectional view of a quay wall with the upper surface presence range clearly indicated. In Figure 14, clusters Cw10 to Cw14 are candidate wall clusters consisting of upper surface points. In this case, the quay wall detection unit 15 considers clusters Cw10 and Cw13, which are outside the upper surface presence range, as clusters of non-wall points. Note that the upper surface presence range fluctuates depending on the tide level and the draft depth of the vessel, so it is set to a range with some margin. The quay wall detection unit 15 may also set a presence range (side presence range) for the sides if the presence range is limited, and classify candidate wall clusters of side points outside the side presence range as clusters of non-wall points.

[0074] Figures 15(A) and 15(B) show the procedures (b) to (d) for selecting wall candidate clusters for the top and side surfaces, respectively. Procedure (a) indicates that surface integration was performed as a preprocessing step by same-surface integration processing. First, in procedure (b), the quay detection unit 15 selects the wall candidate cluster with the lowest representative position among the wall candidate clusters on the top surface that exist within the top surface range. Here, the representative position is, for example, the centroid or geometric center point of the wall candidate cluster. Next, in procedure (c), the quay detection unit 15 excludes the side wall candidate clusters whose representative position is higher than the representative position of the wall candidate cluster corresponding to the top surface selected in procedure (b). Then, the quay detection unit 15 selects the wall candidate cluster with the closest representative position to the target vessel from among the remaining side wall candidate clusters.

[0075] According to these procedures, the quay wall detection unit 15 can accurately select a pair of candidate wall clusters corresponding to the top and side surfaces, respectively.

[0076] (8) Boundary point determination process Near the edge of a quay wall, the normal angle tends to be oblique because points on the top and side surfaces are mixed when calculating the normal. Therefore, it is difficult to accurately determine whether a point near the edge is a top surface point, a side surface point, or a non-wall point based on the normal angle. Figure 16(A) shows the classification results of points on the quay wall. In Figure 16(A), the arrows represent the direction of the normal. In the example in Figure 16(A), the point near the edge at the boundary between the side and top surfaces is determined to be a non-wall point.

[0077] Taking the above into consideration, the quay wall detection unit 15, in the boundary point determination process, re-determines whether the boundary point, which is a non-wall point located at the boundary between the top surface and the side surface, is suitable as a wall point. Figure 16(B) shows the classification result of the measured points after the boundary point determination process. In this case, the non-wall points that existed in Figure 16(A) have been appropriately reclassified as either side points or top points.

[0078] Figures 17(A) and 17(B) show an overview of the boundary point determination process. First, as shown in Figure 17(A), the quay wall detection unit 15 searches for boundary points by scanning from below along each vertical line (i.e., in the direction in which the elevation angle increases from the position where the depression angle of the lidar 3 is at its maximum). Next, as shown in Figure 17(B), the quay wall detection unit 15 calculates the distance from the searched boundary point to the nearest wall point (side point or top point), and if that distance is less than or equal to a predetermined threshold, it re-determines (classifies) the searched boundary point as a wall point. More specifically, in this case, the quay wall detection unit 15 classifies the boundary point that has been re-determined as a wall point as a side point if the nearest wall point is a side point, and as a top point if the nearest wall point is a top point. Furthermore, even if only one side of the top or side is detected, the quay wall detection unit 15 may consider several rows of points to be measured from the endpoint of the wall point as boundary points, and if the shortest distance between the boundary point and the wall point is less than or equal to a threshold, it may re-evaluate the boundary point as a wall point.

[0079] (9) Detailed processing flow Next, we will explain the detailed flow of each process described in the flowchart in Figure 3.

[0080] Figure 18 is an example of a flowchart of the normal calculation process performed in step S2 of Figure 3.

[0081] First, the quay wall detection unit 15 acquires point cloud data to be used in the normal vector calculation process (step S11). Then, the quay wall detection unit 15 executes the processing loop described in steps S12 to S16 below. In this case, the quay wall detection unit 15 sequentially selects all the points to be measured in the acquired point cloud data as points of interest, and repeatedly executes the processing loop until there are no more points to be measured that can be selected as points of interest.

[0082] First, the quay wall detection unit 15 calculates the distance x from the reference point of the lidar 3 to the point of interest (i.e., the measurement distance) (step S12). Next, the quay wall detection unit 15 calculates the normal calculation radius based on the measurement distance x (step S13). Then, the quay wall detection unit 15 searches for other points to be measured (also called "neighboring points") that are within the normal calculation radius from the point of interest (step S14). Then, the quay wall detection unit 15 determines whether the total number of points from the point of interest and neighboring points is equal to or greater than the point threshold (step S15). If the above number of points is equal to or greater than the point threshold (step S15; Yes), the quay wall detection unit 15 calculates the normal of the point of interest based on the point of interest and neighboring points (step S16). On the other hand, if the above number of points is less than the point threshold (step S15; No), the quay wall detection unit 15 does not perform step S16.

[0083] Then, after the completion of the processing loop in steps S12 to S16, the quay wall detection unit 15 outputs a point cloud with normals (normal vectors) associated with each measured point as the result of the normal calculation process (step S17).

[0084] Figure 19 is an example of a flowchart for the wall orientation determination process performed in step S3 of Figure 3.

[0085] First, the quay wall detection unit 15 acquires a point cloud with normals, which is the result of the normal calculation process, as input for the wall orientation determination process (step S21). Next, the quay wall detection unit 15 executes the processing loop from steps S22 to S26. In this case, the quay wall detection unit 15 sequentially selects all the points to be measured in the acquired point cloud data as points of interest, and repeatedly executes the processing loop until there are no more points to be measured that can be selected as points of interest.

[0086] First, the quay wall detection unit 15 determines whether the normal angle θ of the point of interest is greater than the top surface threshold (step S22). If the normal angle θ of the point of interest is greater than the top surface threshold (step S22; Yes), the quay wall detection unit 15 assigns a label (top surface label) to the point of interest to indicate that it is a top surface point (step S23). On the other hand, if the normal angle θ of the point of interest is less than or equal to the top surface threshold (step S22; No), the quay wall detection unit 15 determines whether the normal angle θ of the point of interest is less than the side surface threshold (step S24; Yes). If the normal angle θ of the point of interest is less than the side surface threshold (step S24; Yes), the quay wall detection unit 15 assigns a label (side surface label) to the point of interest to indicate that it is a side surface point (step S25). On the other hand, if the normal angle θ of the point of interest is greater than or equal to the side threshold (step S24; No), the quay wall detection unit 15 assigns a label to the point of interest indicating that it is a non-wall point (non-wall label) (step S26). The above-mentioned labels (top label, side label, non-wall label) are collectively referred to as "wall labels".

[0087] Then, after the completion of the processing loop in steps S22 to S26, the quay wall detection unit 15 outputs a point cloud with normals and wall labels associated with each measurement point as the result of the wall orientation determination process (step S27).

[0088] Figure 20 is an example of a flowchart of the distributed non-wall surface removal process and the same-surface integration process performed in steps S4 and S5 of Figure 3.

[0089] First, the quay wall detection unit 15 acquires a point cloud with normal / wall labels, which is the result of the wall orientation determination process, as input for the distributed non-wall removal process and the same-surface integration process (step S31). Then, the quay wall detection unit 15 performs clustering of the measured points that are wall points (e.g., Euclidean clustering) (step S32). Then, the quay wall detection unit 15 determines whether the number of points in each generated cluster is greater than a predetermined threshold (step S33). Then, the quay wall detection unit 15 determines that clusters with a number of points less than or equal to the predetermined threshold are clusters of non-wall points (step S33; No and step S34). In this case, the quay wall detection unit 15 changes the wall labels of each measured point in the clusters with a number of points less than or equal to the predetermined threshold to non-wall labels.

[0090] Next, the quay wall detection unit 15 separates the top surface points and side surface points and performs clustering (e.g., Euclidean clustering) on ​​each (step S35). Then, the quay wall detection unit 15 executes the processing loop from steps S36 to S45. In this case, the quay wall detection unit 15 sequentially selects the clusters generated in step S35 as clusters of interest, and repeats this processing loop until there are no more clusters to select as clusters of interest.

[0091] First, the quay detection unit 15 calculates a variance-covariance matrix for each measured point in the cluster of interest (step S36). Then, the quay detection unit 15 finds the maximum value of the diagonal elements of the variance-covariance matrix (step S37). Next, the quay detection unit 15 determines whether the maximum value obtained in step S37 is less than a predetermined threshold (step S38). If the maximum value obtained in step S37 is less than the predetermined threshold (step S38; Yes), the quay detection unit 15 determines that the cluster of interest satisfies the conditions for being a cluster of wall points and executes step S39. On the other hand, if the maximum value obtained in step S37 is greater than or equal to the predetermined threshold (step S38; No), the quay detection unit 15 determines that the cluster of interest is a cluster of non-wall points (step S40).

[0092] In step S39, the quay wall detection unit 15 calculates the representative position (representative point) and representative normal of the cluster of interest (step S39). Then, based on the representative normal, the quay wall detection unit 15 determines whether there are other clusters whose representative normal orientation (i.e., wall orientation) is similar to that of the cluster of interest (step S41). If there are other clusters whose wall orientation is similar to that of the cluster of interest (step S41; Yes), the quay wall detection unit 15 calculates the inter-face distance between the cluster of interest and the other cluster (step S42). Then, if the quay wall detection unit 15 determines that the inter-face distance is less than the distance threshold (step S43; Yes), it merges the point clouds of the cluster of interest and the other cluster (step S44), and saves the cluster representing the merged point cloud as a wall candidate cluster in memory 12 or the like (step S45). On the other hand, if the quay wall detection unit 15 determines that there are no other clusters with a similar wall orientation to the cluster of interest (step S41; No), or that the distance between surfaces is greater than or equal to a distance threshold (step S43; No), it saves the cluster of interest as a candidate wall cluster in memory 12 or the like (step S45).

[0093] Then, after the processing loop for the clusters is completed, the quay wall detection unit 15 outputs a group of candidate wall clusters as the result of the distributed non-wall surface removal process and the same-surface integration process (step S46).

[0094] Figure 21 is an example of a flowchart for the area non-wall surface removal process performed in step S6 of Figure 3.

[0095] First, the quay wall detection unit 15 acquires a group of candidate wall clusters, which are the processing results of the distributed non-wall surface removal process and the same-surface integration process, as input for the area non-wall surface removal process (step S51). Then, the quay wall detection unit 15 executes the processing loop of steps S52 to S59. In this case, the quay wall detection unit 15 sequentially selects candidate wall clusters as clusters of interest, and repeats this processing loop until there are no more candidate wall clusters to select as clusters of interest. Note that the quay wall detection unit 15 may execute steps S53 to S55 based on the first pseudo-area calculation method, or it may execute steps S57 and S58 based on the second pseudo-area calculation method.

[0096] First, the quay detection unit 15 performs principal component analysis on the cluster of interest (step S52). Then, if the first pseudo-area calculation method is adopted, the quay detection unit 15 converts the original coordinate values ​​of each measured point constituting the cluster of interest into coordinate values ​​of the first and second principal component axes (step S53). Next, the quay detection unit 15 calculates the length and width w and h of the cluster of interest, assuming it is a rectangle, from the difference between the maximum and minimum values ​​in each axial direction of the cluster of interest (step S54), and calculates the area S1 (step S55). On the other hand, if the second pseudo-area calculation method is adopted, the quay detection unit 15 calculates the standard deviation sd from the eigenvalues ​​λ of the first and second principal components (step S57), and calculates the area S2 (step S58).

[0097] The quay wall detection unit 15 then determines whether the calculated area is greater than a predetermined threshold (step S56). If the calculated area is less than or equal to the predetermined threshold (step S56; Yes), the quay wall detection unit 15 determines the target cluster to be a cluster of non-wall points (step S59).

[0098] Then, after the processing loop for the wall candidate clusters is completed, the quay wall detection unit 15 outputs the group of wall candidate clusters that were not determined to be non-wall points in step S59 as the processing result of the area non-wall removal process (step S60).

[0099] Figure 22 is an example of a flowchart of the optimal wall selection process performed in step S7 of Figure 3.

[0100] First, the quay wall detection unit 15 acquires a group of candidate wall clusters, which are the result of the area non-wall surface removal process, as input for the optimal wall surface selection process (step S61). Next, the quay wall detection unit 15 calculates the representative position of each candidate wall cluster (step S62). Then, the quay wall detection unit 15 assigns a non-wall label to each point in the candidate wall cluster whose representative position is higher than the upper limit of the upper surface existence range, and to each point in the candidate wall cluster whose representative position is lower than the lower limit (step S63).

[0101] The quay wall detection unit 15 then determines whether there is one or more candidate wall clusters consisting of upper surface points (step S64). If there is one or more candidate wall clusters consisting of upper surface points (step S64; Yes), the quay wall detection unit 15 keeps the cluster with the lowest representative position among the candidate wall clusters consisting of upper surface points and assigns non-wall labels to each point in the other clusters (step S65). On the other hand, if there is only one candidate wall cluster consisting of upper surface points (step S64; No), step S65 is not performed.

[0102] Then, the quay wall detection unit 15 assigns a non-wall label to each point in the cluster of wall candidate clusters consisting of side points that is higher (i.e., has a larger Z coordinate) than the representative point of the remaining top point wall candidate cluster (step S66). Then, the quay wall detection unit 15 leaves the cluster with the smallest absolute value of the Y coordinate (i.e., close to the origin) among the remaining side point wall candidate clusters, and assigns a non-wall label to each point in the other clusters (step S67). Then, as a result of the optimal wall selection process, the quay wall detection unit 15 outputs a point cloud with normals and wall labels associated with each measured point (step S68).

[0103] Figure 23 is an example of a flowchart of the boundary point determination process performed in step S8 of Figure 3.

[0104] First, the quay wall detection unit 15 acquires a point cloud with normal / wall labels, which is the result of the optimal wall selection process, as input for the boundary point determination process (step S71). Next, the quay wall detection unit 15 executes the processing loop from steps S72 to S75. In this case, the quay wall detection unit 15 sequentially selects all vertical lines of the point cloud data as lines of interest, and repeats the loop until there are no more vertical lines to select as lines of interest.

[0105] First, the quay wall detection unit 15 searches for non-wall points that exist between the side points and the top points (step S72). Then, for each of the searched non-wall points, the quay wall detection unit 15 calculates the distance to the nearest wall point (top point or side point) (step S73). If the quay wall detection unit 15 determines that there is a non-wall point whose calculated distance is less than a predetermined threshold (step S74; Yes), it sets the wall label of that non-wall point to the same wall label as the nearest wall point (step S75). On the other hand, if the quay wall detection unit 15 determines that there is no non-wall point whose calculated distance is less than a predetermined threshold (step S74; No), it does not execute the process in step S75.

[0106] Then, after completing the loop processing of the vertical lines, the quay wall detection unit 15 outputs a point cloud with wall labels, each of which is associated with a wall label at each measured point, as a result of the boundary point determination processing (step S76).

[0107] (10) Application examples Next, we will explain application examples (Application Example 1 to Application Example 3) that utilize the processing results of the flowchart shown in Figure 3.

[0108] Figures 24 and 25 show an overview of the first application example. The first application example is quay detection when a ship docks at a quay of the shape shown in Figure 24. In this case, the lidar is mounted on the side of the ship and illuminates in the direction of the quay. Since the lidar beam is emitted radially, the spacing between the measured points increases as the distance increases.

[0109] In the first application example, the processing of this embodiment is executed at three points: the first timing when the point cloud of the quay begins to be acquired within the lidar's field of view, the second timing when the section with protrusions on the quay wall surface is being detected, and the third timing when the vessel is close to the quay. In this case, as shown in Figure 25, when the processing of this embodiment is performed at the first timing, the normal calculation radius is increased to search for points, preventing a decrease in the number of points used for normal calculation. Also, by being able to calculate normals even from a long distance, the maximum distance at which the quay can be detected is extended. Furthermore, when the processing of this embodiment is performed at the second timing, protrusions can be removed by the normal dispersion threshold, and only surfaces with high flatness can be extracted. Furthermore, after integrating the separated surfaces, wall surface determination can be performed by area. Furthermore, when the processing of this embodiment is performed at the third timing, the normal calculation radius is reduced to search for points, so local normals can be obtained, improving calculation accuracy. In particular, performance is improved at the boundary points between the top and side surfaces where the normal direction is prone to disturbance.

[0110] Figures 26 and 27 show an overview of the second application example. The second application example takes the case where objects other than walls, such as mooring ropes and buoys, are present within the lidar's field of view. When calculating the normal vectors of these obstacles, depending on the angle, the points of these obstacles may be labeled as top or side points and may be mixed in with the wall candidates. In contrast, by executing the processing of this embodiment, as shown in Figure 27, outlier points can be removed by point filtering (dispersed non-wall removal processing, area non-wall removal processing, etc.). Furthermore, even if an obstacle that could not be removed by the point threshold is mistakenly labeled as a wall, it can be excluded from the wall candidates by determining its area. As a result, as shown in Figure 27, wall labels are applied seamlessly from the side to the top.

[0111] Figures 28 and 29 illustrate an overview of the third application example. The third application example deals with the extraction of a complex-shaped quay, where the edge of the quay is not a straight line, and multiple wall surface candidates appear due to obstacles on the quay. In this case, the surface may be incorrectly identified as the top surface due to the wake. Furthermore, considering the distance to the opposite shore, it is necessary to prioritize identifying wall surface candidate clusters located closer to the ship as wall surfaces. In contrast, by executing the processing according to this embodiment, as shown in Figure 29, it is possible to reduce instances where incorrect surfaces, such as the roofs of buildings on the quay, the sides of containers, or the wake of ships, are mistakenly identified as walls. In addition, by identifying more plausible surfaces as quays, the stability of calculating the distance to the opposite shore (rider-quay distance) using the quay detection results can be improved.

[0112] (11) Calculation of quay detection reliability The berthing parameter calculation unit 16 may calculate the reliability of the quay wall detection result (quay wall detection reliability) from the quay wall detection unit 15 for both the top and side surfaces.

[0113] Figure 30 is a table showing an example of calculating the confidence level for quay wall detection. Here, the coefficient "q3" for "detection / non-detection" is a value depending on whether the top or side surface was detected or not, the coefficient "q2" for "number of wall points" represents the size of the point cloud that has been acquired, with a larger value indicating higher confidence. The coefficient "q1" for "in / outside field of view" represents whether the quay wall is within the field of view of the lidar, with a value of 1 when the field of view margin is large and the quay wall is within the field of view, and -0.5 when there is a possibility that the quay wall extends outside the field of view. The coefficient "q0" for "normal vector variance" represents the degree of variance of the normal vector of the cluster determined to be a quay wall, with a smaller variance indicating higher confidence. And the confidence level for detecting the quay wall on the side is "q s " and the confidence level of detection of the quay on the upper surface "q u The values ​​are calculated by the weighted average of each of the coefficients mentioned above. The weight coefficients "w0" to "w3" used in this case are stored in memory 12, for example, beforehand.

[0114] As described above, the controller 13 of the information processing device 1 mainly comprises an acquisition means, a normal calculation range setting means, and a normal calculation means. The acquisition means acquires measurement data, which is a set of data representing multiple points to be measured measured by a measuring device. The normal calculation range setting means sets a normal calculation range for generating a set of points to be measured that will be used to calculate the normal of each of the points to be measured, based on the measurement distance indicated by the measurement data. The normal calculation means calculates the normal of each of the points to be measured based on the normal calculation range. In this configuration, the controller 13 can suitably calculate the normal of each of the points to be measured measured by the measuring device.

[0115] In the above-described embodiment, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a controller. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic storage medium (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage medium (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0116] The present invention has been described above with reference to the examples, but the present invention is not limited to the above examples. Various modifications to the structure and details of the present invention can be made that can be understood by a person skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents, etc., that has been cited is incorporated herein by reference. [Explanation of Symbols]

[0117] 1. Information Processing Device 2 Sensor Groups 3 Riders

Claims

1. Acquisition means for acquiring measurement data which is a set of data representing multiple points to be measured measured by a measuring device installed on a ship, A normal calculation range setting means sets a normal calculation range for generating a set of points of the measured points to be used to calculate the normal of each of the measured points, based on the measured distance indicated by the measurement data. A normal calculation means that calculates the normal of each of the points to be measured based on the normal calculation range, A quay wall determination means for determining the point to be measured that represents the quay wall based on the aforementioned normal, An information processing device having

2. The information processing apparatus according to claim 1, wherein the normal calculation range setting means increases the normal calculation range as the measurement distance increases.

3. The quay determination means is A clustering means that performs clustering of the measured points based on the normal vector, A determination means for determining whether each of the clusters of the points to be measured formed by the clustering is a candidate for the quay, based on the degree of variation of the normal within the cluster, An information processing apparatus according to claim 1 or 2, having the following features.

4. The quay determination means is A clustering means that performs clustering of the measured points based on the normal vector, A determination means that estimates a plane approximating each of the clusters of the measured points formed by the clustering, and determines whether or not the clusters need to be merged based on the distance between the planes, An information processing apparatus according to any one of claims 1 to 3.

5. The quay determination means is A clustering means that performs clustering of the measured points based on the normal vector, A determination means for determining whether each of the clusters of the points to be measured formed by the clustering is a candidate for the quay, based on the number of points to be measured forming the cluster and the area of ​​the cluster, An information processing apparatus according to any one of claims 1 to 4.

6. The quay determination means is A clustering means that performs clustering of the measured points based on the normal vector, A determination means for determining that, among the clusters of measurement points that are candidates for the upper surface of the quay formed by the aforementioned clustering, the cluster that is located within a predetermined range and at the lowest position is the cluster representing the upper surface. An information processing apparatus according to any one of claims 1 to 5.

7. The information processing device according to claim 6, wherein the determination means determines that the cluster of measurement points that are candidates for the side surface of the quay formed by the clustering, which is located at a lower position than the cluster representing the upper surface and is closest to the measuring device, is the cluster representing the side surface.

8. The quay wall determination means includes a classification means for classifying each of the points to be measured as either a wall point where the quay wall was measured or a non-wall point where something other than the quay wall was measured, based on the normal line. The information processing apparatus according to any one of claims 1 to 7, wherein the classification means reclassifies the non-wall points that exist between the wall points as wall points based on the shortest distance between the non-wall points and the wall points.

9. A control method performed by a computer, We acquire measurement data, which is a collection of data representing multiple measurement points measured by measuring devices installed on the ship. Based on the measurement distance indicated by the measurement data, a normal calculation range is set for generating a set of points of the measured points to be used for calculating the normal of each of the measured points. Based on the normal calculation range, the normals of each of the points to be measured are calculated. Based on the aforementioned normal, the measurement point representing the quay wall is determined. Control method.

10. Obtain measurement data which is a set of data representing multiple points to be measured measured by a measuring device installed on a ship, Based on the measurement distance indicated by the measurement data, a normal calculation range is set for generating a set of points of the measured points to be used for calculating the normal of each of the measured points. Based on the normal calculation range, the normals of each of the points to be measured are calculated. A program that causes a computer to perform a process to determine the point to be measured, which represents a quay, based on the aforementioned normal.

11. A storage medium storing the program described in claim 10.

Images