Information processing device, control method, program, and storage medium

Abstract

To acquire a self position and peripheral information without being affected by sunlight.SOLUTION: In an information processing device, point cloud data acquiring means acquires point cloud data generated by a measuring device provided on a ship. Solar position information acquiring means acquires solar position information regarding a position of the sun at the time that the point cloud data is acquired. Erroneous detection data extracting means extracts erroneous detection data from the point cloud data based on the solar position information.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] This disclosure relates to the processing of data measured on a vessel. [Background technology]

[0002] Conventionally, techniques have been known for estimating the self-position of a moving object by comparing (matching) shape data of surrounding objects measured using measuring devices such as laser scanners with map information in which the shapes of surrounding objects are pre-stored. For example, Patent Document 1 discloses an autonomous mobile system that determines whether a detected object in a voxel, which is divided into spaces according to a predetermined rule, is a stationary or moving object, and performs matching between map information and measurement data for voxels in which stationary objects exist. Patent Document 2 discloses a scan matching method that estimates the self-position by matching voxel data, which includes the mean vector and covariance matrix of stationary objects for each voxel, with point cloud data output by a lidar. Furthermore, Patent Document 3 describes a method for estimating the performance degradation of sensors mounted on a vehicle due to the incidence of sunlight. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International release WO2013 / 076829 [Patent Document 2] International release WO2018 / 221453 [Patent Document 3] Japanese Patent Publication No. 2019-148531 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] Recently, not only in the automotive field but also in ships, the study of driving support systems has been underway. For example, systems are being developed to support safe docking by measuring the distance to the pier, detecting obstacles near the ship to prompt avoidance, and detecting following waves for safe crossing.

[0005] The above-described driving support system measures and detects the position and surrounding information of the ship by detecting data around the ship itself. However, during the daytime on sunny days, sunlight reflected from the water surface may reach measurement devices such as lidars. In this case, when collecting data near the water surface, it will be affected by sunlight.

[0006] The present disclosure has been made to solve the above problems, and the main object is to provide an information processing device capable of acquiring its own position, surrounding information, etc. without being affected by sunlight.

Means for Solving the Problems

[0007] The invention according to the claim is an information processing device, comprising: point cloud data acquisition means for acquiring first point cloud data generated by a measurement device provided on a ship; sun position information acquisition means for acquiring sun position information at the time of acquisition of the first point cloud data; misdetection data extraction means for extracting misdetection data from the first point cloud data based on the sun position information; and following wave detection means for detecting following waves existing around the ship using second point cloud data obtained by removing the misdetection data from the first point cloud data.

[0008] Also, the invention according to the claim is a control method executed by a computer, which acquires first point cloud data generated by a measurement device provided on a ship, acquires sun position information at the time of acquisition of the first point cloud data, extracts misdetection data from the first point cloud data based on the sun position information, and detects following waves existing around the ship using second point cloud data obtained by removing the misdetection data from the first point cloud data.

[0009] Furthermore, the invention described in the claim is a program that causes a computer to perform the following processes: acquire first point cloud data generated by a measuring device installed on a ship; acquire solar position information at the time of acquisition of the first point cloud data; extract false detection data from the first point cloud data based on the solar position information; and use second point cloud data obtained by removing the false detection data from the first point cloud data to detect wake present around the ship. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of the driver assistance system. [Figure 2] This is a block diagram showing the configuration of an information processing device. [Figure 3] This figure shows the self-position that the self-position estimation unit should estimate, represented in three-dimensional Cartesian coordinates. [Figure 4] An example of a general data structure for voxel data is shown. [Figure 5] (A) An overhead view of the target vessel showing an example of the arrangement of the riders installed on the vessel. (B) A view of the target vessel from the rear, in close proximity to a structure. [Figure 6] (A) A rear view of the target vessel, clearly indicating the voxels in which voxel data exists. (B) A rear view of the target vessel, clearly indicating the dashed line representing the height position of the lidar and the dashed line representing the water surface position. (C) A diagram showing the reflected components of sunlight. [Figure 7] This diagram illustrates a method for removing the reflective component of sunlight. [Figure 8] This diagram illustrates a method for adjusting the range of removal of the reflective component of sunlight. [Figure 9] This is an example of a functional block for the self-localization unit. [Figure 10] This is an example of a flowchart illustrating the procedure for self-localization. [Figure 11] This diagram illustrates a method for detecting obstacles and undertows. [Figure 12] This is another diagram illustrating the method for detecting obstacles and undertows. [Figure 13] This is another diagram illustrating the method for detecting obstacles and undertows. [Figure 14] This is a flowchart for obstacle / undertow detection processing. [Figure 15] This shows an example of detecting obstacles and undertow waves. [Figure 16] This diagram illustrates a method for changing the search range, such as detecting obstacles. [Figure 17] This is a diagram illustrating the method for detecting piers. [Modes for carrying out the invention]

[0011] In one preferred embodiment of the present invention, the information processing device includes: point cloud data acquisition means for acquiring point cloud data generated by a measuring device installed on a ship; solar position information acquisition means for acquiring solar position information relating to the position of the sun at the time the point cloud data is acquired; and false detection data extraction means for extracting false detection data from the point cloud data based on the solar position information.

[0012] In the above-described information processing device, the point cloud data acquisition means acquires point cloud data generated by a measuring device installed on the ship. The solar position information acquisition means acquires solar position information regarding the position of the sun at the time the point cloud data is acquired. The false detection data extraction means extracts false detection data from the point cloud data based on the solar position information. This makes it possible to eliminate the influence of sunlight.

[0013] In one embodiment of the information processing device described above, the false detection data is data generated when sunlight is reflected from the water surface and incident on the measuring device, and the false detection data extraction means removes the false detection data from the point cloud data. In this embodiment, false detection data caused by components of sunlight reflected from the water surface can be removed.

[0014] In another embodiment of the information processing device described above, the false detection data extraction means estimates a solar reflection region, which is the area where sunlight is reflected from the water surface and incident on the measuring device, based on the solar position information, and removes point cloud data corresponding to the solar reflection region from the point cloud data as the false detection data. In this embodiment, false detection data can be removed based on the area where sunlight is reflected from the water surface.

[0015] In another embodiment of the information processing device described above, the solar position information includes the azimuth and elevation angle of the sun, and the false detection data extraction means estimates the solar reflection region based on the azimuth and elevation angle of the sun. In this embodiment, the region to which sunlight is reflected is estimated based on the azimuth and elevation angle of the sun.

[0016] Another embodiment of the information processing device described above includes a self-position estimation means for estimating the ship's own position using point cloud data from which the false detection data has been removed. In this embodiment, the influence of sunlight can be eliminated, and the ship's own position can be estimated with high accuracy.

[0017] Another embodiment of the information processing device described above includes a detection means for detecting at least one of obstacles and wakes present around the vessel using point cloud data from which the false detection data has been removed. In this embodiment, the effects of sunlight can be eliminated, and obstacles and wakes around the vessel can be detected with high accuracy.

[0018] Another embodiment of the information processing device described above includes a calculation means for calculating the distance to the object the vessel intends to dock at using point cloud data from which the false detection data has been removed. In this embodiment, the influence of sunlight can be eliminated, and the distance the vessel intends to dock at can be calculated with high accuracy.

[0019] Another preferred embodiment of the present invention is a control method performed by a computer, which acquires point cloud data generated by a measuring device installed on a ship, acquires solar position information relating to the position of the sun at the time the point cloud data is acquired, and removes falsely detected data from the point cloud data based on the solar position information. This makes it possible to eliminate the influence of sunlight.

[0020] Another preferred embodiment of the present invention is a program that acquires point cloud data generated by a measuring device installed on a ship, acquires solar position information relating to the position of the sun at the time the point cloud data is acquired, and causes a computer to perform a process to remove falsely detected data from the point cloud data based on the solar position information. By executing this program on a computer, the above information processing device can be realized. This program can be stored and handled on a storage medium. [Examples]

[0021] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. Note that any character with "^" or "-" above it will be referred to as "A^" or "A" in this specification for convenience. - This is represented as (where "A" is any letter).

[0022] (1) Overview of the driver assistance system Figure 1 shows a schematic configuration of the driver assistance system according to this embodiment. The driver assistance system comprises an information processing device 1 that moves together with the ship, which is a moving object, and a group of sensors 2 mounted on the ship. Hereafter, the ship that moves together with the information processing device 1 will also be referred to as the "target ship".

[0023] The information processing device 1 is electrically connected to the sensor group 2 and estimates the position of the target vessel (also called "self-position") based on the outputs of the various sensors included in the sensor group 2. Based on the self-position estimation result, the information processing device 1 provides driving support such as automatic driving control of the target vessel. Driving support includes docking support such as automatic docking. Here, "docking" includes not only docking the target vessel at a quay but also docking the target vessel at a structure such as a pier. The information processing device 1 may be a navigation device installed on the target vessel or an electronic control device built into the vessel.

[0024] Furthermore, the information processing device 1 stores a map database (DB:DataBase) 10 containing voxel data "VD". Voxel data VD is data that records position information of stationary structures for each voxel, which represents a cube (normal grid) that is the smallest unit of three-dimensional space. Voxel data VD includes data that represents the measured point cloud data of stationary structures within each voxel using a normal distribution, and is used for scan matching using NDT (Normal Distributions Transform), as described later. The information processing device 1 uses NDT scan matching to estimate, for example, the position on the plane, height position, yaw angle, pitch angle, and roll angle of the target vessel. Unless otherwise specified, the self-position is assumed to include attitude angles such as the yaw angle of the target vessel.

[0025] Sensor group 2 includes various external and internal sensors installed on the target vessel. In this embodiment, sensor group 2 includes a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3, a speed sensor 4 for detecting the speed of the target vessel, a GPS (Global Positioning Satellite) receiver 5, and an inertial measurement unit (IMU) 6 for measuring the acceleration and angular velocity of the target vessel in three axes.

[0026] The lidar 3 discretely measures the distance to an object in the external environment by emitting a pulsed laser within a predetermined angular range in the horizontal and vertical directions, and generates three-dimensional point cloud data indicating the position of the object. In this case, the lidar 3 has an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data (points that constitute the point cloud data, hereafter referred to as "measurement points") based on the light receiving signal output by the light receiving unit. The measurement points are generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light specified based on the light receiving signal described above. Generally, the accuracy of the lidar's distance measurement is higher the closer the distance to the object, and lower the accuracy the farther the distance. The lidar 3 is an example of a "measurement device" in the present invention. The speed sensor 4 may be, for example, a speedometer using Doppler or a speedometer using GNSS.

[0027] Furthermore, the sensor group 2 may have a receiver that generates positioning results from a GNSS other than GPS, instead of the GPS receiver 5.

[0028] (2) Configuration of an information processing device Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly consists of an interface 11, a memory 12, and a controller 13. Each of these elements is interconnected via a bus line.

[0029] Interface 11 performs interface operations related to the exchange of data between the information processing device 1 and external devices. In this embodiment, Interface 11 acquires output data from each sensor in the sensor group 2, such as the lidar 3, speed sensor 4, GPS receiver 5, and IMU 6, and supplies it to the controller 13. Interface 11 also supplies signals related to the control of the target vessel, generated by the controller 13, to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel includes a drive source such as an engine or electric motor, a propeller that generates thrust in the direction of travel based on the driving force of the drive source, a thruster that generates thrust in the lateral direction based on the driving force of the drive source, and a rudder, etc., which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation such as automatic docking, Interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, Interface 11 supplies control signals generated by the controller 13 to the electronic control device. Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices via cables, etc. Interface 11 may also perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0030] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0031] Memory 12 also stores map DB 10, which includes voxel data VD. In addition to voxel data VD, map DB 10 includes, for example, information about berthing locations (including shores and piers) and information about waterways to which ships can move. Map DB 10 may also be stored in an external storage device of the information processing device 1, such as a hard disk connected to the information processing device 1 via interface 11. The storage device may be a server device that communicates with the information processing device 1. The storage device may also consist of multiple devices. Map DB 10 may also be updated periodically. In this case, for example, controller 13 receives partial map information about the area to which its own position belongs from the server device that manages map information via interface 11 and reflects it in map DB 10.

[0032] In addition to the map DB 10, the memory 12 also stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, the memory 12 stores information used to set the downsampling size when downsampling is performed on the point cloud data obtained when the lidar 3 performs one cycle of scanning.

[0033] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to self-localization and driving assistance by executing programs stored in memory 12, etc.

[0034] Furthermore, the controller 13 functionally includes a self-position estimation unit 15, an obstacle / wake detection unit 16, and a pier detection unit 17. The controller 13 also functions as a "point cloud data acquisition means," a "solar position information acquisition means," a "false detection data extraction means," a "detection means," a "calculation means," and a computer that executes programs.

[0035] The self-position estimation unit 15 estimates its own position by performing NDT-based scan matching (NDT scan matching) based on point cloud data based on the output of the LIDA 3 and voxel data VD corresponding to the voxel to which the point cloud data belongs. Here, the point cloud data to be processed by the self-position estimation unit 15 may be point cloud data generated by the LIDA 3, or it may be point cloud data after downsampling the point cloud data.

[0036] The obstacle / wake detection unit 16 uses the point cloud data output by the LIDA 3 to detect obstacles and wakes around the vessel. The pier detection unit 17 uses the point cloud data output by the LIDA 3 to detect the pier where the vessel intends to dock and calculates the distance to the pier.

[0037] (3) NDT Scan Matching Next, we will explain the position estimation based on NDT scan matching performed by the self-position estimation unit 15.

[0038] Figure 3 shows the self-position to be estimated by the self-position estimation unit 15, represented in three-dimensional Cartesian coordinates. As shown in Figure 3, the self-position on a plane defined on the three-dimensional Cartesian coordinate system (x, y, z) is represented by the coordinates "(x, y, z)", the roll angle "φ", the pitch angle "θ", and the yaw angle (direction) "ψ". Here, the roll angle φ is defined as the rotation angle around the direction of travel of the target vessel, the pitch angle θ is the elevation angle of the direction of travel of the target vessel relative to the xy plane, and the yaw angle ψ is defined as the angle between the direction of travel of the target vessel and the x-axis. The coordinates (x, y, z) are, for example, absolute positions corresponding to a combination of latitude, longitude, and altitude, or world coordinates indicating a position with a predetermined point as the origin. The self-position estimation unit 15 then performs self-position estimation using these x, y, z, φ, θ, and ψ as estimation parameters.

[0039] Next, we will explain the voxel data VD used for NDT scan matching. The voxel data VD includes data representing the measured point cloud data of stationary structures within each voxel according to a normal distribution.

[0040] Figure 4 shows an example of the schematic data structure of the voxel data VD. The voxel data VD includes information on parameters when representing a point cloud within a voxel by a normal distribution. In this embodiment, as shown in FIG. 4, it includes a voxel ID, voxel coordinates, a mean vector, and a covariance matrix.

[0041] The "voxel coordinates" indicate the absolute three-dimensional coordinates of a reference position such as the center position of each voxel. Each voxel is a cube obtained by dividing space into a grid pattern, and since its shape and size are determined in advance, it is possible to specify the space of each voxel using the voxel coordinates. The voxel coordinates may be used as the voxel ID.

[0042] The "mean vector" and "covariance matrix" indicate the mean vector and covariance matrix corresponding to the parameters when representing the point cloud within the target voxel by a normal distribution. Incidentally, if the coordinates of an arbitrary point "i" within an arbitrary voxel "n" are X n (i)=[x n (i), y n (i), z n (i)]<00岁0006> is defined, and if the number of point clouds within voxel n is "N n ", then the mean vector "μ n " and covariance matrix "V n " within voxel n are represented by the following equations (1) and (2), respectively.

[0043]

Equation

[0044]

Equation

[0045] Next, an overview of NDT scan matching using the voxel data VD will be described.

[0046] Scan matching using NDT for ships involves estimation parameters that include the amount of movement in the horizontal plane (here, x and y coordinates) and the orientation of the ship. P=[t x t y t z t φ t θ t ψ ] T This will lead to the estimation of "t x " is the amount of movement in the x direction, "t y " is the amount of movement in the y direction, "t z " is the amount of movement in the z direction, "t φ " is the roll angle, "t θ " is the pitch angle, "t ψ The symbol " indicates the yaw angle.

[0047] Also, the coordinates of the point cloud data output by the LIDA3 are X L (j) = [x n (j), y n (j), z n (j)] T Therefore, X L (j) Average value "L' n This can be expressed by the following equation (3).

[0048]

number

[0049] Then, the self-position estimation unit 15 searches for voxel data VD that corresponds to the point cloud data converted to an absolute coordinate system (also called the "world coordinate system") which is the same coordinate system as the map DB10, and the mean vector μ contained in that voxel data VD. n and the covariance matrix V n Using this, the evaluation function value of voxel n (also called the "individual evaluation function value") "E nThe self-position estimation unit 15 calculates the individual evaluation function value E of voxel n based on the following equation (4). n Calculate.

[0050]

number

[0051] The self-localization unit 15 then calculates an overall evaluation function value (also called a "score value") "E(k)" for all voxels subject to matching, as shown by the following equation (5). The score value E serves as an indicator of the goodness of fit of the matching.

[0052]

number

[0053]

number

[0054] (4) Self-location estimation (4-1) Removal of water surface reflection data The self-position estimation unit 15 performs a water surface reflection data removal process to remove data generated by the reception of light emitted by the lidar 3 reflected from the water surface (also called "water surface reflection data") from the point cloud data used to calculate the NDT position. As a result, the self-position estimation unit 15 considers the water surface reflection data, which represents an inaccurate location different from the actual location, as noise data (also called false detection data or false alarm data), and removes it appropriately from the point cloud data used for position estimation, thereby improving the accuracy of processing such as position estimation.

[0055] First, we will explain how water surface reflection data can be misdetected, referring to Figures 5(A) and 5(B).

[0056] Figure 5(A) is an overhead view of a target vessel showing an example of the arrangement of the lidars 3 installed on the vessel. In Figure 5(A), four lidars 3 are installed on the target vessel as an example, and each lidar 3 emits pulsed light within a predetermined angular range while changing its angle. Note that the number and arrangement of lidars 3 are not limited to the example shown in Figure 5(A).

[0057] Figure 5(B) is a view of the target vessel in close proximity to structure 50, observed from the rear. Of the emitted light, the downward-facing portion hardly returns to the lidar 3 if there are no objects on the water surface, but some of the light returns to the lidar 3 and is detected as data. In Figure 5(B), the measurement positions "P1" to "P10" represented by each measurement point that constitutes the trajectory of the light emitted by the lidar 3 and the point cloud data generated by the lidar 3 are clearly indicated. Furthermore, positions "Pa8" to "Pa10" represent the positions where the light from the lidar 3 used for measurement positions P8 to P10 was reflected off the water surface and irradiated onto structure 50.

[0058] As mentioned above, some of the light emitted from the downward-facing Rider 3 may reflect off the water surface and return to the Rider 3. In this case, (a) A pattern in which light reflected from the water surface returns directly to the RIDA3 and is received by the RIDA3. (b) A pattern in which light reflected from the water surface hits a structure, some of the scattered light is reflected again from the water surface, and is received by the LIDA3. There are two patterns. The point cloud data generated by these patterns using LIDA3 becomes the water surface reflection data.

[0059] The measurement positions P1 to P3 in Figure 5(B) correspond to the positions measured based on pattern (a). In this case, the lidar 3 generates water surface reflection data that makes it appear as if an object is present at the water surface positions corresponding to the measurement positions P1 to P3.

[0060] Furthermore, measurement positions P8 to P10 are positions measured by the reflection of light from LiDAR 3 at positions Pa8 to Pa10 of structure 50 and returning to LiDAR 3, respectively, and correspond to positions measured based on pattern (b). In this case, LiDAR 3 cannot recognize that the light is reflected from the water surface, so LiDAR 3 generates inaccurate water surface reflection data that makes it appear as if there is an object below the water surface.

[0061] On the other hand, measurement positions P5 to P7 are positions measured by the lidar 3 receiving a portion of the scattered light that was directly shone on the structure 50 without being reflected by the water surface, and correspond to the surface position of the structure 50. Therefore, the point cloud data representing measurement positions P5 to P7 is not false detection data, but rather data that accurately represents the actual position of the object being measured.

[0062] Next, the correspondence between water surface reflection data and voxel data will be explained with reference to Figure 6(A). Figure 6(A) is a diagram that clearly indicates the voxels in Figure 5(B) where voxel data exists.

[0063] Here, the water surface reflection data corresponding to measurement positions P1 to P3 generated by pattern (a) is not originally intended for NDT scan matching. However, since there is no voxel data at these positions, the target voxels cannot be found during the mapping process, and the data is not used to calculate the NDT position. Thus, although the water surface reflection data based on pattern (a) does not affect the performance of NDT scan matching, it results in unnecessary processing during the mapping process and other steps in NDT scan matching.

[0064] On the other hand, the water surface reflection data based on pattern (b) corresponding to measurement positions P8 to P10 may be associated with voxels that have voxel data nearby, and may be used in the calculation of the NDT position. Therefore, the water surface reflection data based on pattern (b) can cause a deterioration in NDT matching performance. Specifically, while it would be sufficient to calculate the NDT position based on the point cloud data corresponding to measurement positions P4 to P7, using the water surface reflection data corresponding to measurement positions P8 to P10 means that it will be associated with nearby voxels and used in the calculation, which will be a source of error in the NDT position.

[0065] Furthermore, voxel data corresponding to coastal areas is generally created at low tide. This is done to create voxels as close to the seawall as possible. As a result, regardless of the tide level during operation, there will always be voxel data that corresponds to the point cloud data.

[0066] Next, an overview of the water surface reflection data removal process will be explained with reference to Figure 6(B). Figure 6(B) is a diagram in which the dashed line 51 representing the height position of the lidar 3 and the dashed line 52 representing the water surface position are clearly indicated in Figure 6(A).

[0067] The self-position estimation unit 15 acquires multiple water surface reflection data based on pattern (a) (data corresponding to measurement positions P1 to P3 in Figure 6(B)) and calculates the height from the lidar 3 to the water surface (also called "water surface height Hw") by averaging its z component. The self-position estimation unit 15 then considers the point cloud data near the coast that is located below the water surface height Hw calculated above (data corresponding to measurement positions P8 to P10 in Figure 6(B)) as water surface reflection data (specifically, water surface reflection data based on pattern (b)). The self-position estimation unit 15 then determines that the point cloud data considered as water surface reflection data is falsely detected data and excludes it from NDT scan matching. In this case, the self-position estimation unit 15 also determines that the water surface reflection data based on pattern (a) used to calculate the water surface height Hw is falsely detected data and excludes it from NDT scan matching.

[0068] As a result, the self-position estimation unit 15 can avoid using inaccurate data for matching during NDT position calculation, and can perform NDT position calculation using only correct data. Furthermore, since the ship floats on the water surface, the amount it sinks into the water changes depending on the number of passengers and the amount of cargo, and even if the water surface height Hw changes, the self-position estimation unit 15 can accurately calculate the water surface height Hw.

[0069] (4-2) Removal of reflective components of sunlight Next, we will explain the removal of the reflected sunlight component. As mentioned above, the self-position estimation unit 15 acquires multiple water surface reflection data based on pattern (a) (data corresponding to measurement positions P1 to P3 in Figure 6(B)) and calculates the water surface height Hw from the lidar 3 to the water surface by averaging its z component. However, as shown in Figure 7(A), on sunny days, sunlight reflected from the water surface may enter the lidar 3. This is because the wavelength of the laser used in the lidar 3 is also included in sunlight. In this case, the lidar 3 receives the reflected sunlight component mixed with the reflected light that returns to the lidar 3 after its own emitted light is reflected from the water surface, so it is affected by sunlight when collecting data near the water surface. Since the lidar 3 cannot distinguish whether what it receives is its own emitted light that has returned or the reflected sunlight component, data at various distances will be measured depending on the timing of sunlight reception.

[0070] Figure 6(C) shows the water surface reflection component of sunlight as indicated by the dashed line 70. The water surface reflection component of sunlight includes parts that are above the water surface position. Therefore, simply estimating the water surface position and removing data below the water surface height Hw is not sufficient to remove the water surface reflection component of sunlight.

[0071] Figure 7(B) shows an example of the reflected component of sunlight. LIDA3 detects point cloud data around the target vessel, and on a clear day, the point cloud data detected by LIDA3 contains a region 71 that contains a large amount of the reflected component of sunlight (hereinafter also referred to as the "sunlight reflection region"). Point cloud data within the sunlight reflection region 71 cannot be used as data indicating the correct water surface position because it contains a large amount of the reflected component of sunlight.

[0072] Therefore, the self-position estimation unit 15 extracts and removes point cloud data containing the sunlight reflection component from the point cloud data measured by the lidar 3 as sunlight reflection data (false detection data). Figure 7(C) schematically shows the method for extracting point cloud data containing the sunlight reflection component. The position of the sun (azimuth and elevation angle) can be determined based on the date and time. Therefore, the self-position estimation unit 15 obtains the azimuth and elevation angle of the sun based on the current date and time. Point cloud data measured near the sun's azimuth and elevation angle θ as seen from the ship contains a large amount of sunlight reflection component. Therefore, the self-position estimation unit 15 estimates the area on the water surface where sunlight of that azimuth and elevation angle is reflected as the sunlight reflection region 71. Then, the self-position estimation unit 15 sets the sunlight reflection region 71 as the removal range and removes point cloud data detected within the removal range as sunlight reflection data. Note that the point cloud data detected in the sunlight reflection region 71 may actually include reflection data of the light emitted by the lidar 3 rather than sunlight reflection data, but it is difficult to distinguish between them. Therefore, in this embodiment, all point cloud data detected in the sunlight reflection region 71 is removed as false detection data. Then, the self-position estimation unit 15 uses the point cloud data after the false detection data has been removed to calculate the aforementioned water surface position (water surface height Hw).

[0073] The self-position estimation unit 15 changes the removal range according to the elevation angle of the sun. As shown in Figure 8(A), when the elevation angle of the sun is large, the sunlight reflection area 71 narrows in the direction of the sun's azimuth, so the self-position estimation unit 15 narrows the removal range. On the other hand, as shown in Figure 8(B), when the elevation angle of the sun is small, the sunlight reflection area 71 widens in the direction of the sun's azimuth, so the self-position estimation unit 15 widens the removal range.

[0074] Furthermore, the self-position estimation unit 15 may change its removal range according to the wave height around the vessel. As shown in Figure 8(C), when the wave height is small, the water surface is flat, so the self-position estimation unit 15 narrows the removal range. On the other hand, as shown in Figure 8(D), when the wave height is high, the reflected component of sunlight is received over a wide area, so the self-position estimation unit 15 widens the removal range. The magnitude of the wave height can be determined by calculating the dispersion of the reflected component of the water surface when estimating the water surface position.

[0075] (4-3) Functional Blocks Figure 9 shows an example of the functional blocks of the self-position estimation unit 15. As shown in Figure 9, the self-position estimation unit 15 includes a dead reckoning block 21, a coordinate transformation block 22, a sunlight reflection data removal block 23, a water surface position data extraction block 24, a height direction processing block 25, a water surface position calculation block 26, a filter block 27, a water surface reflection data removal block 28, and an NDT position calculation block 29.

[0076] The dead reckoning block 21 calculates the DR position based on the signals output by the sensor group 2. Specifically, the dead reckoning block 21 uses the movement speed and angular velocity of the target vessel, based on the outputs of the speed sensor 4 and IMU 6, etc., to determine the distance traveled and the change in direction from the previous time. Then, the dead reckoning block 21 calculates the estimated self-position X for time k-1, which is the previous processing time relative to the current processing time k. ^ For (k-1), the DR position X at time k is obtained by adding the distance traveled and the change in direction from the previous time. DR Calculate (k). This DR position X DR (k) is the self-position at time k based on dead reckoning, and the predicted self-position X - This corresponds to (k). Note that this is immediately after the start of self-localization, and the estimated self-localization X at time k-1. ^ If (k-1) does not exist, the dead reckoning block 21, for example, based on the signal output by the GPS receiver 5, determines the DR position X DR Determine (k).

[0077] The coordinate transformation block 22 transforms the point cloud data based on the output of the lidar 3 into the world coordinate system, which is the same coordinate system as the map DB 10. In this case, the coordinate transformation block 22 performs the coordinate transformation of the point cloud data at time k, for example, based on the predicted self-position output by the dead reckoning block 21 at time k. Processes for transforming point cloud data in a coordinate system based on a lidar installed on a moving object (a ship in this embodiment) into the coordinate system of the moving object, and processes for transforming from the coordinate system of the moving object into the world coordinate system, are disclosed, for example, in International Publication WO2019 / 188745.

[0078] The solar reflection data removal block 23 obtains the sun's azimuth and elevation angle based on the current date and time, estimates the solar reflection region 71 based on that azimuth and elevation angle, and identifies the point cloud data corresponding to the solar reflection region 71 as solar reflection data. Then, the solar reflection data removal block 23 removes the solar reflection data from the point cloud data output by the coordinate transformation block 22. This makes it possible to eliminate the influence of sunlight.

[0079] The water surface position data extraction block 24 extracts sample water surface position data (also called "water surface position data") used to calculate the water surface position (water surface height Hw) from the point cloud data output by the sunlight reflection data removal block 23. In this case, the water surface position data extraction block 24 obtains information on the height of the water surface position from a server device that provides information on the absolute height (elevation) of the water surface position, and extracts measurement points that represent heights within a predetermined range from the absolute height of the water surface position identified based on the obtained information, as the water surface position data. For example, the water surface position can be determined by obtaining tidal level information provided by the Japan Meteorological Agency, etc.

[0080] Next, the water surface position data extraction block 24 extracts point cloud data measured at a position close to the target vessel. A position close to the target vessel refers to a position within a predetermined range from the target vessel's own position. By using point cloud data measured at a position close to the target vessel, it becomes possible to calculate the water surface height with high accuracy using point cloud data (water surface reflection data based on the aforementioned pattern (a)) obtained by measuring light that has directly reflected from the water surface and returned directly to the lidar 3. The water surface position data extraction block 24 then outputs the extracted point cloud data as water surface position data to the height direction processing block 25.

[0081] The height direction processing block 25 calculates the average and variance values ​​of the z-coordinate values ​​of the positions in the z-direction represented by each measurement point extracted by the water surface position data extraction block 24. Then, the water surface position calculation block 26 calculates the current water surface position based on the average and variance values ​​calculated by the height direction processing block 25.

[0082] Specifically, the water surface position calculation block 26 updates the water surface position using the average value calculated by the height direction processing block 25 if the variance value calculated by the height direction processing block 25 is smaller than a predetermined threshold. On the other hand, if the variance value calculated by the height direction processing block 25 is greater than or equal to a predetermined threshold, the water surface position calculation block 26 determines that the reliability of the average value calculated by the height direction processing block 25 is low and does not update the water surface position, instead using the previously calculated water surface position.

[0083] Here, we will provide a supplementary explanation of the processing of the height direction processing block 25 and the water surface position calculation block 26. Generally, when the variance of the z-coordinate values ​​represented by each measurement point extracted by the water surface position data extraction block 24 is large, it is highly likely that the waves have become larger due to the passage of other ships or that floating objects are present. Taking the above into consideration, the water surface position calculation block 26 determines whether or not it is necessary to update the water surface position based on the variance value calculated by the height direction processing block 25.

[0084] The filter block 27 determines the current water surface position by averaging or filtering the water surface position calculated by the water surface position calculation block 26 with previously calculated water surface positions. In this case, for example, the filter block 27 calculates the current water surface position as the average value (which may be a weighted average considering proximity to the current time) of the most recent predetermined water surface position calculated by the water surface position calculation block 26 in the past and the water surface position calculated by the water surface position calculation block 26 at the current processing time. In another example, the filter block 27 determines that the reliability of the water surface position calculated by the water surface position calculation block 26 at the current processing time is low if the difference between the average of the most recent predetermined water surface positions calculated by the water surface position calculation block 26 in the past and the water surface position calculated by the water surface position calculation block 26 at the current processing time is greater than or equal to a predetermined value. In this case, the filter block 27 does not adopt the water surface position calculated by the water surface position calculation block 26 at the current processing time and considers the water surface position from the previous time as the current water surface position. In this way, the filter block 27 can determine an appropriate water surface position based on the latest predetermined number of water surface positions calculated by the water surface position calculation block 26.

[0085] Furthermore, the water surface position calculation block 26 or the filter block 27 may be set to an offset that is a predetermined distance above the set water surface position in order to reliably remove false detection data. This offset is stored in advance in, for example, memory 12. In this case, the water surface position calculation block 26 or the filter block 27 determines the water surface position to be output to the next processing block by adding the offset stored in advance in memory 12 to the water surface position determined by the above calculation.

[0086] Note that the presence of filter block 27 is not mandatory and it does not need to be provided.

[0087] The water surface reflection data removal block 28 determines each measurement point in the point cloud data representing a position below the water surface position supplied by the filter block 27 (including the same height, the same applies hereinafter) (i.e., a position with the same or lower z-coordinate value) to be water surface reflection data, and removes the determined water surface reflection data from the point cloud data. Furthermore, the water surface reflection data removal block 28 may also consider the water surface position data extracted by the water surface position data extraction block 24 as water surface reflection data and exclude it from the point cloud data to be supplied to the NDT position calculation block 29. This allows the water surface reflection data removal block 28 to accurately identify the water surface reflection data corresponding to pattern (a) and pattern (b), respectively, and exclude it from the data used by the NDT position calculation block 29.

[0088] The NDT position calculation block 29 calculates the NDT position based on the point cloud data after water surface reflection data removal supplied from the water surface reflection data removal block 28. In this case, the NDT position calculation block 29 matches the point cloud data in the world coordinate system supplied from the water surface reflection data removal block 28 with the voxel data VD represented in the same world coordinate system to establish a correspondence between the point cloud data and the voxels. Then, for each voxel that has been matched with the point cloud data, the NDT position calculation block 29 calculates an individual evaluation function value based on equation (4) and calculates the estimated parameter P that maximizes the score value E(k) based on equation (5). Then, based on equation (6), the NDT position calculation block 29 calculates the DR position X output by the dead reckoning block 21. DR The NDT position X at time k is determined by applying the estimated parameter P obtained at time k to (k). NDT (k) is calculated. The NDT position calculation block 29 calculates the NDT position X NDT (k) is output as the estimated self-position X^(k) at time k. In this case, the NDT position calculation block 29 calculates the NDT position X based on the point cloud data from which the water surface reflection data, which is false detection data, has been removed. NDT (k) can be calculated with high accuracy. In addition, in the correspondence between point cloud data and voxels, water surface reflection data, which is falsely detected data, is excluded from the correspondence, thus reducing the processing load.

[0089] Generally, ships experience motion, and fluctuations in height and attitude can cause errors when calculating the water surface position. Therefore, in Figure 9, the point cloud data generated by LIDA3 is converted to the world coordinate system using the coordinate transformation block 22, then the sunlight reflection data is removed using the sunlight reflection data removal block 23, and finally the water surface position data is extracted using the water surface position data extraction block 24. Alternatively, the sunlight reflection data and water surface reflection data can be removed from the point cloud data generated by LIDA3 without performing a coordinate transformation to the world coordinate system. In that case, the point cloud data generated by LIDA3 is input to the sunlight reflection data removal block 23 without going through the coordinate transformation block 22. After that, the water surface reflection data is removed via the water surface position data extraction block 24, height direction processing block 25, water surface position calculation block 26, filter block 27, and water surface reflection data removal block 28, and then input to the coordinate transformation block 22 to convert to the world coordinate system. In this case, the effect of ship movement can be reduced by setting a wider range for removing sunlight and by lengthening the filter time constant of filter block 27.

[0090] (4-4) Processing Flow Figure 10 is an example flowchart showing the procedure for the self-position estimation process performed by the self-position estimation unit 15 of the information processing device 1. The self-position estimation unit 15 starts the process shown in the flowchart of Figure 10 when it becomes necessary to perform self-position estimation, such as when the power is turned on.

[0091] First, immediately after the start of the self-position estimation process, the self-position estimation unit 15 calculates a predicted self-position from the GPS positioning result based on the data output by the GPS receiver 5 (step S11).

[0092] Next, the self-position estimation unit 15 performs dead reckoning based on the movement speed and angular velocity of the target vessel detected by the speed sensor 4 and IMU 6, etc., and the previously estimated self-position, to determine the predicted self-position, DR position X DRThe calculation is performed (step S12). Next, the self-localization unit 15 converts the point cloud data based on the output of the lidar 3 into point cloud data in the world coordinate system (step S13).

[0093] Next, the self-position estimation unit 15 acquires the sun's azimuth and elevation angle based on the date and time, as described above, to estimate the sunlight reflection region 71, and removes the point cloud data corresponding to the sunlight reflection region 71 from the point cloud data converted in step S13 as sunlight reflection data (step S14). Next, it extracts water surface position data from the point cloud data after removing the sunlight reflection data (step S15). Then, the self-position estimation unit 15 collects water surface position data for multiple frames (i.e., water surface position data obtained by scanning over the most recent multiple cycles of the Lidar 3), and when the number of water surface position data (number of measurement points) reaches a predetermined number of data, it calculates the mean and variance in the z direction (step S16). If the number of water surface position data does not reach a predetermined number of data, for example, the self-position estimation unit 15 considers the water surface position determined in step S18 immediately before as the current water surface position and proceeds to step S19.

[0094] Next, the self-position estimation unit 15 determines whether the variance value calculated in step S16 is less than a predetermined threshold (step S17). If the variance value calculated in step S16 is less than the predetermined threshold (step S17; Yes), the self-position estimation unit 15 updates the water surface position using the average value calculated in step S16 (step S18). On the other hand, if the variance value calculated in step S16 is greater than or equal to the predetermined threshold (step S17; No), the self-position estimation unit 15 proceeds to step S19 without updating the water surface position. The self-position estimation unit 15 may further perform averaging or filtering processing as performed by the filter block 27 in Figure 9 to determine the final water surface position.

[0095] The self-position estimation unit 15 then determines that data located below the determined water surface position is water surface reflection data (step S19). In this case, the self-position estimation unit 15 preferably also determines that the water surface position data extracted in step S15 is water surface reflection data. The self-position estimation unit 15 then excludes the data determined to be water surface reflection data from the point cloud data to be processed by NDT (step S20).

[0096] Then, the self-position estimation unit 15 uses the DR position X calculated in step S12. DR Using this as the initial value, the NDT matching process is performed after the water surface reflection data is excluded in step S20, thereby determining the NDT position X which will be the estimated self-position X^. NDT The self-position estimation unit 15 may also perform downsampling on the point cloud data after the water surface reflection data has been excluded, immediately before step S21.

[0097] Next, the self-position estimation unit 15 determines whether or not to terminate the self-position estimation process (step S22). If the self-position estimation unit 15 determines that the self-position estimation process should be terminated (step S22; Yes), it terminates the process in the flowchart. On the other hand, if the self-position estimation unit 15 decides to continue the self-position estimation process (step S22; No), it returns to step S12 and uses the estimated self-position X^ calculated in step S21 to estimate the self-position at the next processing time.

[0098] (4-5) Variations The following describes suitable modifications of the above-described embodiments. These modifications may be applied in combination to these embodiments.

[0099] (Variation 1) The method for calculating the water surface height Hw is not limited to a calculation method based on the mean (and variance) of the water surface position data in the z direction.

[0100] For example, the information processing device 1 receives an image of the target vessel from an external device (e.g., a camera installed on the shore), analyzes the image to recognize the relative water surface position to the target vessel, and calculates the water surface height Hw based on the recognized water surface position and the installation position of the lidar 3 on the target vessel. In this case, the information processing device 1 may perform the above-mentioned recognition of the water surface position using an inference model that outputs an inference result regarding the water surface position when an image of the vessel is input. This inference model is, for example, a model that has been pre-trained based on deep learning, and the trained parameters are pre-stored in memory 12, etc. The installation position of the lidar 3 on the target vessel is, for example, pre-stored in memory 12, etc. In another example, if a sensor for detecting the relative water surface position to the target vessel is installed on the target vessel, the information processing device 1 may calculate the water surface height Hw by recognizing the water surface position based on the output of the sensor.

[0101] (Modification 2) In addition to removing water surface reflection data from the point cloud data, the information processing device 1 may also exclude voxel data VD of voxels located below the water surface (in the height direction) from the target of correspondence with the point cloud data in NDT scan matching.

[0102] Here, voxels located below the estimated water surface are submerged and therefore not included in the self-position estimation. Taking the above into consideration, in this modified example, the information processing device 1 calculates the NDT position by excluding (i.e., treating as nonexistent) the voxel data VD of the above voxels from the target of correspondence with point cloud data. As a result, the information processing device 1 can eliminate unnecessary processing when performing correspondence between point cloud data and voxels, and can effectively suppress a decrease in position estimation accuracy caused by errors in correspondence.

[0103] (Variation 3) Voxel data VD is not limited to a data structure that includes a mean vector and a covariance matrix, as shown in Figure 4. For example, voxel data VD may include the point cloud data used to calculate the mean vector and covariance matrix.

[0104] (Modification 4) The use of the point cloud data, from which water surface reflection data has been removed as false detection data, is not limited to position estimation. Instead, the information processing device 1 may be used for various purposes such as detecting docking locations in docking support such as automatic docking, and detecting obstacles. Even in this case, the information processing device 1 can suitably remove false detection data and perform various processes using the point cloud data with high accuracy.

[0105] (Variation 5) In the embodiments described above, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a controller. Non-transitory computer-readable mediums include various types of tangible storage mediums. Examples of non-transitory computer-readable mediums include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0106] (5) Obstacle / wake detection Next, the detection of obstacles and wakes by the obstacle / wake detection unit 16 will be explained. The obstacle / wake detection unit 16 uses the water surface height calculated by the self-position estimation unit 15 to detect obstacles and wakes. When there are obstacles near a vessel, it is necessary to navigate while avoiding collision or contact with the obstacle. Obstacles include, for example, other ships, pilings, bridge piers, buoys, nets, and debris. Also, if there is a wake from another ship nearby, caution is required when navigating. Therefore, the obstacle / wake detection unit 16 uses the water surface height to detect the presence of obstacles or wakes near the vessel.

[0107] Specifically, as shown in Figure 11(A), the obstacle / wake detection unit 16 acquires point cloud data belonging to the search range R set around the vessel and detects clusters of point cloud data by clustering based on the distance between points (Euclidean clustering). As shown in Figure 11(B), if there are no obstacles or wakes near the vessel, the point cloud data obtained near the vessel will not contain any clusters of point cloud data. On the other hand, as shown in Figure 12(A), obstacles can be detected as clusters of point cloud data. Also, as shown in Figure 12(B), wakes can be detected as linear clusters of point cloud data. Therefore, the obstacle / wake detection unit 16 can detect obstacles and wakes by detecting clusters of point cloud data obtained around the vessel.

[0108] When acquiring point cloud data around a vessel, the obstacle / wake detection unit 16 determines the search range based on the vessel's own position and water surface height Hw. As shown in Figures 11(A) and 11(B), the search range R is defined by a planar range Rp on the water surface and a height range Rh. The planar range Rp is a range in the xy plane and can be, for example, a circular or polygonal range centered on the vessel's own position. Figure 11(B) shows an example of a circular planar range Rp. The height range Rh is a range in the z-axis direction and can be a range of a predetermined distance vertically centered on the water surface height Hw.

[0109] When detecting obstacles and wakes, it is necessary to eliminate the influence of the reflected component of sunlight. As shown in Figure 13(A), on sunny days, sunlight is reflected from the water surface and incident on the lidar 3. In Figure 13(A), on the left side of the ship, the lidar 3 is not affected by sunlight and receives reflected light from the obstacle (buoy) 72. On the other hand, on the right side of the ship, the lidar 3 receives the reflected component of sunlight mixed with the reflected light from the water surface of the outgoing light. In this case, as shown in Figure 13(B), the obstacle / wake detection unit 16 can correctly detect the buoy 72. However, because the reflected component of sunlight is detected in the sunlight reflection region 71, the obstacle / wake detection unit 16 may determine the sunlight reflection region 71 as a cluster and mistakenly identify that there is some kind of obstacle. Furthermore, if the sunlight reflection region 71 is determined to be a cluster, the wake 73 cannot be determined to be linear, which may result in the wake 73 not being detected.

[0110] Therefore, the obstacle / wake detection unit 16 removes sunlight reflection data from the point cloud data acquired around the vessel before detecting obstacles and wakes. The method for identifying sunlight reflection data is basically the same as the method described in section (4-2) Removal of sunlight reflection data.

[0111] Figure 14 is a flowchart of the obstacle / wake detection process using the method described above. This process is achieved when the controller 13 executes a pre-prepared program and operates as the obstacle / wake detection unit 16. This process is repeated at predetermined intervals.

[0112] First, the controller 13 acquires the ship's own position and water surface height Hw, and sets the search range R based on the ship's own position and water surface height (step S31). Specifically, the controller 13 sets a predetermined horizontal range from the ship's own position as the planar range Rp, and a predetermined vertical range around the water surface height Hw as the height range Rh.

[0113] Next, the controller 13 extracts point cloud data belonging to the search range R (step S32). Next, the controller 13 obtains the sun's azimuth and elevation angle based on the date and time, and removes sunlight reflection data from the point cloud data extracted in step S32 (step S33). Next, the controller 13 detects clusters of point cloud data by clustering or the like (step S34). If no clusters of point cloud data are detected (step S34: No), the process ends. On the other hand, if a cluster of point cloud data is detected (step S34: Yes), the controller 13 determines whether the cluster was detected for a predetermined time, i.e., over multiple frames (step S35).

[0114] If no cluster of point cloud data is detected across multiple frames (Step S35: No), the process terminates. On the other hand, if a cluster of point cloud data is detected across multiple frames (Step S35: Yes), the controller 13 determines whether the cluster of point cloud data is linear or not, for example, using a method such as the Hough transform (Step S36). If the cluster of point cloud data is not linear (Step S36: No), the controller 13 determines that the cluster is an obstacle (Step S37). On the other hand, if the cluster of point cloud data is linear (Step S36: Yes), the controller 13 determines that the cluster is a wake (Step S38). Then, the process terminates.

[0115] Figure 15(A) shows an example where a solar reflection region 71 is near an obstacle. In this case, the obstacle / wake detection unit 16 identifies the solar reflection region 71 based on the sun's azimuth and elevation angle, and removes all point cloud data acquired within the solar reflection region 71. Although the solar reflection region 71 may contain point cloud data of the actual obstacle, since ships and obstacles usually move at different speeds, the obstacle is correctly detected by performing detection over multiple frames as described above.

[0116] Figure 15(B) shows an example where the sunlight reflection region 71 is on the undertow. In this case as well, the obstacle / undertow detection unit 16 removes all point cloud data acquired within the sunlight reflection region 71. When applying the Hough transform to the clusters detected by clustering, by allowing straight lines to be dashed, it becomes possible to detect undertows even after removing the sunlight reflection region 71. Allowing straight lines to be dashed means slightly relaxing the threshold for straight line determination in the Hough transform, and by relaxing the threshold appropriately, short straight lines and dashed lines can also be detected.

[0117] As described above, the obstacle / wake detection unit 16 sets a height range Rh based on the horizontal height Hw and extracts point cloud data belonging to the search range R. However, when the waves are large, obstacles floating on the water surface may fall outside the height range Rh and become undetectable. On the other hand, if the height range Rh is increased to prevent this, irrelevant data will be included when the waves are small, reducing the detection accuracy.

[0118] Therefore, the obstacle / wake detection unit 16 estimates the wave height using the variance of the z values ​​of the point cloud data belonging to the search range R, and sets the height range Rh according to the wave height. Specifically, if the variance of the z values ​​of the point cloud data is small, the wave height is presumed to be small, as shown in Figure 16(A). In this case, the obstacle / wake detection unit 16 narrows the height range Rh. For example, the obstacle / wake detection unit 16 sets the height range Rh to near the water surface height Hw. This reduces the inclusion of noise, thereby improving the detection accuracy of obstacles and wakes.

[0119] On the other hand, if the variance of the z values ​​of the point cloud data belonging to the search range R is large, it is presumed that the wave height is large, as shown in Figure 16(B). Therefore, the obstacle / wake detection unit 16 widens the height range Rh. That is, the obstacle / wake detection unit 16 sets a wider height range Rh than when the wave height is small, centered on the water surface height Hw. For example, as shown in Figure 16(C), the obstacle / wake detection unit 16 may calculate the standard deviation σ of the z values ​​of the point cloud data obtained in the search range R and determine the height range Rh to be, for example, a range of ±3σ centered on the water surface height Hw. This allows the search range to be widened even when the waves are high, thereby preventing the detection of obstacles and wakes from being missed.

[0120] (6) Pier detection Next, a method for eliminating the influence of sunlight in the detection of piers by the pier detection unit 17 will be described. When a vessel docks at a pier, sequentially measuring the distance, speed, angle, etc. from the vessel to the pier and providing this information to the operator or driving support system is useful for safe and smooth docking. Normally, piers are located slightly above the water surface, so the pier detection unit 17 performs pier detection using measurement data from the lidar 3 that is slightly above the water surface, thereby calculating the distance to the pier, speed, angle, etc.

[0121] Since the pier has a nearly flat top surface and no significant irregularities on its sides, the pier detection unit 17 detects that the object is a pier by evaluating the direction and variance of the normal vector of the measurement data from the lidar 3. If it is determined that the object is a pier, the pier detection unit 17 calculates the distance from the ship to the pier using methods such as nearest neighbor search, and calculates the ship's speed based on the change in that distance over time. Also, since the sides of the pier are generally flat, the pier detection unit 17 calculates the relative angle with the ship by finding a straight line along that line.

[0122] However, as mentioned above, when detecting a pier, on sunny days, the lidar 3 may receive reflected sunlight. Figures 17(A) and 17(B) show examples of reflected sunlight incident on the lidar 3. In Figure 17(A), sunlight is reflected from the water surface and puddles on the pier and incident on the lidar 3 as shown by the dashed line. In Figures 17(A) and 17(B), the point cloud measured from the light emitted from the lidar 3 is shown as black dots 75, and the point cloud measured as a result of reflected sunlight incident on the lidar 3 is shown as white dots 76. As a result of sunlight being reflected from the water surface and the top of the quay and incident on the lidar 3, a large number of white dots 76 are measured.

[0123] If the measurement data from Lida3 includes sunlight reflection data, the amount of measurement data other than that of the pier will increase, leading to greater variation in the normal vector and potentially making it impossible to identify the object as a pier. Furthermore, even if the object is correctly identified as a pier, the sunlight reflection data may cause the shortest distance to the pier obtained by nearest neighbor search to deviate from the correct distance.

[0124] Therefore, the pier detection unit 17 removes the sunlight reflection data from the measurement data of the lidar 3 and detects the pier. The pier detection unit 17 basically uses the same method as described in section (4-2) Removal of sunlight reflection data. That is, the pier detection unit 17 obtains the sun's azimuth and elevation angle from the current date and time and removes the sunlight reflection data included in the measurement data of the lidar 3.

[0125] Figure 17(C) shows an example of pier detection after removing solar reflection data. The pier detection unit 17 identifies the range in which sunlight enters the lidar 3 based on the azimuth and elevation angle of the sun. In the example in Figure 17(C), the pier detection unit 17 removes point cloud data measured in the range of angles α corresponding to the azimuth of the sun in the horizontal direction, and in the range of angles near the elevation angle θ of the sun in the vertical direction, as solar reflection data. Then, the pier detection unit 17 detects the pier surface using the point cloud data after removing the solar reflection data and creates a straight line L of the pier. Furthermore, the pier detection unit 17 calculates the shortest distance from the ship to the pier, that is, the length of the perpendicular D from the ship's own position to the straight line L, as the distance from the ship to the pier.

[0126] In this way, by removing the reflective component of sunlight, piers can be accurately detected. Furthermore, this method can be applied to the detection of various objects where ships attempt to dock, such as wharves, in addition to piers.

[0127] The present invention has been described above with reference to the examples, but the present invention is not limited to the above examples. Various modifications to the structure and details of the present invention can be made that can be understood by a person skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents, etc., that has been cited is incorporated herein by reference. [Explanation of Symbols]

[0128] 1. Information Processing Device 2 Sensor Groups 3 Riders 4 Speed ​​Sensor 5 GPS receivers 6 IMU 10 Map Database 13 Controllers 15 Self-position estimation part 16 Obstacle / Surface Wave Detection Unit 17 Pier detection unit

Claims

1. A point cloud data acquisition means for acquiring first point cloud data generated by a measuring device installed on a ship, A means for acquiring solar position information that acquires solar position information at the time of acquiring the first point cloud data, Based on the aforementioned solar position information, a false detection data extraction means extracts false detection data from the first point cloud data, A wake detection means for detecting wakes present around the vessel using second point cloud data obtained by removing the false detection data from the first point cloud data, An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, wherein the trailing wave detection means detects a cluster of linear point cloud data that has been continuously detected for a predetermined time in the second point cloud data as the trailing wave.

3. The system further includes a height calculation means that uses the second point cloud data to calculate the water surface height corresponding to the height from the measuring device to the water surface, The information processing apparatus according to claim 1 or 2, wherein the wake detection means detects the wake within a height range set with respect to the water surface height.

4. A control method performed by a computer, The first point cloud data generated by the measuring device installed on the ship is acquired, The solar position information at the time of acquiring the first point cloud data is obtained, Based on the aforementioned solar position information, false detection data is extracted from the first point cloud data. A control method for detecting wakes present around a vessel using second point cloud data obtained by removing the false detection data from the first point cloud data.

5. The first point cloud data generated by the measuring device installed on the ship is acquired, The solar position information at the time of acquiring the first point cloud data is obtained, Based on the aforementioned solar position information, false detection data is extracted from the first point cloud data. A program that causes a computer to perform a process to detect wakes present around a vessel using second point cloud data obtained by removing the false detection data from the first point cloud data.

6. A storage medium storing the program described in claim 5.

Images