Information processing device, control method, program and storage medium

Abstract

To acquire a self position and peripheral information without being affected by sunlight.SOLUTION: In an information processing device, point cloud data acquiring means acquires point cloud data generated by a measuring device provided on a ship. Solar position information acquiring means acquires solar position information regarding a position of the sun at the time that the point cloud data is acquired. Erroneous detection data extracting means extracts erroneous detection data from the point cloud data based on the solar position information.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to processing of data measured on board a vessel. [Background technology]

[0002] Conventionally, there have been known techniques for estimating the self-position of a moving object by comparing (matching) shape data of surrounding objects measured using a measurement device such as a laser scanner with map information in which the shapes of surrounding objects are stored in advance. For example, Patent Document 1 discloses an autonomous mobile system that determines whether a detected object in a voxel obtained by dividing a space according to a predetermined rule is a stationary object or a moving object, and matches the map information with the measurement data for voxels in which a stationary object exists. Furthermore, Patent Document 2 discloses a scan matching method that estimates the self-position by comparing voxel data including the mean vector and covariance matrix of a stationary object for each voxel with point cloud data output by a lidar. Furthermore, Patent Document 3 describes a method for estimating the performance degradation of a sensor mounted on a vehicle due to the incidence of sunlight. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication WO2013 / 076829 [Patent Document 2] International Publication No. WO2018 / 221453 [Patent Document 3] Japanese Patent Application Publication No. 2019-148531 Summary of the Invention [Problem to be solved by the invention]

[0004] Recently, driver assistance systems have been studied not only in the automotive field but also in ships. For example, development is underway for systems that support safe docking by measuring the distance to the pier, detect obstacles near the ship and encourage it to avoid them, and detect undertows and use them to safely overcome them.

[0005] Driving assistance systems like those mentioned above measure and detect the ship's position and surrounding information by detecting data around the ship. However, on clear days, sunlight reflected from the water surface can reach measuring devices such as LIDAR. In this case, the sunlight can affect the data collected near the water surface.

[0006] The present disclosure has been made to solve the above-mentioned problems, and its main purpose is to provide an information processing device that can acquire its own position and surrounding information without being affected by sunlight. [Means for solving the problem]

[0007] The invention described in the claims is an information processing device comprising: a point cloud data acquisition means for acquiring first point cloud data generated by a measuring device installed on a ship; a solar position information acquisition means for acquiring solar position information at the time of acquisition of the first point cloud data; a false detection data extraction means for extracting false detection data from the first point cloud data based on the solar position information; and a backwash detection means for detecting backwash present around the ship using second point cloud data obtained by removing the false detection data from the first point cloud data.

[0008] The invention described in the claims is a control method executed by a computer, which acquires first point cloud data generated by a measuring device installed on a ship, acquires solar position information at the time the first point cloud data is acquired, extracts false detection data from the first point cloud data based on the solar position information, and detects backwash existing around the ship using second point cloud data from which the false detection data has been removed.

[0009] The invention described in the claims is also a program that causes a computer to execute a process of acquiring first point cloud data generated by a measuring device installed on a ship, acquiring solar position information at the time the first point cloud data is acquired, extracting false detection data from the first point cloud data based on the solar position information, and detecting wakes present around the ship using second point cloud data from which the false detection data has been removed. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic configuration diagram of a driving assistance system. [Figure 2] FIG. 1 is a block diagram showing a configuration of an information processing device. [Figure 3] 2 is a diagram showing a self-position to be estimated by a self-position estimation unit in three-dimensional orthogonal coordinates. FIG. [Figure 4] 1 shows an example of a schematic data structure of voxel data. [Figure 5] 1A is a bird's-eye view of a target vessel showing an example of the arrangement of lidars provided on the target vessel, and FIG. 1B is a view of the target vessel approaching a structure, observed from behind. [Figure 6] (A) A rear view of the target vessel, clearly indicating voxels for which voxel data exists. (B) A rear view of the target vessel, clearly indicating the dashed line representing the height position of the lidar and the dashed line representing the water surface position. (C) A diagram showing the reflected component of sunlight. [Figure 7] FIG. 10 is a diagram illustrating a method for removing reflected components of sunlight. [Figure 8] 10A and 10B are diagrams illustrating a method for adjusting the removal range of the reflected component of sunlight. [Figure 9] 3 is an example of a functional block of a self-position estimation unit. [Figure 10] 10 is an example of a flowchart illustrating a procedure of a self-position estimation process. [Figure 11] 10A and 10B are diagrams illustrating a method for detecting obstacles and undertows. [Figure 12] FIG. 10 is another diagram illustrating a method for detecting an obstacle and a backwash. [Figure 13] FIG. 10 is another diagram illustrating a method for detecting an obstacle and a backwash. [Figure 14] 10 is a flowchart of an obstacle / undertow detection process. [Figure 15] An example of detecting obstacles and undertow is shown. [Figure 16] 10A and 10B are diagrams illustrating a method for changing the search range for obstacles and the like. [Figure 17] FIG. 10 is a diagram illustrating a method for detecting a pier. DETAILED DESCRIPTION OF THE INVENTION

[0011] In one preferred embodiment of the present invention, the information processing device comprises: a point cloud data acquisition means for acquiring point cloud data generated by a measuring device installed on a ship; a solar position information acquisition means for acquiring solar position information regarding the position of the sun at the time of acquisition of the point cloud data; and a false detection data extraction means for extracting false detection data from the point cloud data based on the solar position information.

[0012] In the information processing device, the point cloud data acquisition means acquires point cloud data generated by a measurement device installed on the ship. The solar position information acquisition means acquires solar position information relating to the position of the sun at the time of acquisition of the point cloud data. The false detection data extraction means extracts false detection data from the point cloud data based on the solar position information. This makes it possible to remove the effects of sunlight.

[0013] In one aspect of the information processing device, the false detection data is data generated by sunlight reflected by a water surface and incident on the measuring device, and the false detection data extraction means removes the false detection data from the point cloud data. In this aspect, it is possible to remove false detection data caused by components of sunlight reflected by the water surface.

[0014] In another aspect of the information processing device, the false detection data extraction means estimates a sunlight reflection area, which is an area where sunlight is reflected from the water surface and enters the measurement device, based on the solar position information, and removes point cloud data corresponding to the sunlight reflection area as the false detection data from the point cloud data. In this aspect, the false detection data can be removed based on the area where sunlight is reflected from the water surface.

[0015] In another aspect of the information processing device, the solar position information includes the azimuth and elevation angle of the sun, and the erroneous detection data extraction means estimates the solar reflection area based on the azimuth and elevation angle of the sun. In this aspect, the solar reflection area is estimated based on the azimuth and elevation angle of the sun.

[0016] In another aspect of the information processing device, the information processing device further includes a self-position estimation unit that estimates the self-position of the ship using the point cloud data from which the erroneous detection data has been removed. In this aspect, the influence of sunlight can be eliminated and the self-position of the ship can be estimated with high accuracy.

[0017] Another aspect of the information processing device includes a detection unit that detects at least one of an obstacle and a wake existing around the ship using the point cloud data from which the erroneous detection data has been removed. In this aspect, the influence of sunlight can be eliminated and the obstacles and wake around the ship can be detected with high accuracy.

[0018] Another aspect of the information processing device includes a calculation unit that calculates the distance to an object where the ship is to dock, using the point cloud data from which the erroneous detection data has been removed. In this aspect, the influence of sunlight can be eliminated and the distance to the object where the ship is to dock can be calculated with high accuracy.

[0019] Another preferred embodiment of the present invention is a control method executed by a computer, which acquires point cloud data generated by a measurement device installed on a ship, acquires solar position information relating to the position of the sun at the time of acquisition of the point cloud data, and removes erroneous detection data from the point cloud data based on the solar position information, thereby removing the effects of sunlight.

[0020] Another preferred embodiment of the present invention is a program that causes a computer to acquire point cloud data generated by a measurement device installed on a ship, acquire solar position information regarding the position of the sun at the time of acquisition of the point cloud data, and remove erroneous detection data from the point cloud data based on the solar position information. By executing this program on a computer, the above-mentioned information processing device can be realized. This program can be stored in a storage medium and used. [Example]

[0021] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. For convenience, in this specification, a character with "^" or "-" added above any symbol will be referred to as "A^" or "A - " (where "A" is any letter).

[0022] (1) Overview of the driving assistance system Figure 1 shows a schematic configuration of a driving assistance system according to this embodiment. The driving assistance system includes an information processing device 1 that moves together with a ship, which is a moving body, and a sensor group 2 mounted on the ship. Hereinafter, the ship that moves together with the information processing device 1 will also be referred to as the "target ship."

[0023] The information processing device 1 is electrically connected to the sensor group 2, and estimates the position of the target ship on which the information processing device 1 is installed (also referred to as "self-position") based on the outputs of various sensors included in the sensor group 2. Then, the information processing device 1 performs driving assistance such as automatic driving control of the target ship based on the results of estimating its self-position. Driving assistance also includes docking assistance such as automatic docking. Here, "docking" includes not only docking the target ship at a quay, but also docking the target ship at a structure such as a pier. The information processing device 1 may be a navigation device installed on the target ship, or an electronic control device built into the ship.

[0024] The information processing device 1 also stores a map database (DB: DataBase) 10 containing voxel data "VD." The voxel data VD is data that records position information and the like of stationary structures for each voxel, which represents a cube (regular lattice), the smallest unit of three-dimensional space. The voxel data VD includes data that expresses measured point cloud data of stationary structures within each voxel using a normal distribution, and is used for scan matching using NDT (Normal Distributions Transform), as described below. The information processing device 1 estimates, for example, the planar position, height position, yaw angle, pitch angle, and roll angle of the target ship through NDT scan matching. Unless otherwise specified, the self-position is assumed to also include the attitude angle, such as the yaw angle, of the target ship.

[0025] The sensor group 2 includes various external and internal sensors provided on the target ship. In this embodiment, the sensor group 2 includes a Lidar (Light Detection and Ranging, or Laser Illuminated Detection and Ranging) 3, a speed sensor 4 that detects the speed of the target ship, a GPS (Global Positioning Satellite) receiver 5, and an Inertial Measurement Unit (IMU) 6 that measures the acceleration and angular velocity of the target ship in three axial directions.

[0026] The LIDAR 3 emits a pulsed laser beam over a predetermined angular range in the horizontal and vertical directions to discretely measure the distance to an object in the external world and generate three-dimensional point cloud data indicating the position of the object. In this case, the LIDAR 3 includes an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scan data (points that constitute the point cloud data, hereinafter referred to as "measurement points") based on the light receiving signal output by the light receiving unit. The measurement points are generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light identified based on the above-mentioned light receiving signal. Note that, in general, the closer the distance to the object, the higher the accuracy of the LIDAR's distance measurement value, and the farther the distance, the lower the accuracy. The LIDAR 3 is an example of a "measurement device" in the present invention. The speed sensor 4 may be, for example, a Doppler-based speedometer or a GNSS-based speedometer.

[0027] Instead of the GPS receiver 5, the sensor group 2 may include a receiver that generates positioning results of a GNSS other than GPS.

[0028] (2) Configuration of information processing device 2 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly includes an interface 11, a memory 12, and a controller 13. These elements are connected to each other via a bus line.

[0029] The interface 11 performs interface operations related to the exchange of data between the information processing device 1 and an external device. In this embodiment, the interface 11 acquires output data from each sensor in the sensor group 2, such as the lidar 3, speed sensor 4, GPS receiver 5, and IMU 6, and supplies the data to the controller 13. The interface 11 also supplies, for example, signals related to the control of the target vessel generated by the controller 13 to each component of the target vessel that controls the operation of the target vessel. For example, the target vessel includes a drive source such as an engine or electric motor, a screw that generates a forward thrust based on the drive force of the drive source, a thruster that generates a lateral thrust based on the drive force of the drive source, and a rudder, which is a mechanism for freely determining the direction of travel of the vessel. During automatic operation, such as automatic docking, the interface 11 supplies control signals generated by the controller 13 to each of these components. If the target vessel is equipped with an electronic control device, the interface 11 supplies the control signal generated by the controller 13 to the electronic control device. The interface 11 may be a wireless interface such as a network adapter for wireless communication, or may be a hardware interface for connecting to an external device via a cable, etc. The interface 11 may also perform interface operations with various peripheral devices such as an input device, a display device, and a sound output device.

[0030] The memory 12 is configured by various types of volatile and non-volatile memory, such as a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk drive, and a flash memory. The memory 12 stores programs for the controller 13 to execute predetermined processes. The programs executed by the controller 13 may be stored in a storage medium other than the memory 12.

[0031] The memory 12 also stores a map DB 10 including the voxel data VD. In addition to the voxel data VD, the map DB 10 also includes, for example, information on docking locations (including shores and piers) and information on waterways that ships can navigate. The map DB 10 may be stored in an external storage device of the information processing device 1, such as a hard disk connected to the information processing device 1 via the interface 11. The storage device may be a server device that communicates with the information processing device 1. The storage device may also be composed of multiple devices. The map DB 10 may also be updated periodically. In this case, for example, the controller 13 receives partial map information on an area to which its own position belongs from a server device that manages map information via the interface 11, and reflects the partial map information in the map DB 10.

[0032] The memory 12 also stores information necessary for the processing executed by the information processing device 1 in this embodiment, in addition to the map DB 10. For example, the memory 12 stores information used to set the downsampling size when downsampling is performed on point cloud data obtained when the LIDAR 3 performs one scanning cycle.

[0033] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 executes programs stored in the memory 12 or the like to perform processing related to self-position estimation, driving assistance, and the like.

[0034] Moreover, functionally, the controller 13 has a self-position estimation unit 15, an obstacle / undertow detection unit 16, and a pier detection unit 17. The controller 13 functions as a "point cloud data acquisition means," a "sunlight position information acquisition means," a "false detection data extraction means," a "detection means," a "calculation means," and a computer that executes a program.

[0035] The self-position estimation unit 15 estimates its own position by performing scan matching based on NDT (NDT scan matching) based on the point cloud data based on the output of the LIDAR 3 and the voxel data VD corresponding to the voxels to which the point cloud data belongs. Here, the point cloud data to be processed by the self-position estimation unit 15 may be point cloud data generated by the LIDAR 3, or may be point cloud data obtained by downsampling the point cloud data.

[0036] The obstacle / undertow detection unit 16 detects obstacles and undertows around the ship using the point cloud data output by the LIDAR 3. The pier detection unit 17 uses the point cloud data output by the LIDAR 3 to detect the pier where the ship is about to dock, and calculates the distance to the pier, etc.

[0037] (3) NDT Scan Matching Next, the position estimation based on NDT scan matching executed by the self-position estimation unit 15 will be described.

[0038] FIG. 3 is a diagram showing the self-position to be estimated by the self-position estimation unit 15 in three-dimensional Cartesian coordinates. As shown in FIG. 3, the self-position on a plane defined on the three-dimensional Cartesian coordinates of xyz is expressed by the coordinates "(x, y, z)", the roll angle "φ", pitch angle "θ", and yaw angle (azimuth) "ψ" of the target ship. Here, the roll angle φ is defined as the rotation angle around the axis of the target ship's traveling direction, the pitch angle θ is defined as the elevation angle of the traveling direction of the target ship relative to the xy plane, and the yaw angle ψ is defined as the angle between the traveling direction of the target ship and the x-axis. The coordinates (x, y, z) are, for example, absolute positions corresponding to a combination of latitude, longitude, and altitude, or world coordinates indicating a position with a predetermined point as the origin. The self-position estimation unit 15 then performs self-position estimation using these x, y, z, φ, θ, and ψ as estimation parameters.

[0039] Next, we will explain the voxel data VD used in NDT scan matching. The voxel data VD includes data in which measured point cloud data of a stationary structure in each voxel is expressed using a normal distribution.

[0040] Fig. 4 shows an example of a schematic data structure of the voxel data VD. The voxel data VD includes parameter information when expressing a point group in a voxel using a normal distribution, and in this embodiment, as shown in Fig. 4, includes a voxel ID, voxel coordinates, a mean vector, and a covariance matrix.

[0041] "Voxel coordinates" indicate the absolute three-dimensional coordinates of a reference position, such as the center position of each voxel. Each voxel is a cube that divides space into a grid, and since its shape and size are predetermined, it is possible to identify the space of each voxel using its voxel coordinates. Voxel coordinates may also be used as a voxel ID.

[0042] The "mean vector" and "covariance matrix" refer to the mean vector and covariance matrix, which are parameters when expressing the point cloud in the target voxel as a normal distribution. Note that the coordinates of an arbitrary point "i" in an arbitrary voxel "n" are X n (i)=[x n (i), y n (i), z n (i)] T and the number of points in voxel n is defined as "N n ", then the mean vector at voxel n is "μ n ” and the covariance matrix “V n " are expressed by the following formulas (1) and (2), respectively.

[0043]

number

[0044]

number

[0045] Next, an overview of NDT scan matching using voxel data VD will be explained.

[0046] Scan matching using NDT assuming a ship is performed using estimated parameters that are based on the amount of movement in the horizontal plane (here, xy coordinates) and the orientation of the ship. P=[t x , t y , t z , t φ , t θ , t ψ ] T Here, "t x " is the amount of movement in the x direction, "t y " is the amount of movement in the y direction, "t z " is the amount of movement in the z direction, "t φ ” is the roll angle, “t θ ” is the pitch angle, “t ψ " indicates the yaw angle.

[0047] In addition, the coordinates of the point cloud data output by the lidar 3 are X L (j)=[x n (j), y n (j), z n (j)] T Then, X L The average value of (j) "L' n " is expressed by the following equation (3).

[0048]

number

[0049] Then, the self-position estimation unit 15 searches for voxel data VD that is associated with the point cloud data converted into an absolute coordinate system (also called a "world coordinate system") that is the same coordinate system as the map DB 10, and calculates the mean vector μ n and the covariance matrix V n Using the above, the evaluation function value of voxel n (also called "individual evaluation function value") "E nIn this case, the self-position estimation unit 15 calculates the individual evaluation function value E of the voxel n based on the following equation (4): n Calculate.

[0050]

number

[0051] Then, the self-position estimation unit 15 calculates a comprehensive evaluation function value (also called a "score value") "E(k)" for all voxels to be matched, as shown in the following formula (5). The score value E is an index showing the compatibility of the matching.

[0052]

number

[0053]

number

[0054] (4) Self-location estimation (4-1) Removal of water surface reflection data The self-position estimation unit 15 performs a water surface reflection data removal process to remove data (also called "water surface reflection data") generated when the LIDAR 3 receives the light emitted from the LIDAR 3 reflected on the water surface from the point cloud data used to calculate the NDT position. As a result, the self-position estimation unit 15 regards water surface reflection data that represents an inaccurate location different from the actual location as noise data (also called false detection data or false alarm data), and appropriately removes it from the point cloud data used for position estimation, etc., to improve the accuracy of processing such as position estimation.

[0055] First, the reason why water surface reflection data becomes erroneous detection data will be described with reference to FIGS. 5(A) and 5(B).

[0056] Fig. 5(A) is an overhead view of a target ship showing an example of the arrangement of LIDARs 3 installed on the target ship. In Fig. 5(A), as an example, four LIDARs 3 are installed on the target ship, and each LIDAR 3 emits pulsed light within a predetermined angle range while changing its angle. Note that the number and arrangement of the LIDARs 3 are not limited to the example shown in Fig. 5(A).

[0057] FIG. 5(B) is a view of a target vessel approaching a structure 50, observed from behind. If there is no object on the water surface, most of the emitted light directed downward will not return to the LIDAR 3, but some of the light will return to the LIDAR 3 and be detected as data. FIG. 5(B) clearly shows the trajectory of the light emitted by the LIDAR 3 and the measurement positions "P1" to "P10" represented by the measurement points that make up the point cloud data generated by the LIDAR 3. In addition, positions "Pa8" to "Pa10" represent the positions where the light from the LIDAR 3 used to measure measurement positions P8 to P10 was irradiated on the structure 50 after being reflected by the water surface.

[0058] As described above, a portion of the light emitted from the lidar 3 pointing downward may be reflected by the water surface and return to the lidar 3. In this case, (a) Light reflected from the water surface returns directly to LIDAR 3 and is received by LIDAR 3. (b) A pattern in which light reflected from the water surface hits a structure, some of the scattered light is reflected again from the water surface, and is received by LIDAR 3. The point cloud data of the lidar 3 generated by these patterns becomes water surface reflection data.

[0059] 5(B) correspond to positions measured based on pattern (a). In this case, water surface reflection data is generated by the lidar 3 as if there were objects at the water surface positions corresponding to the measurement positions P1 to P3.

[0060] Furthermore, measurement positions P8 to P10 are positions measured by the light of the lidar 3 being reflected at positions Pa8 to Pa10 of the structure 50 and returning to the lidar 3, respectively, and correspond to positions measured based on pattern (b). In this case, the lidar 3 cannot recognize that the light has been reflected from the water surface, so the lidar 3 generates inaccurate water surface reflection data that makes it appear as if there is an object below the water surface.

[0061] On the other hand, measurement positions P5 to P7 are positions measured by the lidar 3 receiving a portion of the scattered light of light that is directly irradiated onto the structure 50 without being reflected by the water surface, and correspond to the surface positions of the structure 50. Therefore, the point cloud data representing measurement positions P5 to P7 is not erroneous detection data, but data that accurately represents the actual position of the object to be measured.

[0062] Next, the correspondence between water surface reflection data and voxel data will be described with reference to Fig. 6(A), which is a diagram clearly showing voxels in Fig. 5(B) for which voxel data exists.

[0063] Here, the water surface reflection data corresponding to measurement positions P1 to P3 generated by pattern (a) is not originally the target of NDT scan matching, but because there is no voxel data at those positions, the target voxel cannot be found in the matching process and is not used to calculate the NDT position. In this way, although the water surface reflection data based on pattern (a) does not affect the NDT scan matching performance, it does result in unnecessary processing in the matching process in NDT scan matching.

[0064] On the other hand, the water surface reflection data based on pattern (b) corresponding to measurement positions P8 to P10 has voxels with nearby voxel data, so the data and voxels may be associated and used to calculate the NDT position. Therefore, the water surface reflection data based on pattern (b) causes deterioration of NDT matching performance. Specifically, while it would be sufficient to calculate the NDT position based on the point cloud data corresponding to measurement positions P4 to P7, using the water surface reflection data corresponding to measurement positions P8 to P10 also causes the data to be associated with nearby voxels and used in calculations, resulting in errors in the NDT position.

[0065] Voxel data corresponding to the coast is generally created at low tide. This is to create voxels as far below the quay as possible. This ensures that voxel data that can be associated with point cloud data exists regardless of the tide level during operation.

[0066] Next, an overview of the water surface reflection data removal process will be described with reference to Fig. 6(B). Fig. 6(B) is a diagram clearly showing the dashed line 51 representing the height position of the lidar 3 and the dashed line 52 representing the water surface position in Fig. 6(A).

[0067] The self-localization unit 15 acquires multiple pieces of water surface reflection data based on pattern (a) (data corresponding to measurement positions P1 to P3 in FIG. 6(B)) and calculates the height from the LIDAR 3 to the water surface (also referred to as "water surface height Hw") by averaging the z components. The self-localization unit 15 then considers data located below the calculated water surface height Hw (data corresponding to measurement positions P8 to P10 in FIG. 6(B)) among the point cloud data near the coast as water surface reflection data (more specifically, water surface reflection data based on pattern (b)). The self-localization unit 15 then determines that the point cloud data considered to be water surface reflection data is false detection data and excludes it from the target of NDT scan matching. In this case, the self-localization unit 15 also determines that the water surface reflection data based on pattern (a) used to calculate the water surface height Hw is false detection data and excludes it from the target of NDT scan matching.

[0068] This allows the self-position estimation unit 15 to avoid using inaccurate data for matching when calculating the NDT position, and enables calculation processing of the NDT position using only correct data. Furthermore, because a ship floats on the water surface, the amount of water it sinks into the water changes depending on the number of passengers and cargo, and even if this changes the water surface height Hw, the self-position estimation unit 15 can accurately calculate the water surface height Hw.

[0069] (4-2) Removal of reflected sunlight Next, the removal of the reflected component of sunlight will be described. As described above, the self-position estimation unit 15 acquires multiple pieces of water surface reflection data based on pattern (a) (in FIG. 6(B), data corresponding to measurement positions P1 to P3) and calculates the water surface height Hw from the LIDAR 3 to the water surface by averaging the z components. However, as shown in FIG. 7(A), on a clear day, sunlight reflected from the water surface may enter the LIDAR 3. This is because the wavelength of the laser used in the LIDAR 3 is also contained in sunlight. In this case, the LIDAR 3 receives the reflected component of sunlight mixed with the reflected light that reflects off the water surface and returns to the LIDAR 3, and is therefore affected by sunlight when collecting data near the water surface. Because the LIDAR 3 cannot distinguish whether the light it receives is its own emitted light or a reflected component of sunlight, it ends up measuring data for various distances depending on the timing of receiving the sunlight.

[0070] In Figure 6(C), the water surface reflection component of sunlight is indicated by a dashed line 70. The water surface reflection component of sunlight also includes sunlight that is above the water surface position. Therefore, simply estimating the water surface position and removing data below the water surface height Hw cannot remove the water surface reflection component of sunlight.

[0071] Figure 7(B) shows an example of the reflected component of sunlight. The lidar 3 detects point cloud data around the target ship, and on a clear day, an area 71 containing a large amount of reflected sunlight (hereinafter also referred to as a "sunlight reflection area") is detected in the point cloud data detected by the lidar 3. The point cloud data within the sunlight reflection area 71 contains a large amount of reflected sunlight, and therefore cannot be used as data indicating the correct water surface position.

[0072] Therefore, the self-location estimation unit 15 extracts and removes point cloud data containing sunlight reflection components from the point cloud data measured by the LIDAR 3 as sunlight reflection data (falsely detected data). Figure 7(C) schematically shows a method for extracting point cloud data containing sunlight reflection components. The position of the sun (azimuth and elevation angle) can be identified based on the date and time. Therefore, the self-location estimation unit 15 acquires the solar azimuth and elevation angle based on the current date and time. Point cloud data measured near the solar azimuth and solar elevation angle θ as seen from the ship contains a large amount of sunlight reflection components. Therefore, the self-location estimation unit 15 estimates the area on the water surface where sunlight is reflected at that azimuth and elevation angle as a sunlight reflection area 71. The self-location estimation unit 15 then sets the sunlight reflection area 71 as an exclusion range and removes point cloud data detected within the exclusion range as sunlight reflection data. Note that the point cloud data detected in the sunlight reflection area 71 may actually include reflection data of the light emitted by the LIDAR 3 rather than sunlight reflection data, but it is difficult to distinguish between them. Therefore, in this embodiment, all point cloud data detected in the sunlight reflecting region 71 is removed as false detection data. Then, the self-position estimation unit 15 calculates the water surface position (water surface height Hw) described above using the point cloud data after the false detection data has been removed.

[0073] The self-position estimation unit 15 changes the elimination range depending on the solar elevation angle. As shown in Fig. 8(A), when the solar elevation angle is large, the sunlight reflecting area 71 narrows in the azimuth direction of the sun, so the self-position estimation unit 15 narrows the elimination range. On the other hand, as shown in Fig. 8(B), when the solar elevation angle is small, the sunlight reflecting area 71 widens in the azimuth direction of the sun, so the self-position estimation unit 15 widens the elimination range.

[0074] The self-position estimation unit 15 may also change the removal range depending on the wave height around the ship. As shown in Figure 8(C), when the wave height is small, the water surface is flat, so the self-position estimation unit 15 narrows the removal range. On the other hand, as shown in Figure 8(D), when the wave height is high, the reflected component of sunlight is received over a wide range, so the self-position estimation unit 15 widens the removal range. The size of the wave height can be known by calculating the variance of the reflected component of the water surface when estimating the water surface position.

[0075] (4-3) Functional Blocks Fig. 9 shows an example of functional blocks of the self-position estimation unit 15. As shown in Fig. 9, the self-position estimation unit 15 has a dead reckoning block 21, a coordinate transformation block 22, a solar light reflection data removal block 23, a water surface position data extraction block 24, a height direction processing block 25, a water surface position calculation block 26, a filter block 27, a water surface reflection data removal block 28, and an NDT position calculation block 29.

[0076] The dead reckoning block 21 calculates the DR position based on the signals output by the sensor group 2. Specifically, the dead reckoning block 21 uses the moving speed and angular velocity of the target ship based on the outputs of the speed sensor 4, IMU 6, etc. to determine the moving distance and change in heading from the previous time. Then, the dead reckoning block 21 calculates the estimated self-position X at time k-1, which is the processing time immediately preceding the current processing time k. ^ (k-1), the DR position X at time k, which is the distance traveled since the previous time and the change in direction DR (k) is calculated. DR (k) is the self-position obtained at time k based on dead reckoning, and the predicted self-position X - (k) corresponds to the estimated self-position X at time k-1 immediately after the start of self-position estimation. ^ If there is no DR position X (k-1), the dead reckoning block 21 determines the DR position X based on the signal output by the GPS receiver 5, for example. DR (k) is defined.

[0077] The coordinate transformation block 22 transforms the point cloud data based on the output of the LIDAR 3 into a world coordinate system, which is the same coordinate system as the map DB 10. In this case, the coordinate transformation block 22 performs coordinate transformation of the point cloud data at time k, for example, based on the predicted self-position output by the dead reckoning block 21 at time k. Note that the process of transforming point cloud data in a coordinate system based on a LIDAR installed on a moving body (a ship in this embodiment) into the coordinate system of the moving body, and the process of transforming from the coordinate system of the moving body to the world coordinate system are disclosed in, for example, International Publication WO2019 / 188745.

[0078] The sunlight reflection data removal block 23 acquires the azimuth and elevation angle of the sun based on the current date and time, estimates the sunlight reflection area 71 based on the azimuth and elevation angle, and identifies the point cloud data corresponding to the sunlight reflection area 71 as sunlight reflection data. The sunlight reflection data removal block 23 then removes the sunlight reflection data from the point cloud data output by the coordinate transformation block 22. This makes it possible to eliminate the influence of sunlight.

[0079] The water surface position data extraction block 24 extracts data (also referred to as "water surface position data") representing the water surface position to be used as a sample for calculating the water surface position (water surface height Hw) from the point cloud data output by the solar reflection data removal block 23. In this case, the water surface position data extraction block 24 acquires information about the height of the water surface position from a server device that provides information about the absolute height (altitude) of the water surface position, for example, and extracts measurement points that represent heights within a predetermined range from the absolute height of the water surface position identified based on the acquired information as the above-mentioned water surface position data. For example, it is possible to determine the water surface position by acquiring tide level information provided by the Japan Meteorological Agency or the like.

[0080] Next, the water surface position data extraction block 24 extracts point cloud data measured at a position close to the target ship. A position close to the target ship refers to a position within a predetermined range from the target ship's own position. By using the point cloud data measured at a position close to the target ship, it becomes possible to calculate the water surface height with high accuracy using point cloud data measuring light that is directly reflected by the water surface and directly returned to the LIDAR 3 (water surface reflection data based on the above-mentioned pattern (a)). Then, the water surface position data extraction block 24 outputs the extracted point cloud data to the height direction processing block 25 as water surface position data.

[0081] The height direction processing block 25 calculates the average value and variance value of the z direction position (z coordinate value) represented by each measurement point extracted by the water surface position data extraction block 24. Then, the water surface position calculation block 26 calculates the current water surface position based on the average value and variance value calculated by the height direction processing block 25.

[0082] Specifically, when the variance calculated by the height direction processing block 25 is smaller than a predetermined threshold, the water surface position calculation block 26 updates the water surface position using the average value calculated by the height direction processing block 25. On the other hand, when the variance calculated by the height direction processing block 25 is equal to or greater than the predetermined threshold, the water surface position calculation block 26 determines that the reliability of the average value calculated by the height direction processing block 25 is low, and uses the water surface position calculated previously, without updating the water surface position.

[0083] Here, we will provide additional explanation about the processing of the height direction processing block 25 and the water surface position calculation block 26. Generally, when the variance value of the z coordinate values ​​represented by each measurement point extracted by the water surface position data extraction block 24 is large, it is highly likely that this corresponds to cases where waves have become large due to the passage of another ship, or where floating objects are present. Taking the above into consideration, the water surface position calculation block 26 determines whether or not the water surface position needs to be updated based on the variance value calculated by the height direction processing block 25.

[0084] The filter block 27 determines the current water surface position by averaging or filtering the water surface position calculated by the water surface position calculation block 26 with previously calculated water surface positions. In this case, for example, the filter block 27 calculates the current water surface position as the average (which may be a weighted average taking into account proximity to the current time) of the most recent predetermined number of water surface positions calculated in the past by the water surface position calculation block 26 and the water surface position calculated by the water surface position calculation block 26 at the current processing time. In another example, if the difference between the average of the most recent predetermined number of water surface positions calculated in the past by the water surface position calculation block 26 and the water surface position calculated by the water surface position calculation block 26 at the current processing time is equal to or greater than a predetermined value, the filter block 27 determines that the reliability of the water surface position calculated by the water surface position calculation block 26 at the current processing time is low. In this case, the filter block 27 does not adopt the water surface position calculated by the water surface position calculation block 26 at the current processing time, and instead considers the water surface position at the previous time to be the current water surface position. In this way, the filter block 27 can determine an appropriate water surface position based on a predetermined number of latest water surface positions calculated by the water surface position calculation block 26.

[0085] Furthermore, in order to reliably remove erroneous detection data, the water surface position calculation block 26 or the filter block 27 may provide an offset to set the water surface position a predetermined distance above the set water surface position. This offset is pre-stored, for example, in the memory 12. In this case, the water surface position calculation block 26 or the filter block 27 adds the offset pre-stored in the memory 12 to the water surface position determined by the above-mentioned calculation, and determines the result as the water surface position to be output to the next processing block.

[0086] The filter block 27 is not essential and may not be provided.

[0087] The water surface reflection data removal block 28 determines that each measurement point of the point cloud data representing a position below (including the same height, the same applies hereinafter) the water surface position supplied from the filter block 27 (i.e., a position with the same or lower z coordinate value) is water surface reflection data, and removes the determined water surface reflection data from the point cloud data. Note that the water surface reflection data removal block 28 may also regard the water surface position data extracted by the water surface position data extraction block 24 as water surface reflection data, and may exclude it from the point cloud data to be supplied to the NDT position calculation block 29. In this way, the water surface reflection data removal block 28 can accurately identify water surface reflection data corresponding to pattern (a) and pattern (b), respectively, and exclude it from the data used by the NDT position calculation block 29.

[0088] The NDT position calculation block 29 calculates the NDT position based on the point cloud data after water surface reflection data removal, which is supplied from the water surface reflection data removal block 28. In this case, the NDT position calculation block 29 matches the point cloud data in the world coordinate system supplied from the water surface reflection data removal block 28 with the voxel data VD expressed in the same world coordinate system, thereby associating the point cloud data with the voxels. Then, the NDT position calculation block 29 calculates an individual evaluation function value based on equation (4) for each voxel associated with the point cloud data, and calculates an estimated parameter P that maximizes the score value E(k) based on equation (5). Then, the NDT position calculation block 29 calculates the DR position X output by the dead reckoning block 21 based on equation (6). DR (k), the NDT position X at time k is determined by applying the estimated parameter P obtained at time k. NDT (k). The NDT position calculation block 29 calculates the NDT position X NDT (k) is output as the estimated self-position X^(k) at time k. In this case, the NDT position calculation block 29 calculates the NDT position X based on the point cloud data from which the water surface reflection data, which is false detection data, has been removed. NDT (k) can be calculated with high accuracy. Furthermore, in the correspondence between point cloud data and voxels, water surface reflection data, which is falsely detected data, is excluded from the correspondence, thereby suitably reducing the processing load.

[0089] In general, ships sway, and height fluctuations and changes in attitude can cause errors when calculating the water surface position. Therefore, in FIG. 9 , the point cloud data generated by the LIDAR 3 is converted to the world coordinate system by the coordinate transformation block 22, after which the solar reflection data is removed by the solar reflection data removal block 23, and the water surface position data is extracted by the water surface position data extraction block 24. Alternatively, the solar reflection data and water surface reflection data may be removed from the point cloud data generated by the LIDAR 3 without converting the data to the world coordinate system. In this case, the point cloud data generated by the LIDAR 3 is input to the solar reflection data removal block 23 without passing through the coordinate transformation block 22. The data then passes through the water surface position data extraction block 24, height direction processing block 25, water surface position calculation block 26, filter block 27, and water surface reflection data removal block 28, where the water surface reflection data is removed and the data is input to the coordinate transformation block 22 for conversion to the world coordinate system. In this case, the influence of the rolling of the ship can be reduced by setting the sunlight removal range to a wider range and lengthening the filter time constant of the filter block 27.

[0090] (4-4) Processing flow Fig. 10 is an example of a flowchart showing the procedure of a self-location estimation process executed by the self-location estimation unit 15 of the information processing device 1. The self-location estimation unit 15 starts the process of the flowchart in Fig. 10 when it becomes necessary to perform self-location estimation, for example, when the power is turned on.

[0091] First, immediately after the start of the self-position estimation process, the self-position estimation unit 15 calculates a predicted self-position from the GPS positioning result based on the data output by the GPS receiver 5 (step S11).

[0092] Next, the self-position estimation unit 15 performs dead reckoning based on the movement speed and angular velocity of the target ship detected based on the speed sensor 4 and IMU 6, etc., and the previous estimated self-position, and calculates a DR position X DR(Step S12). Next, the self-position estimation unit 15 converts the point cloud data based on the output of the LIDAR 3 into point cloud data in the world coordinate system (Step S13).

[0093] Next, as described above, the self-position estimation unit 15 acquires the solar azimuth and elevation angle based on the date and time to estimate the solar reflection area 71, and removes the point cloud data corresponding to the solar reflection area 71 as solar reflection data from the point cloud data converted in step S13 (step S14). Next, water surface position data is extracted from the point cloud data after the solar reflection data removal (step S15). Then, the self-position estimation unit 15 collects water surface position data for multiple frames (i.e., water surface position data obtained by scanning over the most recent multiple cycles of the LIDAR 3), and when the number of water surface position data (number of measurement points) reaches a predetermined number of data, it calculates the average and variance in the z direction (step S16). Note that if the water surface position data does not reach the predetermined number of data, for example, the self-position estimation unit 15 regards the water surface position determined immediately before in step S18 as the current water surface position, and proceeds to step S19.

[0094] Next, the self-position estimation unit 15 determines whether the variance value calculated in step S16 is less than a predetermined threshold value (step S17). If the variance value calculated in step S16 is less than the predetermined threshold value (step S17; Yes), the self-position estimation unit 15 updates the water surface position using the average value calculated in step S16 (step S18). On the other hand, if the variance value calculated in step S16 is equal to or greater than the predetermined threshold value (step S17; No), the self-position estimation unit 15 proceeds to step S19 without updating the water surface position. Note that the self-position estimation unit 15 may further perform averaging or filtering executed by the filter block 27 in Fig. 9 to determine the final water surface position.

[0095] Then, the self-position estimation unit 15 determines that data located below the determined water surface position is water surface reflection data (step S19). In this case, the self-position estimation unit 15 preferably also determines that the water surface position data extracted in step S15 is water surface reflection data. Then, the self-position estimation unit 15 excludes the data determined to be water surface reflection data from the point cloud data to be subjected to NDT processing (step S20).

[0096] Then, the self-position estimation unit 15 calculates the DR position X DR is used as the initial value, and the NDT matching process is performed after the water surface reflection data is removed in step S20, and the NDT position X NDT (Step S21). Immediately before step S21, the self-position estimation unit 15 may perform downsampling on the point cloud data after the water surface reflection data has been removed.

[0097] Next, the self-position estimation unit 15 determines whether or not to end the self-position estimation process (step S22). If the self-position estimation unit 15 determines that the self-position estimation process should end (step S22; Yes), it ends the process of the flowchart. On the other hand, if the self-position estimation unit 15 determines that the self-position estimation process should continue (step S22; No), it returns the process to step S12 and estimates the self-position at the next processing time using the estimated self-position X^ calculated in step S21.

[0098] (4-5) Modifications The following describes preferred modifications of the above-described embodiment. The following modifications may be applied to these embodiments in combination.

[0099] (Variation 1) The method for calculating the water surface height Hw is not limited to the calculation method based on the average (and variance) of the water surface position data in the z direction.

[0100] For example, the information processing device 1 receives an image of the target ship from an external device (e.g., a camera installed on a shore), analyzes the image, and recognizes the water surface position relative to the target ship. Then, the information processing device 1 calculates the water surface height Hw based on the recognized water surface position and the installation position of the LIDAR 3 on the target ship. In this case, the information processing device 1 may recognize the water surface position using an inference model that outputs an inference result regarding the water surface position when an image of a ship is input. This inference model may be a model that has been trained in advance based on, for example, deep learning, and the trained parameters are stored in advance in the memory 12, etc. In addition, the installation position of the LIDAR 3 on the target ship is stored in advance in the memory 12, etc. In another example, if a sensor for detecting the water surface position relative to the target ship is installed on the target ship, the information processing device 1 may calculate the water surface height Hw by recognizing the water surface position based on the output of the sensor.

[0101] (Variation 2) In addition to removing the water surface reflection data from the point cloud data, the information processing device 1 may exclude voxel data VD of voxels located below the water surface position (in the height direction) from the targets for association with point cloud data in NDT scan matching.

[0102] Here, voxels that exist below the estimated water surface position are below the water surface and are therefore not included in the self-location estimation. Taking the above into consideration, in this modification, the information processing device 1 calculates the NDT position by excluding the voxel data VD of the above voxels from the objects of correspondence with the point cloud data (i.e., by treating them as non-existent). This allows the information processing device 1 to omit unnecessary processing when correlating the point cloud data with the voxels, and can suitably suppress a decrease in position estimation accuracy caused by correspondence errors.

[0103] (Variation 3) The voxel data VD is not limited to a data structure including a mean vector and a covariance matrix as shown in Fig. 4. For example, the voxel data VD may include point cloud data used to calculate the mean vector and the covariance matrix.

[0104] (Variation 4) The use of point cloud data from which water surface reflection data has been removed as false detection data is not limited to position estimation. Instead, the information processing device 1 may be used for various purposes such as detecting a docking location in docking assistance such as automatic docking, detecting obstacles, etc. Even in this case, the information processing device 1 can preferably remove false detection data and perform various processes using the point cloud data with high accuracy.

[0105] (Variation 5) In the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer controller or the like. The non-transitory computer-readable media includes various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0106] (5) Obstacle / undertow detection Next, detection of obstacles and undertow by the obstacle / undertow detection unit 16 will be described. The obstacle / undertow detection unit 16 uses the water surface height calculated by the self-position estimation unit 15 to detect obstacles and undertow. If there is an obstacle near the ship, it is necessary to navigate while avoiding collision or contact with the obstacle. Obstacles include, for example, other ships, piles, bridge piers, buoys, nets, and garbage. Furthermore, if there is a undertow caused by another ship nearby, the ship must navigate with caution. Therefore, the obstacle / undertow detection unit 16 uses the water surface height to detect the presence of an obstacle or undertow near the ship.

[0107] Specifically, as shown in FIG. 11(A), the obstacle / wake detection unit 16 acquires point cloud data belonging to a search range R set around the ship, and detects clusters of point cloud data using clustering based on inter-point distance (Euclidean clustering) or the like. As shown in FIG. 11(B), if there are no obstacles or wakes near the ship, the point cloud data acquired around the ship will not contain any clusters of point cloud data. On the other hand, as shown in FIG. 12(A), obstacles can be detected as clusters of point cloud data. Furthermore, as shown in FIG. 12(B), wakes can be detected as linear clusters of point cloud data. Therefore, the obstacle / wake detection unit 16 can detect obstacles and wakes by detecting clusters of point cloud data acquired around the ship.

[0108] When acquiring point cloud data around the ship, the obstacle / wake detection unit 16 determines a search range based on the ship's own position and water surface height Hw. As shown in Figures 11(A) and 11(B), the search range R is defined by a planar range Rp on the water surface and a height range Rh. The planar range Rp is a range on the xy plane and can be, for example, a circular or polygonal range centered on the ship's own position. Figure 11(B) shows an example of a circular planar range Rp. The height range Rh is a range in the z-axis direction and can be a range of a predetermined distance in the vertical direction centered on the water surface height Hw.

[0109] When detecting obstacles and backwash, it is also necessary to eliminate the influence of reflected sunlight. As shown in FIG. 13(A), on a clear day, sunlight reflects off the water surface and enters the LIDAR 3. In FIG. 13(A), on the left side of the vessel, the LIDAR 3 is not affected by sunlight and receives light reflected by an obstacle (buoy) 72. On the other hand, on the right side of the vessel, the LIDAR 3 receives the reflected sunlight component mixed with the emitted light reflected off the water surface. In this case, as shown in FIG. 13(B), the obstacle / backwash detection unit 16 can correctly detect the buoy 72. However, because the reflected sunlight component is detected in the sunlight reflection region 71, the obstacle / backwash detection unit 16 may mistakenly determine that the sunlight reflection region 71 is a cluster and thus erroneously determine that an obstacle exists. Furthermore, if the sunlight reflection region 71 is determined to be a cluster, the backwash 73 may not be determined to be linear, which may result in the backwash 73 being missed.

[0110] Therefore, the obstacle / wake detection unit 16 removes the sunlight reflection data from the point cloud data acquired around the ship before detecting obstacles and wakes. The method for identifying the sunlight reflection data is basically the same as the method explained in the section "(4-2) Removal of sunlight reflection data."

[0111] 14 is a flowchart of the obstacle / wake detection process using the above method. This process is realized by the controller 13 executing a pre-prepared program and operating as the obstacle / wake detection unit 16. This process is repeatedly executed at predetermined time intervals.

[0112] First, the controller 13 acquires the ship's own position and water surface height Hw, and sets a search range R based on the ship's own position and water surface height (step S31). Specifically, the controller 13 sets a predetermined range in the horizontal direction from the ship's own position as a planar range Rp, and a range a predetermined distance above and below the water surface height Hw as a height range Rh.

[0113] Next, the controller 13 extracts point cloud data belonging to the search range R (step S32). Next, the controller 13 acquires the solar azimuth and elevation angle based on the date and time, and removes solar reflection data from the point cloud data extracted in step S32 (step S33). Next, the controller 13 detects clusters of point cloud data by clustering or the like (step S34). If no clusters of point cloud data are detected (step S34: No), the processing ends. On the other hand, if a cluster of point cloud data is detected (step S34: Yes), the controller 13 determines whether the clusters have been detected continuously for a predetermined period of time, i.e., across multiple frames (step S35).

[0114] If a block of point cloud data is not detected across multiple frames (step S35: No), the process ends. On the other hand, if a block of point cloud data is detected across multiple frames (step S35: Yes), the controller 13 determines whether the block of point cloud data is linear using a method such as a Hough transform (step S36). If the block of point cloud data is not linear (step S36: No), the controller 13 determines the block to be an obstacle (step S37). On the other hand, if the block of point cloud data is linear (step S36: Yes), the controller 13 determines the block to be a wake (step S38). Then, the process ends.

[0115] 15(A) shows an example in which there is a sunlight reflecting area 71 near an obstacle. In this case, the obstacle / undertow detection unit 16 identifies the sunlight reflecting area 71 based on the azimuth and elevation angle of the sun, and removes all point cloud data acquired within the sunlight reflecting area 71. Note that although the sunlight reflecting area 71 may contain point cloud data of an actual obstacle, since the ship and obstacles usually move at different speeds, the obstacle can be correctly detected by performing detection over multiple frames as described above.

[0116] FIG. 15(B) shows an example in which there is a sunlight reflection region 71 on the backwash. In this case, too, the obstacle / backwash detection unit 16 removes all point cloud data acquired within the sunlight reflection region 71. When applying the Hough transform to the clusters detected by clustering, by applying the Hough transform while allowing straight lines to be dashed, it becomes possible to detect backwash even if the sunlight reflection region 71 is removed. Allowing straight lines to be dashed means slightly relaxing the threshold for determining straight lines in the Hough transform; by relaxing the threshold appropriately, even short straight lines and dashed lines can be detected.

[0117] As described above, the obstacle / undertow detection unit 16 sets the height range Rh based on the horizontal height Hw and extracts point cloud data belonging to the search range R. However, when the waves are large, obstacles floating on the water surface may fall outside the height range Rh and become undetectable. On the other hand, if the height range Rh is increased to prevent this, irrelevant data will be included when the waves are small, reducing detection accuracy.

[0118] Therefore, the obstacle / undertow detection unit 16 estimates the wave height (wave height) using the variance of the z values ​​of the point cloud data belonging to the search range R, and sets a height range Rh according to the wave height. Specifically, when the variance of the z values ​​of the point cloud data is small, the wave height is estimated to be small, as shown in FIG. 16(A). In this case, the obstacle / undertow detection unit 16 narrows the height range Rh. For example, the obstacle / undertow detection unit 16 sets the height range Rh to be close to the water surface height Hw. This reduces noise contamination, improving the accuracy of detecting obstacles and undertows.

[0119] On the other hand, if the variance of the z values ​​of the point cloud data belonging to the search range R is large, it is estimated that the wave height is large, as shown in FIG. 16(B). Therefore, the obstacle / undertow detection unit 16 widens the height range Rh. That is, the obstacle / undertow detection unit 16 sets a wider height range Rh centered on the water surface height Hw than when the wave height is small. As an example, as shown in FIG. 16(C), the obstacle / undertow detection unit 16 may calculate the standard deviation σ of the z values ​​of the point cloud data obtained in the search range R and determine the height Rh range to be, for example, ±3σ centered on the water surface height Hw. This allows the search range to be widened even when the waves are high, thereby preventing obstacles and undertows from being missed.

[0120] (6) Pier detection Next, we will explain a method for eliminating the influence of sunlight when detecting a pier by the pier detection unit 17. When a ship docks at a pier, successively measuring the distance, speed, angle, etc. from the ship to the pier and providing this information to the operator or a driving assistance system is useful for safe and smooth docking. Since piers are usually located slightly above the water surface, the pier detection unit 17 detects the pier using measurement data obtained by the lidar 3 slightly above the water surface, thereby calculating the distance, speed, angle, etc. to the pier.

[0121] The upper surface of a pier is almost flat, and the sides are not significantly uneven, so the pier detection unit 17 detects that the object is a pier by evaluating the direction and variance of the normal vector of the measurement data from the LIDAR 3. If it is determined that the object is a pier, the pier detection unit 17 calculates the distance from the ship to the pier using a method such as nearest neighbor search, and calculates the speed of the ship based on the change in that distance over time. In addition, because the sides of the pier are almost flat, the pier detection unit 17 calculates the relative angle with the ship by determining the straight line.

[0122] However, when detecting a pier as described above, the LIDAR 3 may receive reflected sunlight, especially during the daytime on a clear day. Figures 17(A) and 17(B) show an example of reflected sunlight incident on the LIDAR 3. In Figure 17(A), sunlight is reflected from the water surface and puddles on the pier and enters the LIDAR 3 as shown by the dashed lines. In Figures 17(A) and 17(B), black dots 75 represent point clouds measured by light emitted from the LIDAR 3, and white dots 76 represent point clouds measured as a result of the reflected sunlight entering the LIDAR 3. As a result of sunlight reflecting off the water surface and the top surface of the pier and entering the LIDAR 3, many white dots 76 are measured.

[0123] If the measurement data from LIDAR 3 includes sunlight reflection data, the amount of measurement data other than the pier will increase, the variation in normal vectors will increase, and it may become impossible to determine that the object is a pier. Furthermore, even if the object is correctly determined to be a pier, the sunlight reflection data may cause the shortest distance to the pier obtained by nearest neighbor search to deviate from the correct distance.

[0124] Therefore, the pier detection unit 17 detects the pier by removing the sunlight reflection data from the measurement data of the LIDAR 3. The pier detection unit 17 basically uses the same method as the method described in the section "(4-2) Removal of sunlight reflection data." That is, the pier detection unit 17 obtains the solar azimuth and elevation angle from the current date and time, and removes the sunlight reflection data included in the measurement data of the LIDAR 3.

[0125] FIG. 17(C) shows an example of pier detection after removing sunlight reflection data. The pier detection unit 17 determines the range in which sunlight is incident on the LIDAR 3 based on the azimuth and elevation angle of the sunlight. In the example of FIG. 17(C), the pier detection unit 17 removes point cloud data measured within a horizontal angle range α corresponding to the azimuth of the sunlight and a vertical angle range near the solar elevation angle θ as sunlight reflection data. The pier detection unit 17 then detects the pier surface using the point cloud data after removing the sunlight reflection data and creates a straight line L of the pier. Furthermore, the pier detection unit 17 calculates the shortest distance from the ship to the pier, i.e., the length of a perpendicular line D from the ship's current position to the straight line L, as the distance from the ship to the pier.

[0126] In this way, by removing the reflected sunlight component, the pier can be detected correctly. This method can also be applied to the detection of various objects, such as quays, where ships are about to dock.

[0127] Although the present invention has been described above with reference to the examples, the present invention is not limited to the above examples. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art in accordance with the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference. [Explanation of symbols]

[0128] 1. Information processing equipment 2 Sensor group 3 Rider 4 Speed ​​Sensor 5 GPS receiver 6 IMU 10 Map DB 13 Controller 15 Self-position estimation part 16 Obstacle / undertow detector 17 Pier detection unit

Claims

1. a point cloud data acquisition means for acquiring first point cloud data generated by a measuring device provided on the ship; a solar position information acquisition means for acquiring solar position information at the time of acquiring the first point cloud data; an erroneous detection data extraction means for extracting erroneous detection data from the first point cloud data based on the solar position information; a backwash detection means for detecting backwash existing around the ship using second point cloud data obtained by removing the erroneous detection data from the first point cloud data; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the backwash detection means detects, as the backwash, a linear cluster of point cloud data that has been detected continuously for a predetermined time in the second point cloud data.

3. The method further includes a height calculation means for calculating a water surface height corresponding to a height from the measuring device to a water surface using the second point cloud data, 3. The information processing device according to claim 1, wherein the backwash detection means detects the backwash within a height range set with the water surface height as a reference.

4. A computer-implemented control method comprising: acquiring first point cloud data generated by a measuring device installed on the ship; acquiring solar position information at the time of acquiring the first point cloud data; extracting false detection data from the first point cloud data based on the solar position information; A control method for detecting wakes present around the ship using second point cloud data obtained by removing the erroneous detection data from the first point cloud data.

5. acquiring first point cloud data generated by a measuring device installed on the ship; acquiring solar position information at the time of acquiring the first point cloud data; extracting false detection data from the first point cloud data based on the solar position information; A program that causes a computer to execute a process of detecting wakes present around the ship using second point cloud data obtained by removing the erroneous detection data from the first point cloud data.

6. A storage medium storing the program according to claim 5.

Images