Information processing device, control method, program, and storage medium

Through information processing devices and lidar measurement data, the closest point of the docking site is extracted, and the straight line along the docking site is calculated, which solves the accuracy of the calculation of distance, speed and angle when the ship is docked in the prior art. Improves the safety and accuracy of automatic docking.

CN118900804APending Publication Date: 2024-11-05PIONEER IP +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380025984.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-15
Filing Date
2023-03-09
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

It is difficult to accurately calculate the distance, speed and angle of a ship when docking, especially in automatic navigation and driving assistance systems, it is difficult to calculate the straight lines along the side of the docking site with high accuracy.

Method used

An information processing device is designed to extract the nearest point of the docking site by obtaining measurement data, calculate the straight line along the docking site based on these data, and eliminate noise points and reduce weight points through lidar measurement data. Ensure calculation accuracy.

Benefits of technology

It realizes the calculation of the distance between the opposite shore and the straight line on the side of the shore with high accuracy during the ship's docking process, which improves the safety and smoothness of the docking, and is suitable for automatic docking and driving assistance systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118900804A_ABST
    Figure CN118900804A_ABST
Patent Text Reader

Abstract

A controller (13) of an information processing device (1) functions as an acquisition means, an extraction means, and a straight line calculation means. The acquisition means acquires dot matrix data generated by a lidar (3) provided in a target ship, the dot matrix data being a set of data specified by a group of horizontal numbers that are indexes indicating lateral positions and vertical numbers that are indexes indicating vertical positions. The extraction unit extracts, for each horizontal number, data indicating a point to be measured at a docking location, data indicating a nearest point closest to a target ship for each vertical line from the data indicating the point to be measured at the docking location. A straight line calculation means calculates a shore side straight line (Ls), which is a straight line along the shore site, on the basis of a neighboring point set, which is a set of data representing the nearest neighboring points.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to handling of a ship when it docks. Background Art

[0002] In the past, there are known technologies for assisting the berthing (landing) of ships. For example, Patent Document 1 describes the following method: In an automatic berthing device for automatically berthing a ship, the posture of the ship is controlled to change in such a way that light irradiated from a laser radar is reflected by objects around the berthing position and received by the laser radar.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-59403 Summary of the invention

[0006] Problems to be solved by the invention

[0007] In ships, it is important to land safely and smoothly at a berthing place, and in particular, the realization of a landing assistance system for ship driving assistance and automatic navigation is desired. Therefore, it is necessary to calculate parameters such as the distance, speed, and angle relative to the berthing place to land with high accuracy. In addition, in order to calculate these parameters, it is required to accurately calculate the straight line along the side of the berthing place.

[0008] The present disclosure has been made to solve the above-mentioned problems, and a main object of the present disclosure is to provide an information processing device that can appropriately calculate a straight line along the side surface of a landing place.

[0009] Means for solving problems

[0010] The invention described in the technical solution is an information processing device having:

[0011] an acquisition unit that acquires measurement data generated by a measurement device installed on the ship, the measurement data being a set of data identified by a combination of an index indicating a transverse position and an index indicating a longitudinal position;

[0012] an extracting unit that extracts the data indicating the nearest point to the ship for each index indicating the lateral position from the data indicating the measured point of the berthing place; and

[0013] The straight line calculation unit calculates a straight line along the landing place based on a set of neighboring points that is a set of the data indicating the nearest points.

[0014] In addition, the invention described in the technical solution is a control method, which is a control method executed by a computer, wherein:

[0015] obtaining measurement data generated by a measurement device installed on the ship, the measurement data being a set of data identified by a combination of an index indicating a lateral position and an index indicating a longitudinal position,

[0016] extracting the data indicating the nearest point closest to the ship for each index indicating the horizontal position from the data indicating the measured point at the berthing place,

[0017] A straight line along the landing place is calculated based on a set of neighboring points that is a set of the data representing the nearest points.

[0018] In addition, the invention described in the technical solution is a program that enables a computer to perform the following processing:

[0019] obtaining measurement data generated by a measurement device installed on the ship, the measurement data being a set of data identified by a combination of an index indicating a lateral position and an index indicating a longitudinal position,

[0020] extracting the data indicating the nearest point closest to the ship for each index indicating the horizontal position from the data indicating the measured point at the berthing place,

[0021] A straight line along the landing place is calculated based on a set of neighboring points that is a set of the data representing the nearest points. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the driving assistance system.

[0023] Figure 2 is a block diagram showing the hardware configuration of an information processing device.

[0024] Figure 3 It is a functional block diagram related to the docking auxiliary processing.

[0025] Figure 4 (A) shows an example of a hull coordinate system based on the hull of the target ship. (B) shows a situation where the laser radar captures the shore to be docked. (C) is a three-dimensional diagram of a structure with a normal vector clearly shown.

[0026] Figure 5 (A) is a stereoscopic view of a quayside wall showing the measured points of the quayside wall as the berthing place measured by laser radar and the nearest points of each longitudinal line. (B) is a stereoscopic view of a quayside wall showing the berthing side straight lines generated from the nearest points.

[0027] Figure 6(A) shows the shore side straight line Ls when the obtained dot matrix data are only the measured points on the side of the shore site. (B) shows the shore side straight line Ls when the obtained dot matrix data are only the measured points on the upper surface of the shore site.

[0028] Figure 7 (A) shows the shore side straight line before the update. (B) shows the shore side straight line after the update.

[0029] Figure 8 This is a diagram showing one frame of dot matrix data on a virtual plane facing the laser radar.

[0030] Fig. 9 (A) shows a front view of the target ship when the laser radar measurement range is the range from the upper surface to the side of the berthing place. (B) is a stereoscopic view of the berthing place showing the measurement range when the laser radar measurement range is the range from the upper surface to the side of the berthing place, the measurement range when there are vertical lines where only the upper surface is within the measurement range of the laser radar, and the nearest point of each vertical line.

[0031] Fig.10 (A) shows a top view of the target ship and the berthing place, which clearly shows the nearest points of each vertical line when the laser radar measurement range is the range from the upper surface to the side of the berthing place, and the vertical line with only the upper surface in the laser radar measurement range, and the berthing side line generated based on them. (B) shows a top view of the target ship and the berthing place, which clearly shows the nearest points selected when the laser radar measurement range is the range from the upper surface to the side of the berthing place, and the vertical line with only the upper surface in the laser radar measurement range, and the berthing side line generated based on them. (C) shows a front view of the target ship when the pier with a thin upper surface and difficult to detect sides is the berthing place.

[0032] Fig.11 This is a diagram showing one frame of dot matrix data on a virtual plane facing the laser radar.

[0033] Fig.12 (A) shows a front view of the target ship when the laser radar measurement range is the range from the upper surface to the side of the berthing place in the quay wall with the side inclined inward. (B) is a perspective view of the berthing place where the laser radar measurement range is the range from the upper surface to the side of the berthing place, and there are vertical lines where only the side is within the laser radar measurement range, and the measurement range of the laser radar 3 and the nearest point of each vertical line.

[0034] Fig.13(A) shows a plan view of the target vessel and the berthing place, which clearly shows the nearest points of each vertical line when the laser radar measurement range is the range from the upper surface to the side of the berthing place, and the vertical line with only the side in the measurement range of the laser radar 3, and the berthing side line generated based on them. (B) shows a plan view of the target vessel and the berthing place, which clearly shows the nearest points with reduced weights and the berthing side line generated when the laser radar measurement range is the range from the upper surface to the side of the berthing place, and the vertical line with only the side in the measurement range of the laser radar.

[0035] Fig.14 An example of the data structure of the reliability information is shown.

[0036] Fig.15 This is an example of a flowchart showing an outline of the docking support process.

[0037] Fig.16 This is a bird's-eye view of the target ship with its outline points clearly indicated.

[0038] Fig.17 This is a diagram in which the distance from the contour point to the straight line close to the shore side is clearly indicated by arrows.

[0039] Fig.18 This is a diagram that clearly shows the approach points and the shortest distance.

[0040] Fig.19 This is an example of a flowchart of the shore side straight line generation process.

[0041] Fig. 20 Indicators and reliability included in reliability information according to a modification are shown.

[0042] Fig.21 It is clearly stated Fig.19 A bird's-eye view of the target ship and the berthing place for the indicator shown. DETAILED DESCRIPTION

[0043] According to a preferred embodiment of the present disclosure, the information processing device has: an acquisition unit that acquires measurement data generated by a measuring device installed on a ship, the measurement data being a set of data determined by a group of indices representing a lateral position and an indices representing a longitudinal position; an extraction unit that extracts the data representing the nearest point closest to the ship for each index representing the lateral position from the data representing the measured point of the berthing place; and a straight line calculation unit that calculates a straight line along the berthing place based on a set of the data representing the nearest point, i.e., a set of adjacent points. According to this scheme, the information processing device can appropriately calculate a straight line along the berthing place based on the measurement data generated by the measuring device.

[0044] In one embodiment of the information processing device, the straight line calculation unit updates the straight line based on the set of neighboring points after excluding the nearest points whose distances to the straight line are greater than a predetermined distance. With this embodiment, the information processing device can accurately calculate a straight line along the berthing place based on the set of neighboring points after excluding the nearest points that are noise.

[0045] In another embodiment of the information processing device, the straight line calculation unit calculates the straight line based on the neighboring point set after excluding the data corresponding to the index representing the longitudinal position and the index having the largest depression angle in the measurement direction of the measuring device. With this embodiment, the information processing device can accurately calculate the straight line along the berthing place even when the measurement range of the measuring device spans the upper surface and the side surface.

[0046] In another embodiment of the information processing device, the straight line calculation unit calculates the straight line by reducing the weight of the data of the neighboring point set corresponding to the index representing the longitudinal position and the index having the largest elevation angle in the measurement direction of the measuring device compared to the other data in the neighboring point set. With this embodiment, the information processing device can accurately calculate the straight line along the berthing place when the measurement range of the measuring device spans the upper surface and the side surface and the side surface of the berthing place is inclined inward.

[0047] In another embodiment of the information processing device, the information processing device further comprises a shore distance calculation unit, which calculates the distance between the ship and the berthing place, i.e., the shore distance, based on the length of a perpendicular line drawn from the measuring device to the straight line. With this embodiment, the information processing device can accurately calculate the shore distance, i.e., an important parameter used as a reference when berthing the ship at the berthing place.

[0048] In another embodiment of the above-mentioned information processing device, the information processing device further comprises: a shortest distance to the shore calculation unit, which extracts the shortest length of the perpendicular lines drawn from the contour point of the ship to the straight line to calculate the shortest distance from the hull of the ship to the berthing place; and a close position determination unit, which determines the part of the ship closest to the berthing place. With this embodiment, the information processing device can appropriately calculate the shortest distance from the hull of the ship to the berthing place.

[0049] In another embodiment of the information processing device, the acquisition unit acquires data generated by the measuring device after performing at least one of removal of water surface reflection data generated by reflection of light emitted by the measuring device on the water surface and reduction of the number of data points by downsampling as the measurement data. With this embodiment, the information processing device can appropriately acquire the measurement data after noise is removed or after adjustment in a manner to avoid excessive data volume, and calculate the distance to the other shore.

[0050] According to another preferred embodiment of the present disclosure, there is a control method executed by a computer, wherein measurement data generated by a measuring device installed on a ship is obtained, the measurement data is a set of data determined by a group of indices representing lateral positions and indices representing longitudinal positions, the data representing the nearest neighboring point closest to the ship is extracted from the data representing the measured points of the berthing place for each index representing the lateral position, and a straight line along the berthing place is calculated based on the set of the data representing the nearest neighboring points, that is, the set of neighboring points. By executing the control method, the computer can appropriately calculate the straight line along the berthing place.

[0051] According to another preferred embodiment of the present disclosure, the program causes a computer to perform the following processing: obtaining measurement data generated by a measuring device installed on a ship, the measurement data being a set of data determined by a group of indices representing lateral positions and indices representing longitudinal positions, extracting the data representing the nearest neighboring point closest to the ship for each index representing the lateral position from the data representing the measured point of the berthing place, and calculating a straight line along the berthing place based on the set of the data representing the nearest neighboring points, i.e., the set of neighboring points. By executing the program, the computer can appropriately calculate a straight line along the berthing place. Preferably, the above-mentioned program is stored in a storage medium.

[0052] Example

[0053] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0054] (1) Overview of Driver Assistance Systems

[0055] Figure 1 (A)~ Figure 1 (C) is a schematic structure of the navigation assistance system involved in this embodiment. Specifically, Figure 1 (A) shows a block diagram of the navigation assistance system, Figure 1 (B) is a top view illustrating a visual field range (also referred to as a “measuring range” or “measurable range”) 90 of a ship included in the navigation assistance system and a laser radar 3 described later. Figure 1(C) is a diagram showing a ship and a field of view 90 of the laser radar 3 from the rear side. The navigation assistance system includes an information processing device 1 that moves with a ship as a mobile body and a sensor group 2 mounted on the ship. Hereinafter, a ship equipped with a driving assistance system is also referred to as a "target ship".

[0056] The information processing device 1 is electrically connected to the sensor group 2, and performs navigation assistance for the target ship based on the outputs of various sensors included in the sensor group 2. The navigation assistance includes docking assistance such as automatic docking (landing). Here, "docking" includes not only the case of bringing the target ship closer to the quay, but also the case of bringing the target ship closer to structures such as piers. In addition, hereinafter, "docking place" is a general term for structures such as quays and piers that are targets for docking. The information processing device 1 can be a navigation device installed in a ship, or it can be an electronic control device built into the ship.

[0057] The sensor group 2 includes various external sensors and internal sensors installed on the ship. In this embodiment, the sensor group 2 includes a Lidar (Light Detection and Ranging or Laser Illuminated Detection And Ranging) 3, for example.

[0058] The laser radar 3 is a radar that detects the horizontal direction within a predetermined angle range (see Figure 1 (B)) and the specified angle range in the vertical direction (refer to Figure 1 (C)) An external sensor that emits pulsed laser light to discretely measure the distance to an object in the outside world and generates three-dimensional dot matrix data representing the position of the object. Figure 1 (B) and Figure 1 In the example of (C), as the laser radar 3, a laser radar facing the front of the port side of the ship and a laser radar facing the front of the starboard side of the ship are respectively installed on the ship. It should be noted that the configuration of the laser radar 3 is not limited to Figure 1 (B) and Figure 1 For example, the target ship may have multiple laser radars 3 (for example, laser radars installed in front and behind the target ship) that measure the same side direction in such a way that multiple measurement data of the berthing place are obtained simultaneously when berthing. In addition, the number of laser radars 3 installed on the target ship is not limited to two, and may be one, or may be three or more.

[0059] The laser radar 3 includes an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scanning data based on the light receiving signal output by the light receiving unit. The data measured for each direction (scanning position) of the irradiated laser light is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light determined based on the above-mentioned light receiving signal. Hereinafter, the point measured by being irradiated with laser light within the measurement range of the laser radar 3 or its measurement data is also referred to as a "measured point".

[0060] Here, the dot matrix data can be regarded as an image (frame) with each measurement direction as a pixel and the measured distance and reflection intensity value in each measurement direction as the pixel value. In this case, the emission direction of the laser at the elevation angle (i.e., the measurement direction) is different in the longitudinal arrangement of the pixels, and the emission direction of the laser at the horizontal angle is different in the lateral arrangement of the pixels. Hereinafter, the measured points corresponding to the columns of pixels (i.e., vertical columns) that coincide with the positions of the horizontal indexes when the dot matrix data is regarded as an image are also referred to as "vertical lines". In addition, the horizontal index when the dot matrix data is regarded as an image is referred to as the "horizontal number", and the vertical index is referred to as the "vertical number".

[0061] It should be noted that the laser radar 3 is not limited to the above-mentioned scanning laser radar, and may also be a flash laser radar that generates three-dimensional data by diffusing laser light into the field of view of a two-dimensional array sensor. The laser radar 3 is an example of the "measuring device" in the present invention.

[0062] (2) Structure of information processing device

[0063] Figure 2 1 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly includes an interface 11, a memory 12, and a controller 13. These elements are connected to each other via a bus.

[0064] The interface 11 performs interface operations related to the exchange of data between the information processing device 1 and the external device. In this embodiment, the interface 11 obtains output data from each sensor of the sensor group 2 and supplies it to the controller 13. In addition, the interface 11 supplies, for example, a signal related to the control of the target ship generated by the controller 13 to each component of the target ship that controls the driving of the target ship. For example, the target ship has a driving source such as an engine or an electric motor, a propeller that generates a propulsion force in the direction of travel based on the driving force of the driving source, a thruster that generates a lateral propulsion force based on the driving force of the driving source, and a mechanism for freely determining the direction of travel of the ship, i.e., a rudder. In addition, during automatic driving such as automatic docking, the interface 11 supplies the control signal generated by the controller 13 to these components. It should be noted that when the target ship is provided with an electronic control device, the interface 11 supplies the control signal generated by the controller 13 to the electronic control device. The interface 11 can be a wireless interface such as a network adapter for wireless communication, or a hardware interface for connecting to an external device through a cable or the like. In addition, the interface 11 can also perform interface operations with various peripheral devices such as an input device, a display device, and a sound output device.

[0065] The memory 12 is composed of various volatile memories and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. The memory 12 stores a program for the controller 13 to execute a predetermined process. It should be noted that the program executed by the controller 13 may also be stored in a storage medium other than the memory 12.

[0066] In addition, the memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, map data including information related to the location of the docking place may be stored in the memory 12. In another example, the memory 12 stores information related to the size of downsampling when the dot matrix data obtained when the laser radar 3 performs one cycle of scanning is downsampled.

[0067] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to driving assistance of the target ship by executing a program stored in the memory 12 or the like.

[0068] In addition, the controller 13 functionally has a docking place detection unit 15 and a docking parameter calculation unit 16. The docking place detection unit 15 performs processing related to the detection of the docking place based on the dot matrix data output by the laser radar 3. The docking parameter calculation unit 16 calculates the parameters required for docking at the docking place (also referred to as "dockage parameters"). Here, the docking parameters include the distance from the target ship to the docking place (the opposite shore distance), the entry angle of the target ship to the docking place, the speed at which the target ship approaches the docking place (the docking speed), etc. In addition, the docking parameter calculation unit 16 calculates information indicating the reliability related to the docking at the docking place (also referred to as "reliability information") based on the processing results of the docking place detection unit 15 and the docking parameters. In addition, the controller 13 functions as an "acquisition unit", "extraction unit", "straight line calculation unit", "opposite shore distance calculation unit", and a computer that executes a program.

[0069] It should be noted that the processing performed by the controller 13 is not limited to being implemented by software based on the program, but can also be implemented by any combination of hardware, firmware and software. In addition, the processing performed by the controller 13 can also be implemented using an integrated circuit that can be programmed by a user, such as an FPGA (Field-Programmable Gate Array) or a microcomputer. In this case, the program executed by the controller 13 in this embodiment can also be implemented using the integrated circuit.

[0070] (3) Overview of Assisted Beaching

[0071] Next, the outline of the docking assistance processing performed by the information processing device 1 is described. The information processing device 1 generates a straight line along the side of the docking place (also referred to as "dockage side straight line Ls") based on the dot matrix data of the laser radar 3 measured in the direction of the docking place. In addition, the information processing device 1 calculates docking parameters such as the distance to the opposite shore based on the docking side straight line Ls.

[0072] Figure 3 This is a functional block diagram of a docking place detection unit 15 and a docking parameter calculation unit 16 related to the docking auxiliary processing. The docking place detection unit 15 functionally includes a normal vector calculation block 20, a field of view·detection surface determination block 21, a normal number determination block 22, an average·variance calculation block 23, and a docking condition determination block 24. In addition, the docking parameter calculation unit 16 functionally includes a neighboring point search block 26, a straight line generation block 27, an opposite shore distance calculation block 28, an approach angle calculation block 29, a docking speed calculation block 30, and a reliability information generation block 40.

[0073] The normal vector calculation block 20 calculates the normal vector of the surface formed by the berthing place (also referred to as the "berthing surface") based on the dot matrix data generated by the laser radar 3 in the direction of the berthing place. In this case, the normal vector calculation block 20 calculates the normal vector based on the dot matrix data generated by the laser radar 3 that includes the berthing side in the measurement range of the target ship, for example. Information related to the measurement range of the laser radar 3 and the direction of the berthing place can be pre-registered in, for example, the memory 12.

[0074] In this case, it is preferable that the normal vector calculation block 20 performs downsampling of the dot data and removal of data obtained by reflection of the laser light on the water surface (also referred to as “water surface reflection data”).

[0075] In this case, first, the normal vector calculation frame 20 removes the data that exists below the water surface position as water surface reflection data (i.e., false detection data) for the dot matrix data generated by the laser radar 3. It should be noted that the normal vector calculation frame 20 estimates the water surface position based on, for example, the average value in the height direction of the dot matrix data generated by the laser radar 3 when there are no objects other than the water surface in the surrounding area. Then, the normal vector calculation frame 20 downsamples the dot matrix data after the water surface reflection data is removed. Downsampling is a process of integrating the measured points for each grid space of a specified size. Then, the normal vector calculation frame 20 calculates the normal vector for each measured point represented by the downsampled dot matrix data using multiple surrounding measured points. It should be noted that downsampling can also be performed before removing the data obtained by water surface reflection.

[0076] The field of view·detection surface determination frame 21 determines the surface of the docking place existing in the field of view angle of the laser radar 3 (also referred to as "field of view internal surface") and the surface of the docking place detected based on the normal vector calculated by the normal vector calculation frame 20 (also referred to as "detection surface"). In this case, the field of view·detection surface determination frame 21 determines whether the field of view internal surface and the detection surface include the upper surface and / or the side surface of the docking place.

[0077] The normal number determination block 22 extracts the normal vectors in the vertical direction and the normal vectors in the direction perpendicular thereto (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation block 20, and calculates the number of normal vectors in the vertical direction and the number of normal vectors in the horizontal direction. Here, the normal number determination block 22 regards the normal vector in the vertical direction as a vector representing the normal of the measured point relative to the upper surface of the berthing place, and regards the normal vector in the horizontal direction as a vector representing the normal of the measured point relative to the side surface of the berthing place, and calculates the number of each as an index of the reliability related to the berthing place.

[0078] The mean / variance calculation frame 23 extracts the normal vectors in the vertical direction and the normal vectors in the direction perpendicular to the vertical direction (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation frame 20, and calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction.

[0079] The docking condition determination frame 24 obtains the processing results of the field of view and detection surface determination frame 21, the number of normals determination frame 22, and the average and variance calculation frame 23 determined or calculated based on the same dot matrix data as the determination result indicating the detection condition of the docking place at the time point of generation of the dot matrix data. In addition, the docking condition determination frame 24 supplies the processing results of the field of view and detection surface determination frame 21, the number of normals determination frame 22, and the average and variance calculation frame 23 to the docking parameter calculation unit 16 as the determination result of the detection condition of the docking place.

[0080] The neighboring point search block 26 performs processing for searching the nearest neighboring point to the target ship for each vertical line from the measured points constituting the dot matrix data.

[0081] The straight line generation frame 27 generates a straight line along the side of the berthing place, that is, a shore side straight line Ls (in detail, an equation representing a straight line) based on the nearest point determined by the neighboring point search frame 26. The opposite shore distance calculation frame 28 calculates the opposite shore distance equivalent to the shortest distance between the target ship and the berthing place based on the shore side straight line Ls generated by the straight line generation frame 27. Here, when there are multiple laser radars 3 that can measure the berthing place, the opposite shore distance calculation frame 28 aggregates the dot matrix data of the multiple laser radars 3 to generate the shore side straight line Ls, and calculates the shortest distance to each laser radar 3 as the opposite shore distance. Alternatively, the shore side straight line Ls can be generated for the dot matrix data of each laser radar 3, and the shortest distance between each shore side straight line Ls and each laser radar 3 can be calculated as the opposite shore distance. In addition, the shortest distance from a reference point such as the center position of the ship to the shore side straight line Ls can also be calculated as the opposite shore distance. It should be noted that the shore distance calculation frame 28 can also determine the shortest distance of each laser radar 3 as the shore distance instead of considering the shortest distance of each laser radar 3 as the shore distance, or determine the average of these shortest distances as the shore distance.

[0082] The approach angle calculation block 29 generates an approach angle of the target ship relative to the berthing place based on the berthing side straight line Ls generated by the straight line generation block 27. Specifically, the approach angle calculation block 29 calculates the approach angle using a function "atan2" for obtaining an inverse tangent (inverse tangent) from two independent variables of a predetermined tangent. More specifically, the approach angle calculation block 29 calculates the approach angle by calculating the function atan2 based on the direction vector of the berthing side straight line Ls.

[0083] The approach speed calculation block 30 calculates the approach speed, which is the speed at which the target ship approaches the berthing place, based on the shore distance calculated by the shore distance calculation block 28. For example, the approach speed calculation block 30 calculates the time change of the shore distance (shortest distance) as the approach speed.

[0084] The reliability information generation block 40 generates reliability information based on the processing results of the approaching state determination block 24, the straight line generation block 27, the opposite shore distance calculation block 28, and the approach angle calculation block 29. Details of the reliability information will be described later.

[0085] Next, for a specific example of the processing of the normal vector calculation block 20, the normal number determination block 22, and the mean / variance calculation block 23, refer to Figure 4 To explain.

[0086] Figure 4 (A) shows an example of a hull coordinate system based on the hull of the target ship. Figure 4 As shown in (A), the front (forward) direction of the target ship is set as the "x" coordinate, the side direction of the target ship is set as the "y" coordinate, and the height direction of the target ship is set as the "z" coordinate. In addition, the measurement data of the coordinate system based on the laser radar 3 measured by the laser radar 3 is transformed into Figure 4 The hull coordinate system shown in (A). It should be noted that the process of transforming the dot matrix data of the coordinate system based on the laser radar installed on the mobile body into the coordinate system of the mobile body is disclosed in, for example, the international publication WO2019 / 188745.

[0087] Figure 4 (B) shows a situation where the laser radar 3 captures the quay wall serving as the berthing place. Figure 4 (C) is a stereoscopic view of the berthing place (the quay in this case) showing the measured points indicating the measurement positions measured by the laser radar 3 and the normal vectors calculated based on the measured points relative to the quay as the berthing place. Figure 4 In (C), the measured point is represented by a circle, and the normal vector is represented by an arrow. Here, an example in which both the upper surface and the side surface of the bank are measured by the laser radar 3 is shown.

[0088] like Figure 4As shown in (C), the normal vector calculation frame 20 calculates the normal vector of the measured point relative to the side and upper surface of the quay. The normal vector is a vector perpendicular to the plane or curved surface set as the object, so it is calculated using a plurality of measured points that can constitute a surface. Therefore, a grid with a predetermined length in the vertical and horizontal directions or a circle with a predetermined radius is set, and the calculation is performed using the measured points existing inside it. In this case, the normal vector calculation frame 20 can calculate the normal vector for each measured point, or it can calculate the normal vector at predetermined intervals. In addition, the normal number determination frame 22 determines the normal vector whose z component is larger than the predetermined threshold as the normal vector facing the vertical direction. It should be noted that the normal vector is unitized. In addition, the normal vector whose z component is less than the predetermined threshold is determined as the normal vector facing the horizontal direction. In addition, the normal number determination frame 22 determines the number of normal vectors in the vertical direction (here 5) and the number of normal vectors in the horizontal direction (here 4). The mean and variance calculation block 23 calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction. It should be noted that, regarding the edge portion, the measured points around it are the upper surface or the side surface, and are drawn in an oblique direction.

[0089] (4) Details of the method for generating the shore side straight line

[0090] Next, the generation method (first generation method, second generation method, and third generation method) of the shore side surface straight line Ls will be described.

[0091] (4-1) The first generation method

[0092] In the first generation method, the landing parameter calculation unit 16 performs a nearest neighbor search for each vertical line of the dot matrix data of one frame, and creates a collection of points near the edge of the landing place (specifically, the boundary between the upper surface and the side surface), that is, a neighboring point set. Then, the landing parameter calculation unit 16 performs a principal component analysis on the neighboring point set, and finds the vector of the principal component axis with the largest eigenvalue, thereby calculating the landing side straight line Ls.

[0093] Figure 5 (A) is a perspective view of a quayside serving as a berthing place, clearly showing a measured point of the quayside and the nearest point of each longitudinal line measured by the laser radar 3 . Figure 5 (B) is a three-dimensional diagram of the shore wall clearly showing the shore side line Ls generated based on the nearest point.

[0094] like Figure 5 As shown in (A), the landing parameter calculation unit 16x performs a nearest neighbor search for each vertical line of the dot matrix data of one frame, and determines the nearest neighbor point for each vertical line. Figure 5As shown in (B), the shore parameter calculation unit 16 regards the set of nearest points determined for each longitudinal line as a set of neighboring points and performs principal component analysis on the set of neighboring points. Then, the shore parameter calculation unit 16 calculates the vector of the principal component axis with the largest eigenvalue as the shore side straight line Ls.

[0095] Specifically, the berthing parameter calculation unit 16 calculates the n nearest points “p i =[x i y i z i ] T ", obtain the average vector "μ" representing the centroid of the neighboring point set as shown in the following formula (1).

[0096]

[0097] Then, the landing parameter calculation unit 16 uses these nearest points p i and the mean vector μ, and calculate the covariance matrix C shown in the following formula (2).

[0098]

[0099] When the eigenvalue "λ" and the eigenvector "v" of the covariance matrix C are used, the following equation (3) holds.

[0100]

[0101] Therefore, the approach parameter calculation unit 16 solves the characteristic equation shown in the following equation (4) to obtain the eigenvalue, and calculates the eigenvector based on the eigenvalue.

[0102]

[0103] If the three solutions of the equation, i.e., the eigenvalues, are set in descending order as "λ1", "λ2", and "λ3", the eigenvector v obtained by substituting the eigenvalue λ1 into the equation (3) becomes the direction vector "u" of the shore side line Ls. Then, the shore side parameter calculation unit 16 calculates the shore side line Ls as shown in the following equation (5) using the direction vector u, the average vector μ, and the mediating variable "t".

[0104]

[0105] Preferably, the shore parameter calculation unit 16 performs an update process of the shore side straight line Ls. In this case, if noise that is not data on the shore wall surface is mixed in the neighboring point set, it will affect the generated shore side straight line Ls. Therefore, the shore parameter calculation unit 16 first calculates the distance from the generated shore side straight line Ls to each nearest point of each longitudinal line. Then, when the maximum distance among the calculated distances is greater than the specified distance, the shore parameter calculation unit 16 determines that the nearest point is a noise point different from the data on the shore wall surface and excludes it from the neighboring point set, and updates the shore side straight line Ls by performing principal component analysis on the excluded neighboring point set again. Then, when the maximum distance from the shore side straight line Ls becomes less than the specified distance, the distances from the remaining nearest points to the shore side straight line Ls are all less than the specified distance, so the shore parameter calculation unit 16 ends the update process of the shore side straight line Ls. For example, it can be removed in the case of including points that capture wave spray, buoys, etc. In this way, the shore side surface straight line Ls is updated by executing the principal component analysis again on the neighboring point set excluding the nearest points whose distance from the shore side surface straight line Ls is greater than or equal to a predetermined distance.

[0106] Here, the effect of the first generation method is supplemented. As other methods for calculating the distance to the shore, the following first method and second method can be considered. In the first method, the nearest points within one frame are respectively extracted from the dot matrix data of the berthing place generated by the laser radar 3 respectively set in front and behind the target ship, and the shore side straight line Ls is calculated by generating a straight line connecting the two points, and the distance to the shore is calculated based on the shore side straight line Ls. In addition, in the second method, the nearest point within one frame extracted from the dot matrix data of the laser radar 3 and the normal vector of the side of the berthing place are calculated respectively, and the distance to the shore is calculated by the inner product of the distance from the target ship to the nearest point and the normal vector. In the case of the second method, the distance to the shore can be calculated even when only one laser radar 3 detects the berthing place. However, in the case of a berth with a thin side wall such as a pier, only the upper surface of the berth is detected, and the measured points on the side of the berth cannot be obtained, and the normal vector of the side of the berth required in the second method cannot be obtained. Therefore, in the case where only one laser radar 3 detects the berth and only the dot matrix data of the measured points on the upper surface of the berth can be obtained, the distance to the opposite shore cannot be obtained by the above-mentioned first method or second method.

[0107] In contrast, according to the calculation method of the shore side straight line Ls based on the first generation method, even if the obtained dot matrix data is only the measured points on the side of the shore location or only the measured points on the upper surface, the shore parameter calculation unit 16 can appropriately generate the shore side straight line Ls. Figure 6(A) shows the shore side straight line Ls when the obtained dot matrix data is only the measured points on the side of the shore place. Figure 6 (B) shows the shore side straight line Ls when the obtained dot matrix data is only the measured points on the upper surface of the shore location. Figure 6 (A) and Figure 6 As shown in (B), the shore parameter calculation unit 16 can appropriately generate the shore side straight line Ls regardless of whether the obtained dot matrix data is only the measured points on the side of the shore location or the obtained dot matrix data is only the measured points on the upper surface. In addition, even when only one laser radar 3 detects the shore location, the shore parameter calculation unit 16 can calculate the shore side straight line Ls and obtain the distance to the opposite shore regardless of whether the normal vector of the side surface can be detected.

[0108] Furthermore, the shore parameter calculation unit 16 updates the shore side line Ls by excluding the nearest point far from the shore side line Ls from the neighbor point set based on the distance from the shore side line Ls, thereby avoiding the influence of noise mixing on the generation of the shore side line Ls. Figure 7 (A) shows the shore side straight line Ls before the update, Figure 7 (B) shows the updated shore side straight line Ls. Figure 7 (A) and Figure 7 As shown in (B), the updated shore side straight line Ls calculated by excluding the noise points becomes a straight line that more accurately follows the side of the shore wall. Figure 7 By generating a perpendicular line to the shore side straight line Ls as shown in (C), the distance to the other shore can be accurately calculated.

[0109] In addition, as a secondary effect of the first generation method, the shore-side parameter calculation unit 16 can compare the direction vector of the shore-side straight line Ls generated by this method with the direction vector of the straight line formed by connecting the nearest points based on the dot matrix data of the plurality of laser radars 3 installed in front and behind the target ship, and determine the reliability of the shore-side straight line Ls by the difference between these direction vectors. For example, the shore-side parameter calculation unit 16 can digitize the above reliability based on the difference of each element of the x, y, and z of the orientation quantity based on the maximum value thereof.

[0110] (4-2) Second generation method

[0111] In the second generation method, in addition to the processing of the first generation method, the shore parameter calculation unit 16 reads the vertical number of the nearest neighbor point of each vertical line, and excludes the nearest neighbor point (also called "lowest neighbor point") whose vertical number indicates the lowest point (that is, the point indicating the measurement direction with the largest depression angle in each vertical line) from the neighbor point set. Then, the shore parameter calculation unit 16 calculates the shore side straight line Ls by performing principal component analysis on the neighbor point set after excluding the lowest neighbor point.

[0112] Figure 8 This is a diagram showing one frame of dot matrix data on a virtual plane facing the laser radar 3. Here, one frame contains 10×10 data. In this case, the landing parameter calculation unit 16 regards the nearest point whose vertical number is the vertical number indicating the lowest point (here, "10") among the nearest points of each vertical line as the lowest nearest point and excludes it from the set of nearest points. Figure 8 In the example of , the shore parameter calculation unit 16 describes each data as "D (vertical number, horizontal number)" using a combination of a vertical number and a horizontal number, and regards D(10, 1), D(10, 2), D(10, 3), D(10, 4), and D(10, 5) as the lowest neighboring point. Then, the shore parameter calculation unit 16 calculates the shore side straight line Ls based on D(9, 6), D(8, 7), D(6, 8), D(6, 9), and D(5, 10) other than the lowest neighboring point.

[0113] Then, the approach parameter calculation unit 16 calculates the mean vector and the covariance matrix based on k (k is an integer less than or equal to n) points excluding the lowest neighboring point, using the following equations (6) and (7).

[0114]

[0115] Then, the shore approach parameter calculation unit 16 uses the results obtained by equations (6) and (7) to perform principal component analysis through equations (3) and (4), obtains the vector u of the first principal component axis, and generates the shore approach side straight line Ls according to equation (5).

[0116] Here, the effect of the second generation method will be described.

[0117] When the target ship is close to the berthing place, the distance from the laser radar 3 to the berthing place is short, so the measurement range of the laser radar 3 is narrow. If the side of the berthing place is viewed from the laser radar 3 in an oblique direction, the measurement range becomes the range from the upper surface of the berthing place to the side. Fig. 9 (A) shows a front view of the target ship when the measurement range of the laser radar 3 is a range from the upper surface to the side of the berthing place.

[0118] Furthermore, in such a situation, there is also a longitudinal line that does not include the edge of the berthing place. Fig. 9 (B) is a three-dimensional diagram of the docking place, which clearly shows the measurement range of the laser radar 3 when it is from the upper surface to the side of the docking place and the nearest point of each vertical line. Here, the measurement range of the laser radar 3 is shown in a mesh shape, indicating that there is dot matrix data at the intersection of the mesh. Fig. 9 In the example shown in (B), the left vertical line does not include the edge of the docking place. Therefore, in such a situation, the shore side straight line Ls obtained by applying principal component analysis to the set of nearest points of each vertical line, that is, the set of neighboring points, does not become a straight line along the side of the docking place. Fig.10 (A) shows a top view of the target ship and the berthing place, which clearly shows the nearest points of each vertical line when the measurement range of the laser radar 3 is the range from the upper surface to the side of the berthing place and the berthing side line Ls generated based on them. In this example, since the measured point that is not a point at the edge of the berthing place becomes the nearest point, the calculated berthing side line Ls does not become a straight line along the side of the berthing place.

[0119] On the other hand, a method of calculating the shore side straight line Ls based only on the nearest point on the side determined to be the shore place can also be considered. However, since the side is less likely to be detected as it approaches the shore place, there is a problem that the number of points used in the shore side straight line Ls is reduced, and the calculation accuracy of the shore side straight line Ls will deteriorate.

[0120] In consideration of the above, the landing parameter calculation unit 16 according to the second generation method calculates the landing side surface straight line Ls by excluding the lowest neighboring point whose vertical number indicates the lowest point from the neighboring point set including the nearest neighboring point determined to be the upper surface of the landing place.

[0121] Fig.10 (B) shows a top view of the target ship and the berthing place, which clearly shows the nearest points selected when the measurement range of the laser radar 3 is the range from the upper surface to the side of the berthing place and the berthing side straight line Ls generated based on them. In this example, the berthing parameter calculation unit 16 selects the nearest points other than the lowest nearest points whose vertical numbers represent the lowest points to calculate the berthing side straight line Ls. As a result, the berthing parameter calculation unit 16 can calculate the berthing side straight line Ls with high accuracy while maintaining the stability of the generation of the berthing side straight line Ls. As a result, by generating a perpendicular line relative to the berthing side straight line Ls, the distance to the opposite shore can be accurately calculated.

[0122] Fig.10(C) shows a front view of the target ship in the case where a pier with a thin upper surface and difficult to detect the side is a docking place. In such a pier with a thin upper surface and difficult to detect the side, there are no measured points determined to be the side, and only measured points determined to be the upper surface. In such a case, according to the second generation method, it is also possible to extract the valid measured points at the edge of the pier on the upper surface as the nearest point, so that the docking side straight line Ls can be stably generated for the pier.

[0123] (4-3) The third generation method

[0124] In the third generation method, in addition to the processing of the first generation method, the shore parameter calculation unit 16 reads the vertical number of the nearest point of each vertical line, and calculates the shore side straight line Ls by reducing the weight of the nearest point (also called "uppermost neighbor point") whose vertical number is the vertical number indicating the uppermost point (that is, the point indicating the measurement direction with the largest elevation angle in each vertical line) compared with the weights of other nearest points. Hereinafter, the weight of each nearest point is represented by "w". It should be noted that the third generation method can also be executed in combination with the second generation method.

[0125] Fig.11 This is a diagram showing one frame of dot matrix data on a virtual plane facing the laser radar 3. Here, one frame contains 10×10 data. In this case, the berthing parameter calculation unit 16 regards the nearest point of each vertical line whose vertical number is the vertical number indicating the uppermost point (here, "1") as the uppermost nearest point, and reduces the weight w (for example, w=0.1). Fig.11 In the example of , the landing parameter calculation unit 16 makes the weights w of D(1,6), D(1,7), D(1,8), D(1,9), and D(1,10) corresponding to the uppermost neighboring points less than 1. On the other hand, the landing parameter calculation unit 16 keeps the weights w of the nearest neighboring points D(5,1), D(5,2), D(4,3), D(2,4), and D(2,5) other than the uppermost neighboring point at the initial value (w=1.0).

[0126] Then, the shore approach parameter calculation unit 16 calculates the shore approach side line Ls by weighted principal component analysis. In this case, the shore approach parameter calculation unit 16 calculates the weighted mean vector and the weighted covariance matrix by the following equations (8) and (9).

[0127]

[0128] Then, the shore-approaching parameter calculation unit 16 uses the results obtained by equations (8) and (9) to perform principal component analysis (weighted principal component analysis) by equations (3) and (4), obtains the vector u of the first principal component axis, and generates the shore-approaching side straight line Ls according to equation (5). It should be noted that in the second generation method, the same equation can be used by setting the weight w of the lowest neighboring point in equations (8) and (9) to 0.

[0129] Here, the effect of the third generation method will be described.

[0130] For each vertical line, as a vertical line not including an edge, there is a case where only the side of the docking place is within the measurement range of the laser radar 3. In this case, if it is considered as a two-dimensional plane, there seems to be no problem in the generation of the docking side line Ls, but when the side of the docking place is not flat, the nearest point of the vertical line not including the edge will deviate from the edge of the docking place.

[0131] Fig.12 (A) shows a front view of a target vessel approaching a berthing location with its side tilted inwards, Fig.12 (B) is a stereoscopic view of a docking place, clearly showing the measurement range of the laser radar 3 and the nearest point of each vertical line when there is a vertical line in the measurement range of the laser radar 3 only on the side of the docking place. Fig.12 The landing place shown in (A) has become a quay wall that enters the inside from the side. In such a case, if the shore side straight line Ls is calculated including the nearest point of the vertical line in which only the side of the landing place is within the measurement range of the laser radar 3, the calculation accuracy of the shore side straight line Ls decreases. Fig.13 (A) shows a top view of the target ship and the berthing place, which clearly shows the nearest points of each longitudinal line and the berthing side straight line Ls generated based on the nearest points of each longitudinal line when there is a longitudinal line where only the side of the berthing place is within the measurement range of the laser radar 3. In this example, the berthing side straight line Ls is calculated because the nearest points on the side of the quay wall entering the inside are included, and the berthing side straight line Ls does not become a straight line along the side of the berthing place.

[0132] In consideration of the above, the shore parameter calculation unit 16 according to the third generation method calculates the shore side surface straight line Ls by reducing the weight of the uppermost neighboring point whose vertical number indicates the uppermost point. Fig.13(B) shows a top view of the target ship and the berthing place, which clearly shows the nearest point with a reduced weight w and the generated berthing side straight line Ls when there is a vertical line where only the side of the berthing place is within the measurement range of the laser radar 3. In this example, the berthing parameter calculation unit 16 sets the weight w of the uppermost nearest point whose vertical number represents the uppermost point to 0.1 to calculate the berthing side straight line Ls. As a result, the berthing parameter calculation unit 16 can calculate the berthing side straight line Ls with high accuracy while maintaining the stability of the generation of the berthing side straight line Ls. As a result, by generating a perpendicular line relative to the berthing side straight line Ls, the distance to the opposite shore can be accurately calculated.

[0133] (5) Generation of reliability information

[0134] Next, a specific example of generating reliability information is described. The reliability information generating frame 40 generates flags for each element such as the viewing angle during the detection of the docking place, the surface detection of the docking place, the number of normal vectors and the variance, and generates the vector of the generated flags as reliability information. In the future, when the flag is "1", it means that the reliability of the corresponding element is high, and when the flag is "0", it means that the reliability of the corresponding element is low.

[0135] Fig.14 FIG. 4 shows an example of the data structure of the reliability information generated by the reliability information generation block 40. Fig.14 As shown, the reliability information has items of "upper surface", "side", "straight line", "distance", and "angle". In addition, the item "upper surface" has sub-items of "viewing angle", "detection", "number of normals", and "variance", and the item "side" has sub-items of "viewing angle", "detection", "number of normals", and "variance". In addition, the item "straight line" has sub-items of "absolute value", the item "distance" has sub-items of "change amount" and "change rate", and the item "angle" has sub-item of "change amount".

[0136] Here, the reliability information generation frame 40 registers the sub-item “viewing angle” of the item “upper surface” as “1” when the upper surface of the berthing place is within the viewing angle range, and as “0” when the upper surface is outside the viewing angle. In addition, the reliability information generation frame 40 registers the sub-item “detection” of the item “upper surface” as “1” when the upper surface of the berthing place is a detection surface, and as “0” when the upper surface is not a detection surface. In addition, the reliability information generation frame 40 registers the sub-item “number of normals” of the item “upper surface” as “1” when the number of normal vectors relative to the upper surface of the berthing place is greater than a predetermined threshold value (e.g., 10), and as “0” when the number is less than the threshold value. In addition, the reliability information generation block 40 registers a flag that becomes "1" when the variances of the x, y, and z components of the normal vector to the upper surface of the berthing site are all less than a predetermined threshold value (e.g., 1.0) for the sub-item "variance" of the item "upper surface", and becomes "0" when any of the variances is greater than the threshold value. In addition, the reliability information generation block 40 also registers a flag determined by the same rule as that of each sub-item of the item "upper surface" in each sub-item of the item "side".

[0137] In addition, the reliability information generation frame 40 registers a flag indicating the reliability of the shore side straight line Ls calculated by any one of the first generation method to the third generation method for the sub-item "absolute value" of the item "straight line". For example, when the laser radar 3 is respectively installed in front and behind the target ship, the reliability information generation frame 40 registers a flag that becomes "1" when the difference between each component of the direction vector of the shore side straight line Ls calculated by any one of the first generation method to the third generation method and the direction vector of the straight line obtained by connecting the nearest points of the laser radar 3 in front and behind the target ship is less than the threshold value, and becomes "0" when any of the differences is greater than the threshold value.

[0138] In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "amount of change" of the item "distance" when the amount of change of the opposite shore distance calculated by the opposite shore distance calculation frame 28 relative to 1 moment ago is less than a predetermined threshold value (e.g., 1.0 m), and a flag "0" when the amount of change is greater than the threshold value. In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "rate of change" of the item "distance" when the rate of change of the opposite shore distance calculated by the opposite shore distance calculation frame 28 relative to 1 moment ago is less than a predetermined threshold value (e.g., ±10%), and a flag "0" when the rate of change is greater than the threshold value. In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "amount of change" of the item "angle" when the amount of change of the entry angle calculated by the entry angle calculation frame 29 relative to 1 moment ago is less than a predetermined threshold value (e.g., 1.0 degree), and a flag "0" when the amount of change is greater than the threshold value.

[0139] It should be noted that the above-mentioned threshold values ​​are set to adaptive values ​​pre-stored in the memory 12 , etc. In addition, the reliability information may be generated for each laser radar 3 .

[0140] According to the reliability information having such a data structure, the reliability of the calculated distance to the opposite shore, the berthing speed, and the approach angle can be grasped. It should be noted that when each sub-item of the reliability information is "1", the reliability is the highest. In addition, the information processing device 1 adjusts the output of the driving source when berthing based on the reliability information. For example, the information processing device 1 can determine the upper limit value of the speed of the target ship when berthing based on the total value of each sub-item shown in the reliability information. In this case, for example, the smaller the above-mentioned total value is, the information processing device 1 determines that the reliability of the information related to the berthing place is lower, and it is necessary to dock carefully, and the upper limit value of the speed of the target ship when berthing is reduced.

[0141] (6) Processing Flow

[0142] Fig.15 This is an example of a flowchart showing the outline of the docking support process in this embodiment. The information processing device 1 repeatedly executes Fig.15 Flowchart of the process.

[0143] First, the information processing device 1 obtains dot matrix data in the direction of the berthing place (step S11). In this case, the information processing device 1 obtains, for example, dot matrix data generated by the laser radar 3 that includes the berthing side in the measurement range of the target ship. In addition, the information processing device 1 may further perform downsampling of the obtained dot matrix data and remove data obtained by reflection on the water surface.

[0144] Next, the docking place detection unit 15 of the information processing device 1 calculates the normal vector based on the dot matrix data obtained in step S11 (step S12). In addition, the docking place detection unit 15 calculates the number of normal vectors and the variance of the normal vectors in step S12. In addition, the docking place detection unit 15 determines the inner surface of the field of view and the detection surface based on the processing result of step S12 (step S13).

[0145] Next, the approach parameter calculation unit 16 performs a nearest neighbor search for each longitudinal line based on the dot matrix data acquired in step S11 to obtain the nearest point for each longitudinal line (step S14).

[0146] Next, the shore parameter calculation unit 16 calculates the shore side straight line Ls using the nearest neighbor point of each longitudinal line obtained in step S14 (step S15). In this case, the shore parameter calculation unit 16 may exclude the nearest neighbor point that is far from the shore side straight line Ls and update the shore side straight line Ls based on the first generation method, or may exclude the lowest nearest neighbor point and / or the highest nearest neighbor point or reduce the weight to calculate the shore side straight line Ls based on the second generation method and / or the third generation method. The detailed processing of step S15 will be referred to. Fig.19 And then described.

[0147] Next, the shore approach parameter calculation unit 16 calculates shore approach parameters, namely, the shore distance, the approach angle, and the shore approach speed, using the shore approach side line Ls calculated in step S15 (step S16 ).

[0148] Then, the berthing parameter calculation unit 16 generates reliability information based on the determination result of the visual field inner surface and the detection surface in step S13 and the calculation result of the berthing parameter in step S16 (step S17). Thereafter, the information processing device 1 controls the ship based on the reliability information (step S18). Thus, the information processing device 1 can accurately perform the control of the ship related to the berthing based on the reliability that accurately reflects the berthing condition.

[0149] Then, the information processing device 1 determines whether the target ship has docked (landed) (step S19). In this case, the information processing device 1 determines whether the target ship has docked, for example, based on the output signal of the sensor group 2 or the user input via the interface 11. And, if the information processing device 1 determines that the target ship has docked (step S19; yes), the processing of the flowchart ends. On the other hand, if the target ship has not docked (step S19; no), the information processing device 1 returns the processing to step S11.

[0150] Fig.19 is with Fig.18 FIG. 1 is an example of a flowchart related to the calculation process of the shore side line Ls in step S15.

[0151] First, the landing parameter calculation unit 16 sets the nearest point of each longitudinal line as a neighboring point set (step S21). Then, the landing parameter calculation unit 16 reads the vertical number of each point in the neighboring point set (step S22).

[0152] Next, the docking parameter calculation unit 16 determines whether there is a lowest neighboring point in the neighboring point set (step S23). If there is a lowest neighboring point (step S23; yes), the docking parameter calculation unit 16 excludes the lowest neighboring point from the neighboring point set (step S24). On the other hand, if there is no lowest neighboring point (step S23; no), the docking parameter calculation unit 16 advances the process to step S25.

[0153] Next, the docking parameter calculation unit 16 determines whether there is an uppermost neighboring point in the neighboring point set (step S25). If there is an uppermost neighboring point (step S25; yes), the docking parameter calculation unit 16 reduces the weight of the uppermost neighboring point compared to the weights of other neighboring points (step S26). On the other hand, if there is no uppermost neighboring point (step S25; no), the docking parameter calculation unit 16 advances the processing to step S27.

[0154] Then, the docking parameter calculation unit 16 generates a straight line through principal component analysis of the neighboring point set (step S27). Then, the docking parameter calculation unit 16 calculates the distance from each point in the neighboring point set to the above-mentioned straight line and extracts the maximum distance (step S28). Then, the docking parameter calculation unit 16 determines whether the maximum distance is longer than a specified value (step S29). And, when the maximum distance is longer than the specified value (step S29; yes), the docking parameter calculation unit 16 determines that the point in the neighboring point set corresponding to the maximum distance is a noise point and excludes it from the neighboring point set (step S31). Then, the docking parameter calculation unit 16 causes the processing to enter step S27 again.

[0155] On the other hand, when the maximum distance is equal to or smaller than the predetermined value (step S29; No), the shore approach parameter calculation unit 16 sets the straight line calculated in step S27 as the shore approach side straight line Ls (step S30).

[0156] (7) Modifications

[0157] (Variant 1)

[0158] The shore-approaching parameter calculation unit 16 may calculate the shore-approaching side straight line Ls from the set of nearest points of each longitudinal line, i.e., the set of nearest points, based on the principal component analysis, and may calculate the shore-approaching side straight line Ls from the set of nearest points based on the least square method or other regression analysis techniques. Thus, the shore-approaching parameter calculation unit 16 can also accurately calculate the shore-approaching side straight line Ls based on the nearest points of each longitudinal line.

[0159] (Variant 2)

[0160] The information processing device 1 may also grasp the portion of the ship body closest to the quay (approaching portion) and the shortest distance from the approaching portion to the quay, thereby facilitating the navigation of the ship when landing or leaving the quay.

[0161] In this case, the information processing device 1 determines the above-mentioned close location and the shortest distance through the following (step 1) to (step 3).

[0162] (Step 1) Calculate the distance from a plurality of points representing the outline of the vessel (also referred to as "outline points Po") to the shore side straight line Ls.

[0163] (Step 2) Extract the minimum value from the calculated distance from the contour point Po to the shore side straight line Ls

[0164] (Step 3) The extracted minimum value is set as the shortest distance to the bank, and the contour point Po that becomes the minimum value is determined as the portion close to the bank.

[0165] Fig.16 This is a bird's-eye view of the target ship showing the outline points Po. For example, the memory 12 stores the position data indicating the outline position of the target ship, i.e., the outline data. The outline data is data in which a plurality of (here, 24) outline points Po indicating the outline of the target ship are recorded as coordinates of the ship coordinate system. Here, the front (forward) direction of the target ship is set to "X". b " coordinates, set the side direction of the target ship to "Y b " coordinates, and the vertical direction of the target ship is set to "Z b " coordinates. In addition, the measurement data of the coordinate system based on the laser radar 3 measured by the laser radar 3 is transformed into the ship coordinate system. It should be noted that the processing of transforming the dot matrix data of the coordinate system based on the laser radar set on the mobile body into the coordinate system of the mobile body is disclosed in, for example, the international disclosure WO2019 / 188745.

[0166] Fig.17This is a diagram showing the distances from the contour point Po to the shore side straight line Ls by arrows. The calculation of these distances is similar to the above-mentioned method, and is implemented by finding the length of the perpendicular line from each contour point Po to the shore side straight line Ls. Here, arrows representing the above-mentioned distances relative to 24 contour points Po are shown. Fig.18 Here, the contour point Po corresponding to the minimum value extracted in (step 2) is highlighted by a frame 90 as the close part, and the shortest distance from the close part to the bank is indicated by an arrow 91.

[0167] Therefore, according to Modification 2, the position of the ship closest to the target quay and the distance from the quay can be grasped, thereby contributing to safer and smoother ship driving assistance.

[0168] (Variant 3)

[0169] Fig.19 Indicators and reliability included in reliability information according to a modification are shown. Fig. 20 It is clearly stated Fig.19 A bird's-eye view of the target ship and the berth of the indicator shown. It should be noted that, here, the measured point near the edge of the berth is also called the "target point".

[0170] The index "c3" is an index based on the number of object points, and is represented here as a linear function of the number of object points as an example. The index "c2" is an index based on the standard deviation of the object point, and is represented here as a linear function of the standard deviation of the object point as an example. The index "c1" is an index indicating whether the shore walls on both the front side and the rear side are measured by two laser radars 3. As an example, the case where both sides are measured is set to "1.0" and the case where only one side is measured is set to "0.0". The index "c0" is an index based on the interval between the two ends of the object point (the two ends in the direction of the shore side straight line Ls), and is represented as a linear function of the interval between the above-mentioned two ends. In addition, the indicators c0~c3 are calculated in a manner limited to the range of 0~1.

[0171] The calculation reliability "c" is based on the reliability of each of the above-mentioned indicators c0 to c3, and here, it becomes the weighted average value of the indicators c0 to c3 using the weighting coefficients "w0" to "w3" corresponding to the importance of the indicators c0 to c3. In addition, an example of the setting value of the weighting coefficients w0 to w3 is shown in the figure. Since each of the indicators c0 to c3 is in the range of 0 to 1, the weighted average value of them, that is, the calculation reliability c, is also calculated as a value in the range of 0 to 1.

[0172] The comprehensive reliability "r" is based on the reliability related to the detection of the side of the quay wall, that is, the side detection reliability "qs ", the reliability related to the detection of the upper surface of the quay wall, i.e., the upper surface detection reliability "q u " and the reliability of the calculated reliability c. Here, it is used with respect to each reliability q s ,q u , the weighted coefficient "w qs " " qu " " c The reliability of s ,q u , c. In addition, the weighting coefficient w is shown in the figure. qs 、w qu 、w c It should be noted that the information processing device 1 can, for example, set the side detection reliability q s based on Fig.14 The reliability information item "side" is calculated by taking the upper surface detection reliability q u The side detection reliability q is calculated based on the item "upper surface" of the reliability information in the figure. s and upper surface detection reliability q u All are calculated so as to fall within the range of 0 to 1. In addition, when both the front and rear banks are detected, the side detection reliability q of the front bank is s0 , side detection reliability of the rear quay wall q s1 , the upper surface detection reliability of the front quay wall q u0 , upper surface detection reliability of the rear quay wall q u1 Since the side detection reliability q s , upper surface detection reliability q u and the calculation reliability c are both in the range of 0 to 1, so their weighted average, i.e., the comprehensive reliability r, is also calculated as a value in the range of 0 to 1. Therefore, it can be seen that the closer the comprehensive reliability r is to 1, the higher the reliability of the calculated berthing parameters, and the closer the comprehensive reliability r is to 0, the lower the reliability of the calculated berthing parameters.

[0173] The information processing device 1 can appropriately adjust the output of the driving source when approaching the berth, etc., based on the reliability information including the various indices and the reliability according to the modification.

[0174] As described above, the controller 13 of the information processing device 1 functions as an acquisition unit, an extraction unit, and a calculation unit. The acquisition unit acquires the dot matrix data generated by the laser radar 3 installed on the target ship, and the dot matrix data is a set of data determined by a group of indices representing the horizontal position, i.e., horizontal numbers, and indices representing the longitudinal position, i.e., vertical numbers. The extraction unit extracts data representing the nearest point closest to the target ship for each horizontal number (i.e., for each vertical line) from the data representing the measured points of the berthing place. And, the calculation unit calculates a straight line along the berthing place, i.e., a shore side straight line Ls, based on a set of data representing the nearest points, i.e., a set of adjacent points. Through this scheme, the controller 13 can accurately calculate the berthing parameters such as the distance to the opposite shore.

[0175] It should be noted that in the above-mentioned embodiments, the program can be stored and provided to a controller of a computer, etc. using various types of non-transitory computer readable media (Non-Trasitory Computer Readable Medium). Non-transitory computer readable media include various types of tangible storage media (Tangible Storage Medium). Examples of non-transitory computer readable media include magnetic storage media (e.g., floppy disks, magnetic tapes, hard disk drives), optical magnetic storage media (e.g., optical magnetic disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory).

[0176] The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention. That is, the present invention certainly includes various deformations and modifications that can be made according to the entire disclosure and technical ideas including the claims by those skilled in the art. In addition, the above-mentioned patent documents and other disclosures cited are incorporated into this application by reference.

[0177] Description of Reference Numerals

[0178] 1 Information processing device

[0179] 2 Sensor Groups

[0180] 3 Lidar.

Claims

1. An information processing device, comprising: an acquisition unit that acquires measurement data generated by a measurement device installed on the ship, the measurement data being a set of data identified by a combination of an index indicating a transverse position and an index indicating a longitudinal position; an extracting unit that extracts the data indicating the nearest point to the ship for each index indicating the lateral position from the data indicating the measured point of the berthing place; and The straight line calculation unit calculates a straight line along the landing place based on a set of neighboring points that is a set of the data indicating the nearest points.

2. The information processing device according to claim 1, The straight line calculation unit updates the straight line based on the neighboring point set excluding the nearest point whose distance from the straight line is equal to or greater than a predetermined distance.

3. The information processing device according to claim 1 or 2, The straight line calculation unit calculates the straight line based on the neighboring point set from which the index indicating the longitudinal position and the data corresponding to the index having the largest depression angle in the measurement direction of the measurement device are excluded.

4. The information processing device according to any one of claims 1 to 3, The straight line calculation unit calculates the straight line by reducing the weight of the data of the neighboring point set corresponding to the index indicating the longitudinal position and the index having the largest elevation angle in the measurement direction of the measuring device compared to the other data in the neighboring point set.

5. The information processing device according to any one of claims 1 to 4, The information processing device further includes a shore distance calculation unit that calculates a shore distance that is a distance between the ship and the berthing place based on a length of a perpendicular line drawn from the measuring device to the straight line.

6. The information processing device according to any one of claims 1 to 5, further comprising: a shortest distance to the shore calculation unit that extracts the shortest length of a perpendicular line drawn from the contour point of the ship to the straight line to calculate the shortest distance from the hull of the ship to the berthing place; and The approaching position specifying unit specifies a position of the ship that is closest to the berthing place.

7. The information processing device according to any one of claims 1 to 6, The acquisition unit acquires, as the measurement data, data generated by the measurement device by performing at least one of removal of water surface reflection data generated by reflection of light emitted by the measurement device on the water surface and reduction of the number of data points by downsampling.

8. A control method is a control method executed by a computer, wherein: obtaining measurement data generated by a measurement device installed on the ship, the measurement data being a set of data identified by a combination of an index indicating a lateral position and an index indicating a longitudinal position, extracting the data indicating the nearest point closest to the ship for each index indicating the horizontal position from the data indicating the measured point at the berthing place, A straight line along the landing place is calculated based on a set of neighboring points that is a set of the data representing the nearest points.

9. A program that causes a computer to execute the following processing: obtaining measurement data generated by a measurement device installed on the ship, the measurement data being a set of data identified by a combination of an index indicating a lateral position and an index indicating a longitudinal position, extracting the data indicating the nearest point closest to the ship for each index indicating the horizontal position from the data indicating the measured point at the berthing place, A straight line along the landing place is calculated based on a set of neighboring points that is a set of the data representing the nearest points.

10. A storage medium storing the program according to claim 9.