Information processing device, control method, program, and storage medium

Through information processing devices and lidar technology, the distance between the ship and the docking site is calculated, which solves the problem of inaccuracy in the distance calculation in the existing technology, and improves the safety and accuracy of the automatic docking system.

CN119053511APending Publication Date: 2024-11-29PIONEER IP +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280093270.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-03-15
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The prior art is difficult to calculate the distance between the ship and the docking site with high accuracy, which affects the safety and accuracy of the automatic docking system.

Method used

An information processing device is designed to calculate the straight line along the ship's docking by obtaining data from multiple measured points, and calculate the distance between the ship and the docking site based on the distance between the straight line and the measured point. The distance is to use the docking lateral straight lines to generate the lattice data captured by the lidar, and extract the closest point and the tentative straight lines to ensure the accuracy of distance calculation.

Benefits of technology

The distance between the ship and the docking site is calculated with high accuracy, the safety and accuracy of the automatic docking system is improved, and the smooth docking of the ship is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119053511A_ABST
    Figure CN119053511A_ABST
Patent Text Reader

Abstract

A controller (13) of an information processing device (1) has at least an acquisition means, a straight line calculation means, and a distance calculation means. The acquisition means acquires measurement data, which is a set of data indicating a plurality of measurement points measured by a measurement device provided in a ship. The straight line calculation means calculates a straight line along a landing location at which the ship is to land on the basis of the measurement data. The distance calculation means calculates the opposite-shore distance between the ship and the landing site on the basis of the distance between the straight line and each of the measured points.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to handling of a ship when it docks. Background Art

[0002] In the past, there are known technologies for assisting the berthing (landing) of ships. For example, Patent Document 1 describes the following method: In an automatic berthing device for automatically berthing a ship, the posture of the ship is controlled to change in such a way that light irradiated from a laser radar is reflected by objects around the berthing position and received by the laser radar.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-59403 Summary of the invention

[0006] Problems to be solved by the invention

[0007] In ships, it is important to land safely and smoothly at a berth, and in particular, there is a desire to implement a landing assistance system for assisting ship driving and automatic navigation. Therefore, it is necessary to calculate the distance to the berth with high accuracy.

[0008] The present disclosure has been made to solve the above-mentioned problems, and a main object of the present disclosure is to provide an information processing device that can appropriately calculate the distance from a ship to a berth.

[0009] Means for solving problems

[0010] The invention described in the technical solution is an information processing device having:

[0011] an acquisition unit that acquires measurement data, the measurement data being a collection of data representing a plurality of measurement points measured by a measurement device installed on the ship;

[0012] a straight line calculation unit that calculates a straight line along a berthing location where the ship is to dock based on the measurement data;

[0013] The distance calculation unit calculates the distance between the ship and the berth based on the distance between the straight line and each of the measured points.

[0014] In addition, the invention described in the technical solution is a control method, which is a control method executed by a computer, wherein:

[0015] Acquiring measurement data, the measurement data being a collection of data representing a plurality of measurement points measured by a measurement device installed on the ship,

[0016] Based on the measurement data, a straight line along the berthing location where the ship is to dock is calculated;

[0017] The distance between the ship and the berthing place is calculated based on the distance between the straight line and each of the measured points.

[0018] In addition, the invention described in the technical solution is a program that enables a computer to perform the following processing:

[0019] Acquiring measurement data, the measurement data being a collection of data representing a plurality of measurement points measured by a measurement device installed on the ship,

[0020] Based on the measurement data, a straight line along the berthing location where the ship is to dock is calculated;

[0021] The distance between the ship and the berthing place is calculated based on the distance between the straight line and each of the measured points. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the driving assistance system.

[0023] Figure 2 is a block diagram showing the hardware configuration of an information processing device.

[0024] Figure 3 It is a functional block diagram related to the docking auxiliary processing.

[0025] Figure 4 (A) shows an example of a hull coordinate system based on the hull of the target ship. (B) shows a situation where the laser radar captures the shore to be docked. (C) is a three-dimensional diagram of a structure with a normal vector clearly shown.

[0026] Figure 5 (A) to (C) are diagrams showing the processing flow of the first generation method.

[0027] Figure 6 (A) is a plan view showing the first shore side surface straight line L1 and the nearest point p1. (B) is a diagram showing the unit vector u used in searching for the nearest point p1.

[0028] Figure 7 (A) to (C) are diagrams showing the flow of processing for calculating the provisional straight line L′ based on the nearest point p1 and the nearest point p2 and calculating the distance to the other bank based on the provisional straight line L′.

[0029] Figure 8 The flow of processing for calculating the tentative straight line L″ obtained by updating the tentative straight line L′ is shown.

[0030] Fig. 9(A) is a diagram showing the nearest point to be extracted. (B) shows an example of extracting the data of the measured point with the shortest measurement distance of the laser radar as the nearest point. (C) shows an example of comparing the values ​​of only the y coordinate of the ship coordinate system and extracting the point closest to the ship origin as the nearest point.

[0031] Fig.10 This is a diagram showing the first shore-approaching side surface straight line L1 and the measured points of the side surface dot matrix data of the front laser radar.

[0032] Fig.11 (A) to (C) are diagrams showing the flow of processing when there is a shore side straight line L generated within a predetermined time in the past.

[0033] Fig.12 (A) to (C) are diagrams showing the flow of processing when there is no shore side surface straight line L generated within the past predetermined time.

[0034] Fig.13 An example of the data structure of the reliability information is shown.

[0035] Fig.14 This is an example of a flowchart showing an outline of the docking support process in the embodiment.

[0036] Fig.15 This is an example of a flowchart of the shore side straight line calculation process based on the first generation method.

[0037] Fig.16 This is an example of a flowchart of the shore side straight line calculation process based on the second generation method.

[0038] Fig.17 This is a bird's-eye view of the target ship with its outline points clearly indicated.

[0039] Fig.18 This is a diagram in which the distance from the contour point to the straight line close to the shore side is clearly indicated by arrows.

[0040] Fig.19 This is a diagram that clearly shows the approach points and the shortest distance.

[0041] Fig. 20 Indicators and reliability included in reliability information according to a modification are shown.

[0042] Fig.21 It is clearly stated Fig. 20 A bird's-eye view of the target ship and the berthing place for the indicator shown. DETAILED DESCRIPTION

[0043] According to a preferred embodiment of the present disclosure, the information processing device comprises: an acquisition unit that acquires measurement data, the measurement data being a collection of data representing a plurality of measured points measured by a measurement device installed on a ship; a straight line calculation unit that calculates a straight line along a berthing place where the ship is to dock based on the measurement data; and a distance calculation unit that calculates the distance between the ship and the berthing place based on the distance between the straight line and each of the measured points. According to this scheme, the information processing device can appropriately calculate the distance between the ship and the berthing place based on the measurement data generated by the measurement device.

[0044] In one embodiment of the above-mentioned information processing device, the information processing device further comprises a nearest point search unit, which searches for the nearest point with respect to the ship from the measured point based on the straight line, and the distance calculation unit calculates the distance based on the nearest point. With this embodiment, the information processing device can appropriately calculate the distance between the ship and the berthing place using the nearest point calculated based on the straight line.

[0045] In another embodiment of the information processing device, the measuring device includes a first measuring device and a second measuring device, the straight line calculation unit calculates a second straight line connecting a first nearest point to the ship searched based on the straight line from a measured point of first measurement data measured by the first measuring device and a second nearest point to the ship searched based on the straight line from a measured point of second measurement data measured by the second measuring device, and the distance calculation unit calculates the distance based on the second straight line. With this embodiment, the information processing device can accurately calculate a straight line along the berthing place.

[0046] In another embodiment of the information processing device, when the second straight line is not close to the straight line calculated before the second straight line, the straight line calculation unit re-searches the first nearest point and the second nearest point based on the second straight line, calculates a third straight line connecting the re-searched first nearest point and the second nearest point, and the distance calculation unit calculates the distance based on the third straight line. With this embodiment, the information processing device can more accurately calculate the straight line along the berthing place.

[0047] In another embodiment of the information processing device, the straight line calculation unit repeatedly performs the re-search of the first nearest point and the second nearest point and the calculation of the straight line until the calculated straight line converges. With this embodiment, the information processing device can appropriately calculate a straight line that is suitable as a straight line along the berthing place.

[0048] In another aspect of the above-mentioned information processing device, when the index indicating the deviation of the measured point from the straight line is equal to or greater than a threshold value, the straight line calculation unit calculates the straight line based on the nearest point searched from the measured point based on the straight line calculated by the straight line calculation unit before the current processing time. With this aspect, the information processing device can accurately calculate the straight line along the berthing place even when there is a deviation of the measurement data from the straight line.

[0049] In another embodiment of the information processing device, when the index indicating the deviation of the measured point from the straight line is equal to or greater than a threshold value and the straight line does not exist within a predetermined time relative to the current processing time, the straight line calculation unit calculates the straight line based on the nearest point searched from the measured point based on a straight line parallel to the direction of the ship. With this embodiment, the information processing device can accurately calculate a straight line along the berthing place even when there is a deviation of the measurement data from the straight line and there is no nearest calculated straight line.

[0050] In another embodiment of the above-mentioned information processing device, the information processing device further comprises: a shortest distance to shore calculation unit, which extracts the shortest length of the perpendicular lines drawn from the contour point of the ship to the straight line, and calculates the shortest distance from the hull of the ship to the berthing place; and a close position determination unit, which determines the part of the ship closest to the berthing place. With this embodiment, the information processing device can appropriately calculate the shortest distance from the hull of the ship to the berthing place, and determine the part of the ship closest to the berthing place.

[0051] According to another preferred embodiment of the present disclosure, there is a control method executed by a computer, wherein measurement data is obtained, the measurement data being a collection of data representing a plurality of measured points measured by a measuring device installed on a ship, a straight line along a berthing place where the ship is to dock is calculated based on the measurement data, and the distance between the ship and the berthing place is calculated based on the distance between the straight line and each of the measured points. By executing the control method, the computer can appropriately calculate the distance between the ship and the berthing place.

[0052] According to another preferred embodiment of the present disclosure, the program causes a computer to perform the following processing: obtaining measurement data, which is a collection of data representing a plurality of measured points measured by a measuring device installed on a ship, that is, measurement data, calculating a straight line along a berthing place where the ship is to dock based on the measurement data, and calculating the distance between the ship and the berthing place based on the distance between the straight line and each of the measured points. By executing the program, the computer can appropriately calculate the distance between the ship and the berthing place. Preferably, the above-mentioned program is stored in a storage medium.

[0053] Example

[0054] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0055] (1) Overview of Driver Assistance Systems

[0056] Figure 1 (A)~ Figure 1 (C) is a schematic structure of the navigation assistance system involved in this embodiment. Specifically, Figure 1 (A) shows a block diagram of the navigation assistance system, Figure 1 (B) is a top view illustrating a visual field range (also referred to as a “measuring range” or “measurable range”) 90 of a ship included in the navigation assistance system and a laser radar 3 described later. Figure 1 (C) is a diagram showing a ship and a field of view 90 of the laser radar 3 from the rear side. The navigation assistance system includes an information processing device 1 that moves with a ship as a mobile body and a sensor group 2 mounted on the ship. Hereinafter, a ship equipped with a driving assistance system is also referred to as a "target ship".

[0057] The information processing device 1 is electrically connected to the sensor group 2, and performs navigation assistance for the target ship based on the outputs of various sensors included in the sensor group 2. The navigation assistance includes docking assistance such as automatic docking (landing). Here, "docking" includes not only the case of bringing the target ship closer to the quay, but also the case of bringing the target ship closer to structures such as piers. In addition, hereinafter, "docking place" is a general term for structures such as quays and piers that are targets for docking. The information processing device 1 can be a navigation device installed in a ship, or it can be an electronic control device built into the ship.

[0058] The sensor group 2 includes various external sensors and internal sensors installed on the ship. In this embodiment, the sensor group 2 includes a Lidar (Light Detection and Ranging or Laser Illuminated Detection And Ranging) 3, for example.

[0059] The laser radar 3 is a radar that detects the horizontal direction within a predetermined angle range (see Figure 1 (B)) and the specified angle range in the vertical direction (refer to Figure 1 (C)) is an external sensor that emits pulsed laser light to discretely measure the distance to an object existing in the outside world and generates three-dimensional dot matrix data representing the position of the object.

[0060] The laser radar 3 includes an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scanning data based on the light receiving signal output by the light receiving unit. The data measured for each direction (scanning position) of the irradiated laser light is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light determined based on the above-mentioned light receiving signal. Hereinafter, the point or its data measured by being irradiated with laser light within the measurement range of the laser radar 3 is also referred to as a "measured point".

[0061] exist Figure 1 (B) and Figure 1 In the example of (C), as laser radars 3, a laser radar facing the front of the port side of the target ship, a laser radar facing the rear of the port side of the target ship, a laser radar facing the front of the starboard side of the target ship, and a laser radar facing the rear of the starboard side of the target ship are respectively installed on the ship. And, when the target ship approaches the berthing place, the laser radar 3 in front of the target ship and the laser radar 3 in the rear of the target ship, which are installed on the side where the target ship is berthing toward the berthing place, respectively generate dot matrix data measuring the berthing place. Hereinafter, the laser radar 3 measuring the front of the berthing place will be referred to as the "front laser radar", and the laser radar 3 measuring the rear of the berthing place will be referred to as the "rear laser radar". The front measuring laser radar and the rear measuring laser radar are examples of the "first measuring device" and the "second measuring device". It should be noted that the configuration of the laser radar 3 is not limited to Figure 1 (B) and Figure 1 Example of (C).

[0062] The laser radar 3 is not limited to the scanning laser radar described above, but may be a flash laser radar that generates three-dimensional data by diffusing laser light into the field of view of a two-dimensional array sensor. The laser radar 3 is an example of the "measuring device" in the present invention.

[0063] (2) Structure of information processing device

[0064] Figure 2 1 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly includes an interface 11, a memory 12, and a controller 13. These elements are connected to each other via a bus.

[0065] The interface 11 performs interface operations related to the exchange of data between the information processing device 1 and the external device. In this embodiment, the interface 11 obtains output data from each sensor of the sensor group 2 and supplies it to the controller 13. In addition, the interface 11 supplies, for example, a signal related to the control of the target ship generated by the controller 13 to each component of the target ship that controls the driving of the target ship. For example, the target ship has a driving source such as an engine or an electric motor, a propeller that generates a propulsion force in the direction of travel based on the driving force of the driving source, a thruster that generates a lateral propulsion force based on the driving force of the driving source, and a mechanism for freely determining the direction of travel of the ship, i.e., a rudder. In addition, during automatic driving such as automatic docking, the interface 11 supplies the control signal generated by the controller 13 to these components. It should be noted that when the target ship is provided with an electronic control device, the interface 11 supplies the control signal generated by the controller 13 to the electronic control device. The interface 11 can be a wireless interface such as a network adapter for wireless communication, or a hardware interface for connecting to an external device through a cable or the like. In addition, the interface 11 can also perform interface operations with various peripheral devices such as an input device, a display device, and a sound output device.

[0066] The memory 12 is composed of various volatile memories and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. The memory 12 stores a program for the controller 13 to execute a predetermined process. It should be noted that the program executed by the controller 13 may also be stored in a storage medium other than the memory 12.

[0067] In addition, the memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, map data including information related to the location of the docking place may be stored in the memory 12. In another example, the memory 12 stores information related to the size of downsampling when the dot matrix data obtained when the laser radar 3 performs one cycle of scanning is downsampled.

[0068] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to driving assistance of the target ship by executing a program stored in the memory 12 or the like.

[0069] In addition, the controller 13 functionally has a docking place detection unit 15 and a docking parameter calculation unit 16. The docking place detection unit 15 performs processing related to the detection of the docking place based on the dot matrix data output by the laser radar 3. The docking parameter calculation unit 16 calculates the parameters required for docking at the docking place (also referred to as "dockage parameters"). Here, the docking parameters include the distance from the target ship to the docking place (distance to the opposite shore), the entry angle of the target ship to the docking place, the speed at which the target ship approaches the docking place (dockage speed), etc. In addition, the docking parameter calculation unit 16 calculates information indicating the reliability related to the docking at the docking place (also referred to as "reliability information") based on the processing results of the docking place detection unit 15 and the docking parameters. In addition, the controller 13 functions as an "acquisition unit", "straight line calculation unit", "nearest point search unit", "distance calculation unit", and a computer that executes a program.

[0070] It should be noted that the processing executed by the controller 13 is not limited to being implemented by software based on the program, but may also be implemented by any combination of hardware, firmware, and software. In addition, the processing executed by the controller 13 may also be implemented using an integrated circuit that can be programmed by a user, such as an FPGA (Field-Programmable Gate Array) or a microcomputer. In this case, the program executed by the controller 13 in this embodiment may also be implemented using the integrated circuit.

[0071] (3) Overview of Assisted Beaching

[0072] Next, the outline of the docking assistance processing performed by the information processing device 1 is described. The information processing device 1 generates a straight line along the side of the docking place (also referred to as "dockage side straight line L") based on the dot matrix data generated by the front laser radar and the dot matrix data generated by the rear laser radar. In addition, the information processing device 1 calculates docking parameters such as the distance to the opposite shore based on the docking side straight line L.

[0073] Figure 3 This is a functional block diagram of a docking place detection unit 15 and a docking parameter calculation unit 16 related to the docking auxiliary processing. The docking place detection unit 15 functionally includes a normal vector calculation block 20, a field of view·detection surface determination block 21, a normal number determination block 22, an average·variance calculation block 23, and a docking condition determination block 24. In addition, the docking parameter calculation unit 16 functionally includes a straight line calculation block 27, an opposite shore distance calculation block 28, an approach angle calculation block 29, a docking speed calculation block 30, and a reliability information generation block 40.

[0074] The normal vector calculation block 20 calculates the normal vector of the surface formed by the berthing place (also referred to as the "berthing surface") based on the dot matrix data generated by the laser radar 3 in the direction of the berthing place. In this case, the normal vector calculation block 20 calculates the normal vector based on the dot matrix data generated by the front laser radar and the rear laser radar that include the berthing side in the measurement range of the target ship. Information related to the measurement range of the laser radar 3 and the direction of the berthing place can be pre-registered in, for example, the memory 12.

[0075] In this case, it is preferable that the normal vector calculation block 20 performs downsampling of the dot data and removal of data obtained by reflection of the laser light on the water surface (also referred to as “water surface reflection data”).

[0076] In this case, first, the normal vector calculation frame 20 removes the data that exists below the water surface position as water surface reflection data (i.e., false detection data) for the dot matrix data generated by the laser radar 3. It should be noted that the normal vector calculation frame 20 estimates the water surface position based on, for example, the average value in the height direction of the dot matrix data generated by the laser radar 3 when there are no objects other than the water surface in the surrounding area. Then, the normal vector calculation frame 20 downsamples the dot matrix data after the water surface reflection data is removed. Downsampling is a process of integrating the measured points for each grid space of a specified size. Then, the normal vector calculation frame 20 calculates the normal vector for each measured point represented by the downsampled dot matrix data using multiple measured points in the surrounding area. It should be noted that downsampling can also be performed before removing the data obtained by water surface reflection.

[0077] The field of view·detection surface determination frame 21 determines the surface of the docking place existing in the field of view angle of the laser radar 3 (also referred to as "field of view internal surface") and the surface of the docking place detected based on the normal vector calculated by the normal vector calculation frame 20 (also referred to as "detection surface"). In this case, the field of view·detection surface determination frame 21 determines whether the field of view internal surface and the detection surface include the upper surface and / or the side surface of the docking place.

[0078] The normal number determination block 22 extracts the normal vectors in the vertical direction and the normal vectors in the direction perpendicular thereto (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation block 20, and calculates the number of normal vectors in the vertical direction and the number of normal vectors in the horizontal direction. Here, the normal number determination block 22 regards the normal vector in the vertical direction as a vector representing the normal of the measured point relative to the upper surface of the berthing place, and regards the normal vector in the horizontal direction as a vector representing the normal of the measured point relative to the side surface of the berthing place, and calculates the number of each as an index of the reliability related to the berthing place.

[0079] The mean / variance calculation frame 23 extracts the normal vectors in the vertical direction and the normal vectors in the direction perpendicular to the vertical direction (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation frame 20, and calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction.

[0080] The docking condition determination frame 24 obtains the processing results of the field of view and detection surface determination frame 21, the number of normals determination frame 22, and the average and variance calculation frame 23 determined or calculated based on the same dot matrix data as the determination result indicating the detection condition of the docking place at the time point of generation of the dot matrix data. In addition, the docking condition determination frame 24 supplies the processing results of the field of view and detection surface determination frame 21, the number of normals determination frame 22, and the average and variance calculation frame 23 to the docking parameter calculation unit 16 as the determination result of the detection condition of the docking place.

[0081] The straight line calculation frame 27 extracts the dot data (also referred to as "side dot data") constituting the side of the berthing place from the dot data generated by the front laser radar and the dot data generated by the rear laser radar based on the processing result of the normal vector calculation frame 20. For example, the straight line calculation frame 27 extracts the data of the measured point that becomes the normal vector facing the horizontal direction as the side dot data. In addition, the straight line calculation frame 27 generates the shore side straight line L (in detail, the formula representing the straight line) based on the nearest point searched from the side dot data of the front laser radar and the nearest point searched from the side dot data of the rear laser radar. The details of the calculation method of the shore side straight line L will be described later.

[0082] The shore distance calculation frame 28 calculates the shore distance equivalent to the shortest distance between the target ship and the berthing place based on the shore side straight line L generated by the linear generation frame 27. Here, the shore distance calculation frame 28, for example, calculates the shortest distance between the shore side straight line L and each laser radar 3 (i.e., each of the front laser radar and the rear laser radar) as the shore distance. In addition, the shortest distance from a reference point such as the center position of the ship to the shore side straight line L can also be calculated as the shore distance. It should be noted that the shore distance calculation frame 28 can also determine the shortest distance among the shortest distances of each laser radar 3 as the shore distance instead of considering the shortest distance of each laser radar 3 as the shore distance, and can also determine the average of these shortest distances as the shore distance.

[0083] The approach angle calculation block 29 calculates the approach angle of the target ship relative to the berthing place based on the berthing side straight line L generated by the straight line generation block 27. Specifically, the approach angle calculation block 29 calculates the approach angle using a function "atan2" that obtains an arctangent from two independent variables of a predetermined tangent. More specifically, the approach angle calculation block 29 calculates the approach angle based on the direction vector of the berthing side straight line L by calculating the function atan2.

[0084] The approach speed calculation block 30 calculates the approach speed, which is the speed at which the target ship approaches the berthing place, based on the shore distance calculated by the shore distance calculation block 28. For example, the approach speed calculation block 30 calculates the time change of the shore distance (shortest distance) as the approach speed.

[0085] The reliability information generation block 40 generates reliability information based on the processing results of the approaching state determination block 24, the straight line calculation block 27, the opposite bank distance calculation block 28, and the approach angle calculation block 29. Details of the reliability information will be described later.

[0086] Next, for a specific example of the processing of the normal vector calculation block 20, the normal number determination block 22, and the mean / variance calculation block 23, refer to Figure 4 To explain.

[0087] Figure 4 (A) shows an example of a hull coordinate system based on the hull of the target ship. Figure 4 As shown in (A), the front (forward) direction of the target ship is set as the "x" coordinate, the side direction of the target ship is set as the "y" coordinate, and the height direction of the target ship is set as the "z" coordinate. In addition, the measurement data of the coordinate system based on the laser radar 3 measured by the laser radar 3 is transformed into Figure 4 The hull coordinate system shown in (A). It should be noted that the process of transforming the dot matrix data of the coordinate system based on the laser radar installed on the mobile body into the coordinate system of the mobile body is disclosed in, for example, the international publication WO2019 / 188745.

[0088] Figure 4 (B) shows a situation where the laser radar 3 captures the quay wall serving as the berthing place. Figure 4 (C) is a stereoscopic view of the berthing place (the quay in this case) showing the measured points indicating the measured positions measured by the laser radar 3 and the normal vectors calculated based on the measured points relative to the quay as the berthing place. Figure 4 In (C), the measured point is represented by a circle, and the normal vector is represented by an arrow. Here, an example in which both the upper surface and the side surface of the bank are measured by the laser radar 3 is shown.

[0089] like Figure 4 As shown in (C), the normal vector calculation frame 20 calculates the normal vector of the measured point relative to the side and upper surface of the quay. The normal vector is a vector perpendicular to the plane or curved surface set as the object, so it is calculated using a plurality of measured points that can be formed into a surface. Therefore, a grid with a predetermined length in length and width or a circle with a predetermined radius is set, and the measured points existing inside it are used for calculation. In this case, the normal vector calculation frame 20 can calculate the normal vector for each measured point, or it can calculate the normal vector at predetermined intervals. In addition, the normal number determination frame 22 determines the normal vector whose z component is larger than the predetermined threshold as the normal vector facing the vertical direction. It should be noted that the normal vector is unitized. In addition, the normal vector whose z component is smaller than the predetermined threshold is determined as the normal vector facing the horizontal direction. In addition, the normal number determination frame 22 determines the number of normal vectors in the vertical direction (here 5) and the number of normal vectors in the horizontal direction (here 4). The mean and variance calculation block 23 calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction. It should be noted that, regarding the edge portion, since the measured points around it are the upper surface or the side surface, the direction is oblique.

[0090] (4) Details of the method for generating the shore side straight line

[0091] Next, the generation method (first generation method and second generation method) of the shore side surface straight line L will be described.

[0092] (4-1) The first generation method

[0093] In the first generation method, first, the shore parameter calculation unit 16 generates a tentative shore side straight line L (also referred to as "first shore side straight line L1") based on the side dot matrix data of the front laser radar, and generates a tentative shore side straight line L (also referred to as "second shore side straight line L2") based on the side dot matrix data of the rear laser radar. Then, the shore parameter calculation unit 16 searches for the nearest point from the side dot matrix data of the front laser radar based on the first shore side straight line L1, and searches for the nearest point from the side dot matrix data of the rear laser radar based on the second shore side straight line L2. Thereafter, the shore parameter calculation unit 16 generates a straight line connecting these nearest points to each other, uses the generated straight line to search for the nearest point again through the same process, and generates a straight line connecting the searched nearest points. Then, when it is determined that the formula of the generated straight line no longer changes and has converged, the shore parameter calculation unit 16 sets the converged straight line as the shore side straight line L. Through this processing, the docking parameter calculation unit 16 extracts the point at the top of the docking place as the nearest point, and appropriately determines the straight line passing through the top of the docking place as the docking side straight line L. It should be noted that, as described later, the process of repeatedly calculating the straight line is preferably executed and continued only when there is a margin from the next dot matrix data output time (i.e., the next processing time based on the frame period of the dot matrix data).

[0094] Figure 5 (A) to (C) are diagrams showing the process flow of the first generation method. First, Figure 5 As shown in (A), the berthing parameter calculation unit 16 generates a first berthing side straight line L1 based on the side dot matrix data of the front laser radar, and uses this straight line to extract the nearest point "p1" closest to the target ship from the measured points of the side dot matrix data of the front laser radar. Similarly, the berthing parameter calculation unit 16 generates a second berthing side straight line L2 based on the side dot matrix data of the rear laser radar, and uses this straight line to extract the nearest point "p2" closest to the target ship from the measured points of the side dot matrix data of the rear laser radar.

[0095] Then, if Figure 5 As shown in (B), the berthing parameter calculation unit 16 generates a provisional straight line "L'" connecting the nearest point p1 and the nearest point p2, and uses the provisional straight line L' to extract the nearest point "p'1" from the measured point of the side dot matrix data of the front laser radar, and extract the nearest point "p'2" from the measured point of the side dot matrix data of the rear laser radar. Then, as Figure 5As shown in (C), the shore parameter calculation unit 16 generates a line "L" connecting the nearest point p'1 and the nearest point p'2. Since the provisional straight line L' and the provisional straight line L'' are approximate, the provisional straight line L'' finally calculated is set as the shore side straight line L. It should be noted that in the approximate judgment of the provisional straight line L' and the provisional straight line L'', for example, the shore parameter calculation unit 16 takes the difference of each element of the x, y, and z of each other's unit vectors, and judges that they are not approximate when the maximum value is greater than the threshold, and judges that they are approximate when it is less than the threshold. Thereafter, the shore parameter calculation unit 16 calculates the distance to the opposite shore based on the shore side straight line L. The provisional straight line L' is an example of the "first straight line", and the provisional straight line L'' is an example of the "second straight line". In addition, the nearest point p1 and the nearest point p'1 are examples of the "first nearest point", and the nearest point p2 and the nearest point p'2 are examples of the "second nearest point".

[0096] Here, about Figure 5 For details of the method for calculating the nearest point p1 shown in (A), refer to Figure 6 (A) and Figure 6 (B) to illustrate. Figure 6 (A) is a top view showing the first shore side straight line L1 and the nearest point p1. Figure 6 (B) is a diagram clearly showing the unit vector "u" used in the search for the nearest point p1.

[0097] First, the shore approach parameter calculation unit 16 generates a first shore approach side surface straight line L1 represented by the following equation (1) by applying principal component analysis and least square method to the side surface dot matrix data of the front laser radar.

[0098]

[0099] Here, "[x0 y0 z0] T " represents the center of gravity of the measured points represented by the side dot matrix data of the front laser radar, "[abc] T " represents a direction vector, and "t" represents a mediating variable. For example, in the case of principal component analysis, the eigenvector corresponding to the largest eigenvalue becomes the direction vector of the first shore side straight line L1.

[0100] Then, the shore approach parameter calculation unit 16 calculates a unit vector u perpendicular to the first shore approach side surface straight line L1 in the two-dimensional plane as shown in the following equation (2).

[0101]

[0102] Furthermore, since the inner product of the (x, y) component of the data p of the measured point representing the side dot matrix data of the front laser radar and the unit vector u is equivalent to the projection of the data p onto the vector u, Figure 6 The following equation (3) related to the distance "d" shown in (B) holds.

[0103]

[0104] Thus, the approach parameter calculation unit 16 calculates the distance d based on the following equation (4).

[0105]

[0106] Then, the shore parameter calculation unit 16 calculates the distance based on the formula (4) for each measured point represented by the side dot matrix data of the front laser radar, and extracts the data of the measured point representing the shortest distance as the nearest point p1. Similarly, the shore parameter calculation unit 16 calculates the distance based on the formula (4) for each measured point represented by the side dot matrix data of the rear laser radar, and extracts the data of the measured point representing the shortest distance as the nearest point p2.

[0107] Next, supplementary explanation will be given on the provisional straight line L′ based on the nearest point p1 and the nearest point p2. Figure 7 (A)~ Figure 7 (C) is a diagram showing the flow of calculation of the provisional straight line L' based on the nearest point p1 and the nearest point p2 and calculation of the distance to the other bank based on the provisional straight line L'. Figure 7 (A) is a diagram showing the nearest point p1 searched based on the first shore side straight line L1 and the nearest point p2 searched based on the second shore side straight line L2. Figure 7 (B) is a diagram showing a provisional straight line L′ connecting the nearest point p1 and the nearest point p2. Figure 7 (C) shows the process of calculating the length of the perpendicular line from the position of the front laser radar to the provisional straight line L' and the length of the perpendicular line from the position of the rear laser radar to the provisional straight line L' as the distance to the other side.

[0108] Here, if Figure 7 (A)~ Figure 7As shown in (C), the provisional straight line L′ connecting the nearest point p1 searched from the side dot matrix data of the front laser radar and the nearest point p2 searched from the side dot matrix data of the rear laser radar is roughly parallel to the side of the docking place. Therefore, by using such a provisional straight line L′ as the docking side straight line L, the information processing device 1 can accurately calculate various docking parameters such as the distance to the opposite shore. It should be noted that in order to calculate the length of the perpendicular line from the front and rear reference points to the docking side straight line L, respectively, Figure 6 The point p in (B) may be set to be either the nearest point p1 or the nearest point p2 on the shore side straight line L, and can be calculated by equation (4).

[0109] In addition, it is preferable that the shore-reaching parameter calculation unit 16 updates the provisional straight line L' to make the shore-reaching side straight line L more accurate when there is a margin to the next dot matrix data output time. In this case, for example, when the length of time from the current time to the next dot matrix data output time is greater than a predetermined threshold, the shore-reaching parameter calculation unit 16 determines that there is a margin to the next dot matrix data output time.

[0110] Figure 8 (A)~ Figure 8 (C) shows the flow of processing for calculating the tentative straight line L'' after updating the tentative straight line L'. Figure 8 (A) is a diagram showing the nearest point p1 searched based on the first shore side straight line L1 and the nearest point p2 searched based on the second shore side straight line L2. Figure 8 (B) is a diagram showing the provisional straight line L′ connecting the nearest point p1 and the nearest point p2, the nearest point p′1 searched from the side dot matrix data of the front laser radar based on the provisional straight line L′, and the nearest point p′2 searched from the side dot matrix data of the rear laser radar based on the provisional straight line L′. Figure 8 (C) is a diagram showing a provisional straight line L″ connecting the nearest point p′1 and the nearest point p′2.

[0111] In this case, the shore approach parameter calculation unit 16 compares the direction vectors of the first shore approach side line L1 and the second shore approach side line L2 with the direction vector of the provisional line L', and when the difference between them is greater than a predetermined threshold, it is determined that the processing has not converged and the provisional line L' needs to be updated. Figure 8 As shown in (B), the shore parameter calculation unit 16 uses the provisional straight line L' to search for the nearest point p'1 from the side dot matrix data of the front laser radar and the nearest point p'2 from the side dot matrix data of the rear laser radar. Then, as Figure 8As shown in (C), the shore parameter calculation unit 16 calculates the provisional straight line L'' connecting the nearest point p'1 and the nearest point p'2, and regards the provisional straight line L'' as the shore side straight line L to calculate the distance to the shore. It can be seen that: Figure 8 The two arrows in (C) that indicate the distance to the opposite shore are Figure 7 By doing so, the shore approach parameter calculation unit 16 can more accurately calculate the shore approach side straight line L and accurately calculate various shore approach parameters such as the opposite shore distance.

[0112] It should be noted that the detailed processing flow of the first generation method will be referred to Fig.15 And then described.

[0113] Here, the effect of the first generation method is supplemented. Generally speaking, in order to find the distance to the opposite shore, it is necessary to extract the point closest to the top of the berthing place (i.e., the nearest point) from the dot matrix data irradiated to the berthing place in a two-dimensional plane. On the other hand, depending on the measurement direction of the laser radar 3 and the orientation of the ship relative to the berthing place, it is impossible to accurately extract the nearest point by simply comparing the distance from the laser radar 3 to each point.

[0114] Fig. 9 (A) is a diagram that clearly indicates the nearest points to be extracted. Fig. 9 (B) shows an example of extracting the data of the measured point with the shortest measurement distance of the laser radar 3 as the nearest point. Fig. 9 (C) shows an example of comparing the values ​​of only the y coordinates of the ship coordinate system and extracting the point closest to the ship origin as the nearest point. In the method of extracting the data of the measured point with the shortest measurement distance of the laser radar 3 as the nearest point (refer to Fig. 9 (B)) A method of comparing only the y coordinate values ​​of the ship coordinate system and extracting the point closest to the ship origin as the nearest point (refer to Fig. 9 In (C)), the point closest to the top of the quay wall (refer to Fig. 9 (A)) is not accurately extracted as the nearest point. Considering the above, in the first generation method, the docking parameter calculation unit 16 respectively searches for the nearest point p1 searched using the first docking side straight line L1 based on the dot data of the front laser radar and the nearest point p2 searched using the second docking side straight line L2 based on the dot data of the rear laser radar, and calculates the straight line connecting them. Thereby, the docking side straight line L passing through the top of the docking place can be generated, so various docking parameters such as the distance to the opposite shore can be accurately calculated. In addition, even if the docking place is a quay wall that lacks flatness, the docking side straight line L passing through the top of the docking place can be generated by repeatedly executing the process of updating the straight line.

[0115] (4-2) Second generation method

[0116] In the second generation method, after calculating the first shore side straight line L1 and the second shore side straight line L2, the shore parameter calculation unit 16 searches for the nearest point based on the shore side straight line L calculated in the past when the variance of the side dot matrix data with respect to each straight line is greater than a predetermined threshold value. It should be noted that "variance" is an example of "an index indicating deviation", and in the following, the standard deviation may be used instead of the variance.

[0117] Generally speaking, when the shore captured by the laser radar 3 is a shore location that lacks flatness, the deviation of the side dot matrix data increases, and the generated shore side straight line L becomes inaccurate. In addition, if the shore side straight line L is generated based on the side dot matrix data relative to the shore wall that lacks flatness, the variance of the measured points represented by the side dot matrix data relative to the shore side straight line L becomes larger.

[0118] Taking the above into consideration, in the second generation method, when the standard deviation of each point of the side lattice data obtained by the shore parameter calculation unit 16 at the current processing time relative to the first shore side straight line L1 and the second shore side straight line L2 is greater than a specified threshold (for example, σ>0.5), the calculation of the shore side straight line L based on the side lattice data is not performed. Fig.10 : is a diagram showing the first bank side straight line L1 and the measured points of the side dot matrix data of the front laser radar. Fig.10 In the figure, the variance of the measured points of the side dot matrix data of the front laser radar relative to the first shore side straight line L1 is clearly shown as “σ 2 ”.

[0119] Furthermore, when there is a shore side straight line L generated within a predetermined time in the past (for example, within 1 second), the shore side parameter calculation unit 16 searches for the nearest point at the current processing time and calculates the shore side straight line L based on the shore side straight line L. On the other hand, when there is no shore side straight line L generated within a predetermined time in the past (for example, within 1 second), the shore side parameter calculation unit 16 searches for the nearest point at the current processing time and calculates the shore side straight line L based on a straight line parallel to the direction of the target ship.

[0120] Fig.11 (A)~ Fig.11 (C) is a diagram showing a process flow when there is a shore side straight line L generated within a predetermined time in the past.

[0121] First, if there is a past shore side straight line L generated within a predetermined time, as shown in FIG. Fig.11As shown in (A), the shore-approaching parameter calculation unit 16 searches for the nearest point p1 and the nearest point p2 from the side dot matrix data of the front laser radar and the side dot matrix data of the rear laser radar, respectively, based on the past shore-approaching side straight line L. In this case, the shore-approaching parameter calculation unit 16 determines the nearest point p1 and the nearest point p2 by calculating the distance to each measured point using a vector perpendicular to the direction vector of the past shore-approaching side straight line L.

[0122] Then, if Fig.11 As shown in (B), the berthing parameter calculation unit 16 generates a provisional straight line L' connecting the nearest point p1 and the nearest point p2, and searches for the nearest point p'1 and the nearest point p'2 from the side dot matrix data of the front laser radar and the side dot matrix data of the rear laser radar respectively based on the provisional straight line L'. Fig.11 As shown in (C), the docking parameter calculation unit 16 generates a provisional straight line L'' passing through the nearest point p'1 and the nearest point p'2. Afterwards, when the provisional straight line L' and the provisional straight line L'' are not approximate, the docking parameter calculation unit 16 repeatedly performs the following processing: using the provisional straight line L'' to search for the nearest point again through the same process, and generates a straight line connecting the searched nearest points. On the other hand, when the docking parameter calculation unit 16 determines that the formula of the generated straight line no longer changes but has converged (that is, the straight lines before and after the update are approximate), the converged straight line is set as the docking side straight line L, and the distance to the shore is calculated, etc. And, in Fig.11 (A)~ Fig.11 In the example of (C), the following situation can be seen: the nearest point gradually approaches the top part of the bank wall, and the straight line approaches the straight line passing through the top part of the bank wall.

[0123] Fig.12 (A)~ Fig.12 (C) is a diagram showing a process flow when there is no shore side surface straight line L generated within the past predetermined time.

[0124] In this case, if Fig.12 As shown in (A), the berthing parameter calculation unit 16 sets the straight lines parallel to the direction of the target ship as the first berthing side straight line L1 and the second berthing side straight line L2, and searches for the nearest point p1 and the nearest point p2 based on the first berthing side straight line L1 and the second berthing side straight line L2. Fig.12 As shown in (B), the berthing parameter calculation unit 16 generates a provisional straight line L' connecting the nearest point p1 and the nearest point p2, and searches for the nearest point p'1 and the nearest point p'2 from the side dot matrix data of the front laser radar and the side dot matrix data of the rear laser radar respectively based on the provisional straight line L'. Fig.12As shown in (C), the docking parameter calculation unit 16 generates a provisional straight line L'' passing through the nearest point p'1 and the nearest point p'2. Afterwards, when the provisional straight line L' and the provisional straight line L'' are not approximate, the docking parameter calculation unit 16 repeatedly performs the following processing: using the provisional straight line L'' to search for the nearest point again through the same process, and generates a straight line connecting the searched nearest points. On the other hand, when the docking parameter calculation unit 16 determines that the formula of the generated straight line no longer changes but has converged (that is, the straight lines before and after the update are approximate), the converged straight line is set as the docking side straight line L, and the distance to the shore is calculated, etc. And, in Fig.12 (A)~ Fig.12 In the example of (C), it can be seen that the nearest point gradually approaches the top portion of the bank wall, and the straight line approaches the straight line passing through the top portion of the bank wall.

[0125] It should be noted that, if there is no shore side straight line L generated within the past specified time, if the angle between the target ship and the side of the quay is large, the difference between the straight line of the initial value and the shore side straight line L that should have been calculated is large, so there is a tendency that the number of repetitions until convergence increases. Therefore, it is preferred that the shore parameter calculation unit 16 stops updating the straight line halfway when there is no margin before the output time of the next laser radar 3, and uses the last calculated straight line as the shore side straight line L. In this case, since the straight line close to convergence is used as the next initial value, by continuing this process, convergence can also be completed within the time.

[0126] For the detailed processing flow of the second generation method, please refer to Fig.16 And then described.

[0127] (5) Generation of reliability information

[0128] Next, a specific example of generating reliability information is described. The reliability information generating frame 40 generates flags for each element such as the viewing angle during the detection of the docking place, the surface detection of the docking place, the number of normal vectors and the variance, and generates the vector of the generated flags as reliability information. In the future, when the flag is "1", it means that the reliability of the corresponding element is high, and when the flag is "0", it means that the reliability of the corresponding element is low.

[0129] Fig.13 FIG. 4 shows an example of the data structure of the reliability information generated by the reliability information generation block 40. Fig.13As shown, the reliability information has items of "upper surface", "side", "straight line", "distance", and "angle". In addition, the item "upper surface" has sub-items of "viewing angle", "detection", "number of normals", and "variance", and the item "side" has sub-items of "viewing angle", "detection", "number of normals", and "variance". In addition, the item "straight line" has a sub-item of "variance", the item "distance" has sub-items of "change amount" and "change rate", and the item "angle" has a sub-item of "change amount".

[0130] Here, the reliability information generation frame 40 registers the sub-item “viewing angle” of the item “upper surface” as “1” when the upper surface of the berthing place is within the viewing angle range, and as “0” when the upper surface is outside the viewing angle. In addition, the reliability information generation frame 40 registers the sub-item “detection” of the item “upper surface” as “1” when the upper surface of the berthing place is a detection surface, and as “0” when the upper surface is not a detection surface. In addition, the reliability information generation frame 40 registers the sub-item “number of normals” of the item “upper surface” as “1” when the number of normal vectors relative to the upper surface of the berthing place is greater than a predetermined threshold value (e.g., 10), and as “0” when the number is less than the threshold value. In addition, the reliability information generation block 40 registers a flag that becomes "1" when the variances of the x, y, and z components of the normal vector to the upper surface of the berthing site are all less than a predetermined threshold value (e.g., 1.0) for the sub-item "variance" of the item "upper surface", and becomes "0" when any of the variances is greater than the threshold value. In addition, the reliability information generation block 40 also registers a flag determined by the same rule as that of each sub-item of the item "upper surface" in each sub-item of the item "side".

[0131] In addition, the reliability information generation block 40 registers a flag indicating the reliability of the shore side straight line L calculated by either the first generation method or the second generation method for the sub-item "variance" of the item "straight line". For example, the reliability information generation block 40 registers a flag that becomes "1" when the variance of the side dot matrix data with respect to the shore side straight line L is less than a predetermined threshold value, and becomes "0" when the variance is greater than the threshold value.

[0132] In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "amount of change" of the item "distance" when the amount of change of the opposite shore distance calculated by the opposite shore distance calculation frame 28 relative to 1 moment ago is less than a predetermined threshold value (e.g., 1.0 m), and a flag "0" when the amount of change is greater than the threshold value. In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "rate of change" of the item "distance" when the rate of change of the opposite shore distance calculated by the opposite shore distance calculation frame 28 relative to 1 moment ago is less than a predetermined threshold value (e.g., ±10%), and a flag "0" when the rate of change is greater than the threshold value. In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "amount of change" of the item "angle" when the amount of change of the entry angle calculated by the entry angle calculation frame 29 relative to 1 moment ago is less than a predetermined threshold value (e.g., 1.0 degree), and a flag "0" when the amount of change is greater than the threshold value.

[0133] It should be noted that the above-mentioned threshold values ​​are set to adaptive values ​​pre-stored in the memory 12 , etc. In addition, the reliability information may be generated for each laser radar 3 .

[0134] According to the reliability information having such a data structure, the reliability of the calculated distance to the opposite shore, the berthing speed, and the approach angle can be grasped. It should be noted that when each sub-item of the reliability information is "1", the reliability is the highest. In addition, the information processing device 1 adjusts the output of the driving source when berthing based on the reliability information. For example, the information processing device 1 can determine the upper limit value of the speed of the target ship when berthing based on the total value of each sub-item shown in the reliability information. In this case, for example, the smaller the above-mentioned total value is, the information processing device 1 determines that the reliability of the information related to the berthing place is lower, and it is necessary to dock carefully, and the upper limit value of the speed of the target ship when berthing is reduced.

[0135] (6) Processing Flow

[0136] (6-1) Processing Overview

[0137] Fig.14 This is an example of a flowchart showing the outline of the docking support process in this embodiment. The information processing device 1 repeatedly executes Fig.14 Flowchart of the process.

[0138] First, the information processing device 1 obtains dot matrix data in the direction of the berthing place (step S11). In this case, the information processing device 1 obtains, for example, dot matrix data generated by the laser radar 3 that includes the berthing side in the measurement range of the target ship. In addition, the information processing device 1 may further perform downsampling of the obtained dot matrix data and remove data obtained by reflection on the water surface.

[0139] Next, the docking place detection unit 15 of the information processing device 1 calculates the normal vector based on the dot matrix data obtained in step S11 (step S12). In addition, the docking place detection unit 15 calculates the number of normal vectors and the variance of the normal vectors in step S12. In addition, the docking place detection unit 15 determines the inner surface of the field of view and the detection surface based on the processing result of step S12 (step S13).

[0140] Next, the shore parameter calculation unit 16 performs a shore side straight line calculation process for calculating the shore side straight line L (step S14). Fig.15 As will be described later, regarding the calculation process of the shore side straight line based on the second generation method, reference will be made to Fig.16 And then described.

[0141] Next, the shore approach parameter calculation unit 16 calculates the shore approach distance, approach angle, and shore approach speed as shore approach parameters using the shore approach side surface straight line L calculated in step S15 (step S15 ).

[0142] Then, the berthing parameter calculation unit 16 generates reliability information based on the processing results in steps S13 to S15 (step S16). Thereafter, the information processing device 1 controls the ship based on the reliability information (step S17). Thus, the information processing device 1 can accurately perform the control of the ship related to the berthing based on the reliability that accurately reflects the berthing condition.

[0143] Then, the information processing device 1 determines whether the target ship has docked (landed) (step S18). In this case, the information processing device 1 determines whether the target ship has docked, for example, based on the output signal of the sensor group 2 or the user input via the interface 11. And, if the information processing device 1 determines that the target ship has docked (step S18; yes), the processing of the flowchart ends. On the other hand, if the target ship has not docked (step S18; no), the information processing device 1 returns the processing to step S11.

[0144] (6-2) Calculation of the shore side straight line based on the first generation method

[0145] Fig.15 This is an example of a flowchart of the calculation process of the shore side straight line based on the first generation method. Fig.15 The process shown in the flowchart is Fig.14 The process is performed in step S14.

[0146] First, the shore parameter calculation unit 16 extracts side dot matrix data from the dot matrix data generated by the front laser radar and the rear laser radar respectively (step S21). Then, the shore parameter calculation unit 16 generates a first shore side straight line L1 based on the side dot matrix data of the front laser radar, and calculates a second shore side straight line L2 based on the side dot matrix data of the rear laser radar (step S22).

[0147] Then, the shore parameter calculation unit 16 searches for the nearest point p1 and the nearest point p2 corresponding to the first shore side straight line L1 and the second shore side straight line L2, respectively (step S23). In this case, the shore parameter calculation unit 16 determines the nearest point p1 based on the inner product of the vector perpendicular to the first shore side straight line L1 and each data of the side dot matrix data of the front laser radar (position vector of the measured point), and determines the nearest point p2 based on the inner product of the vector perpendicular to the second shore side straight line L2 and each data of the side dot matrix data of the rear laser radar.

[0148] Then, the shore parameter calculation unit 16 calculates the provisional straight line L′ connecting the nearest point p1 and the nearest point p2 (step S24). Then, the shore parameter calculation unit 16 determines whether the provisional straight line L′ is approximate to both the first shore side straight line L1 and the second shore side straight line L2 (step S25). For example, the shore parameter calculation unit 16 determines that the provisional straight line L′ is approximate to the first shore side straight line L1 and the second shore side straight line L2 when the maximum difference between the elements of the unit vectors of the provisional straight line L′ and the first shore side straight line L1 is less than a specified value and the maximum difference between the elements of the unit vectors of the provisional straight line L′ and the second shore side straight line L2 is less than a specified value, and determines that the provisional straight line L′ is not approximate to the first shore side straight line L1 and the second shore side straight line L2 when the above conditions are not met.

[0149] Furthermore, when the provisional straight line L' is not close to at least one of the first shore side straight line L1 and the second shore side straight line L2 (step S25; No), the shore parameter calculation unit 16 determines whether there is a time margin from the output time of the next laser radar 3 dot matrix data (step S26). Furthermore, when the provisional straight line L' is close to both the first shore side straight line L1 and the second shore side straight line L2 (step S25; Yes), or when there is no margin from the output time of the next laser radar 3 dot matrix data (step S26; No), the shore parameter calculation unit 16 sets the provisional straight line as the shore side straight line L (step S30).

[0150] On the other hand, when there is a time margin (step S26; yes), the docking parameter calculation unit 16 searches for the nearest points (nearest point p1 and nearest point p2) relative to the provisional straight line (step S27). Then, the docking parameter calculation unit 16 newly calculates a provisional straight line connecting the nearest points (step S28). Then, the docking parameter calculation unit 16 determines whether the newly calculated provisional straight line is approximate to the previous provisional straight line (step S29). And, when the newly calculated provisional straight line is approximate to the previous provisional straight line (step S29; yes), the docking parameter calculation unit 16 sets the provisional straight line as the shore side straight line L (step S30). On the other hand, when the newly calculated provisional straight line is not approximate to the previous provisional straight line (step S29; no), the docking parameter calculation unit 16 returns the processing to step S26.

[0151] (6-3) Calculation of the shore side straight line based on the second generation method

[0152] Fig.16 This is an example of a flowchart of the calculation process of the shore side straight line based on the second generation method. Fig.16 The process shown in the flowchart is Fig.14 The process is performed in step S14.

[0153] First, the shore-approaching parameter calculation unit 16 extracts side dot matrix data from the dot matrix data generated by the front laser radar and the rear laser radar respectively (step S31). Then, the shore-approaching parameter calculation unit 16 calculates the first shore-approaching side straight line L1 based on the side dot matrix data of the front laser radar, and calculates the second shore-approaching side straight line L2 based on the side dot matrix data of the rear laser radar (step S32).

[0154] Then, the shore parameter calculation unit 16 calculates the variance of the side dot matrix data of the front laser radar with respect to the first shore side straight line L1 and the variance of the side dot matrix data of the front laser radar with respect to the second shore side straight line L2 (step S33). Then, the shore parameter calculation unit 16 determines whether any of the calculated variances is greater than a specified value (step S34).

[0155] Furthermore, when any one of the above-mentioned variances is greater than or equal to a predetermined value (step S34; yes), the shore parameter calculation unit 16 determines that the first shore side straight line L1 and the second shore side straight line L2 should not be used, and the processing proceeds to step S42. Then, the shore parameter calculation unit 16 determines whether there is a previous shore side straight line L generated within the past predetermined time (step S42). And, when there is a previous shore side straight line L generated within the past predetermined time (step S42; yes), the shore side straight line L is set as a provisional straight line (step S44), and then the processing proceeds to step S39. On the other hand, when there is no shore side straight line L generated within the past predetermined time (step S42; no), the shore parameter calculation unit 16 generates straight lines parallel to the direction of the target ship as the first shore side straight line L1 and the second shore side straight line L2 (step S43).

[0156] On the other hand, when the above-mentioned variances are all less than the prescribed value (step S34; No), or after the execution of step S43, the shore-approaching parameter calculation unit 16 searches for the nearest points corresponding to the first shore-approaching side straight line L1 and the second shore-approaching side straight line L2, respectively (step S35). Then, after step S35, the shore-approaching parameter calculation unit 16 calculates a provisional straight line connecting the nearest points (step S36).

[0157] Then, the shore parameter calculation unit 16 determines whether the provisional straight line is close to both the first shore side straight line L1 and the second shore side straight line L2 (step S37). Furthermore, when the provisional straight line is not close to at least one of the first shore side straight line L1 and the second shore side straight line L2 (step S37; No), the shore parameter calculation unit 16 determines whether there is a time margin from the output time of the next laser radar 3 dot matrix data (step S38). Furthermore, when the provisional straight line L′ is close to both the first shore side straight line L1 and the second shore side straight line L2 (step S37; Yes), or when there is no margin from the output time of the next laser radar 3 dot matrix data (step S38; No), the shore parameter calculation unit 16 sets the provisional straight line as the shore side straight line L (step S45).

[0158] On the other hand, when there is a time margin (step S38; yes), the docking parameter calculation unit 16 searches for the nearest points (nearest point p1 and nearest point p2) relative to the provisional straight line (step S39). Then, the docking parameter calculation unit 16 newly calculates a provisional straight line connecting the nearest points (step S40). Then, the docking parameter calculation unit 16 determines whether the newly calculated provisional straight line is similar to the previous provisional straight line (step S41). And, when the newly calculated provisional straight line is similar to the previous provisional straight line (step S41; yes), the docking parameter calculation unit 16 sets the provisional straight line as the shore side straight line L (step S45). On the other hand, when the newly calculated provisional straight line is not similar to the previous provisional straight line (step S41; no), the docking parameter calculation unit 16 returns the processing to step S38.

[0159] (7) Modifications

[0160] (Variant 1)

[0161] The information processing device 1 may also grasp the portion of the ship body closest to the quay (approaching portion) and the shortest distance from the approaching portion to the quay, thereby facilitating the navigation of the ship when landing or leaving the quay.

[0162] In this case, the information processing device 1 determines the above-mentioned close location and the shortest distance through the following (step 1) to (step 3).

[0163] (Step 1) Calculate the distance from a plurality of points representing the outline of the vessel (also referred to as "outline points Po") to the shore side straight line L.

[0164] (Step 2) Extract the minimum value from the calculated distances from the contour point Po to the shore side straight line L

[0165] (Step 3) The extracted minimum value is set as the shortest distance to the bank, and the contour point Po that becomes the minimum value is determined as the portion close to the bank.

[0166] Fig.17 This is a bird's-eye view of the target ship showing the outline points Po. For example, the memory 12 stores the position data indicating the outline position of the target ship, i.e., the outline data. The outline data is data in which a plurality of (here, 24) outline points Po indicating the outline of the target ship are recorded as coordinates of the ship coordinate system. Here, the front (forward) direction of the target ship is set to "X". b " coordinates, set the side direction of the target ship to "Y b " coordinates, and the vertical direction of the target ship is set to "Z b" coordinates. In addition, the measurement data of the coordinate system based on the laser radar 3 measured by the laser radar 3 is transformed into the ship coordinate system. It should be noted that the processing of transforming the dot matrix data of the coordinate system based on the laser radar set on the mobile body into the coordinate system of the mobile body is disclosed in, for example, the international disclosure WO2019 / 188745.

[0167] Fig.18 This is a diagram showing the distances from the contour point Po to the shore side straight line L by arrows. These distances are calculated by finding the length of the perpendicular line from each contour point Po to the shore side straight line L in the same manner as described above. Here, arrows showing the above distances relative to 24 contour points Po are shown. Fig.19 Here, the contour point Po corresponding to the minimum value extracted in (step 2) is highlighted by a frame 90 as the close part, and the shortest distance from the close part to the bank is indicated by an arrow 91.

[0168] Therefore, according to Modification 1, the position of the ship closest to the target quay and the distance from the quay can be grasped, thereby contributing to safer and smoother ship driving assistance.

[0169] (Variant 2)

[0170] Fig. 20 Indicators and reliability included in reliability information according to a modification are shown. Fig.21 It is clearly stated Fig. 20 A bird's-eye view of the target ship and the berth of the indicator shown. It should be noted that the measured point of the berth is also referred to as the "target point" here.

[0171] The index "c3" is an index based on the number of object points, and is represented here as a linear function of the number of object points as an example. The index "c2" is an index based on the standard deviation of the object point, and is represented here as a linear function of the standard deviation of the object point as an example. The index "c1" is an index indicating whether the shore walls on both the front side and the rear side are measured by two laser radars 3. Here, as an example, the case where both sides are measured is set to "1.0", and the case where only one side is measured is set to "0.0". The index "c0" is an index based on the interval between the two ends of the object point (the two ends in the direction of the straight line L on the shore side), and is represented as a linear function of the interval between the above-mentioned two ends. In addition, the indicators c0~c3 are calculated in a manner limited to the range of 0~1.

[0172] The calculation reliability "c" is based on the reliability of each of the above-mentioned indicators c0 to c3, and here, it becomes the weighted average value of the indicators c0 to c3 using the weighting coefficients "w0" to "w3" corresponding to the importance of the indicators c0 to c3. In addition, an example of the setting value of the weighting coefficients w0 to w3 is shown in the figure. Since each indicator c0 to c3 is in the range of 0 to 1, the weighted average value of them, that is, the calculation reliability c, is also calculated as a value in the range of 0 to 1.

[0173] The comprehensive reliability "r" is based on the reliability related to the detection of the side of the quay wall, that is, the side detection reliability "q s ", the reliability related to the detection of the upper surface of the quay wall, i.e., the upper surface detection reliability "q u " and the reliability of the calculated reliability c. Here, it is used with respect to each reliability q s ,q u The weighting coefficient "w" corresponding to the importance of m0, m1, and c qs " " qu " " c The reliability of s ,q u , c. In addition, the weighting coefficient w is shown in the figure. qs 、w qu 、w m0 、w m1 、w c It should be noted that the information processing device 1 can, for example, set the side detection reliability q s based on Fig.14 The reliability information item "side" is calculated by taking the upper surface detection reliability q u The side detection reliability q is calculated based on the item "upper surface" of the reliability information in the figure. s and upper surface detection reliability q u All are calculated so as to fall within the range of 0 to 1. In addition, when both the front and rear banks are detected, the side detection reliability q of the front bank is s0 , side detection reliability of the rear quay wall q s1 , the upper surface detection reliability of the front quay wall q u0 , upper surface detection reliability of the rear quay wall q u1 Since the side detection reliability q s , upper surface detection reliability q uand the calculation reliability c are both in the range of 0 to 1, so their weighted average, i.e., the comprehensive reliability r, is also calculated as a value in the range of 0 to 1. Therefore, it can be seen that the closer the comprehensive reliability r is to 1, the higher the reliability of the calculated berthing parameters, and the closer the comprehensive reliability r is to 0, the lower the reliability of the calculated berthing parameters.

[0174] The information processing device 1 can appropriately adjust the output of the driving source when approaching the berth, etc., based on the reliability information including the various indices and the reliability according to the modification.

[0175] As described above, the controller 13 of the information processing device 1 has at least an acquisition unit, a straight line calculation unit, and a distance calculation unit. The acquisition unit acquires measurement data, which is a collection of data representing a plurality of measured points measured by a measurement device installed on the ship. The straight line calculation unit calculates a straight line along the berthing place where the ship is to dock based on the measurement data. The distance calculation unit calculates the distance between the ship and the berthing place on the other side based on the distance between the straight line and each measured point. By performing this processing, the controller 13 can accurately calculate the distance between the ship and the berthing place on the other side based on the target measurement device.

[0176] It should be noted that in the above-mentioned embodiments, the program can be stored and supplied to a controller or the like as a computer using various types of non-transitory computer readable media (Non-Trasitory Computer Readable Medium). Non-transitory computer readable media include various types of tangible storage media (Tangible Storage Medium). Examples of non-transitory computer readable media include magnetic storage media (e.g., floppy disks, magnetic tapes, hard disk drives), optical magnetic storage media (e.g., optical magnetic disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory).

[0177] The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention. That is, the present invention certainly includes various deformations and modifications that can be made according to the entire disclosure and technical ideas including the claims by those skilled in the art. In addition, the above-mentioned patent documents and other disclosures cited are incorporated into this specification by reference.

[0178] Description of Reference Numerals

[0179] 1 Information processing device

[0180] 2 Sensor Groups

[0181] 3 Lidar.

Claims

1. An information processing device comprising: an acquisition unit that acquires measurement data, the measurement data being a collection of data representing a plurality of measurement points measured by a measurement device installed on the ship; a straight line calculation unit that calculates a straight line along a berthing location where the ship is to dock based on the measurement data; and The distance calculation unit calculates the distance between the ship and the berth based on the distance between the straight line and each of the measured points.

2. The information processing device according to claim 1, The information processing device further includes a nearest point search unit configured to search for a nearest point with respect to the ship from the measured point based on the straight line. The distance calculation unit calculates the distance based on the nearest point.

3. The information processing device according to claim 2, The measuring device includes a first measuring device and a second measuring device. The straight line calculation unit calculates a second straight line connecting a first nearest point to the ship searched based on the straight line from a measured point of first measurement data measured by the first measurement device and a second nearest point to the ship searched based on the straight line from a measured point of second measurement data measured by the second measurement device, The distance calculation unit calculates the distance based on the second straight line.

4. The information processing device according to claim 3, The straight line calculation unit re-searches the first nearest point and the second nearest point based on the second straight line when the second straight line is not close to the straight line calculated before the second straight line, and calculates a third straight line connecting the re-searched first nearest point and the second nearest point. The distance calculation unit calculates the distance based on the third straight line.

5. The information processing device according to claim 4, The straight line calculation unit repeatedly performs re-searching of the first nearest point and the second nearest point and calculation of the straight line until the calculated straight line converges.

6. The information processing device according to any one of claims 1 to 5, The straight line calculation unit calculates the straight line based on the nearest point searched from the measured point based on the straight line calculated by the straight line calculation unit before a current processing time when an index indicating a deviation of the measured point from the straight line is equal to or larger than a threshold value.

7. The information processing device according to any one of claims 1 to 6, The straight line calculation unit calculates the straight line based on the nearest point searched from the measured point according to the straight line parallel to the direction of the ship, when an index representing the deviation of the measured point with respect to the straight line is greater than a threshold value and the straight line does not exist within a specified time relative to the current processing time.

8. The information processing device according to any one of claims 1 to 7, further comprising: a shortest distance to the shore calculation unit that extracts the shortest length of a perpendicular line drawn from the contour point of the ship to the straight line to calculate the shortest distance from the hull of the ship to the berthing place; and The approaching position specifying unit specifies a position of the ship that is closest to the berthing place.

9. A control method is a control method executed by a computer, wherein: Acquiring measurement data, the measurement data being a collection of data representing a plurality of measurement points measured by a measurement device installed on the ship, Based on the measurement data, a straight line along the berthing location where the ship is to dock is calculated; The distance between the ship and the berthing place is calculated based on the distance between the straight line and each of the measured points.

10. A program that causes a computer to execute the following processing: Acquiring measurement data, the measurement data being a collection of data representing a plurality of measurement points measured by a measurement device installed on the ship, Based on the measurement data, a straight line along the berthing location where the ship is to dock is calculated; The distance between the ship and the berthing place is calculated based on the distance between the straight line and each of the measured points.

11. A storage medium storing the program according to claim 10.