Information processing device, control method, program, and storage medium

The measurement data is obtained through lidar and the edge search range of the docking site is solved, and the accuracy of the calculation of docking parameters in the automatic docking system of ships is improved, and the safety and efficiency of the docking process are improved.

CN119053512APending Publication Date: 2024-11-29PIONEER IP +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202280093272.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-03-15
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing automatic ship docking system is difficult to calculate the distance, speed and angle parameters of the docking site with high accuracy, which affects the safety and efficiency of the shore auxiliary system.

Method used

An information processing device is designed to obtain measurement data through lidar, the marking position acquisition unit obtains the marking position of the docking site, the search range setting unit sets the edge dot matrix search range of the docking site, and the computer executes the control method to Calculate the docking parameters with high accuracy.

Benefits of technology

High-precision landing parameter calculation in the automatic ship docking system is realized, the safety and efficiency of the landing process is improved, and the accurate docking between the ship and the landing site is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119053512A_ABST
    Figure CN119053512A_ABST
Patent Text Reader

Abstract

The information processing apparatus includes an acquisition unit, a mark position acquisition unit, and a search range setting unit. The acquisition means acquires measurement data generated by a measurement device provided in a ship. The mark position acquisition means acquires the position of at least one mark provided in the landing location on the basis of the measurement data. The search range setting means sets the search range of the dot matrix of the edge portion of the landing site on the basis of the position of at least one marker.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to handling of a ship when it docks. Background Art

[0002] In the past, there are known technologies for assisting the berthing (landing) of ships. For example, Patent Document 1 describes the following method: In an automatic berthing device for automatically berthing a ship, the posture of the ship is controlled to change in such a way that light irradiated from a laser radar is reflected by objects around the berthing position and received by the laser radar.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-59403 Summary of the invention

[0006] Problems to be solved by the invention

[0007] In ships, it is important to land safely and smoothly at a berthing place, and the realization of a landing assistance system for ship driving assistance and automatic navigation is particularly desired. Therefore, in the landing assistance system, it is necessary to calculate parameters such as the distance, speed, and angle relative to the berthing place to land with high accuracy.

[0008] The present disclosure has been made to solve the above-mentioned problems, and a main object of the present disclosure is to provide an information processing device capable of calculating parameters used in landing assistance with high accuracy.

[0009] Means for solving problems

[0010] The invention described in the claims is an information processing device, which comprises: an acquisition unit for acquiring measurement data generated by a measuring device installed on a ship; a mark position acquisition unit for acquiring the position of at least one mark installed at a berthing place based on the measurement data; and a search range setting unit for setting the search range of the dot matrix of the edge part of the berthing place based on the position of the at least one mark.

[0011] In addition, the invention described in the claims is a computer-implemented control method, wherein measurement data generated by a measuring device installed on a ship is obtained, the position of at least one mark installed at a berthing place is obtained based on the measurement data, and the search range of the dot matrix of the edge portion of the berthing place is set based on the position of the at least one mark.

[0012] In addition, the invention described in the claims is a program, wherein a computer is caused to perform the following processing: obtaining measurement data generated by a measuring device installed on a ship, obtaining the position of at least one mark installed at a berthing place based on the measurement data, and setting a search range of a dot matrix of an edge portion of the berthing place based on the position of the at least one mark. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1A It is a block diagram of the navigation assistance system.

[0014] Figure 1B This is a top view illustrating the field of view of a ship and a laser radar included in the navigation assistance system.

[0015] Figure 1C This is a diagram showing the field of view of a ship and a laser radar from the rear.

[0016] Figure 2 This is a block diagram showing an example of the hardware configuration of the information processing device.

[0017] Figure 3 It is a functional block diagram related to the docking auxiliary processing.

[0018] Figure 4A This is a diagram showing a situation where the laser radar captures the shore where the shore is about to approach.

[0019] Figure 4B It is a three-dimensional view of the quay wall showing the straight line of the shore side.

[0020] Figure 5A This is a diagram showing an example of the data structure of reliability information.

[0021] Figure 5B This is a diagram showing an example of the index and reliability included in the reliability information.

[0022] Figure 5C It is clearly stated Figure 5B A bird's-eye view of the target ship and the berthing area for the indicators shown.

[0023] Fig. 6A This is a diagram showing an example of a hull coordinate system based on the hull of the target ship.

[0024] Figure 6B It is a three-dimensional diagram of a structure with its normal vector clearly indicated.

[0025] Figure 7 This is a diagram showing an example of a berthing area where a front mark and a rear mark are provided.

[0026] Figure 8 It is a top view clearly showing the extraction window EWA.

[0027] Fig.9A It is a diagram showing the nearest point of each vertical line in the berthing place SBP.

[0028] Fig. 9B FIG. 1 is a diagram showing an example of each nearest point acquired as a landing place edge point matrix by the extraction window EWA.

[0029] Fig. 10A It indicates the height h mf and height h mr .

[0030] Fig. 10B It is clear that the marking interval r m Top view of the .

[0031] Fig. 10C It indicates the distance w mf and distance w mr Top view of the .

[0032] Fig.11 3 is a diagram showing an example of each nearest point acquired when it is determined that the mark CM is a front mark.

[0033] Fig.12 2 is a diagram showing an example of a straight line generated assuming that the mark CM is a front mark.

[0034] Fig.13 2 is a diagram showing an example of a straight line generated when it is assumed that the marker CM is a rear marker.

[0035] Fig.14A This is a flowchart showing an overview of the docking support process.

[0036] Fig. 14B This is a flowchart showing an example of processing related to acquisition of a landing place edge matrix.

[0037] Fig.15A FIG. 1 is a diagram showing an example of a case where an incorrect edge straight line is generated.

[0038] Fig. 15B This is a diagram showing an example of a case where a correct edge straight line is generated.

[0039] Fig. 15C This is a diagram showing an example of a case where a correct edge straight line is generated.

[0040] Fig.16 This is a diagram showing an example of processing related to acquisition of a landing place edge matrix.

[0041] Fig.17A It clearly indicates the origin O and the distance df and distance d r , a top view of the front end portion FE and the rear end portion RE.

[0042] Fig. 17B It is clear that at a distance d f and distance d r Graph of the unit vector u used in the calculation of .

[0043] Fig.18 is the distance d f and distance d r A diagram of an example of processing involved in the calculation of . DETAILED DESCRIPTION

[0044] In a preferred embodiment of the present invention, the information processing device comprises: an acquisition unit for acquiring measurement data generated by a measuring device installed on a ship; a mark position acquisition unit for acquiring the position of at least one mark installed at a berthing place based on the measurement data; and a search range setting unit for setting the search range of a dot matrix of an edge portion of the berthing place based on the position of the at least one mark.

[0045] The above-mentioned information processing device has an acquisition unit, a marker position acquisition unit, and a search range setting unit. The acquisition unit acquires measurement data generated by a measurement device installed on the ship. The marker position acquisition unit acquires the position of at least one marker installed at the berthing place based on the measurement data. The search range setting unit sets the search range of the dot matrix of the edge part of the berthing place based on the position of the at least one marker. In this way, the parameters used in the landing assistance can be calculated with high accuracy.

[0046] In one embodiment of the above-mentioned information processing device, the mark position acquisition unit acquires the position of a first mark set on the front side of the ship and the position of a second mark set on the rear side of the ship, and the search range setting unit sets a roughly rectangular area or a roughly cuboid area corresponding to the position of the first mark and the position of the second mark as the search range.

[0047] In one aspect of the above-mentioned information processing device, the search range setting unit sets the search range using a distance from a straight line along an edge of the landing place to the first mark and a distance from the straight line to the second mark.

[0048] In one embodiment of the above-mentioned information processing device, the mark position acquisition unit acquires the position of one of the two markers set at the berthing place, which is set on the front side or the rear side of the ship, and the search range setting unit sets the search range based on the interval between the two markers and the position of the one marker.

[0049] In one aspect of the above-mentioned information processing device, the search range setting unit sets the search range using a distance from a straight line along an edge portion of the landing place to the one mark.

[0050] In one aspect of the above-mentioned information processing device, the search range setting unit sets the search range using the entire length of the ship.

[0051] In one embodiment of the above-mentioned information processing device, the mark position acquisition unit acquires the position of a first mark set on the front side of the ship and the position of a second mark set on the rear side of the ship, and the information processing device also has a distance calculation unit, which calculates the distance from the reference point of the ship to the end of the berthing area based on the position of the first mark, the position of the second mark and a straight line along the edge of the berthing place, and the berthing area is the area where the ship should be located when docking at the berthing place.

[0052] In one aspect of the above-mentioned information processing device, two marks are provided at positions in the docking place at different heights from the upper surface of the docking place.

[0053] In another embodiment of the present invention, in a control method executed by a computer, measurement data generated by a measurement device installed on a ship is obtained, the position of at least one mark installed at a berthing place is obtained based on the measurement data, and the search range of the dot matrix of the edge portion of the berthing place is set based on the position of the at least one mark. In this way, parameters used in landing assistance can be calculated with high accuracy.

[0054] In another embodiment of the present invention, the program causes a computer to execute the following processing: obtaining measurement data generated by a measurement device installed on a ship, obtaining the position of at least one mark installed at a berthing place based on the measurement data, and setting the search range of the dot matrix of the edge portion of the berthing place based on the position of the at least one mark. By using a computer to execute the program, the above-mentioned information processing device can be realized. The program can be stored in a storage medium for use.

[0055] Example

[0056] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0057] <First Embodiment>

[0058] First, the first embodiment will be described below.

[0059] [Overview of Driving Assistance Systems]

[0060] Figure 1A to Figure 1C This is a schematic structure of the navigation assistance system involved in this embodiment. Specifically, Figure 1A The block diagram of the navigation assistance system is shown in FIG. Figure 1B 1 is a top view illustrating a visual field range (also referred to as a “measurement range” or “measurable range”) 90 of a ship included in the navigation assistance system and a laser radar 3 described later. Figure 1C The figure shows the ship and the field of view 90 of the laser radar 3 from the rear side. The navigation support system includes an information processing device 1 that moves with the ship as a mobile body and a sensor group 2 mounted on the ship. Hereinafter, the ship equipped with the driving support system is also referred to as a "target ship".

[0061] The information processing device 1 is electrically connected to the sensor group 2, and performs navigation assistance for the target ship based on the outputs of various sensors included in the sensor group 2. The navigation assistance includes docking assistance such as automatic docking (landing). Here, "docking" includes not only the case of docking the target ship to the quay, but also the case of docking the target ship to a structure such as a pier. In addition, hereinafter, "docking place" is a general term for structures such as quays and piers that are the targets of docking. The information processing device 1 can be a navigation device built into the ship, or it can be an electronic control device built into the ship.

[0062] The sensor group 2 includes various external sensors and internal sensors installed on the ship. In this embodiment, the sensor group 2 includes a Lidar (Light Detection and Ranging or Laser Illuminated Detection And Ranging) 3, for example.

[0063] The laser radar 3 is a radar that detects the horizontal direction within a predetermined angle range (see Figure 1B ) and the specified angle range in the vertical direction (refer to Figure 1C ) is an external sensor that emits pulsed laser to discretely measure the distance to an object existing in the outside world and generate three-dimensional dot matrix data representing the position of the object.

[0064] The laser radar 3 includes an irradiation unit that irradiates laser light while changing the irradiation direction, a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs scanning data based on the light receiving signal output by the light receiving unit. The data measured for each direction (scanning position) of the irradiated laser light is generated based on the irradiation direction corresponding to the laser light received by the light receiving unit and the response delay time of the laser light determined based on the above-mentioned light receiving signal. Hereinafter, the point or its data measured by being irradiated with laser light within the measurement range of the laser radar 3 is also referred to as a "measured point".

[0065] exist Figure 1B and Figure 1C In the example, as laser radars 3, a laser radar facing the front of the port side of the target ship, a laser radar facing the rear of the port side of the target ship, a laser radar facing the front of the starboard side of the target ship, and a laser radar facing the rear of the starboard side of the target ship are respectively installed on the ship. And, when the target ship approaches the berthing place, the laser radar 3 in front of the target ship and the laser radar 3 in the rear of the target ship, which are installed on the side where the target ship is berthing to the berthing place, respectively generate dot matrix data measuring the berthing place. Hereinafter, the laser radar 3 measuring the front of the berthing place will be referred to as the "front laser radar", and the laser radar 3 measuring the rear of the berthing place will be referred to as the "rear laser radar". The front measuring laser radar and the rear measuring laser radar are examples of the "first measuring device" and the "second measuring device". It should be noted that the configuration of the laser radar 3 is not limited to Figure 1B and Figure 1C Example.

[0066] Here, the dot matrix data can be regarded as an image (frame) with each measurement direction as a pixel and the measured distance and reflection intensity value in each measurement direction as the pixel value. In this case, the emission direction of the laser at the elevation angle (i.e., the measurement direction) is different in the longitudinal arrangement of the pixels, and the emission direction of the laser at the horizontal angle is different in the lateral arrangement of the pixels. Hereinafter, the measured points corresponding to the columns of pixels (i.e., vertical columns) that coincide with the positions of the horizontal indexes when the dot matrix data is regarded as an image are also referred to as "vertical lines". In addition, the horizontal index when the dot matrix data is regarded as an image is referred to as the "horizontal number", and the vertical index is referred to as the "vertical number".

[0067] The laser radar 3 is not limited to the scanning laser radar described above, but may be a flash laser radar that generates three-dimensional data by diffusing laser light into the field of view of a two-dimensional array sensor. The laser radar 3 is an example of the "measuring device" in the present invention.

[0068] [Structure of information processing device]

[0069] Figure 2 1 is a block diagram showing an example of the hardware configuration of the information processing device 1. The information processing device 1 mainly includes an interface 11, a memory 12, and a controller 13. These elements are connected to each other via a bus.

[0070] The interface 11 performs interface operations related to the exchange of data between the information processing device 1 and the external device. In this embodiment, the interface 11 obtains output data from each sensor of the sensor group 2 and supplies it to the controller 13. In addition, the interface 11 supplies, for example, a signal related to the control of the target ship generated by the controller 13 to each component of the target ship that controls the driving of the target ship. For example, the target ship has a driving source such as an engine or an electric motor, a propeller that generates a propulsion force in the direction of travel based on the driving force of the driving source, a thruster that generates a lateral propulsion force based on the driving force of the driving source, and a mechanism for freely determining the direction of travel of the ship, i.e., a rudder. In addition, during automatic driving such as automatic docking, the interface 11 supplies the control signal generated by the controller 13 to these components. It should be noted that when the target ship is provided with an electronic control device, the interface 11 supplies the control signal generated by the controller 13 to the electronic control device. The interface 11 can be a wireless interface such as a network adapter for wireless communication, or a hardware interface for connecting to an external device through a cable or the like. In addition, the interface 11 can also perform interface operations with various peripheral devices such as an input device, a display device, and a sound output device.

[0071] The memory 12 is composed of various volatile memories and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. The memory 12 stores a program for the controller 13 to execute a predetermined process. It should be noted that the program executed by the controller 13 may also be stored in a storage medium other than the memory 12.

[0072] In addition, the memory 12 stores information necessary for the processing performed by the information processing device 1 in this embodiment. For example, map data including information related to the location of the docking place may be stored in the memory 12. In another example, the memory 12 stores information related to the size of downsampling when the dot matrix data obtained when the laser radar 3 performs one cycle of scanning is downsampled.

[0073] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 performs processing related to driving assistance of the target ship by executing a program stored in the memory 12 or the like.

[0074] In addition, the controller 13 functionally has a docking place detection unit 15 and a docking parameter calculation unit 16. The docking place detection unit 15 performs processing related to the detection of the docking place based on the dot matrix data output by the laser radar 3. The docking parameter calculation unit 16 calculates the parameters required for docking at the docking place (also referred to as "dockage parameters"). Here, the docking parameters include the distance between the target ship and the docking place (distance to the shore), the angle of entry of the target ship to the docking place, the speed at which the target ship approaches the docking place (dockage speed), etc. In addition, the docking parameter calculation unit 16 calculates information indicating the reliability related to the docking at the docking place (also referred to as "reliability information") based on the processing results of the docking place detection unit 15 and the docking parameters. In addition, the controller 13 functions as an "acquisition unit", "mark position acquisition unit", "search range setting unit", "distance calculation unit", and a computer that executes a program.

[0075] It should be noted that the processing executed by the controller 13 is not limited to being implemented by software based on the program, but may also be implemented by any combination of hardware, firmware, and software. In addition, the processing executed by the controller 13 may also be implemented using an integrated circuit that can be programmed by a user, such as an FPGA (Field-Programmable Gate Array) or a microcomputer. In this case, the program executed by the controller 13 in this embodiment may also be implemented using the integrated circuit.

[0076] [Overview of the docking assistance process]

[0077] Next, the outline of the docking assistance processing performed by the information processing device 1 is described. The information processing device 1 generates a straight line along the side of the docking place (also referred to as "dockage side straight line L") based on the dot matrix data of the laser radar 3 measured in the direction of the docking place. That is, the docking side straight line L is a straight line along the side of the quay wall of the docking place. In addition, the information processing device 1 calculates docking parameters such as the distance to the opposite shore based on the docking side straight line L.

[0078] Figure 3 This is a functional block diagram of a docking place detection unit 15 and a docking parameter calculation unit 16 related to the docking auxiliary processing. The docking place detection unit 15 functionally includes a normal vector calculation frame 20, a field of view·detection surface determination frame 21, a normal number determination frame 22, an average·variance calculation frame 23, and a docking condition determination frame 24. In addition, the docking parameter calculation unit 16 functionally includes a neighboring point search frame 26, a straight line generation frame 27, an opposite shore distance calculation frame 28, an approach angle calculation frame 29, a docking speed calculation frame 30, a reliability information generation frame 40, a mark detection frame 41, and a docking place edge point matrix extraction frame 42.

[0079] The normal vector calculation block 20 calculates the normal vector of the surface formed by the berthing place (also referred to as the "berthing surface") based on the dot matrix data generated by the laser radar 3 in the direction of the berthing place. In this case, the normal vector calculation block 20 calculates the normal vector based on the dot matrix data generated by the laser radar 3 that includes the berthing side in the measurement range of the target ship, for example. Information related to the measurement range of the laser radar 3 and the direction of the berthing place can be pre-registered in, for example, the memory 12.

[0080] In this case, it is preferable that the normal vector calculation block 20 performs downsampling of the dot data and removal of data obtained by reflection of the laser light on the water surface (also referred to as “water surface reflection data”).

[0081] In this case, first, the normal vector calculation frame 20 removes the data that exists below the water surface position as water surface reflection data (i.e., false detection data) for the dot matrix data generated by the laser radar 3. It should be noted that the normal vector calculation frame 20 estimates the water surface position based on, for example, the average value in the height direction of the dot matrix data generated by the laser radar 3 when there are no objects other than the water surface in the surrounding area. Then, the normal vector calculation frame 20 downsamples the dot matrix data after the water surface reflection data is removed. Downsampling is a process of integrating the measured points for each grid space of a specified size. Then, the normal vector calculation frame 20 calculates the normal vector for each measured point represented by the downsampled dot matrix data using multiple measured points in the surrounding area. It should be noted that downsampling can also be performed before removing the data obtained by water surface reflection.

[0082] The field of view·detection surface determination frame 21 determines the surface of the docking place existing in the field of view angle of the laser radar 3 (also referred to as "field of view internal surface") and the surface of the docking place detected based on the normal vector calculated by the normal vector calculation frame 20 (also referred to as "detection surface"). In this case, the field of view·detection surface determination frame 21 determines whether the field of view internal surface and the detection surface include the upper surface and / or the side surface of the docking place.

[0083] The normal number determination block 22 extracts the normal vectors in the vertical direction and the normal vectors in the direction perpendicular thereto (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation block 20, and calculates the number of normal vectors in the vertical direction and the number of normal vectors in the horizontal direction. Here, the normal number determination block 22 regards the normal vector in the vertical direction as a vector representing the normal of the measured point relative to the upper surface of the berthing place, and regards the normal vector in the horizontal direction as a vector representing the normal of the measured point relative to the side surface of the berthing place, and calculates the number of each as an index of the reliability related to the berthing place.

[0084] The mean / variance calculation frame 23 extracts the normal vectors in the vertical direction and the normal vectors in the direction perpendicular to the vertical direction (i.e., the horizontal direction) from the normal vectors calculated by the normal vector calculation frame 20, and calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction.

[0085] The docking condition determination frame 24 obtains the processing results of the field of view and detection surface determination frame 21, the number of normals determination frame 22, and the average and variance calculation frame 23 determined or calculated based on the same dot matrix data as the determination result indicating the detection condition of the docking place at the time point of generation of the dot matrix data. In addition, the docking condition determination frame 24 supplies the processing results of the field of view and detection surface determination frame 21, the number of normals determination frame 22, and the average and variance calculation frame 23 to the docking parameter calculation unit 16 as the determination result of the detection condition of the docking place.

[0086] The mark detection frame 41 detects at least one mark installed at the landing place based on the measured points constituting the dot matrix data, and acquires the coordinate position of the detected at least one mark.

[0087] The neighboring point search block 26 performs a process of searching for the nearest neighboring point to the target ship for each vertical line from the measured points constituting the dot matrix data. Figure 4A As shown, when the laser radar 3 captures the quay to be docked, the point closest to the ship, i.e., the nearest point, becomes the edge between the upper surface and the side of the quay. The set of points obtained by searching the nearest point for each longitudinal line gathers points near the edge of the quay.

[0088] The landing place edge dot matrix extraction block 42 extracts the dot matrix of the edge portion of the landing place within the search range based on the coordinate position of at least one marker acquired by the marker detection block 41 and the nearest point searched by the nearest point search block 26 .

[0089] The straight line generation block 27 generates a straight line along the side of the berthing place, that is, a berthing side straight line L, using principal component analysis and the least square method based on the nearest point determined by the neighboring point search block 26. According to such processing, the straight line generation block 27 can generate, for example, Figure 4B The shore side straight line L is shown. Figure 4B It is a three-dimensional view of the quay wall showing the straight line of the shore side.

[0090] The shore distance calculation frame 28 calculates the shore distance equivalent to the shortest distance between the target ship and the berthing place based on the shore side straight line L generated by the straight line generation frame 27. Here, when there are multiple laser radars 3 that can measure the berthing place, the shore distance calculation frame 28 collects the dot matrix data of the multiple laser radars 3 to generate the shore side straight line L, and calculates the shortest distance with each laser radar 3 as the shore distance. Alternatively, the shore distance calculation frame 28 can also generate the shore side straight line L for the dot matrix data of each laser radar 3, and calculate the shortest distance between each shore side straight line L and each laser radar 3 as the shore distance. In addition, the shore distance calculation frame 28 can also calculate the shortest distance from the reference point such as the center position of the ship to the shore side straight line L as the shore distance. It should be noted that the shore distance calculation frame 28 can also determine the shortest distance among the shortest distances of each laser radar 3 as the shore distance instead of considering the shortest distance of each laser radar 3 as the shore distance, and can also determine the average of these shortest distances as the shore distance. It should be noted that the shortest distance to the shore side straight line L is obtained by calculating the length of a perpendicular line to the shore side straight line L in the xy plane.

[0091] The approach angle calculation block 29 generates an approach angle of the target ship relative to the berthing place based on the berthing side straight line L generated by the straight line generation block 27. Specifically, the approach angle calculation block 29 calculates the approach angle using a function "atan2" that obtains an arctangent from two independent variables of a predetermined tangent. More specifically, the approach angle calculation block 29 calculates the approach angle based on the direction vector of the berthing side straight line L by calculating the function atan2.

[0092] The approach speed calculation block 30 calculates the approach speed, which is the speed at which the target ship approaches the berthing place, based on the shore distance calculated by the shore distance calculation block 28. For example, the approach speed calculation block 30 calculates the time change of the shore distance (shortest distance) as the approach speed.

[0093] The reliability information generation block 40 generates reliability information based on the processing results of the approaching state determination block 24 , the straight line generation block 27 , the opposite shore distance calculation block 28 , and the approach angle calculation block 29 .

[0094] Here, the first specific example of generating reliability information is described. The reliability information generating frame 40 generates a flag for each element such as the viewing angle during the detection of the docking place, the surface detection of the docking place, the number of normal vectors and the variance, and generates the vector of the generated flag as the reliability information. In the future, when the flag is "1", it means that the reliability of the corresponding element is high, and when the flag is "0", it means that the reliability of the corresponding element is low.

[0095] Figure 5A 4 is a diagram showing an example of the data structure of the reliability information generated by the reliability information generation block 40. Figure 5A As shown, the reliability information has items of "upper surface", "side", "straight line", "distance", and "angle". In addition, the item "upper surface" has sub-items of "viewing angle", "detection", "number of normals", and "variance", and the item "side" has sub-items of "viewing angle", "detection", "number of normals", and "variance". In addition, the item "straight line" has sub-items of "absolute value", the item "distance" has sub-items of "change amount" and "change rate", and the item "angle" has sub-item of "change amount".

[0096] Here, the reliability information generation frame 40 registers the sub-item “viewing angle” of the item “upper surface” as “1” when the upper surface of the berthing place is within the viewing angle range, and as “0” when the upper surface is outside the viewing angle. In addition, the reliability information generation frame 40 registers the sub-item “detection” of the item “upper surface” as “1” when the upper surface of the berthing place is a detection surface, and as “0” when the upper surface is not a detection surface. In addition, the reliability information generation frame 40 registers the sub-item “number of normals” of the item “upper surface” as “1” when the number of normal vectors relative to the upper surface of the berthing place is greater than a predetermined threshold value (e.g., 10), and as “0” when the number is less than the threshold value. In addition, the reliability information generation block 40 registers a flag that becomes "1" when the variances of the x, y, and z components of the normal vector to the upper surface of the berthing site are all less than a predetermined threshold value (e.g., 1.0) for the sub-item "variance" of the item "upper surface", and becomes "0" when any of the variances is greater than the threshold value. In addition, the reliability information generation block 40 also registers a flag determined by the same rule as that of each sub-item of the item "upper surface" in each sub-item of the item "side".

[0097] In addition, the reliability information generation block 40 registers a flag indicating the reliability of the shore side straight line L generated by the generation method described later for the sub-item "absolute value" of the item "straight line". For example, when the laser radar 3 is provided at the front and rear of the target ship, the reliability information generation block 40 registers a flag that becomes "1" when the difference between each component of the direction vector of the shore side straight line L generated by the generation method described later and the direction vector of the straight line connecting the nearest points of the laser radar 3 at the front and rear of the target ship is less than a threshold value, and becomes "0" when any of the differences is greater than the threshold value.

[0098] In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "amount of change" of the item "distance" when the amount of change of the opposite shore distance calculated by the opposite shore distance calculation frame 28 relative to 1 moment ago is less than a predetermined threshold value (e.g., 1.0 m), and a flag "0" when the amount of change is greater than the threshold value. In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "rate of change" of the item "distance" when the rate of change of the opposite shore distance calculated by the opposite shore distance calculation frame 28 relative to 1 moment ago is less than a predetermined threshold value (e.g., ±10%), and a flag "0" when the rate of change is greater than the threshold value. In addition, the reliability information generation frame 40 registers a flag "1" for the sub-item "amount of change" of the item "angle" when the amount of change of the entry angle calculated by the entry angle calculation frame 29 relative to 1 moment ago is less than a predetermined threshold value (e.g., 1.0 degree), and a flag "0" when the amount of change is greater than the threshold value.

[0099] It should be noted that the above-mentioned threshold values ​​are set to adaptive values ​​pre-stored in the memory 12 , etc. In addition, the reliability information may be generated for each laser radar 3 .

[0100] Next, a second specific example of generating reliability information will be described. Figure 5B This is a diagram showing an example of the index and reliability included in the reliability information. Figure 5C It is clearly stated Figure 5B A bird's-eye view of the target ship and the berthing place of the indicator shown. It should be noted that, here, the quay wall serving as the berthing place is provided with a mark M0 and a mark M1 as a reference for berthing, and the information processing device 1 detects the mark M0 and the mark M1 based on the dot matrix data of the laser radar 3, and uses the measured points (neighboring point set) near the edge of the berthing place between the mark M0 and the mark M1 to perform various processes such as the generation of the berthing side straight line L. Hereinafter, the measured points near the edge of the berthing place between the mark M0 and the mark M1 will also be referred to as "target points". It should be noted that, when the mark M0 and the mark M1 are not detected, for example, all of the neighboring point set will be regarded as target points.

[0101] The index "c3" is an index based on the number of object points, and is represented here as a linear function of the number of object points as an example. The index "c2" is an index based on the standard deviation of the object point, and is represented here as a linear function of the standard deviation of the object point as an example. The index "c1" is an index indicating whether the shore walls on both the front side and the rear side are measured by two laser radars 3. As an example, the case where both sides are measured is set to "1.0" and the case where only one side is measured is set to "0.0". The index "c0" is an index based on the interval between the two ends of the object point (the two ends in the direction of the straight line L on the shore side), and is represented as a linear function of the interval between the above-mentioned two ends. In addition, the indexes c0~c3 are calculated in a manner limited to the range of 0~1.

[0102] The calculation reliability "c" is based on the reliability of each of the above-mentioned indicators c0 to c3, and here, it becomes the weighted average value of the indicators c0 to c3 using the weighting coefficients "w0" to "w3" corresponding to the importance of the indicators c0 to c3. In addition, an example of the setting value of the weighting coefficients w0 to w3 is shown in the figure. Since each of the indicators c0 to c3 is in the range of 0 to 1, the weighted average value of them, that is, the calculation reliability c, is also calculated as a value in the range of 0 to 1.

[0103] The comprehensive reliability "r" is based on the reliability related to the detection of the side of the quay wall, that is, the side detection reliability "q s ", the reliability related to the detection of the upper surface of the quay wall, i.e., the upper surface detection reliability "q u ”, the reliability related to the detection of the mark M0, i.e., the mark detection reliability “m0”, the reliability related to the detection of the mark M1, i.e., the mark detection reliability “m1”, and the reliability of the calculation reliability c. Here, the reliability q is used to determine the reliability of the mark M0. s ,q u The weighting coefficient "w" corresponding to the importance of m0, m1, and c qs " " qu " " m0 " " m1 " " c The reliability of s ,q u , m0, m1, and c. In addition, the weighting coefficient w is shown in the figure. qs 、w qu 、w m0 、w m1 、w c It should be noted that the information processing device 1 can, for example, set the side detection reliability q s based on Figure 5AThe reliability information item "side" is calculated by taking the upper surface detection reliability q u The information processing device 1 can calculate the marker detection reliability m0 and the marker detection reliability m1 based on the number of points of the dot matrix data of the laser radar 3 of the marker M0 and the marker M1, respectively. s , upper surface detection reliability q u , the marker detection reliability m0, and the marker detection reliability m1 are all calculated so as to be in the range of 0 to 1. In addition, when both the front and rear embankments are detected, the side detection reliability q of the front embankment is s0 , side detection reliability of the rear quay wall q s1 , the upper surface detection reliability of the front quay wall q u0 , upper surface detection reliability of the rear quay wall q u1 Since the side detection reliability q s , upper surface detection reliability q u , mark detection reliability m0, mark detection reliability m1, and calculation reliability c are all in the range of 0 to 1, so their weighted average, i.e., comprehensive reliability r, is also calculated as a value in the range of 0 to 1. Therefore, it can be seen that the closer the comprehensive reliability r is to 1, the higher the reliability of the calculated docking parameters, and the closer the comprehensive reliability r is to 0, the lower the reliability of the calculated docking parameters.

[0104] That is, the reliability information generated by the second specific example includes the indicators and reliability calculated by the method described above.

[0105] Next, for a specific example of the processing of the normal vector calculation block 20, the normal number determination block 22, and the mean / variance calculation block 23, refer to Fig. 6A and Figure 6B To explain.

[0106] Fig. 6A FIG. 1 is a diagram showing an example of a hull coordinate system based on the hull of the target ship. Fig. 6A As shown in FIG. 1 , the front (forward) direction of the target ship is set as the “x” coordinate, the side direction of the target ship is set as the “y” coordinate, and the height direction of the target ship is set as the “z” coordinate. Furthermore, the measurement data of the coordinate system based on the laser radar 3 measured by the laser radar 3 is transformed into Fig. 6A The hull coordinate system shown. It should be noted that the process of transforming the dot matrix data of the coordinate system based on the laser radar installed on the moving body into the coordinate system of the moving body is disclosed in, for example, International Publication WO2019 / 188745.

[0107] Figure 6B This is a stereoscopic view of the berthing location (the quay in this case) showing the measured points indicating the measurement positions measured by the laser radar 3 and the normal vectors calculated based on the measured points relative to the quay as the berthing location. Figure 6B In FIG. 1 , the measured point is represented by a circle, and the normal vector is represented by an arrow. Here, an example in which both the upper surface and the side surface of the bank are measured by the laser radar 3 is shown.

[0108] like Figure 6B As shown, the normal vector calculation frame 20 calculates the normal vector of the measured point relative to the side and upper surface of the quay. The normal vector is a vector perpendicular to the plane or curved surface set as the object, so it is calculated using a plurality of measured points that can be formed into a surface. Therefore, a grid with a predetermined length in length and width or a circle with a predetermined radius is set, and the calculation is performed using the measured points existing inside it. In this case, the normal vector calculation frame 20 can calculate the normal vector for each measured point, or it can calculate the normal vector at predetermined intervals. In addition, the normal number determination frame 22 determines the normal vector with a z component greater than a predetermined threshold as a normal vector facing the vertical direction. It should be noted that the normal vector is unitized. In addition, the normal vector with a z component less than a predetermined threshold is determined as a normal vector facing the horizontal direction. In addition, the normal number determination frame 22 determines the number of normal vectors in the vertical direction (here 5) and the number of normal vectors in the horizontal direction (here 4). The mean and variance calculation block 23 calculates the mean and variance of the normal vectors in the vertical direction and the mean and variance of the normal vectors in the horizontal direction. It should be noted that, regarding the edge portion, since the measured points around it are the upper surface or the side surface, the direction is oblique.

[0109] [Details of the method for generating the shore side straight line]

[0110] Next, a method of generating the shore side surface straight line L will be described.

[0111] (First generation method)

[0112] Figure 7 This is a diagram showing an example of a berthing area where a front mark and a rear mark are provided.

[0113] In the present generation method, for example, regarding making the ship move to Figure 7The present invention will be described below with reference to the drawings. The present invention will be described below with reference to the drawings. In addition, in the present generation method, it is assumed that two marks having high reflectivity characteristics such as a retroreflective plate are set at the shore place SBP for description. In addition, in the subsequent description of the present generation method, unless otherwise specified, the mark set at the front side of the ship landing at the shore place SBP is referred to as the front mark FM, and the mark set at the rear side of the ship is referred to as the rear mark RM. In addition, in the present generation method, it is assumed that the front mark FM and the rear mark RM are set at positions with different heights from the upper surface of the shore place SBP for description. It should be noted that, as long as it is possible to identify which side of the front side and the rear side of the ship is located, the front mark FM and the rear mark RM may have different shapes from each other, or may have different sizes from each other.

[0114] The berthing parameter calculation unit 16 obtains the dot matrix data FMG constituting the front mark FM and the dot matrix data RMG constituting the rear mark RM, for example, by extracting the measured points having a predetermined intensity or higher from the dot matrix data generated by the laser radar 3. In addition, the berthing parameter calculation unit 16 obtains the coordinate position M of the front mark FM in the ship coordinate system based on the centroid position of each measured point included in the dot matrix data FMG. f [m fx m fy m fz ] T In addition, the berthing parameter calculation unit 16 obtains the coordinate position M of the rear marker RM in the ship coordinate system based on the centroid position of each measured point included in the dot matrix data RMG. r [m rx m ry m rz ] T .

[0115] Then, the berthing parameter calculation unit 16 calculates the berthing parameter based on the coordinate position M of the front marker FM. f , the coordinate position M of the rear marker RM r A rectangular extraction window EWA is set on the xy plane based on the specified margin Δw.

[0116] Specifically, the extraction window EWA is, for example, Figure 8 The coordinate position FWA [m fx +Δw m ry +Δw] T 、Coordinate position FWB[m fx +Δw m fy -Δw] T , coordinate position RWA[m rx -Δw m ry +Δw]T And coordinate position RWB[m rx -Δwm fy -Δw] T Set to the rectangular area of ​​the vertices. Figure 8 It is a top view clearly showing the extraction window EWA.

[0117] It should be noted that the extraction window EWA is not limited to being set as a rectangular (or substantially rectangular) area on the xy plane, and may be set as a cuboid (or substantially cuboid) area in the xyz space, for example. fz and m rz The relatively small coordinate value in the coordinate position M can be set as the upper limit value of the extraction window EWA in the z direction. f and M r The corresponding rectangular area or cuboid area is set as the search range of the edge of the landing place SBP.

[0118] As described above, the berth parameter calculation unit 16 searches for the nearest point to the target ship for each longitudinal line from the measured points constituting the dot matrix data of the berth site SBP, and acquires the neighboring point set SG of the berth site SBP.

[0119] The landing parameter calculation unit 16 extracts the measured points within the extraction window EWA from the neighboring point set SG to obtain dot matrix data (hereinafter also referred to as "landing place edge dot matrix") including a plurality of measured points (nearest points) constituting the edge (specifically, the boundary between the upper surface and the side surface) of the landing place SBP. According to such processing, for example, it is possible to extract the dot matrix data including the following: Fig.9A The neighboring point set SG of each nearest neighbor point shown in FIG. 1 can be obtained, including Fig. 9B The landing place edge matrix SP of each nearest point is shown. Fig.9A It is a diagram showing the nearest point of each vertical line in the berthing place SBP. Fig. 9B FIG. 1 is a diagram showing an example of each nearest point acquired as a landing place edge point matrix by the extraction window EWA.

[0120] Then, the shore approach parameter calculation unit 16 generates the shore approach side surface straight line L by applying principal component analysis or the least square method to each measured point (nearest point) included in the shore approach site edge point matrix SP.

[0121] It should be noted that, according to the present generation method, the landing parameter calculation unit 16 may also calculate the landing parameter based on, for example, the average value of the z coordinate values ​​of a plurality of measured points corresponding to the upper surface of the landing place SBP and the coordinate position M. f The z coordinate value m fzThe height h of the installation position of the forward mark FM based on the upper surface of the berthing place SBP is calculated. mf In addition, according to the present generation method, the berthing parameter calculation unit 16 may also calculate the berthing parameter based on, for example, the average value of the z coordinate values ​​of a plurality of measured points corresponding to the upper surface of the berthing place SBP and the coordinate position M. r The z coordinate value m rz The height h of the installation position of the rear mark RM is calculated based on the upper surface of the berthing place SBP. mr (Refer to Fig. 10A ). In addition, according to the present generation method, the berthing parameter calculation unit 16 may also be based on the coordinate position M f and M r The mark interval r is calculated as the distance between the front mark FM and the rear mark RM. m (Refer to Fig. 10B ). In addition, according to the present generation method, the shore parameter calculation unit 16 may also calculate, for example, the distance w from the shore side straight line L to the front mark FM. mf and the distance w from the shore side straight line L to the rear mark RM mr (Refer to Fig. 10C ). In addition, according to the present generation method, the landing parameter calculation unit 16 may also calculate the height h mf , height h mr , mark interval r m , distance w mf , distance w mr The calculation results are saved in a storage medium such as memory 12. Fig. 10A This is the height h of the mark setting position. mf , height h mr . Fig. 10B It is clear that the marking interval r m Top view of the . Fig. 10C It indicates the distance w mf and distance w mr Top view of the .

[0122] According to the present generation method, instead of the predetermined margin Δw, a margin w corresponding to (the actual measured value of) the distance from the shore side straight line L to the front mark FM may be used. mf and a margin w corresponding to the distance (measured value) from the shore side straight line L to the rear mark RM mr In this case, the extraction window EWA is generated as the extraction window FWA[m fx +w mf m ry +w mf ] T、Coordinate position FWB[m fx +w mf m fy -w mf ] T , coordinate position RWA[m rx -w mr m ry +w mr ] T And coordinate position RWB[m rx -w mr m fy -w mr ] T That is, the landing parameter calculation unit 16 can set the search range of the edge of the landing place SBP using the distance from the landing side straight line L to the front mark FM and the distance from the landing side straight line L to the rear mark RM.

[0123] (Second generation method)

[0124] In the present generation method, an example will be described in which data corresponding to one of two markers provided at the landing site SBP is obtained, but data corresponding to the other of the two markers is not obtained.

[0125] The berthing parameter calculation unit 16 obtains the dot matrix data CMG corresponding to one of the two markers CM set at the berthing place SBP by, for example, extracting the measured points having a predetermined intensity or higher from the dot matrix data generated by the laser radar 3. In addition, the berthing parameter calculation unit 16 obtains the coordinate position CP [m cx m cy m cz ] T In addition, the berthing parameter calculation unit 16 calculates the z coordinate value m of the coordinate position CP based on the average value of the z coordinate values ​​of the plurality of measured points corresponding to the upper surface of the berthing place SBP and the z coordinate value m of the coordinate position CP. cz The height h of the installation position of the mark CM based on the upper surface of the berthing place SBP is calculated. mc .

[0126] The berthing parameter calculation unit 16 calculates the height h mc and the height h calculated previously mf and h mr By comparison, it is determined whether the mark CM is the front mark FM or the rear mark RM. mf and h mrFor example, the information may be stored in the memory 12 as information related to landing assistance for the target ship.

[0127] For example, the landing parameter calculation unit 16 calculates the height h mc Compared to the height h mr Closer to height h mf In the case where the marker CM is determined to be the front marker FM. In addition, when the landing parameter calculation unit 16 determines that the marker CM is the front marker FM, it extracts the x coordinate value m having the coordinate position CP from the neighboring point set SG. cx The landing parameter calculation unit 16 extracts the distance from the mark CM from each measured point detected as described above to obtain the mark interval r calculated in the past. m That is, when it is determined that the mark CM is the front mark FM, the x coordinate value x that satisfies both the following mathematical expressions (1) and (2) is obtained from the neighboring point set SG. i and the y coordinate value y i It should be noted that, for example, when it is determined that the mark CM is the front mark FM, the data including the following is obtained: Fig.11 The landing place edge matrix SP of each nearest point is shown. Fig.11 3 is a diagram showing an example of each nearest point acquired when it is determined that the mark CM is a front mark.

[0128]

[0129]

[0130] On the other hand, the landing parameter calculation unit 16 calculates the landing parameter at a height h mc Compared to the height h mf Closer to height h mr In the case where the marker CM is determined to be the rear marker RM. In addition, when the landing parameter calculation unit 16 determines that the marker CM is the rear marker RM, it extracts a point having an x ​​coordinate value m from the coordinate position CP from the neighboring point set SG. cx The landing parameter calculation unit 16 extracts the distance from the mark CM from each measured point extracted as described above to become the mark interval r calculated in the past. m That is, when it is determined that the mark CM is the rear mark RM, the x coordinate value x that satisfies both the following mathematical expressions (3) and (4) is obtained from the neighboring point set SG. i and the y coordinate value yi The data is used as the edge point matrix SP of the landing place.

[0131]

[0132]

[0133] The docking parameter calculation unit 16 performs extraction processing based on mathematical formulas (1) and (2) on the neighboring point set SG obtained as described above when the mark CM is determined to be the front mark FM, and performs extraction processing based on mathematical formulas (3) and (4) when the mark CM is determined to be the rear mark RM, thereby obtaining the edge point matrix SP of the docking location.

[0134] Then, the shore approach parameter calculation unit 16 generates the shore approach side surface straight line L by applying principal component analysis or the least square method to each measured point (nearest point) included in the shore approach site edge point matrix SP.

[0135] According to the above-described processing, the landing parameter calculation unit 16 can calculate the landing parameter based on the marking interval r m The search range of the edge point matrix of the landing place SBP is set by the coordinate position CP of the marker CM.

[0136] It should be noted that, according to the present generation method, for example, in the representation mark interval r m If the information is not stored in the memory 12, the mark interval r may be replaced. m The full length of the target ship (length in the x direction) is applied to the above-mentioned mathematical formulas (2) and (4) to obtain the berthing place edge dot matrix SP. For example, by referring to the quay wall map DB, the stored information related to the upper limit of the full length of the ship that can dock is obtained. And, when the full length of the target ship is shorter than the obtained upper limit value, it can be determined that the target ship can dock. Therefore, it can be considered that the mark interval of the quay wall is at least longer than the full length of the target ship. Therefore, by using the full length of the target ship for searching, although the search range will be narrowed, it is possible to prevent the extraction of unnecessary dot matrix. That is, the berthing parameter calculation unit 16 can replace the mark interval r m On the other hand, the search range of the edge of the berth place SBP is set using the full length of the target ship.

[0137] According to the present generation method, in the above-mentioned mathematical formula (1), instead of the predetermined margin Δw, a margin w corresponding to (the actual measured value of) the distance from the shore side straight line L to the front mark FM may be applied. mf In addition, according to the present generation method, in the above-mentioned mathematical formula (3), instead of the predetermined margin Δw, a margin w corresponding to the distance (actually measured value) from the shore side straight line L to the rear mark RM may be applied. mrThat is, the shore-side parameter calculation unit 16 can set the search range of the edge of the shore place SBP using the distance from the shore side surface straight line L to the mark CM.

[0138] According to the present generation method, for example, when marker information MJ, which is information about markers installed at a berthing place, is stored in a storage medium such as the memory 12, the berthing parameter calculation unit 16 may perform processing using information included in the marker information MJ.

[0139] The mark information MJ includes information related to the forward mark FM, for example, the installation position (latitude, longitude, and altitude), the height from the upper surface of the quay wall (equivalent to the height h mf ), the distance from the shore (equivalent to the distance w mf In addition, the mark information MJ includes information related to the rear mark RM, such as the installation position (latitude, longitude and altitude), the height from the upper surface of the quay (equivalent to the height h mr ), the distance from the shore (equivalent to the distance w mr ), shape and size. In addition, the mark information MJ may also include the distance between marks (equivalent to the mark interval r m )

[0140] (Third Generation Method)

[0141] In the present generation method, an example is described of a situation in which data corresponding to one of two marks set at the docking place SBP is obtained, data corresponding to the other of the two marks is not obtained, and it is impossible to determine whether the mark on one side is the front mark FM or the rear mark RM.

[0142] The approach parameter calculation unit 16 assumes that the mark CM is the front mark FM, and performs the same processing as described in the second generation method to obtain the x coordinate value x satisfying both the above-mentioned mathematical expressions (1) and (2). i and the y coordinate value y i The data of each measured point is obtained as the landing place edge point matrix SPD. Then, the landing parameter calculation unit 16 generates a straight line LD which is a straight line indicating the edge of the landing place SBP when the mark CM is assumed to be the front mark FM by applying principal component analysis or least square method to each measured point (nearest point) included in the landing place edge point matrix SPD. According to such processing, for example, a straight line having the following characteristics is obtained: Fig.12 In addition, according to the above-mentioned processing, for example, the following is generated: Fig.12 The straight line LD is shown. Fig.12 2 is a diagram showing an example of a straight line generated assuming that the mark CM is a front mark.

[0143] The approach parameter calculation unit 16 obtains an x-coordinate value x satisfying both the above-mentioned equations (3) and (4) by performing the same processing as described in the second generation method, assuming that the marker CM is the rear marker RM. i and the y coordinate value y i The landing parameter calculation unit 16 generates a straight line LE which represents the edge of the landing place SBP assuming that the mark CM is the rear mark RM by applying principal component analysis or least square method to each measured point (nearest point) included in the landing place edge point matrix SPE. Fig.13 In addition, according to the above-mentioned processing, for example, the following is generated: Fig.13 The straight line LE shown. Fig.13 2 is a diagram showing an example of a straight line generated when it is assumed that the marker CM is a rear marker.

[0144] The landing parameter calculation unit 16 calculates the variance VD of each measured point in the landing site edge point matrix SPD with respect to the straight line LD and the variance VE of each measured point included in the landing site edge point matrix SPE with respect to the straight line LE. The landing parameter calculation unit 16 compares the variances VD and VE and selects the straight line with the calculated relatively small variance as the landing side straight line L along the edge of the landing site SBP. It should be noted that Fig.12 and Fig.13 In the example shown, since the variance VE is smaller than the variance VD, the straight line LE is selected as the shore side straight line L.

[0145] It should be noted that even when there is only one single mark in the landing area, if the map data in the memory 12 stores information indicating that it is a single mark and stores the distance d ahead from the mark as information related to the landing area, mf And the rear distance d mr , then the edge point matrix SP of the docking place can also be obtained. It should be noted that the front distance d mf And the rear distance d mr , described later. Fig.16 As shown, the landing parameter calculation unit 16 reads the mark information of the landing target area from the map data, recognizes that it is a single mark, and uses the coordinates of the mark CM and the front distance d from the mark to the neighboring point set SG obtained as described above. mf and rear distance dmr The extraction processing based on the following mathematical formulas (5) and (6) and the extraction processing based on (7) and (8) are performed, and the extraction results are combined to obtain the landing place edge point matrix SP. Fig.16 This is a diagram showing an example of processing related to acquisition of a landing place edge matrix.

[0146]

[0147]

[0148]

[0149]

[0150] [Processing Flow]

[0151] Fig.14A The information processing device 1 repeatedly executes Fig.14A Flowchart of the process.

[0152] First, the information processing device 1 obtains dot matrix data in the direction of the berthing place (step S11). In this case, the information processing device 1 obtains, for example, dot matrix data generated by the laser radar 3 that includes the berthing side in the measurement range of the target ship. In addition, the information processing device 1 may further perform downsampling of the obtained dot matrix data and remove data obtained by reflection on the water surface.

[0153] Next, the docking place detection unit 15 of the information processing device 1 calculates the normal vector based on the dot matrix data obtained in step S11 (step S12). In addition, the docking place detection unit 15 calculates the number of normal vectors and the variance of the normal vectors in step S12. In addition, the docking place detection unit 15 determines the inner surface of the field of view and the detection surface based on the processing result of step S12 (step S13).

[0154] Next, the approach parameter calculation unit 16 performs a nearest neighbor search for each longitudinal line based on the dot matrix data acquired in step S11 to obtain the nearest point for each longitudinal line (step S14).

[0155] Next, the landing parameter calculation unit 16 obtains the landing place edge lattice including the nearest point of each longitudinal line based on any one of the first to third generation methods described above (step S15 ).

[0156] Here, an example of the processing performed in step S15 will be described with reference to Fig. 14B While explaining. Fig. 14B This is a flowchart showing an example of processing related to acquisition of a landing place edge matrix.

[0157] After detecting the markers installed at the berthing place (step S31), the berthing parameter calculation unit 16 obtains map data or previously calculated marker information (marker height, distance from the edge of the quay, distance between markers, etc.) from the memory 12 (step S32).

[0158] The approach parameter calculation unit 16 specifies the number of markers detected in step S31 and the positions of the markers detected in step S31 based on the map data or the marker information acquired in step S32 (step S33 ).

[0159] The approach parameter calculation unit 16 determines whether the number of the marker detected in step S31 is 1 or 2 based on the determination result in step S33 (step S34).

[0160] When the number of markers detected in step S31 is one, the docking parameter calculation unit 16 performs the processing of step S36 described later. When the number of markers detected in step S31 is two, the docking parameter calculation unit 16 sets the search range of the edge of the docking place using the extraction window (step S35), and then performs the processing of step S41 described later.

[0161] The landing parameter calculation unit 16 determines whether the marker installed at the landing place is a single marker based on the determination result of step S33 (step S36).

[0162] If the mark set at the berthing place is not an independent mark (step S36: No), the berthing parameter calculation unit 16 performs the processing of step S38 described later. If the mark set at the berthing place is an independent mark (step S36: Yes), the berthing parameter calculation unit 16 uses the front distance d included in the map data obtained in step S32. mf And the rear distance d mr After setting the search range (step S37), the processing of step S41 described later is performed. It should be noted that the berthing parameter calculation unit 16 does not include the front distance d in the map data obtained in step S32. mf And the rear distance d mr If the map data is not obtained in step S32, the processing of step S38 described later is performed.

[0163] The approach parameter calculation unit 16 determines whether the mark detected in step S31 is recognizable as a front mark or a rear mark based on the determination result in step S33 (step S38).

[0164] When the marker detected in step S31 can be identified as a front marker or a rear marker (step S38: Yes), the landing parameter calculation unit 16 uses the marker interval r m After setting the search range based on the x-coordinate of the marker position (step S39), the processing of step S41 described below is performed. In addition, when the marker detected in step S31 cannot be identified as a front marker or a rear marker (step S38: No), the landing parameter calculation unit 16 sets two search ranges corresponding to the case where the marker is a front marker and the case where the marker is a rear marker (step S40), respectively, and then performs the processing of step S41 described below.

[0165] The docking parameter calculation unit 16 performs a process of acquiring the edge matrix of the docking place using the search range set by the process up to step S40 (step S41). Then, the docking parameter calculation unit 16 updates the marking information (marking height, distance from the edge of the quay, distance between markings, etc.) stored in the memory 12 using the process results up to step S41 (step S42), and then continues the process after step S16.

[0166] The landing parameter calculation unit 16 generates a landing side straight line L using the landing place edge point matrix including the nearest point of each longitudinal line obtained in step S15 (step S16). At this time, when a mark is detected that is unclear whether it is the front or the rear, since there are a shore edge point matrix when it is assumed to be the front and a shore edge point matrix when it is assumed to be the rear, a straight line is generated using both of them, and the variance of the shore edge point matrix with respect to each straight line is calculated, and the one with the smaller variance value is taken as the landing side straight line L.

[0167] Next, the shore approach parameter calculation unit 16 calculates the shore approach distance, approach angle, and shore approach speed as shore approach parameters using the shore approach side surface straight line L calculated in step S16 (step S17 ).

[0168] Then, the berthing parameter calculation unit 16 generates reliability information based on the determination result of the visual field inner surface and the detection surface in step S13 and the calculation result of the berthing parameter in step S17 (step S18). Thereafter, the information processing device 1 controls the ship based on the reliability information (step S19). Thus, the information processing device 1 can accurately perform the control of the ship related to the berthing based on the reliability that accurately reflects the berthing condition.

[0169] Then, the information processing device 1 determines whether the target ship has docked (landed) (step S20). In this case, the information processing device 1 determines whether the target ship has docked, for example, based on the output signal of the sensor group 2 or the user input via the interface 11. And, if the information processing device 1 determines that the target ship has docked (step S20; yes), the processing of the flowchart ends. On the other hand, if the target ship has not docked (step S20; no), the information processing device 1 returns the processing to step S11.

[0170] According to the above-described processing, the acquisition unit acquires the measurement data generated by the measurement device installed in the ship. In addition, according to the above-described processing, the mark position acquisition unit acquires the position of at least one of the two marks installed in the berthing place based on the measurement data. In addition, according to the above-described processing, the search range setting unit sets the search range of the edge of the berthing place based on the position of at least one mark.

[0171] It should be noted that, in the description of the first to third generation methods and the above-mentioned flowchart, after generating a neighboring point set that collects the nearest search points of each longitudinal line, the edge point matrix of the docking place is extracted using the mark, but the order can also be changed, and after extracting the object point matrix based on the mark from the point matrix data of the laser radar 3, the nearest search for each longitudinal line is performed on the extracted point matrix data to extract the edge point matrix of the docking place. Even if it is set to this order, the same result can be obtained.

[0172] Here, for example, when the laser radar captures a shore surface that is not the target, since the edge points of the landing place are not arranged in a straight line, even if the straight line is obtained by principal component analysis or least squares method as it is, it is impossible to generate the correct edge straight line of the target shore (refer to Fig.15A ). In addition, when an incorrect edge straight line is generated, an incorrect distance may be calculated. If an incorrect edge straight line is used as the shore side straight line L, even if a perpendicular line to the straight line is obtained and the distance is calculated, it will not be the correct distance to the shore. In contrast, according to this embodiment, even if there are other shore wall surfaces near the shore wall to be landed, a correct straight line can be generated because only the dot matrix located between the marks is used (see Fig. 15B ). Therefore, by calculating the distance to the perpendicular line L of the correctly generated shore side line, the correct distance to the opposite shore can be obtained. In addition, according to this embodiment, for example, even if the laser radar captures a ship at anchor and mistakenly identifies its side as a wall, since the dot matrix of the ship is excluded by using the marker, a correct straight line can be generated (see Fig. 15C Therefore, even in such a case, by calculating the distance to the perpendicular line of the correctly generated shore side straight line L, the correct distance to the opposite shore can be obtained. Fig.15AFIG. 1 is a diagram showing an example of a case where an incorrect edge straight line is generated. Fig. 15B and Fig. 15C 1 is a diagram showing an example of a case where a correct edge straight line is generated. Therefore, according to this embodiment, parameters used in landing assistance such as distance, speed, and angle can be calculated with high accuracy based on the shore side straight line generated as described above.

[0173] It should be noted that, when three or more marks are provided in the area where the landing is to be performed, the marks at both ends of the front side and the rear side of the landing area may be used as the target.

[0174] In order to identify whether the mark is in the front or rear direction, the height of the mark from the upper surface of the quay wall is made different, but this can also be achieved by other methods. For example, the size, shape, and orientation of the mark in the front and rear directions can be made different. By determining the size, shape, and orientation of the mark based on the mark dot matrix data detected by the laser radar 3, it is possible to identify whether the mark is in the front or rear direction. In this case, information such as the size, shape, and orientation of the mark can also be stored in the map DB.

[0175] <Second Embodiment>

[0176] Next, the second embodiment will be described below. It should be noted that in this embodiment, the description of the parts to which the same structure as the first embodiment can be applied is appropriately omitted, and the description is mainly focused on the parts that are different from the first embodiment. Specifically, in this embodiment, the system structure, hardware structure, and functional structure are roughly the same as those of the first embodiment. On the other hand, the content of the processing performed in the berthing parameter calculation unit 16 is different from that of the first embodiment. Therefore, the following mainly describes the processing performed by the berthing parameter calculation unit 16.

[0177] [Processing related to detection of the landing area]

[0178] The approach parameter calculation unit 16 obtains the coordinate position M of the forward marker FM in the ship coordinate system by performing the same processing as that described in the first embodiment. f [m fx m fy m fz ] T and the coordinate position M of the rear marker RM in the ship's coordinate system r [m rx m ry m rz ] T The shore approach parameter calculation unit 16 generates the shore approach side line L by performing the same processing as that described in the first embodiment.f and M r The front end FE and the rear end RE (refer to Fig.17A ) is determined as the landing area. f and M r The distance d from the origin O of the hull coordinate system to the front end of the berth area is calculated by using the berth side straight line L f and the distance d from the origin O to the rear end of the berthing area r The origin O corresponds to the reference point of the target ship, and may be set to, for example, the center position or the center of gravity position of the target ship. In addition, the berthing area is an area where the target ship should be located when berthing at the berthing place. Fig.17A It clearly indicates the origin O and the distance d f and distance d r , a top view of the front end portion FE and the rear end portion RE.

[0179] Here, regarding the distance d f and distance d r For a specific example of the processing, refer to Fig.17A and Fig. 17B While explaining. Fig. 17B It is clear that at a distance d f and distance d r Graph of the unit vector "u" used in the calculation of .

[0180] The shore approach parameter calculation unit 16 generates the shore approach side surface straight line L represented by the following mathematical formula (9) by performing the processing described in the first embodiment.

[0181]

[0182] Here, "[x0 y0 z0] T " represents the center of gravity of the measured point represented by the edge dot matrix of the docking place, "[a bc] T " represents a direction vector, and "t" represents a mediating variable. For example, in the case of principal component analysis, the eigenvector corresponding to the largest eigenvalue becomes the direction vector of the shore side straight line L.

[0183] Then, the shore-approaching parameter calculation unit 16 calculates a unit vector u perpendicular to the shore-approaching side straight line L in a two-dimensional plane as shown in the following mathematical formula (10). The unit vector u is expressed as Fig. 17B Such a vector starts from the origin O.

[0184]

[0185] Coordinate position M fThe inner product of the (x, y) component of the unit vector u is equivalent to the coordinate position M f The projection of to vector u. Therefore, Fig. 17B The distance "d f " The related mathematical formula (11) is established.

[0186]

[0187] Therefore, the berthing parameter calculation unit 16 calculates the distance d f It is calculated based on the following mathematical formula (12).

[0188]

[0189] In addition, the approach parameter calculation unit 16 calculates the distance d by using the same method as described above. r Based on the following mathematical formula (13), Fig.17A Then when the origin O is between FE and RE, the distance d f and distance d r Both become positive values. When the origin O is located in front of FE, the distance d f becomes a negative value and the distance d r becomes a positive value. When the origin O is located behind RE, the distance d f becomes a positive value and the distance d r becomes a negative value.

[0190]

[0191] [Processing Flow]

[0192] The processing flow in this embodiment is the same as that described in the first embodiment ( Fig.14A On the other hand, in this embodiment, for example, in step S17, the distance d is calculated using the shore side straight line L generated in step S16. f and distance d r The distance d f and distance d r This can be used to calculate the approach parameter in step S17.

[0193] As described above, according to the present embodiment, the distance from the current position of the target ship to the end of the berthing area can be calculated with high accuracy as a parameter used in the landing support.

[0194] It should be noted that, when three or more marks are provided in the area where the landing is to be performed, the marks at both ends of the front side and the rear side of the landing area may be used as the target.

[0195] Even if only one mark is set in the area where the landing is to be made, if the map data in the memory 112 stores the distance d ahead from the mark as the landing area, mf and rear distance d mr , then the distance d can also be calculated f and distance d r .like Fig.18 As shown, when the ship is about to land, the berthing parameter calculation unit 16 refers to the mark information from the map data and obtains the front distance d from the mark. mf and rear distance d mr Calculate the distance d of the mark along the line L m , according to the distance d mf and distance d mr , the distance d f and distance d r It is calculated using the following mathematical formulas (14) and (15). Fig.18 is the distance d f and distance d r A diagram of an example of processing involved in the calculation of .

[0196]

[0197]

[0198] In each of the above-mentioned embodiments, the program can be stored using various types of non-transitory computer readable media and supplied to a control unit or the like as a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (e.g., floppy disks, magnetic tapes, hard disk drives), optical magnetic storage media (e.g., optical magnetic disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0199] The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention. That is, the present invention certainly includes various deformations and modifications that can be made according to the entire disclosure and technical ideas including the claims by those skilled in the art. In addition, the above-mentioned patent documents and other disclosures cited are incorporated into this application by reference.

[0200] Description of Reference Numerals

[0201] 1 Information processing device

[0202] 2 Sensor Groups

[0203] 3 Lidar.

Claims

1. An information processing device, comprising: an acquisition unit that acquires measurement data generated by a measurement device installed on the ship; a marker position acquisition unit that acquires the position of at least one marker among the markers provided at the berthing place based on the measurement data; and The search range setting unit sets a search range of a dot matrix at an edge portion of the landing place based on the position of the at least one marker.

2. The information processing device according to claim 1, The mark position acquisition unit acquires the position of a first mark provided on the front side of the ship and the position of a second mark provided on the rear side of the ship. The search range setting unit sets a substantially rectangular region or a substantially cuboid region corresponding to the position of the first marker and the position of the second marker as the search range.

3. The information processing device according to claim 2, The search range setting unit sets the search range using a distance from a straight line along an edge of the landing place to the first mark and a distance from the straight line to the second mark.

4. The information processing device according to claim 1, The mark position acquisition unit acquires the position of a mark provided on the front side or the rear side of the ship. The search range setting unit sets the search range based on an interval between two markers provided at the landing site and a position of the one marker.

5. The information processing device according to claim 4, The search range setting unit sets the search range using a distance from a straight line along an edge of the landing place to the one mark.

6. The information processing device according to claim 4, The search range setting unit sets the search range using the entire length of the ship.

7. The information processing device according to claim 1, The mark position acquisition unit acquires the position of a first mark provided on the front side of the ship and the position of a second mark provided on the rear side of the ship. The information processing device also has a distance calculation unit, which calculates the distance from the reference point of the ship to the end of the berthing area based on the position of the first mark, the position of the second mark and a straight line along the edge of the berthing place. The berthing area is the area where the ship should be located when docking at the berthing place.

8. The information processing device according to any one of claims 1 to 7, Two marks are provided at positions in the berthing area at different heights from the upper surface of the berthing area.

9. A control method is a control method executed by a computer, wherein: Acquire measurement data generated by a measurement device installed on the ship, Based on the measurement data, the position of at least one marker installed at the berthing place is obtained, A search range for an edge of the landing place is set based on the position of the at least one marker.

10. A program that causes a computer to execute the following processing: Acquire measurement data generated by a measurement device installed on the ship, Based on the measurement data, the position of at least one marker installed at the berthing place is obtained, A search range for an edge of the landing place is set based on the position of the at least one marker.

11. A storage medium storing the program according to claim 10.

Citation Information

Patent Citations

  • Information processing device, control method, program, and storage medium

    WO2019188745A1