Ship inland waterway navigation lidar identification system and its operation method

Abstract

The present invention aims to provide a ship inland waterway navigation lidar identification system, comprising a receiver, a storage, a processor and a display. The processor is connected to the receiver and the storage; the display is connected to the processor. In summary, the ship inland waterway navigation lidar identification system can use the preset map data to confirm the point data that can be ignored in the pre-program, so the traditional alignment step can be omitted, thereby improving the calculation speed.

Description

Technical Field

[0001] The present invention relates to a ship navigation lidar identification system and an operation method thereof, and in particular to a ship navigation lidar identification system for inland waterways and an operation method thereof. Background Art

[0002] Route planning is a fundamental problem in the control of unmanned water vehicles. The central computer completely specifies the route that the unmanned water vehicle can follow from the beginning of navigation to the destination, usually choosing not to collide with any obstacles in the navigation environment; or reducing the possibility of collision as much as possible. Given the many interfering factors on the water surface (such as wind direction, water flow, light reflection, etc.), the challenge of route planning will involve the ability to execute route planning in a timely manner at a very fast speed, even in real time, as the characteristics of the navigation environment change.

[0003] For example, the position or shape characteristics of one or more water obstacles in the navigation environment may change over time (e.g., large floating objects may move based on the flow direction); and, similarly, the position of the unmanned water vehicle itself may also change over time. Therefore, the challenge of route planning is that the processing speed of the detection data must be real-time and can be executed continuously to overcome the problem of relative position changes between the detection body (unmanned water vehicle) and the detected object (water obstacle) during operation, so as to be able to correct / adjust the course in a timely manner. Summary of the invention

[0004] To solve at least one of the above problems, some embodiments of the present invention provide a ship and shipping lidar identification system and its operation method, and in particular, a ship and shipping lidar identification system and its operation method for inland waterways, which have the advantage of being able to calculate the detection data in real time and continuously. Specifically, it uses the preset map data to first confirm the point data that can be ignored in the pre-program, so the traditional alignment step (i.e., the step of determining whether the point data is a target detection object) can be omitted; and the calculation speed of the system is improved.

[0005] At least one embodiment of the present invention is a ship inland waterway navigation lidar identification system, which includes a receiver, a storage, a processor and a display, wherein the processor is connected to the receiver and the storage; and the display is connected to the processor.

[0006] At least one embodiment of the present invention is a method for inland waterway lidar identification of ships. The method includes providing the inland waterway lidar identification system; receiving a plurality of point clouds captured by at least one sensor equipped on a transportation vehicle; merging the plurality of point clouds into at least one first point cloud set by a processor; defining at least one second point cloud cluster to be calculated based on the at least one first point cloud set and a map data; merging the points in the at least one second point cloud set into at least one block based on a clustering calculation; implanting at least one mark in the map data by performing calculations on the at least one block; and displaying the map data and the at least one mark therein by the display. In this way, at least one water obstacle will be displayed on the map data, and any of the marks uniquely identifies and represents the water obstacle.

[0007] At least one embodiment of the present invention is characterized by the pre-process of the point cloud. In some cases, in order to define at least one second point cloud set that needs to be calculated, the method further includes: generating a height based on the at least one land area information, wherein if any point in the at least one first point cloud set is higher than (or higher than or equal to) the height, the point is ignored; projecting the at least one first point cloud set to the map data; generating a boundary based on the at least one water area information, wherein if any point in the at least one first point cloud set crosses (or only crosses) the boundary, the point is ignored; and defining the points that are not ignored in the above steps as the at least one second point cloud set.

[0008] The above brief description of the present invention is intended to provide a basic explanation of several aspects and technical features of the present invention. The brief description of the invention is not a detailed description of the present invention. Therefore, its purpose is not to specifically list the key or important components of the present invention, nor is it to define the scope of the present invention. It is only to present several concepts of the present invention in a concise manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0010] Figure 1 It is a schematic diagram of a partial embodiment of the inland waterway ship lidar identification system of the present invention.

[0011] Figure 2 It is a flow chart of some embodiments of the inland waterway ship lidar identification method of the present invention.

[0012] Figure 3It is a schematic diagram of the configuration of some embodiments of the inland waterway ship lidar identification system of the present invention.

[0013] Figure 4 This is a schematic diagram of map data output for some embodiments of the inland waterway vessel lidar identification method of the present invention.

[0014] Figure 5 This is a schematic diagram of map data output for some embodiments of the inland waterway vessel lidar identification method of the present invention.

[0015] Figure 6 This is a schematic diagram of map data output for some embodiments of the inland waterway vessel lidar identification method of the present invention.

[0016] Reference numerals:

[0017] 1…Ship Inland Waterway LiDAR Identification System

[0018] 10…Receiver

[0019] 12…Sensors

[0020] 20… Storage

[0021] 30…Processor

[0022] 40…Display

[0023] 100…Transportation

[0024] 200…Map data

[0025] 300…mark

[0026] 310…object distance

[0027] 320…Object information

[0028] 330a, 330b…prompt line

[0029] (A)~(G)…Steps DETAILED DESCRIPTION

[0030] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention.

[0031] At least one embodiment of the present invention relates to a ship and shipping lidar identification system and an operating method thereof, and more particularly to a ship and shipping lidar identification system for inland waterways and an operating method thereof.

[0032] See also Figure 1The schematic diagram of the inland waterway ship lidar identification system of the present invention is a partial embodiment. The inland waterway ship lidar identification system 1 comprises a receiver 10, a storage 20, a processor 30 and a display 40, the processor 30 is connected to the receiver 10 and the storage 20; the display 40 is connected to the processor 30.

[0033] Figure 1 The receiver 10 is configured to receive point clouds captured by at least one sensor 12; and optionally store one or more point clouds in the storage 20 / send directly to the processor 30. In this embodiment, the sensor 12 is configured in a transport 100 to capture the point cloud in a timely manner along its moving trajectory, wherein the point cloud can be captured using any of a plurality of cameras, video cameras or depth sensors (such as light detection and ranging laser scanning), for example, Velodyne's VLP-16, VLP-32 LiDAR sensors, Quanergy's S3 or M8 LiDAR sensors can be applied; and one or more communication schemes can be used to send / receive the point cloud, for example, using the user datagram protocol (UDP) or other protocols. The point cloud can be represented in different formats tailored to specific applications, such as: xyz or las format, but the present invention is not limited to being represented in a format suitable for all applications.

[0034] Figure 1 The storage 20 in the embodiment is configured to store the point cloud and a map data 200. In the present embodiment, the storage 20 may be any type of large capacity storage (fixed / removable), for example, a storage of volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable or a combination of the above components, which is provided to store any type of information that the processor 30 may need to operate; or one or more computer programs that may be executed. Here, the map data 200 may be an electronic navigation chart (ENC), but is not limited thereto. The map data 200 may also be any satellite imagery (Satellite imagery), Digital line graph (DLG), Digital raster graphics (DRG), Digital orthophotoquadrangle (DOQ) or other maps that can be stored in a computer-readable form that includes at least one land area information (such as a building or a bridge) and at least one water area information (such as a river or a waterway).

[0035] like Figure 1As shown, the processor 30 is configured to: perform access based on each point in the point cloud representing a map; construct at least one first point cloud set representing the point cloud; determine the points that can be ignored in the first point cloud set to determine at least one second point cloud set; and generate at least one block based on the second point cloud set to display the map data 200 including at least one water obstacle mark on the display 40. It should be noted that during the processing, the processor 30 is designed to communicate with the storage 20 to store / retrieve operation-related data, such as one or more point clouds, map data 200, or calculation instructions. Furthermore, the model or type of the processor 30 and the display 40 is not specifically defined herein. For example, the processor 30 can be a microprocessor; it can also be any conventional processor, controller or other component combination / configuration for implementing calculations. For another example, the display 40 can be a computer display / screen, a television device display / screen or a virtual reality device display / screen; it can also be a display itself with a projection imaging function.

[0036] In other possible embodiments, in order to improve the determination accuracy of the second point cloud set, the receiver 10 may optionally include a filter (not shown) to filter out in advance part of the detection data that may cause misjudgment.

[0037] In order to facilitate understanding and explanation of the principle of the actual application of the above-mentioned operation content to the embodiment, the following will be Figure 2 , Figure 3 as well as Figure 4-6 Explain in order.

[0038] first, Figure 2 The flowchart of the embodiment of the method for ship inland waterway navigation lidar identification of the present invention. The method comprises providing a ship inland waterway navigation lidar identification system (step (A)). The embodiment uses the aforementioned Figure 1 A ship inland waterway lidar identification system 1; receiving a plurality of point clouds captured by at least one sensor 12 equipped on a transport vehicle 100 (step (B)); merging the point clouds of step (B) into a first point cloud set by a processor 30 (step (C)); defining a second point cloud set to be calculated based on the aforementioned map data 200 (step (D)), wherein the so-called second point cloud set is a set form after ignoring some points in the first point cloud set; merging the points in the second point cloud set into one or more blocks based on a clustering calculation (step (E)); performing calculations on the blocks to implant at least one mark 300 in the aforementioned map data 200 (step (F)); and displaying the map data 200 after all the marks 300 are implanted by a display 40 (step (G)), so as to identify at least one water obstacle in the map data 200, wherein one mark 300 uniquely identifies and represents a water obstacle.

[0039] Wherein, step (D) is related to an ignoring procedure for the first point cloud set and includes steps (D1)-(D4). Step (D1): Generate a height based on the aforementioned land information, wherein it is assumed that if any point in the first point cloud set is higher than (or higher than or equal to) the height, the point is ignored. Specifically, at this time, a height threshold is determined for the height and a simple algorithm is applied to determine that any point above the height threshold (that is, when the z-axis value of the point is higher than or equal to the height threshold) is a point to be ignored and does not participate in the calculation; Step (D2): Project the first point cloud set to the aforementioned map data 200; Step (D3): Generate a boundary based on the aforementioned water area information, wherein it is assumed that if any point of the at least one first point cloud set crosses (or only crosses) the boundary, the point is ignored; and Step (D4): Define a second point cloud set based on the points that are not ignored in steps (D1) and (D3). Generally speaking, the execution of steps (D1)-(D4) may also involve a morphological iterative algorithm, and in this case, steps (D1) and (D3) may include steps that are understandable to those skilled in the art. In this embodiment, the height threshold is a variable value that will be adjusted in due course according to land information (such as changes in bridge height, etc.); similarly, the boundary is an inland river channel that depends on water area information (such as the width and curvature of the river channel, etc.). Of course, in other embodiments, the height threshold and the boundary may also be definitions of other information, and are not limited to the aforementioned examples.

[0040] And, it can be understood from the previous description that "generating a height" in step (D1) is a selection procedure involving object data in land information. For example, based on water information, an object data in land information that overlaps with its information (such as a bridge on the water surface) is selected; and the shortest distance between the object data and the water surface is set to the aforementioned height to determine the aforementioned height threshold.

[0041] Among them, the so-called clustering calculation in step (E) may involve an iterative algorithm of Kd-tree (K-dimensional tree), and further include steps (E1)-(E3). Step (E1): extract a spatial point in the second point cloud set; step (E2): calculate a distance value between any point in the second point cloud set and the spatial point based on the Kd-tree (K-dimensional tree) method; step (E3): classify the points in the second point cloud set whose distance value is less than or equal to a distance threshold into a block; and step (E4): repeat steps (E1)-(E3) until all points in the second point cloud set are classified (that is, after a plurality of blocks are generated), and then execute step (F). The distance threshold here is a predetermined value, which is only selected according to actual application requirements, and the present invention is not limited to this.

[0042] And wherein, step (E) further includes a step (E21) between step (E2) and step (E3): preferentially designating points in the second point cloud set whose distance values ​​are less than or equal to a regional threshold as the same region, and repeatedly executing steps (E1) and (E21) until the second point cloud set is divided into multiple regions according to the regional threshold, and then executing step (E3) according to individual regions to sequentially classify the points in each region into the blocks. At this time, the regional threshold will be equal to the maximum value of all available distance values. In practical applications, the so-called available distance value refers to the estimated distance value relative to the identified outlier. In this way, the ship inland waterway lidar identification system 1 will be able to more accurately sort and classify the high-density point blocks of the second point cloud set (i.e., the blocks with higher point density in step (E3)).

[0043] Wherein, step (F) further includes steps (F1)-(F3) to calculate the aforementioned blocks one by one. Step (F1): Calculate the coordinate average of all points in any of the blocks to define the coordinates of a point of the center of gravity in any of the blocks, and repeat the calculation until the definition of the coordinates of the center of gravity of each block is completed; Step (F2): Determine an object message representing the coordinates of all the center of gravity points; and Step (F3): Generate at least one corresponding mark 300 through the object message. It should be noted that the present embodiment can use a coding format that complies with the NEMA-0180, NEMA-0182 or NEMA-0183 specifications to represent the object message. For example, assuming that the definition parameters include "$OBS, <1> , <2> , <3> , <4> , <2> , <3> , <4> , …, <2> , <3> , <4> ,* <5> ", those skilled in the art can understand from the content of Table 1 below that the definition parameter includes detecting the current number of water obstacles and at least one corresponding coordinate component, so as to implant at least one correct mark 300 in the map data 200 in real time; of course, the present invention can also adopt other appropriate encoding formats for transmission, and is not limited thereto. In addition, the calculation of the center of gravity of the block is only a common application of a simple algorithm; it is also a technique known to those skilled in the art, and will not be described here in detail.

[0044] <1> There are currently several obstacles <2> Obstacle Number #a <3> Obstacle number #a X component of distance from the ship <4> Obstacle number #a Y component of distance from the ship <5> Checksum

[0045] Table 1

[0046] In addition, the mark 300 in step (F3) may include an object distance 310 and an object information 320 in actual application, but it is not limited thereto. In other actual applications, the mark 300 may also include any object target with a warning function (such as object color, etc.). The object distance 310 is the distance value between the water obstacle and the transportation tool 100; the object information 320 is any shape set with an outline, which is used to indicate the spatial position of the water obstacle.

[0047] It is worth mentioning that, in addition to presenting the mark 300 in step (F3), the map data 200 in step (G) can also selectively present other information represented in a linear manner, such as the detection range of the sensor 12 or the relative position of the water obstacle. In order to effectively identify the mark 300 implanted in the map data 200, the present invention can further color the multiple object distances 310 and the multiple object information 320 according to the attributes before executing step (G). For example, the close distance in the object distance 310 is colored red to enhance the warning effect; similarly, the large water obstacle in the object information 320 (i.e., the one with a larger outline in the aforementioned shape) is colored yellow; and so on. At this time, the so-called attribute can be a further definition of the object distance 310 and the object information 320, such as: the close distance, the long distance of the object distance 310, or the small, medium, and large water obstacles of the object information 320, or other types of definition forms, which are not limited by the present invention. The corresponding relationship between the attribute and the coloring can be a predetermined standard lookup table; or different coloring representations can be randomly given according to actual operation requirements (ie, one attribute corresponds to one coloring).

[0048] In a possible embodiment, when the number of points in the first point cloud set is too large, steps (B1)-(B2) can be selectively performed between steps (B) and (C). Step (B1): read a first contour point within a preset range from the first point cloud set; and read a plurality of second contour points outside the preset range; Step (B2): calculate the distance from any point of the first contour point to any point of the second contour point, and when the minimum value of the distance is greater than a preset threshold, directly ignore the point and the second contour point where it is located. The interpretation of "too large" here depends on the processing speed of the processor 30.

[0049] In a possible embodiment, when the number of points in the first point cloud set is sufficient, the processor 30 may further perform a matching operation between the first point cloud set and the aforementioned map data 200 between steps (B) and (C), for example, using a Normal Distributions Transform (NDT) or an Iterative Closest Point (ICP) algorithm. The "sufficient" number of interpretations here varies depending on the algorithm.

[0050] In a possible embodiment, step (B) may involve a segmentation procedure for the point cloud to eliminate invalid data points (such as reflection points on the water surface); and the so-called segmentation procedure may involve an iterative algorithm of random sample consensus (RANdom Sample Consensus, RANSAC).

[0051] Figure 3 FIG. 1 is a schematic diagram of the configuration of a partial embodiment of the inland waterway ship lidar identification system of the present invention. Figure 3 As shown, the transport 100 is configured to be provided with the ship inland waterway navigation lidar identification system 1 of the present invention. In this embodiment, the transport 100 can be a small water vehicle such as a speedboat, a fishing boat, a sightseeing boat, etc.; of course, the present invention is not limited thereto, and can be applied to other unmanned ships (such as warships, cruise ships, etc., large water vehicles). At this time, the number of the sensors 12 can be one or more, and they are set at the bow position of the transport 100, and the number of sensors 12 depends on the width of the waterway when the transport 100 is running (for example: when the waterway is 1.8m, the number of sensors 12 is 1).

[0052] Please match Figure 2-3 Reference Figure 4-6 , Figure 4-6 This is a schematic diagram of map data output of a partial embodiment of the inland waterway vessel lidar identification method of the present invention, which is used to represent the display of map data 200 including marks 300 on the display 40 to identify a transportation vehicle 100; and the current spatial positions of the water obstacles represented by the marks 300. Figure 4 It shows the output type when the present invention is applied to close-range detection; Figure 5 The output mode when the present invention is applied to medium-range detection is shown; and Figure 6The output mode of the present invention when applied to long-distance detection is shown, wherein the so-called short distance, medium distance and long distance are not limited to any specific distance value, and are only a conceptual representation of the relative distance between water obstacles and transportation vehicles. The sensor 12 used in this embodiment is a VLP-16 LiDAR sensor manufactured by Velodyne, which sends the detection data to the receiver 10 according to the User Datagram Protocol (UDP) packet to analyze at least one feature information (such as the point cloud distance between any ray and any angle in the detection data); and the processor 30 establishes the aforementioned first point cloud set according to the feature information, wherein the first point cloud set is represented by the Cartesian coordinate system.

[0053] Continuing from the above, in order to further improve the calculation speed, the present invention proposes to first determine the points that need to be calculated in the point cloud (that is, determine the points that can be ignored) and then execute the pre-process of subsequent calculations. In this way, in the subsequent process, the processor 30 only needs to calculate some of the points after determination, which greatly reduces the amount of calculation and the amount of related parameters. It should be noted that based on the fact that the present embodiment is applied to unmanned vehicles traveling on water, there are many interference factors on the water surface. Therefore, it is necessary to make real-time and continuous judgments on the detection data in order to make timely adjustments to the channel. The present invention will only group the second point cloud set according to the result of calculating the proximity relationship (such as Kd-tree (K-dimensional tree) calculation), and directly extract the coordinates of the center of gravity point in each group (block) to obtain a mark 300 on the aforementioned map data 200 in a timely manner; and as Figure 4-6 As shown, in this embodiment, the mark 300 includes an object distance 310 showing the distance (m) of the water obstacle and object information 320 indicating the size of the water obstacle; and includes prompt lines 330a, 330b marked to display the detection range of a sensor 12 and the relative position of the water obstacle, wherein the processor 30 converts the detectable spatial boundary value of the sensor 12 into a plurality of prompt lines 330a; and also converts the spatial shortest path between the sensor 12 and any of the center of gravity coordinates into at least one prompt line 330b.

[0054] In summary, although the present invention has been disclosed by way of embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and alterations without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the appended claims.

Claims

1. A LiDAR identification system for inland waterway navigation of ships. It is characterized in that include: a receiver configured to receive a plurality of point clouds captured by at least one sensor; A storage device configured to store the plurality of point clouds and a map data, wherein the map data includes at least one land area information and at least one water area information; a processor connected to the receiver and the storage, the processor being configured to: merge the plurality of point clouds into at least one first point cloud set; determine at least one second point cloud set to be calculated based on the map data, while ignoring a portion of the at least one first point cloud set that includes the at least one land area information; and generate at least one block through the at least one second point cloud set; as well as a display, connected to the processor, the display being configured to display the map data; The land area information generates a height threshold, the height threshold is a variable value, and the height threshold is adjusted according to an object data of the land area information; Among them, a first contour point within a preset range is read from the at least first point cloud set; and multiple second contour points outside the preset range are read, and a distance from any point of the first contour point to any point of the multiple second contour points is calculated. When the minimum value of the distance is greater than a preset threshold, the first contour point and the second contour point where it is located are directly ignored.

2. The inland waterway ship lidar identification system according to claim 1, It is characterized in that The receiver further includes a filter.

3. The inland waterway ship lidar identification system according to claim 1, It is characterized in that The at least one sensor is a LiDAR sensor.

4. The inland waterway ship lidar identification system according to claim 1, It is characterized in that The map data is an electronic navigation chart.

5. A method for identifying inland waterway ship lidar. It is characterized in that The following steps are involved: (A) providing a ship inland waterway navigation lidar identification system as described in claim 1; (B) receiving the plurality of point clouds captured by the at least one sensor equipped on a transportation vehicle; (C) merging the plurality of point clouds into the at least one first point cloud set by the processor; (D) defining the at least one second point cloud set to be calculated based on the at least one first point cloud set and the map data; (E) merging each point in the at least one second point cloud set into at least one block based on a clustering calculation; (F) embedding at least one marker in the map data by performing a calculation on the at least one block; and (G) Displaying the map data and the at least one mark therein by the display.

6. The inland waterway ship lidar identification method according to claim 5, It is characterized in that Wherein step (D) further comprises: (D1) generating a height based on the at least one land area information, wherein if any point in the at least one first point cloud set is higher than or equal to the height, the point is ignored; (D2) projecting the at least one first point cloud onto the map data; (D3) generating a boundary based on the at least one water area information, wherein if any point in the at least one first point cloud set crosses the boundary, the point is ignored; and (D4) Defining at least one second point cloud set based on the points not ignored in steps (D1) and (D2).

7. The inland waterway ship lidar identification method according to claim 5, It is characterized in that Wherein step (E) further comprises: (E1) extracting a spatial point from the at least one second point cloud; (E2) calculating a distance value between any point in the at least one second point cloud set and the spatial point; (E3) classifying points whose distance values ​​are less than or equal to a distance threshold as the same block; and (E4) Repeat steps (E1)-(E3) until all points in the at least one second point cloud set are classified to generate a plurality of blocks.

8. The inland waterway ship lidar identification method according to claim 5, It is characterized in that Wherein step (F) further comprises: (F1) calculating the average coordinates of all points in any of the blocks to define the coordinates of the center of gravity in any of the blocks, and repeatedly performing the calculation until the coordinates of the center of gravity of each of the blocks are defined; (F2) determining an object message representing the coordinates of the plurality of centroid points; and (F3) Generate the corresponding at least one mark through the object message.

9. The inland waterway ship lidar identification method according to claim 7, It is characterized in that Wherein step (E) further includes a step (E21) between step (E2) and step (E3): preferentially designating points in the at least one second point cloud set whose distance value is less than or equal to an area threshold as the same area, and repeatedly executing steps (E1) and (E21) until the at least one second point cloud set is divided into a plurality of areas according to the area threshold.

Images