Tug autonomous accompanying navigation sensing method and system based on multiple sensors

Abstract

The invention provides a tug autonomous accompanying perception method and system based on multiple sensors, and relates to the field of tug autonomous accompanying, and the method comprises the steps: S1, constructing a perception system, enabling the states of the perception system to comprise a personnel operation state, a perception state and an algorithm state, and carrying out the initialization of the states of the perception system; s2, the state of the sensing system is set according to the state information of the target ship, and the set sensing system is obtained; and the sensors in the set sensing system are used for sensing, and a target accompanying position is obtained through calculation. According to the invention, the personnel operation state, the sensing state and the algorithm state of the sensing system are set according to the state information of the target ship, the tug is controlled to carry out autonomous accompanying navigation operation, and the accuracy and the automation degree of the accompanying navigation operation are improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous tugboat escort, and in particular to a tugboat autonomous tugboat escort perception method and system based on multiple sensors. Background Art

[0002] Traditional tugboat operations rely on human drivers, while autonomous escort technology can achieve unmanned or remote control, greatly improving operational efficiency. Tugboats can autonomously adjust their path and speed according to the ship's operating status and channel conditions, thereby completing tasks more quickly.

[0003] At present, although a variety of sensor-assisted technologies have been adopted in tugboat escort operations, these sensors often work independently and fail to effectively utilize their respective optimal detection ranges. In addition, each sensor is equipped with an independent display screen, resulting in frequent problems of information duplication and insufficiency. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a tugboat autonomous escort perception method and system based on multiple sensors, so as to solve the problem of insufficient sensor integration control level in the existing tugboat escort operation.

[0005] The present invention provides a tugboat autonomous escort perception method based on multiple sensors, including:

[0006] S1: Build a perception system. The state of the perception system includes: personnel operation state, perception state and algorithm state. Initialize the state of the perception system.

[0007] S2: The state of the perception system is set according to the state information of the target ship to obtain the perception system after setting; the target escort position is obtained by calculating through each sensor in the perception system after setting.

[0008] Preferred:

[0009] Personnel operation status includes:

[0010] Personnel operation status 1: The static information of the target ship and the preset escort position are not input; Personnel operation status 2: Only the static information of the target ship is input; Personnel operation status 3: The static information of the target ship and the preset escort position are input; Personnel operation status 4: After the radar rotates and the target is selected, the target ship is locked and tracked.

[0011] Preferred:

[0012] Perception states include:

[0013] Perception state 1: initialization state; perception state 2: the AIS information of the target ship is not recognized; perception state 3: the AIS information of the target ship is recognized, but the K-wave radar does not recognize the target ship or the fusion is not completed; perception state 4: the target ship is within 0.5 nautical miles, but the target ship is not recognized; perception state 5: the AIS information of the target ship is recognized, and the K-wave radar recognizes the target ship; perception state 6: the AIS information of the target ship is not received within the preset time, but the K-wave radar recognizes the target ship; perception state 7: the signal to enter the calming state is received, but at this time the distance between the tugboat and the target ship is greater than 100 meters; perception state 8: the signal to enter the calming state is received, and at this time the distance between the tugboat and the target ship is less than 100 meters; perception state 9: the target ship is lost in AIS, K-wave radar and visual camera at the same time.

[0014] Preferred:

[0015] Algorithm states include:

[0016] Algorithm state 1: initialization state; Algorithm state 2: obtain the target escort position based on AIS information calculation; Algorithm state 3: obtain the target escort position based on K-wave radar information calculation; Algorithm state 4: obtain the target escort position based on the fusion calculation of visual camera and lidar.

[0017] Preferably, step S2 specifically comprises:

[0018] S21: Acquire the state information of the target ship input into the perception system, including the static information of the target ship and the preset escort position;

[0019] S22: If only the static information of the target ship is input, and the perception system does not recognize the AIS information of the target ship, the perception system enters perception state 2, personnel operation state 2 and algorithm state 2, the laser radar performs a reset operation, updates the static information of the target ship, and sets the preset escort position to the default value;

[0020] S23: If only the static information of the target ship is input, and the perception system recognizes the AIS information of the target ship, the perception system enters perception state 3, personnel operation state 2 and algorithm state 2, the preset escort position is set to the default value, the laser radar rotates toward the position of the target ship based on the AIS information, and the target escort position is calculated based on the AIS information;

[0021] S24: If the static information of the target ship and the preset escort position are input, and the perception system does not recognize the AIS information of the target ship, the perception system enters perception state 2, personnel operation state 3 and algorithm state 2, the laser radar performs a reset operation, and updates the static information of the target ship;

[0022] S25: If the static information of the target ship and the preset escort position are input, and the perception system recognizes the AIS information of the target ship, the perception system enters perception state 3, personnel operation state 3 and algorithm state 2, and the laser radar rotates toward the position of the target ship based on the AIS information, and the target escort position is calculated based on the AIS information;

[0023] S26: If the target ship is within 0.5 nautical miles from the tugboat, but the K-wave radar does not identify the target ship, the perception system enters perception state 4, personnel operation state 3 and algorithm state 2, and the laser radar rotates toward the position of the target ship based on the AIS information, and calculates the target escort position based on the AIS information;

[0024] S27: If the perception system recognizes the AIS information of the target ship and the K-wave radar recognizes the target ship, the perception system enters perception state 5, personnel operation state 3 and algorithm state 3, the laser radar rotates toward the position of the target ship based on the K-wave radar information, and the target escort position is calculated based on the K-wave radar information;

[0025] S28: If the perception system does not receive the AIS information of the target ship within the preset time, but the K-wave radar recognizes the target ship, the perception system enters perception state 6, personnel operation state 3 and algorithm state 3, the laser radar rotates toward the position of the target ship based on the K-wave radar information, and the target escort position is calculated based on the K-wave radar information;

[0026] S29: If the perception system receives a signal to enter a calm state, and the distance between the tugboat and the target ship is greater than 100 meters, the perception system enters perception state 7, personnel operation state 4 and algorithm state 4, controls the visual camera and laser radar to rotate toward the position of the target ship, and obtains the target escort position based on the fusion calculation of the visual camera and the laser radar;

[0027] S210: If the perception system receives a signal to enter a calm state, and the distance between the tugboat and the target ship is less than 100 meters, the perception system enters perception state 8, personnel operation state 4 and algorithm state 4, controls the visual camera and laser radar to rotate toward the position of the target ship, and obtains the target escort position based on the fusion calculation of the visual camera and the laser radar;

[0028] S211: If the target ship is lost in AIS, K-wave radar and visual camera at the same time, the perception system enters perception state 9 and algorithm state 1, and the perception system issues a warning.

[0029] A tugboat autonomous escort perception system based on multiple sensors, used to implement the tugboat autonomous escort perception method based on multiple sensors, comprising:

[0030] Equipment layer, communication layer, hub layer and service layer;

[0031] The central layer transmits information with the device layer and service layer through the communication layer.

[0032] Preferred:

[0033] The equipment layer includes: visual camera, lidar, K-wave radar, GPS, fiber optic compass, depth sounder, speedometer, wind speed and direction meter, main steam turbine and propeller shaft system.

[0034] Preferred:

[0035] The communication layer includes MQTT communication network, TCP communication network and UDP communication network.

[0036] Preferred:

[0037] The central layer includes: sensor resource allocation module, close-range target recognition module and target tracking module.

[0038] Preferred:

[0039] The service layer includes: user interface, horizontal and vertical adjustment knobs and start and stop devices.

[0040] The present invention has the following beneficial effects:

[0041] A tugboat autonomous escort perception system is designed. The sensors are concentrated in the equipment layer. The central layer controls the communication of the sensors in the equipment layer through the communication layer. Users can manage the sensors in the same way through the service layer, and the information obtained by multiple sensors is integrated and displayed on a unified user interface to improve the management efficiency of each sensor. The personnel operation status, perception status and algorithm status of the perception system are set according to the status information of the target ship, and the tugboat is controlled to perform autonomous escort operations, which improves the accuracy and automation of escort operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0043] Figure 2 Classify the state of the perception system;

[0044] Figure 3 This is a schematic diagram of the relative coordinate information acquisition process;

[0045] Figure 4 It is a schematic diagram of the tugboat coordinate system;

[0046] Figure 5 This is the structure diagram of the perception system;

[0047] Figure 6 Collect sensor data for the device layer and save process diagrams;

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] Reference Figure 1 The present invention provides a tugboat autonomous escort perception method based on multiple sensors, including:

[0051] S1: Build a perception system. The state of the perception system includes: personnel operation state, perception state and algorithm state. Initialize the state of the perception system.

[0052] Specifically, refer to Figure 2 According to the progress of the escort operation, the state of the perception system is divided into personnel operation state, perception state and algorithm state. There are 4, 9 and 5 state numbers to distinguish different working states. Each independent state has corresponding sensor hardware, perception algorithm, user interface and regulatory control system work tasks and operation basis.

[0053] Furthermore, the personnel operation status includes:

[0054] Personnel operation status 1: The static information of the target ship and the preset escort position are not input; Personnel operation status 2: Only the static information of the target ship is input; Personnel operation status 3: The static information of the target ship and the preset escort position are input; Personnel operation status 4: After the radar rotates and the target is selected, the target ship is locked and tracked.

[0055] Furthermore, the perception states include:

[0056] Perception state 1: initialization state; perception state 2: the AIS information of the target ship is not recognized; perception state 3: the AIS information of the target ship is recognized, but the K-wave radar does not recognize the target ship or the fusion is not completed; perception state 4: the target ship is within 0.5 nautical miles, but the target ship is not recognized; perception state 5: the AIS information of the target ship is recognized, and the K-wave radar recognizes the target ship; perception state 6: the AIS information of the target ship is not received within the preset time, but the K-wave radar recognizes the target ship; perception state 7: the signal to enter the calming state is received, but at this time the distance between the tugboat and the target ship is greater than 100 meters; perception state 8: the signal to enter the calming state is received, and at this time the distance between the tugboat and the target ship is less than 100 meters; perception state 9: the target ship is lost in AIS, K-wave radar and visual camera at the same time.

[0057] Furthermore, the algorithm status includes:

[0058] Algorithm state 1: initialization state; Algorithm state 2: obtain the target escort position based on AIS information calculation; Algorithm state 3: obtain the target escort position based on K-wave radar information calculation; Algorithm state 4: obtain the target escort position based on the fusion calculation of visual camera and lidar.

[0059] S2: The state of the perception system is set according to the state information of the target ship to obtain the perception system after setting; the target escort position is obtained by calculating through each sensor in the perception system after setting.

[0060] Specifically, when the tugboat starts autonomous escort, the perception system enters perception state 1, personnel operation state 1, and algorithm state 1. The sensor hardware, perception algorithm, user interface, and regulatory control system have no response to this. With different user operations, the perception system will enter different states, as shown in step S2.

[0061] Furthermore, step S2 is specifically as follows:

[0062] S21: Acquire the state information of the target ship input into the perception system, including the static information of the target ship and the preset escort position;

[0063] S22: If only the static information of the target ship is input, and the perception system does not recognize the AIS information of the target ship, the perception system enters perception state 2, personnel operation state 2 and algorithm state 2, the laser radar performs a reset operation, updates the static information of the target ship, and sets the preset escort position to the default value;

[0064] Specifically, the escort position is the default value - the starboard stern, and the laser radar module is reset; the perception algorithm updates the static information of the target ship and sends information to the user interface for chart display. When this state lasts for 5 seconds, the perception system warns that the target ship is not identified and prompts the user whether the static information is entered incorrectly; the user interface displays the tugboat and surrounding ships on the chart; there is no input to the regulation and control system.

[0065] S23: If only the static information of the target ship is input, and the perception system recognizes the AIS information of the target ship, the perception system enters perception state 3, personnel operation state 2 and algorithm state 2, the preset escort position is set to the default value, the laser radar rotates toward the position of the target ship based on the AIS information, and the target escort position is calculated based on the AIS information;

[0066] Specifically, the escort position is the default value - starboard stern, the lidar module rotates towards the target ship position based on the AIS data, and places the target ship in the center of the screen; the perception algorithm calculates the distance and escort position through the latitude and longitude of the target ship and tugboat; the perception algorithm sends information to the user interface for chart display; the user interface displays the tugboat and the ships around the target ship on the chart, and highlights the target ship; the input data to the regulation and control system is the escort position calculated based on the AIS information.

[0067] S24: If the static information of the target ship and the preset escort position are input, and the perception system does not recognize the AIS information of the target ship, the perception system enters perception state 2, personnel operation state 3 and algorithm state 2, the laser radar performs a reset operation, and updates the static information of the target ship;

[0068] Specifically, the perception algorithm updates the static information of the target ship and sends information to the user interface for chart display. When this state lasts for 5 seconds, the perception system warns that the target ship is not recognized and prompts the user whether the static information is entered incorrectly; the user interface displays the tugboat and surrounding ships on the chart; and there is no input to the control system.

[0069] S25: If the static information of the target ship and the preset escort position are input, and the perception system recognizes the AIS information of the target ship, the perception system enters perception state 3, personnel operation state 3 and algorithm state 2, and the laser radar rotates toward the position of the target ship based on the AIS information, and the target escort position is calculated based on the AIS information;

[0070] Specifically, the lidar module rotates toward the target ship based on the AIS data and places the target ship in the center of the screen; the perception algorithm calculates the distance and escort position through the longitude and latitude of the target ship and tugboat; the perception algorithm sends information to the user interface for chart display; the user interface displays the tugboat and ships around the target ship on the chart and highlights the target ship; the input data to the regulation and control system is the escort position calculated based on the AIS information.

[0071] S26: If the target ship is within 0.5 nautical miles from the tugboat, but the K-wave radar does not identify the target ship, the perception system enters perception state 4, personnel operation state 3 and algorithm state 2, and the laser radar rotates toward the position of the target ship based on the AIS information, and calculates the target escort position based on the AIS information;

[0072] Specifically, the lidar module rotates toward the target ship based on the AIS data and places the target ship in the center of the screen; the perception algorithm calculates the distance and escort position through the longitude and latitude of the target ship and tugboat; the perception algorithm sends information to the user interface for chart display; the user interface displays the tugboat and ships around the target ship on the chart and highlights the target ship; the input data to the regulation and control system is the escort position calculated based on the AIS information.

[0073] S27: If the perception system recognizes the AIS information of the target ship and the K-wave radar recognizes the target ship, the perception system enters perception state 5, personnel operation state 3 and algorithm state 3, the laser radar rotates toward the position of the target ship based on the K-wave radar information, and the target escort position is calculated based on the K-wave radar information;

[0074] Specifically, the lidar module rotates toward the target ship's position based on the K-wave radar data and places the target ship in the center of the screen; the perception algorithm calculates the distance through the longitude and latitude of the target ship and the tugboat, and calculates the escort position through the K-wave radar information; the perception algorithm sends information to the user interface for chart display; the user interface displays the tugboat and ships around the target ship on the chart, and highlights the target ship; the input data to the regulation and control system is the escort position calculated based on the K-wave radar information.

[0075] S28: If the perception system does not receive the AIS information of the target ship within the preset time, but the K-wave radar recognizes the target ship, the perception system enters perception state 6, personnel operation state 3 and algorithm state 3, the laser radar rotates toward the position of the target ship based on the K-wave radar information, and the target escort position is calculated based on the K-wave radar information;

[0076] Specifically, the lidar module rotates toward the target ship's position based on the K-wave radar data and places the target ship in the center of the screen; the perception algorithm predicts the target ship's position through AIS and matches it with the K-wave radar information to obtain the target ship information; the perception algorithm calculates the distance through the longitude and latitude of the target ship and tugboat, and calculates the escort position through the K-wave radar information; the perception algorithm sends information to the user interface for chart display; the user interface displays the tugboat and ships around the target ship on the chart, and highlights the target ship; the input data to the regulation and control system is the escort position calculated based on the K-wave radar information.

[0077] S29: If the perception system receives a signal to enter a calm state, and the distance between the tugboat and the target ship is greater than 100 meters, the perception system enters perception state 7, personnel operation state 4 and algorithm state 4, controls the visual camera and laser radar to rotate toward the position of the target ship, and obtains the target escort position based on the fusion calculation of the visual camera and the laser radar;

[0078] Specifically, the lidar module receives the lidar rotation signal and point coordinates from the user interface to complete the lock and tracking of the target ship; the perception algorithm obtains the relative coordinate information of the target ship identified by the lidar through UDP communication, and calculates the escort position based on it; the perception algorithm sends information to the user interface for nautical chart display; the user interface displays the video screen, and the lidar module control function is turned on; the input data of the regulation and control system is the escort position calculated based on the lidar information.

[0079] S210: If the perception system receives a signal to enter a calm state, and the distance between the tugboat and the target ship is less than 100 meters, the perception system enters perception state 8, personnel operation state 4 and algorithm state 4, controls the visual camera and laser radar to rotate toward the position of the target ship, and obtains the target escort position based on the fusion calculation of the visual camera and the laser radar;

[0080] Specifically, the laser radar module receives the laser radar rotation signal and point selection coordinates from the user interface to complete the locking and tracking of the fairlead hole; the perception algorithm obtains the relative coordinate information of the fairlead hole identified by the laser radar through UDP communication, and calculates the escort position based on it; the perception algorithm sends information to the user interface for nautical chart display; the user interface displays the video screen, and the laser radar module control function is turned on; the input data of the regulation and control system is the escort position calculated based on the laser radar information.

[0081] S211: If the target ship is lost in AIS, K-wave radar and visual camera at the same time, the perception system enters perception state 9 and algorithm state 1, and the perception system issues a warning.

[0082] Specifically, the LiDAR module has no input; the perception system alarms; the user interface displays a warning to remind the user that the target is lost; and the regulation and control system has no input.

[0083] Further, the process of converting the position information into relative coordinate information and outputting it smoothly is as follows: Figure 3 As shown, the specific steps are as follows:

[0084] Step 1: The tugboat coordinate system is as follows Figure 4 As shown, a left-hand coordinate system is established with the center point of the tugboat as the origin, the bow direction as the X-axis, and the vertical direction as the Z-axis;

[0085] Step 2: When the distance between the tugboat and the target ship is far (usually greater than 0.8 nautical miles), the perception system obtains the latitude and longitude of the tugboat through GPS, obtains the latitude and longitude of the target ship through AIS, and converts the latitude and longitude information into relative coordinate information in the tugboat coordinate system.

[0086] Step 3: When the target ship enters the detection range of the K-wave radar (usually 0.8 to 0.1 nautical miles), the perception system obtains the coordinate information of the target ship in the polar coordinate system through the fusion of AIS and K-wave radar, and converts the coordinates into the tugboat coordinate system.

[0087] Step 4: When the target ship enters the detection range of the lidar (usually within 0.1 nautical miles), the perception system obtains the relative coordinate information of the target ship in the tugboat coordinate system through the fusion of the visual camera and the lidar.

[0088] Step 5: Input the relative coordinate information into the Kalman filter, and obtain a smooth and accurate coordinate information output after prediction, updating and iteration.

[0089] The calculation method of the transverse distance and longitudinal distance between the tugboat and the target ship is as follows:

[0090]

[0091] Among them, H is the horizontal distance, V is the longitudinal distance, D is the straight-line distance between the tugboat and the target ship, γ is the relative azimuth between the tugboat and the target ship, w is the width of the tugboat, and l is the length of the tugboat.

[0092] When the distance between the tugboat and the target ship is greater than 0.8 nautical miles, D and γ are calculated using the longitude and latitude. At this time:

[0093]

[0094] γ=tan -1 [(λ o -λ t )÷(φ o -φ t )]-θ

[0095] Where R is the radius of the earth, λ o is the tugboat longitude, λ t is the target ship accuracy, φ o is the tugboat latitude, φ t is the latitude of the target ship, and θ is the heading of the tugboat.

[0096] When the distance between the tugboat and the target ship is 0.8-0.1 nautical miles, D and γ are obtained through K-wave radar. At this time:

[0097] D=distance

[0098] γ=bearing

[0099] Wherein, distance is the relative distance in K-wave radar data, and bearing is the relative azimuth in K-wave radar data.

[0100] When the distance between the tugboat and the target ship is less than 0.1 nautical miles, H and V can be directly obtained through the laser radar. At this time:

[0101] H=y

[0102] V = x

[0103] Wherein, y is the value of the Y axis of the target ship in the tugboat coordinate system, and x is the value of the X axis of the target ship in the tugboat coordinate system.

[0104] refer to Figure 5, a tugboat autonomous escort perception system based on multiple sensors, used to implement the tugboat autonomous escort perception method based on multiple sensors, including:

[0105] Equipment layer, communication layer, hub layer and service layer;

[0106] The central layer transmits information with the device layer and service layer through the communication layer.

[0107] Furthermore, the equipment layer includes: visual camera, lidar, K-wave radar, GPS, fiber optic compass, depth sounder, speedometer, wind speed and direction meter, main steam turbine and propeller shaft system.

[0108] Specifically, refer to Figure 6 ,The process of collecting and saving sensor data at the device layer includes the following steps:

[0109] Step 1: Use various sensor hardware to collect the required data;

[0110] Step 2: Filter, calibrate, and encode the collected sensor data, and timestamp the data to ensure data quality and accuracy;

[0111] Step 3: Transmit the processed data to the server via the ship's dedicated network;

[0112] Step 4: The perception system retrieves the required sensor data from the server through the MQTT protocol;

[0113] Step 5: The perception system connects to the MySQL database and saves the data in the database.

[0114] Furthermore, the communication layer includes an MQTT communication network, a TCP communication network, and a UDP communication network.

[0115] Specifically, the sensor data of the device layer is quickly transmitted to the central layer through the efficient and reliable MQTT communication network to ensure the timeliness and integrity of the data. At the central layer, each module quickly exchanges information through a lightweight and efficient UDP communication network, which greatly improves the real-time perception capability of the system and enables the system to quickly respond to and process various dynamically changing information. At the same time, a reliable TCP communication network is used between the central layer and the service layer for information exchange, ensuring the stability and security of data transmission, thereby ensuring the reliable operation and efficient work of the entire system in a complex environment. This multi-level communication network structure not only effectively optimizes the performance and response speed of the system, but also improves the overall reliability and stability of the system, providing strong technical support for real-time monitoring, data analysis and decision support.

[0116] Furthermore, the central layer includes: a sensor resource allocation module, a close-range target recognition module and a target tracking module.

[0117] Specifically, the sensor resource allocation module realizes the intelligent allocation of various sensor resources according to the current operating status of the ship and the real-time changes of the surrounding environment. This intelligent allocation not only optimizes the efficiency of resource utilization, but also can make timely adjustments to cope with emergencies, ensuring the stability and reliability of the system in a dynamic environment; the target recognition module based on machine learning combines the fusion technology of images and point clouds, bringing the system the ability of accurate recognition and precise ranging. Through the comprehensive analysis of the data collected by different sensors by the deep learning model, the system can quickly and accurately identify the target object, while achieving accurate ranging at close range. This integrated method greatly improves the accuracy of target recognition and provides reliable data support for navigation and safety decision-making; the target tracking module uses advanced tracking algorithms and sensor data fusion technology, so that the system can continuously track the position, speed and heading information of the target ship to ensure a stable and smooth tracking process. This smooth tracking not only helps to achieve safe interaction between ships, but also optimizes the navigation path planning of ships and improves overall transportation efficiency and safety.

[0118] Furthermore, the service layer includes: a user interface, a horizontal and vertical adjustment knob, and a start-stop device.

[0119] Specifically, the user interface integrates the display information of multiple sensors, which can intuitively show the captain the key data and video images required for the current escort process. Through the combination of data level and video level, the captain can fully understand the real-time status of the ship's surrounding environment, including the position, posture and surrounding waters of the target ship, which helps to make timely decisions and adjustments. The transverse and longitudinal distance adjustment knob is set on the operating table, which provides the captain with a convenient operation method, and can quickly adjust the transverse and longitudinal distance between the tugboat and the target ship according to the specific requirements of the escort. This intuitive and real-time adjustment capability enables the captain to flexibly respond to the escort needs in different situations and ensure the safe distance and optimal operating efficiency between the tugboat and the target ship. The start-stop device controls the start and emergency stop of the escort process, which is an important guarantee for the safety and controllability of the system. The captain can control the start of the escort process through simple operations, and can quickly implement emergency stops in case of emergency to ensure the safety of the ship and personnel. The design of this start-stop device effectively simplifies the complex operation process, improves the reliability of operation and the user's operating experience.

[0120] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0121] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0122] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A tugboat autonomous escort perception method based on multiple sensors, characterized in that: include: S1: Build a perception system. The state of the perception system includes: personnel operation state, perception state and algorithm state. Initialize the state of the perception system. S2: The state of the perception system is set according to the state information of the target ship to obtain the perception system after setting; the target escort position is obtained by calculating through each sensor in the perception system after setting.

2. The tugboat autonomous escort perception method based on multiple sensors according to claim 1 is characterized by: Personnel operation status includes: Personnel operation status 1: The static information of the target ship and the preset escort position are not input; Personnel operation status 2: Only the static information of the target ship is input; Personnel operation status 3: The static information of the target ship and the preset escort position are input; Personnel operation status 4: After the radar rotates and the target is selected, the target ship is locked and tracked.

3. The tugboat autonomous escort perception method based on multiple sensors according to claim 1 is characterized by: Perception states include: Perception state 1: initialization state; perception state 2: the AIS information of the target ship is not recognized; perception state 3: the AIS information of the target ship is recognized, but the K-wave radar does not recognize the target ship or the fusion is not completed; perception state 4: the target ship is within 0.5 nautical miles, but the target ship is not recognized; perception state 5: the AIS information of the target ship is recognized, and the K-wave radar recognizes the target ship; perception state 6: the AIS information of the target ship is not received within the preset time, but the K-wave radar recognizes the target ship; perception state 7: the signal to enter the calming state is received, but at this time the distance between the tugboat and the target ship is greater than 100 meters; perception state 8: the signal to enter the calming state is received, and at this time the distance between the tugboat and the target ship is less than 100 meters; perception state 9: the target ship is lost in AIS, K-wave radar and visual camera at the same time.

4. The tugboat autonomous escort perception method based on multiple sensors according to claim 1 is characterized by: Algorithm states include: Algorithm state 1: initialization state; Algorithm state 2: obtain the target escort position based on AIS information calculation; Algorithm state 3: obtain the target escort position based on K-wave radar information calculation; Algorithm state 4: obtain the target escort position based on the fusion calculation of visual camera and lidar.

5. The tugboat autonomous escort perception method based on multiple sensors according to claim 1 is characterized in that: Step S2 is specifically as follows: S21: Acquire the state information of the target ship input into the perception system, including the static information of the target ship and the preset escort position; S22: If only the static information of the target ship is input, and the perception system does not recognize the AIS information of the target ship, the perception system enters perception state 2, personnel operation state 2 and algorithm state 2, the laser radar performs a reset operation, updates the static information of the target ship, and sets the preset escort position to the default value; S23: If only the static information of the target ship is input, and the perception system recognizes the AIS information of the target ship, the perception system enters perception state 3, personnel operation state 2 and algorithm state 2, the preset escort position is set to the default value, the laser radar rotates toward the position of the target ship based on the AIS information, and the target escort position is calculated based on the AIS information; S24: If the static information of the target ship and the preset escort position are input, and the perception system does not recognize the AIS information of the target ship, the perception system enters perception state 2, personnel operation state 3 and algorithm state 2, the laser radar performs a reset operation, and updates the static information of the target ship; S25: If the static information of the target ship and the preset escort position are input, and the perception system recognizes the AIS information of the target ship, the perception system enters perception state 3, personnel operation state 3 and algorithm state 2, and the laser radar rotates toward the position of the target ship based on the AIS information, and the target escort position is calculated based on the AIS information; S26: If the target ship is within 0.5 nautical miles from the tugboat, but the K-wave radar does not identify the target ship, the perception system enters perception state 4, personnel operation state 3 and algorithm state 2, and the laser radar rotates toward the position of the target ship based on the AIS information, and calculates the target escort position based on the AIS information; S27: If the perception system recognizes the AIS information of the target ship and the K-wave radar recognizes the target ship, the perception system enters perception state 5, personnel operation state 3 and algorithm state 3, the laser radar rotates toward the position of the target ship based on the K-wave radar information, and the target escort position is calculated based on the K-wave radar information; S28: If the perception system does not receive the AIS information of the target ship within the preset time, but the K-wave radar recognizes the target ship, the perception system enters perception state 6, personnel operation state 3 and algorithm state 3, the laser radar rotates toward the position of the target ship based on the K-wave radar information, and the target escort position is calculated based on the K-wave radar information; S29: If the perception system receives a signal to enter a calm state, and the distance between the tugboat and the target ship is greater than 100 meters, the perception system enters perception state 7, personnel operation state 4 and algorithm state 4, controls the visual camera and laser radar to rotate toward the position of the target ship, and obtains the target escort position based on the fusion calculation of the visual camera and the laser radar; S210: If the perception system receives a signal to enter a calm state, and the distance between the tugboat and the target ship is less than 100 meters, the perception system enters perception state 8, personnel operation state 4 and algorithm state 4, controls the visual camera and laser radar to rotate toward the position of the target ship, and obtains the target escort position based on the fusion calculation of the visual camera and the laser radar; S211: If the target ship is lost in AIS, K-wave radar and visual camera at the same time, the perception system enters perception state 9 and algorithm state 1, and the perception system issues a warning.

6. A tugboat autonomous escort perception system based on multiple sensors, used to implement the tugboat autonomous escort perception method based on multiple sensors as described in any one of claims 1 to 5, characterized in that: include: Equipment layer, communication layer, hub layer and service layer; The central layer transmits information with the device layer and service layer through the communication layer.

7. The tugboat autonomous escort perception system based on multiple sensors according to claim 6 is characterized by: The equipment layer includes: visual camera, lidar, K-wave radar, GPS, fiber optic compass, depth sounder, speedometer, wind speed and direction meter, main steam turbine and propeller shaft system.

8. The tugboat autonomous escort perception system based on multiple sensors according to claim 6 is characterized by: The communication layer includes MQTT communication network, TCP communication network and UDP communication network.

9. The tugboat autonomous escort perception system based on multiple sensors according to claim 6 is characterized by: The central layer includes: sensor resource allocation module, close-range target recognition module and target tracking module.

10. The tugboat autonomous escort perception system based on multiple sensors according to claim 6 is characterized by: The service layer includes: user interface, horizontal and vertical adjustment knobs and start and stop devices.

Images