Autonomous navigation control method and system for unmanned ship

Through the combination of the autonomous navigation control system of unmanned ships, using technologies such as edge computers, visual perception and lidar, unmanned ships are able to avoid obstacles in complex sea conditions, and solve the collision accident of traditional unmanned ships in complex environments. Problems, improving navigation safety and adaptability.

CN118672276BActive Publication Date: 2024-11-19JIANGSU HANTONG WING HEAVY IND CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411169992.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-19
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Traditional unmanned ships are difficult to deal with emergencies under complex sea conditions and dynamic environments, prone to collision accidents, and it is difficult for existing technology to achieve effective autonomous obstacle avoidance.

Method used

The autonomous navigation control system of the unmanned ship is adopted, combined with edge computers, visual perception systems, three-dimensional lidar and environmental perception systems, and the unmanned ship's autonomous obstacle avoidance ability is achieved through image processing, lidar collision avoidance programs and potential field methods strategies.

Benefits of technology

It improves the autonomous perception, decision-making and control capabilities of unmanned ships in complex sea conditions, enhances navigation safety and adaptability, and reduces collision risks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118672276B_ABST
    Figure CN118672276B_ABST
Patent Text Reader

Abstract

The present invention provides an unmanned ship autonomous navigation control method and system, which relate to the field of navigation control technology, including an unmanned ship hull and an unmanned ship autonomous navigation control system arranged on the unmanned ship hull, the unmanned ship autonomous navigation control system including an unmanned ship control system, an unmanned ship environmental perception system, a communication system, a shore-based control system, an unmanned ship autonomous obstacle avoidance system, a GPS positioning module, an IMU attitude module and a visual perception system; the unmanned ship control system includes an edge computer for processing sensor data, issuing instructions and processing data of the visual perception system, and a control board composed of an ARM chip for executing unmanned ship control instructions; the IMU attitude module is used to accurately determine the movement of the unmanned ship, the current collector is used to collect the current and voltage of the unmanned ship when it is working, convert it into the real-time power of the unmanned ship and combine it with the battery information to obtain the remaining endurance time of the unmanned ship, and the voltage step-down module is used to convert the battery voltage to power the control board and the sensor.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to navigation control technology, and in particular to an autonomous navigation control method and system for an unmanned ship. Background Art

[0002] In recent years, with the rapid development of artificial intelligence, computer vision, control technology and other fields, unmanned driving technology has received extensive attention and application. In the marine field, unmanned ships, as a new type of intelligent navigation equipment, have the advantages of high operating efficiency, low personnel safety risk and low operating cost, and have shown broad application prospects in many aspects such as marine monitoring, marine survey, maritime search and rescue, and military reconnaissance.

[0003] Traditional unmanned ship navigation usually adopts a waypoint-based navigation method, that is, by pre-setting a sequence of waypoints, and relying on GPS positioning and inertial navigation systems to guide the unmanned ship to sail along the waypoints in sequence. Although this navigation method can realize the basic autonomous navigation function of unmanned ships, in complex sea conditions and dynamic environments, unmanned ships are difficult to respond to emergencies and are prone to collision accidents. Summary of the invention

[0004] The embodiments of the present invention provide an unmanned ship autonomous navigation control method and system, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention,

[0006] Provide unmanned ship autonomous navigation control system, including:

[0007] An unmanned ship hull and an unmanned ship autonomous navigation control system arranged on the unmanned ship hull, wherein the unmanned ship autonomous navigation control system includes an unmanned ship control system, an unmanned ship environment perception system, a communication system, a shore-based control system, an unmanned ship autonomous obstacle avoidance system, a GPS positioning module, an IMU attitude module and a visual perception system;

[0008] The unmanned ship control system includes an edge computer, a control board, a current collector and a voltage step-down module. The edge computer is used to process sensor data, issue instructions and process data of the visual perception system. The control board is composed of an ARM chip and is used to execute unmanned ship control instructions and realize the lateral and longitudinal movement of the unmanned ship by controlling the rotation of the motor; the IMU attitude module is used to provide the navigation azimuth, pitch attitude and multi-axis acceleration data of the unmanned ship; the current collector is used to collect the current and voltage of the unmanned ship when it is working, convert it into the real-time power of the unmanned ship and combine it with the battery information to obtain the remaining endurance time of the unmanned ship, and the voltage step-down module is used to convert the battery voltage to power the control board and the sensor;

[0009] The unmanned boat environment perception system includes a flow velocity sensor, a water depth sensor, a wind speed and direction sensor, an air pressure sensor and a light sensor, which are respectively used to collect water flow velocity information, depth information, wind speed and direction information, atmospheric pressure information and light intensity information of the unmanned boat navigation waters, and send the collected information to the edge computer;

[0010] The visual perception system includes a high-definition camera and a binocular camera. The high-definition camera is used to observe the surface conditions of the unmanned ship navigation area, monitor the navigation of the unmanned ship, and send the video information to the shore-based control system through the communication system to realize shore-based monitoring, and at the same time send the video information to the edge computer. The binocular camera is used to obtain the image depth information of the water surface environment of the unmanned ship navigation area and send it to the edge computer; the edge computer integrates and processes the collected data and transmits it to the shore-based control system through the communication system;

[0011] The edge computer is provided with a ship-side target recognition program, which performs image denoising, histogram equalization enhancement, SURF feature extraction, and obstacle recognition on the image information collected by the high-definition camera and the binocular camera, and then determines whether there are obstacles that hinder the navigation of the unmanned ship on the water surface of the current navigation area by analyzing the results of image processing. When there are obstacles, the corresponding obstacle avoidance instructions are sent to the control board, and the control board executes the visual perception obstacle avoidance program to control the unmanned ship to perform obstacle avoidance behavior;

[0012] The autonomous obstacle avoidance system of the unmanned boat detects obstacles ahead through a three-dimensional laser radar installed on the hull of the unmanned boat, and sends obstacle information to the edge computer. The edge computer is provided with a laser radar collision avoidance program, which confirms obstacle information through the laser radar collision avoidance program and sends instructions to the control board. The control board executes the laser radar collision avoidance program to control the unmanned boat to perform obstacle avoidance behavior;

[0013] The shore-based control system includes unmanned ship control and display software, radar rendering display software, a motion controller and a computer. The unmanned ship control and display software is used to control the navigation movement of the unmanned ship and display the working condition information of the unmanned ship. The radar rendering display software is used to display the image scanned by the laser radar. The motion controller is used to control the movement of the unmanned ship.

[0014] In an optional embodiment,

[0015] The edge computer is also used to:

[0016] For each pixel point of the received image, a Hessian matrix consisting of the second-order partial derivatives of the multivariate function f(x, y) is constructed, where f(x, y) represents the pixel value at the coordinate p(x, y). Whether the pixel point is a key point is determined by finding the local extreme value of the discriminant of the Hessian matrix;

[0017] In order to make the key points scale invariant, the received image is filtered by a second-order standard Gaussian function before constructing the Hessian matrix, and the pixel value after filtering is calculated by the second-order standard Gaussian function and the original pixel value;

[0018] For the filtered image, the second-order partial derivative with respect to the coordinates x and y is calculated at the pixel point p to obtain the Hessian matrix at different scales.

[0019] In an optional embodiment,

[0020] The Hessian matrix is ​​shown in the following formula:

[0021] ;

[0022] The Hessian matrix discriminant is shown in the following formula:

[0023] ;

[0024] The Hessian matrix at different scales is shown in the following formula:

[0025] ;

[0026] in, H(f(x,y)) represents the Hessian matrix, f represents a multivariate function, (x, y) Represents the spatial coordinates of the pixel point, L xx (p, σ) Representing images L At the point p and the lower edge of scale σ x The second-order partial derivative in the direction, L xy (p, σ) Representing images L At the point p and mixed second-order partial derivatives under scale σ, L yy (p, σ) Representing images L At the point p and the lower edge of scale σ y The second partial derivative in the direction.

[0027] In an optional embodiment,

[0028] Also includes:

[0029] The three-dimensional laser radar uses the VFH vector field histogram algorithm to extract point cloud features, and calculates the feature components of the viewpoint by counting the histogram of the angle between the viewpoint direction and each normal line, including:

[0030] Use the VFH method for feature extraction, use the entire point cloud object for calculation and estimation, use the point pair between the center point of the object and all other points on the surface of the object as the calculation unit when calculating the feature description, and add additional statistical information between the viewpoint direction and the estimated normal of each point;

[0031] The viewpoint direction is incorporated into the relative normal angle calculation, and the characteristic component of the viewpoint is calculated by statistically analyzing the histogram of the angle between the viewpoint direction and each normal.

[0032] In an optional embodiment,

[0033] The control panel executes the visual perception obstacle avoidance program to control the unmanned boat to perform obstacle avoidance behaviors including:

[0034] The position coordinates of the obstacle are converted into a polar coordinate system centered on the unmanned ship to obtain the azimuth and distance of the obstacle relative to the unmanned ship;

[0035] Combined with the current speed and heading of the unmanned ship, an obstacle avoidance strategy based on the potential field method is adopted to calculate the heading change required for obstacle avoidance. The obstacle corresponds to the repulsive potential field, and the target point corresponds to the gravitational potential field. Under the combined force of the repulsive potential field and the gravitational potential field, the unmanned ship adjusts its heading, stays away from the obstacle, and sails towards the target point.

[0036] The speed change required for obstacle avoidance is calculated based on the concept of collision avoidance safety distance. When the obstacle distance is less than the safety distance, the unmanned boat slows down until it stops. When the obstacle distance is greater than the safety distance, the unmanned boat maintains the current speed or accelerates.

[0037] The control board adjusts the rudder angle and engine throttle of the unmanned boat according to the calculated changes in heading and speed, and controls the unmanned boat to perform obstacle avoidance behavior.

[0038] In an optional embodiment,

[0039] The system further comprises:

[0040] Communication systems, including high-power radios, high-gain antennas, cloud servers, and IoT modules;

[0041] The edge computer is connected to a high-power radio station via a LAN communication interface, the high-power radio station is connected to a high-gain antenna, and the high-power radio stations are connected via microwave wireless communication, so that the data of the edge computer can be transmitted to the shore-based control system via the high-power radio station, thereby realizing long-distance transmission of ship-side data. The cloud server is used to transmit data uploaded by the control panel through the Internet of Things module connected to the 5G network, thereby realizing the cross-domain data transmission function between the control panel and the shore-based control system.

[0042] In the second aspect of the embodiments of the present application,

[0043] Provides an autonomous navigation control method for an unmanned ship, including:

[0044] Start the unmanned boat, check whether the hull and its onboard equipment are normal, turn on the power of the unmanned boat, and check whether the equipment is working normally;

[0045] The edge computer is connected to the high-power radio through the LAN communication interface, and the control board is connected to the 5G network through the IoT module to establish a TCP connection with the cloud server, and the communication system works;

[0046] The navigation water environment information collected by the unmanned ship's environmental perception system is sent to the shore-based control system through the communication system;

[0047] The speed and heading information and attitude information of the unmanned ship collected by the unmanned ship GPS positioning module and the IMU attitude module, as well as the image information acquired by the laser radar and visual perception system are sent to the unmanned ship control system; the edge computer in the unmanned ship control system integrates the received data and sends it to the shore-based control system through the communication system after processing;

[0048] After receiving the data transmitted by the unmanned ship operating system, the shore-based control system selects the automatic navigation mode in the unmanned ship control and display software, and determines the starting and ending positions of the autonomous navigation for the unmanned ship, which are then sent to the unmanned ship control system by the communication system;

[0049] The unmanned ship control system controls the unmanned ship to navigate automatically according to the starting and ending positions determined by the shore-based control system, and the edge computer controls the unmanned ship in real time;

[0050] During navigation, after the unmanned boat obstacle avoidance system detects an obstacle, it executes the corresponding obstacle avoidance program to avoid the obstacle and resume the navigation route.

[0051] According to a third aspect of the embodiments of the present invention,

[0052] An electronic device is provided, comprising:

[0053] processor;

[0054] a memory for storing processor-executable instructions;

[0055] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0056] According to a fourth aspect of the embodiments of the present invention,

[0057] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0058] This application uses edge computers to process sensor data, issue commands, and process visual perception system data. The control board executes unmanned ship control commands, the IMU attitude module provides navigation attitude data, and the current collector and voltage step-down module implement unmanned ship power management, ensuring data processing, control execution, and energy supply during the autonomous navigation of the unmanned ship, and realizing autonomous motion control of the unmanned ship;

[0059] Various sensors in the environmental perception system comprehensively collect environmental information such as flow velocity, water depth, wind speed and direction, air pressure and light in the waters where the unmanned ship is sailing, providing rich perception data for the unmanned ship's autonomous decision-making, and improving the unmanned ship's perception ability and adaptability to the navigation environment;

[0060] The visual perception system uses high-definition cameras and binocular cameras to obtain video images and depth of field information of the navigation waters, and performs image processing and obstacle recognition through the ship-side target recognition program in the edge computer, thereby achieving effective detection of surface obstacles. The visual perception obstacle avoidance program is executed through the control panel, giving the unmanned ship visual perception and obstacle avoidance capabilities, thereby improving navigation safety.

[0061] The unmanned boat's autonomous obstacle avoidance system uses a three-dimensional lidar to detect obstacles ahead, confirms obstacle information through the lidar collision avoidance program in the edge computer, and controls the unmanned boat to avoid obstacles, providing another reliable obstacle avoidance method for the unmanned boat and further enhancing the unmanned boat's obstacle avoidance capability.

[0062] The environmental perception data and visual perception data are integrated and processed by the edge computer and transmitted to the shore-based control system, realizing real-time information exchange between the unmanned ship and the shore-based control center, making it convenient for shore-based personnel to timely grasp the navigation status of the unmanned ship and the surrounding environment information and realize shore-based monitoring.

[0063] By comprehensively utilizing control systems, environmental perception systems, visual perception systems and autonomous obstacle avoidance systems, a complete unmanned ship autonomous navigation system has been built, which enables autonomous perception, autonomous decision-making and autonomous control of unmanned ships in complex sea conditions, improves the flexibility, adaptability and safety of unmanned ships in performing tasks, and promotes the application of unmanned ship technology in the marine field. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a schematic diagram of the structure of an autonomous navigation control system for an unmanned ship according to an embodiment of the present invention;

[0065] Figure 2 The figure is a flow chart of an autonomous navigation control method for an unmanned ship according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0068] Figure 1 FIG. 1 is a schematic diagram of the structure of an autonomous navigation control system for an unmanned ship according to an embodiment of the present invention. Figure 1 As shown, the system comprises:

[0069] An unmanned ship hull and an unmanned ship autonomous navigation control system arranged on the unmanned ship hull, wherein the unmanned ship autonomous navigation control system includes an unmanned ship control system, an unmanned ship environment perception system, a communication system, a shore-based control system, an unmanned ship autonomous obstacle avoidance system, a GPS positioning module, an IMU attitude module and a visual perception system;

[0070] The unmanned ship control system includes an edge computer, a control board, a current collector and a voltage step-down module. The edge computer is used to process sensor data, issue instructions and process data of the visual perception system. The control board is composed of an ARM chip and is used to execute unmanned ship control instructions and realize the lateral and longitudinal movement of the unmanned ship by controlling the rotation of the motor. The IMU attitude module is used to provide the navigation direction, pitch attitude and multi-axis acceleration data of the unmanned ship for precise unmanned ship movement. The current collector is used to collect the current and voltage of the unmanned ship when it is working, convert it into the real-time power of the unmanned ship and combine it with the battery information to obtain the remaining endurance time of the unmanned ship. The voltage step-down module is used to convert the battery voltage to power the control board and the sensor;

[0071] The unmanned boat environment perception system includes a flow velocity sensor, a water depth sensor, a wind speed and direction sensor, an air pressure sensor and a light sensor, which are respectively used to collect water flow velocity information, depth information, wind speed and direction information, atmospheric pressure information and light intensity information of the unmanned boat navigation waters, and send the collected information to the edge computer;

[0072] The visual perception system includes a high-definition camera and a binocular camera. The high-definition camera is used to observe the surface conditions of the unmanned ship navigation area, monitor the navigation of the unmanned ship, and send the video information to the shore-based control system through the communication system to realize shore-based monitoring, and at the same time send the video information to the edge computer. The binocular camera is used to obtain the image depth information of the water surface environment of the unmanned ship navigation area and send it to the edge computer; the edge computer integrates and processes the collected data and transmits it to the shore-based control system through the communication system;

[0073] The edge computer is provided with a ship-side target recognition program, which performs image denoising, histogram equalization enhancement, SURF feature extraction, and obstacle recognition on the image information collected by the high-definition camera and the binocular camera, and then determines whether there are obstacles that hinder the navigation of the unmanned ship on the water surface of the current navigation area by analyzing the results of image processing. When there are obstacles, the corresponding obstacle avoidance instructions are sent to the control board, and the control board executes the visual perception obstacle avoidance program to control the unmanned ship to perform obstacle avoidance behavior;

[0074] The autonomous obstacle avoidance system of the unmanned boat detects obstacles ahead through a three-dimensional laser radar installed on the hull of the unmanned boat, and sends obstacle information to the edge computer. The edge computer is provided with a laser radar collision avoidance program, which confirms obstacle information through the laser radar collision avoidance program and sends instructions to the control board. The control board executes the laser radar collision avoidance program to control the unmanned boat to perform obstacle avoidance behavior. The three-dimensional laser radar uses the VFH method to extract point cloud features, and calculates the feature components of the viewpoint by statistically calculating the histogram of the angle between the viewpoint direction and each normal line.

[0075] The shore-based control system includes unmanned ship control and display software, radar rendering display software, a motion controller and a computer. The unmanned ship control and display software is used to control the navigation movement of the unmanned ship and display the working condition information of the unmanned ship. The radar rendering display software is used to display the image scanned by the laser radar. The motion controller is used to control the movement of the unmanned ship.

[0076] In an optional embodiment,

[0077] The edge computer is also used to:

[0078] For each pixel point of the received image, a Hessian matrix consisting of the second-order partial derivatives of the multivariate function f(x, y) is constructed, where f(x, y) represents the pixel value at the coordinate p(x, y), and whether the pixel point is a key point is determined by finding the local extreme value of the discriminant of the Hessian matrix;

[0079] In order to make the key points scale invariant, the received image is filtered by a second-order standard Gaussian function before constructing the Hessian matrix, and the pixel value after filtering is calculated by the second-order standard Gaussian function and the original pixel value;

[0080] For the filtered image, the second-order partial derivative with respect to the coordinates x and y is calculated at the pixel point p to obtain the Hessian matrix at different scales.

[0081] In an optional embodiment,

[0082] The Hessian matrix is ​​shown in the following formula:

[0083] ;

[0084] The Hessian matrix discriminant is shown in the following formula:

[0085] ;

[0086] The Hessian matrix at different scales is shown in the following formula:

[0087] ;

[0088] in, H(f(x,y)) represents the Hessian matrix, f represents a multivariate function, (x, y) Represents the spatial coordinates of the pixel point, L xx (p, σ) Representing images L At the point p and the lower edge of scale σ x The second-order partial derivative in the direction, L xy (p, σ) Representing images L At the point p and mixed second-order partial derivatives under scale σ, L yy (p, σ) Representing images L At the point p and the lower edge of scale σ y The second partial derivative in the direction.

[0089] In an optional embodiment,

[0090] The three-dimensional laser radar uses the VFH method to extract point cloud features, and calculates the feature components of the viewpoint by statistically analyzing the histogram of the angle between the viewpoint direction and each normal line, including:

[0091] Use the VFH method for feature extraction, use the entire point cloud object for calculation and estimation, use the point pair between the center point of the object and all other points on the surface of the object as the calculation unit when calculating the feature description, and add additional statistical information between the viewpoint direction and the estimated normal of each point;

[0092] The viewpoint direction is incorporated into the relative normal angle calculation, and the characteristic component of the viewpoint is calculated by statistically analyzing the histogram of the angle between the viewpoint direction and each normal.

[0093] In an optional embodiment,

[0094] The control panel executes the visual perception obstacle avoidance program to control the unmanned boat to perform obstacle avoidance behaviors including:

[0095] The position coordinates of the obstacle are converted into a polar coordinate system centered on the unmanned ship to obtain the azimuth and distance of the obstacle relative to the unmanned ship;

[0096] Combined with the current speed and heading of the unmanned ship, an obstacle avoidance strategy based on the potential field method is adopted to calculate the heading change required for obstacle avoidance. The obstacle corresponds to the repulsive potential field, and the target point corresponds to the gravitational potential field. Under the combined force of the repulsive potential field and the gravitational potential field, the unmanned ship adjusts its heading, stays away from the obstacle, and sails towards the target point.

[0097] The speed change required for obstacle avoidance is calculated based on the concept of collision avoidance safety distance. When the obstacle distance is less than the safety distance, the unmanned boat slows down until it stops. When the obstacle distance is greater than the safety distance, the unmanned boat maintains the current speed or accelerates.

[0098] The control board adjusts the rudder angle and engine throttle of the unmanned boat according to the calculated changes in heading and speed, and controls the unmanned boat to perform obstacle avoidance behavior.

[0099] In an optional embodiment,

[0100] The system further comprises:

[0101] The communication system includes a high-power radio, a high-gain antenna, a cloud server, and an Internet of Things module. The edge computer is connected to the high-power radio through a LAN communication interface, the high-power radio is connected to the high-gain antenna, and the high-power radios are connected through microwave wireless communication, so that the data of the edge computer can be transmitted to the shore-based control system via the high-power radio, thereby realizing long-distance transmission of ship-side data. The cloud server is used to transmit the data uploaded by the control panel through the Internet of Things module connected to the 5G network, thereby realizing the cross-domain data transmission function between the control panel and the shore-based control system.

[0102] Figure 2 FIG. 1 is a flow chart of an autonomous navigation control method for an unmanned ship according to an embodiment of the present invention. Figure 2 As shown, the method includes:

[0103] Start the unmanned boat, check whether the hull and its onboard equipment are normal, turn on the power of the unmanned boat, and check whether the equipment is working normally;

[0104] The edge computer is connected to the high-power radio through the LAN communication interface, and the control board is connected to the 5G network through the IoT module to establish a TCP reliable connection with the cloud server, and the communication system works;

[0105] The navigation water environment information collected by the unmanned ship's environmental perception system is sent to the shore-based control system through the communication system;

[0106] The unmanned ship's speed and heading information, attitude information collected by the unmanned ship's GPS positioning module and IMU attitude module, as well as the image information acquired by the laser radar and visual perception system are sent to the unmanned ship control system; the edge computer in the unmanned ship control system integrates the received data and sends it to the shore-based control system through the communication system after processing;

[0107] After receiving the data transmitted by the unmanned ship operating system, the shore-based control system selects the automatic navigation mode in the unmanned ship control and display software, and determines the starting and ending positions of the autonomous navigation for the unmanned ship, which are then sent to the unmanned ship control system by the communication system;

[0108] The unmanned ship control system controls the unmanned ship to navigate automatically according to the starting and ending positions determined by the shore-based control system, and the edge computer controls the unmanned ship in real time;

[0109] During navigation, after the unmanned boat obstacle avoidance system detects an obstacle, it executes the corresponding obstacle avoidance program to avoid the obstacle and resume the navigation route.

[0110] Another embodiment of the present application is as follows:

[0111] Autonomous navigation of unmanned ships is a complex process involving the coordinated work and real-time control of multiple subsystems. An autonomous navigation mission refers to a navigation mission from a starting position to a target position planned by the shore-based control system for the unmanned ship according to mission requirements. The shore-based control system allows operators to select the starting position and target position of the unmanned ship on the electronic chart in a graphical manner through a human-computer interaction interface, and set other attribute parameters of the navigation mission, such as speed, heading, etc.

[0112] The shore-based control system uses unmanned ship control and display software based on B / S architecture. The front-end page is developed using HTML5 and JavaScript, and the back-end service is built using the Flask framework. Operators access the unmanned ship control and display software through a browser, which can realize interactive operations such as zooming and panning of the nautical chart, and mark the starting position and target position on the nautical chart with the mouse.

[0113] After receiving the location information sent by the front-end page, the back-end service first determines whether the location is legal, such as whether it exceeds the boundary of the navigation area, whether it is in a prohibited navigation area, etc. If the location is illegal, an error prompt is returned. If the location is legal, it is converted into longitude and latitude coordinates in the WGS-84 coordinate system, and combined with other navigation parameters to generate a standardized autonomous navigation mission description file. The file uses JSON format and includes parameters such as the latitude and longitude of the starting position, the latitude and longitude of the target position, speed, and heading.

[0114] The generated autonomous navigation mission file is sent to the edge computer in the unmanned ship control system through the communication module of the shore-based control system using a high-power radio. After receiving the mission file, the edge computer parses its content, extracts key parameters, and triggers the subsequent navigation path planning and autonomous navigation control process.

[0115] Navigation path planning is to plan a safe and efficient navigation path based on the starting position and target position in the autonomous navigation mission, combined with the nautical chart information and environmental information of the navigation area. The goal of path planning is to minimize the length of the navigation path or the navigation time while meeting the navigation safety constraints.

[0116] The edge computer in the unmanned ship control system is responsible for the navigation path planning task. The path planning algorithm uses an improved A* search algorithm, the basic idea of ​​which is to discretize the navigation area into a grid map, where each grid represents a navigable location. The A* algorithm starts from the starting position and ends at the target position, searching for a path with the minimum cost on the grid map.

[0117] The improved A* algorithm introduces a heading deviation penalty factor based on the original heuristic function, making the planned path as smooth as possible and reducing sharp turns. At the same time, the algorithm also considers the impact of environmental factors such as water flow, wind and waves on the navigation path, and dynamically adjusts the cost of each position in the grid map to make the planned path adapt to the current navigation environment.

[0118] The computational complexity of the A* algorithm is related to the size and resolution of the grid map. In order to achieve real-time path planning under limited computing resources, the edge computer adopts a multi-resolution path planning strategy. That is, firstly, a global path search is performed on a low-resolution grid map to obtain a rough navigation path; then, the path is optimized on a high-resolution local map to obtain a fine local path. Finally, the local path is spliced ​​with the global path to form the final navigation path.

[0119] The planned navigation path consists of a series of waypoints, each of which includes attributes such as latitude and longitude coordinates, heading and speed. The edge computer sends the planned waypoint sequence to the control board, which controls the heading and speed of the unmanned ship based on the waypoint attributes to achieve path tracking control.

[0120] During the autonomous navigation of unmanned ships, encountering unknown obstacles is a common emergency. In order to ensure navigation safety, unmanned ships must have the ability to autonomously avoid obstacles, that is, to detect obstacles in time and plan a navigation path to avoid obstacles.

[0121] The unmanned ship obstacle avoidance system is mainly composed of laser radar and visual perception system, in which the laser radar is responsible for the accurate detection of close-range obstacles, and the visual perception system is responsible for the identification and tracking of long-range obstacles.

[0122] LiDAR can obtain the distance and direction of obstacles by measuring the flight time of the laser beam, and then build a local point cloud map. The visual perception system uses a binocular camera to calculate the disparity of the images taken by the left and right cameras to obtain the distance information of obstacles, and combines with target detection algorithms such as YOLO to realize the recognition and classification of obstacles.

[0123] The obstacle information collected by the LiDAR and visual perception system is sent to the edge computer. The edge computer runs an autonomous obstacle avoidance algorithm to determine whether there is a collision risk based on the current navigation path and obstacle location. If there is a collision risk, the obstacle avoidance control process is triggered.

[0124] The autonomous obstacle avoidance algorithm uses an obstacle avoidance strategy based on speed obstacles. By analyzing the relative speed vector between the unmanned ship and the obstacle, it determines whether there is a collision risk and calculates a safe obstacle avoidance direction and speed. The obstacle avoidance direction and speed are converted into a series of obstacle avoidance waypoints, which are merged with the original waypoint sequence to generate a new obstacle avoidance path. The obstacle avoidance path is also executed by the control board to control the unmanned ship to avoid obstacles.

[0125] After avoiding obstacles, the edge computer will re-plan a path back to the original route and control the unmanned boat to gradually return to the original route and resume normal navigation. The entire autonomous obstacle avoidance control process is a closed-loop control system. Through the close cooperation of the three links of perception, decision-making, and execution, the unmanned boat can achieve autonomous obstacle avoidance in complex navigation environments.

[0126] Further,

[0127] In the process of autonomous obstacle avoidance of unmanned ships, the obstacle point cloud data obtained by laser radar scanning needs to be feature extracted and identified to facilitate subsequent obstacle avoidance decisions and path planning. The VFH algorithm is a global feature description algorithm that uses the entire point cloud object as a processing unit and generates a fixed-dimensional feature vector by calculating the angle statistical histogram between the viewpoint direction and the point cloud surface normal. Unlike traditional point features or local feature descriptors, the VFH algorithm takes into account the influence of viewpoint direction on feature description, so it has better perspective invariance and robustness.

[0128] Estimate the normal direction of each point. For each point, select a neighborhood around it and use the principal component analysis (PCA) method to estimate the normal direction of the neighborhood; calculate the angle between the viewpoint direction and the normal of each point. Calculate the statistical histogram of the angle. Divide the angle range from 0° to 180° into multiple intervals (bins) of equal width, and count the number of angles falling into each interval to obtain a multi-dimensional histogram feature vector. Normalize the feature vector to obtain the final VFH feature descriptor.

[0129] In practical applications, the computational efficiency and recognition accuracy of the VFH algorithm are affected by multiple factors, such as the selection of neighborhood size, the accuracy of normal estimation, the setting of histogram resolution, etc. In order to obtain better feature extraction performance, this application optimizes the VFH algorithm as follows:

[0130] (1) Adaptive neighborhood selection. The traditional VFH algorithm uses a fixed-size neighborhood to estimate the normal direction of a point. However, for point cloud regions with different densities and curvatures, a fixed neighborhood may lead to inaccurate normal estimation. Therefore, this application adopts an adaptive neighborhood strategy to dynamically adjust the neighborhood size of each point according to its density and curvature characteristics.

[0131] Weighted statistical histogram. In the original VFH algorithm, the contribution of each point to the histogram is the same, but in the actual point cloud, the importance of different points may be different. For example, points close to the viewpoint may contain more perspective-related information, while points far from the viewpoint contribute less to the feature description. In order to highlight the role of important points, this application introduces a weighting coefficient related to the distance from the point to the viewpoint when calculating the histogram. The weighting coefficient decays as the distance from the point to the viewpoint increases, making the contribution of close points to the histogram greater.

[0132] Further,

[0133] The obstacle position information output by the visual perception system is usually expressed in the camera coordinate system or the world coordinate system. In order to facilitate the calculation of the obstacle avoidance strategy, the obstacle position needs to be converted to a polar coordinate system centered on the unmanned ship. First, the position of the obstacle in the world coordinate system is converted to the unmanned ship coordinate system through the coordinate transformation matrix. Then, the azimuth and distance of the obstacle relative to the unmanned ship are calculated using the obstacle position in the unmanned ship coordinate system. The azimuth is calculated using the inverse tangent function, and the distance is calculated using the three-dimensional space distance formula.

[0134] The basic idea of ​​the potential field method is to construct an artificial potential field around the unmanned ship, so that the unmanned ship can move under the action of the potential field force. The potential field is usually composed of a gravitational potential field and a repulsive potential field. The gravitational potential field points to the target point, attracting the unmanned ship to move toward the target point; the repulsive potential field is centered on the obstacle, repelling the unmanned ship away from the obstacle.

[0135] The gravitational potential field function is defined as the quadratic form of the distance to the target point, and the gravitational potential field force is the negative gradient of the gravitational potential field function. The repulsive potential field function is defined as the square of the difference between the inverse of the obstacle distance and a constant. It works when the obstacle distance is less than the repulsive potential field action distance, otherwise no repulsion is generated. The repulsive potential field force is the negative gradient of the repulsive potential field function.

[0136] Under the combined effect of the gravitational potential field and the repulsive potential field, the combined force on the unmanned ship is the vector sum of the gravitational force and the repulsive force of all obstacles. According to the component of the combined force in the unmanned ship coordinate system, the inverse tangent function is used to calculate the change in the heading angle that the unmanned ship needs to adjust.

[0137] In addition to adjusting the heading angle, the unmanned ship also needs to adjust the speed according to the distance of the obstacle to ensure collision avoidance safety. The concept of collision avoidance safety distance is introduced. When the obstacle distance is less than the safety distance, the unmanned ship needs to slow down until it stops; when the obstacle distance is greater than the safety distance, the unmanned ship can maintain the current speed or accelerate.

[0138] The speed change of the unmanned boat is calculated based on the collision avoidance safety distance. When the obstacle distance is less than the safety distance, the speed change is proportional to the difference between the current speed and the stopping speed; when the obstacle distance is greater than the safety distance, the speed change is proportional to the difference between the expected speed and the current speed. The positive or negative speed change determines whether the unmanned boat is accelerating or decelerating.

[0139] According to the calculated heading angle change and speed change, the control board needs to adjust the rudder angle and engine throttle of the unmanned boat accordingly to perform obstacle avoidance. The rudder angle adjustment of the unmanned boat is proportional to the heading angle change, and the throttle adjustment is proportional to the speed change. The control gain of the rudder angle and throttle determines the adjustment amplitude and response speed.

[0140] Through the above steps, the unmanned ship can plan a smooth obstacle avoidance path in time when the visual perception system detects an obstacle, and safely bypass the obstacle by adjusting the heading and speed, and finally reach the target point. The obstacle avoidance strategy based on the potential field method has the advantages of simple calculation and good real-time performance, and is suitable for autonomous navigation and obstacle avoidance tasks of unmanned ships.

[0141] According to a third aspect of the embodiments of the present invention,

[0142] An electronic device is provided, comprising:

[0143] processor;

[0144] a memory for storing processor-executable instructions;

[0145] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0146] A fourth aspect of the embodiments of the present invention is:

[0147] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0148] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. Unmanned ship autonomous navigation control system, characterized in that: include: An unmanned ship hull and an unmanned ship autonomous navigation control system arranged on the unmanned ship hull, wherein the unmanned ship autonomous navigation control system includes an unmanned ship control system, an unmanned ship environment perception system, a communication system, a shore-based control system, an unmanned ship autonomous obstacle avoidance system, a GPS positioning module, an IMU attitude module and a visual perception system; The unmanned ship control system includes an edge computer, a control board, a current collector and a voltage step-down module. The edge computer is used to process sensor data, issue instructions and process data of the visual perception system. The control board is composed of an ARM chip and is used to execute unmanned ship control instructions and realize the lateral and longitudinal movement of the unmanned ship by controlling the rotation of the motor; the IMU attitude module is used to provide the navigation azimuth, pitch attitude and multi-axis acceleration data of the unmanned ship; the current collector is used to collect the current and voltage of the unmanned ship when it is working, convert it into the real-time power of the unmanned ship and combine it with the battery information to obtain the remaining endurance time of the unmanned ship, and the voltage step-down module is used to convert the battery voltage to power the control board and the sensor; The unmanned boat environment perception system includes a flow velocity sensor, a water depth sensor, a wind speed and direction sensor, an air pressure sensor and a light sensor, which are respectively used to collect water flow velocity information, depth information, wind speed and direction information, atmospheric pressure information and light intensity information of the unmanned boat navigation waters, and send the collected information to the edge computer; The visual perception system includes a high-definition camera and a binocular camera. The high-definition camera is used to observe the surface conditions of the unmanned ship navigation area, monitor the navigation of the unmanned ship, and send the video information to the shore-based control system through the communication system to realize shore-based monitoring, and send the video information to the edge computer at the same time. The binocular camera is used to obtain the image depth information of the water surface environment of the unmanned ship navigation area and send it to the edge computer; the edge computer integrates and processes the collected data and transmits it to the shore-based control system through the communication system; The edge computer is provided with a ship-side target recognition program, which performs image denoising, histogram equalization enhancement, SURF feature extraction, and obstacle recognition on the image information collected by the high-definition camera and the binocular camera, and then determines whether there are obstacles that hinder the navigation of the unmanned ship on the water surface of the current navigation area by analyzing the results of image processing. When there are obstacles, the corresponding obstacle avoidance instructions are sent to the control board, and the control board executes the visual perception obstacle avoidance program to control the unmanned ship to perform obstacle avoidance behavior; The autonomous obstacle avoidance system of the unmanned boat detects obstacles ahead through a three-dimensional laser radar installed on the hull of the unmanned boat, and sends obstacle information to the edge computer. The edge computer is provided with a laser radar collision avoidance program, which confirms obstacle information through the laser radar collision avoidance program and sends instructions to the control board. The control board executes the laser radar collision avoidance program to control the unmanned boat to perform obstacle avoidance behavior; The shore-based control system includes unmanned ship control and display software, radar rendering display software, a motion controller and a computer. The unmanned ship control and display software is used to control the navigation movement of the unmanned ship and display the working condition information of the unmanned ship. The radar rendering display software is used to display the image scanned by the laser radar. The motion controller is used to control the movement of the unmanned ship. Also includes: The three-dimensional laser radar uses the VFH vector field histogram algorithm to extract point cloud features, and calculates the feature components of the viewpoint by counting the histogram of the angle between the viewpoint direction and each normal line, including: Use the VFH method for feature extraction, use the entire point cloud object for calculation and estimation, use the point pair between the center point of the object and all other points on the surface of the object as the calculation unit when calculating the feature description, and add additional statistical information between the viewpoint direction and the estimated normal of each point; The viewpoint direction is integrated into the relative normal angle calculation, and the characteristic component of the viewpoint is calculated by statistically analyzing the histogram of the angle between the viewpoint direction and each normal line. The control panel executes the visual perception obstacle avoidance program to control the unmanned boat to perform obstacle avoidance behaviors including: The position coordinates of the obstacle are converted into a polar coordinate system centered on the unmanned ship to obtain the azimuth and distance of the obstacle relative to the unmanned ship; Combined with the current speed and heading of the unmanned ship, an obstacle avoidance strategy based on the potential field method is adopted to calculate the heading change required for obstacle avoidance. The obstacle corresponds to the repulsive potential field, and the target point corresponds to the gravitational potential field. Under the combined force of the repulsive potential field and the gravitational potential field, the unmanned ship adjusts its heading, stays away from the obstacle, and sails towards the target point. The speed change required for obstacle avoidance is calculated based on the concept of collision avoidance safety distance. When the obstacle distance is less than the safety distance, the unmanned boat slows down until it stops. When the obstacle distance is greater than the safety distance, the unmanned boat maintains the current speed or accelerates. The control board adjusts the rudder angle and engine throttle of the unmanned boat according to the calculated changes in heading and speed, and controls the unmanned boat to perform obstacle avoidance behavior.

2. The system according to claim 1, characterized in that The edge computer is also used to: For each pixel point of the received image, a Hessian matrix consisting of the second-order partial derivatives of the multivariate function f(x, y) is constructed, where f(x, y) represents the pixel value at the coordinate p(x, y). Whether the pixel point is a key point is determined by finding the local extreme value of the discriminant of the Hessian matrix; In order to make the key points scale invariant, the received image is filtered by a second-order standard Gaussian function before constructing the Hessian matrix, and the pixel value after filtering is calculated by the second-order standard Gaussian function and the original pixel value; For the filtered image, the second-order partial derivative with respect to the coordinates x and y is calculated at the pixel point p to obtain the Hessian matrix at different scales.

3. The system according to claim 2, characterized in that The Hessian matrix is ​​shown in the following formula: ; The Hessian matrix discriminant is shown in the following formula: ; The Hessian matrix at different scales is shown in the following formula: ; in, H(f(x,y)) represents the Hessian matrix, f represents a multivariate function, (x, y) Represents the spatial coordinates of the pixel point, L xx (p, σ) Representing images L At the point p and the lower edge of scale σ x The second-order partial derivative in the direction, L xy (p, σ) Representing images L At the point p and mixed second-order partial derivatives under scale σ, L yy (p, σ) Representing images L At the point p and the lower edge of scale σ y The second partial derivative in the direction.

4. The system according to claim 1, characterized in that The system further comprises: Communication systems, including high-power radios, high-gain antennas, cloud servers, and IoT modules; The edge computer is connected to a high-power radio station via a LAN communication interface, the high-power radio station is connected to a high-gain antenna, and the high-power radio stations are connected via microwave wireless communication, so that the data of the edge computer can be transmitted to the shore-based control system via the high-power radio station, thereby realizing long-distance transmission of ship-side data. The cloud server is used to transmit data uploaded by the control panel through the Internet of Things module connected to the 5G network, thereby realizing the cross-domain data transmission function between the control panel and the shore-based control system.

5. An autonomous navigation control method for an unmanned ship, used in a system as claimed in any one of claims 1 to 4, characterized in that: include: Start the unmanned boat, check whether the hull and its onboard equipment are normal, turn on the power of the unmanned boat, and check whether the equipment is working normally; The edge computer is connected to the high-power radio through the LAN communication interface, and the control board is connected to the 5G network through the IoT module to establish a TCP connection with the cloud server, and the communication system works; The navigation water environment information collected by the unmanned ship's environmental perception system is sent to the shore-based control system through the communication system; The speed and heading information and attitude information of the unmanned ship collected by the unmanned ship GPS positioning module and the IMU attitude module, as well as the image information acquired by the laser radar and visual perception system are sent to the unmanned ship control system; the edge computer in the unmanned ship control system integrates the received data and sends it to the shore-based control system through the communication system after processing; After receiving the data transmitted by the unmanned ship operating system, the shore-based control system selects the automatic navigation mode in the unmanned ship control and display software, and determines the starting and ending positions of the autonomous navigation for the unmanned ship, which are then sent to the unmanned ship control system by the communication system; The unmanned ship control system controls the unmanned ship to navigate automatically according to the starting and ending positions determined by the shore-based control system, and the edge computer controls the unmanned ship in real time; During navigation, after the unmanned boat obstacle avoidance system detects an obstacle, it executes the corresponding obstacle avoidance program to avoid the obstacle and resume the navigation route.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in claim 5.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method of claim 5 is implemented.