A method and system for intelligent cruise of unmanned vessels based on multimodal perception

Abstract

This invention relates to the field of intelligent cruise technology for unmanned surface vessels (USVs), and discloses an intelligent cruise method and system for USVs based on multimodal perception. The method includes: determining the current navigation direction based on the current USV coordinates and the target mission coordinates; performing edge recognition on remote sensing satellite images to obtain obstacle contour images; smoothing the contour boundaries of the obstacle contour images to obtain obstacle boundary images; performing path feasibility planning based on the obstacle boundary images, radar systems, and sonar systems to obtain a coordinate navigation dataset; and selecting the optimal path based on the coordinate navigation dataset to obtain the optimal navigation path. This invention can improve the globality and efficiency of USV path planning and reduce fuel consumption and travel time during USV cruise.

Description

Technical Field

[0001] This invention relates to the field of unmanned vessel intelligent cruise technology, and in particular to an unmanned vessel intelligent cruise method and system based on multimodal perception. Background Technology

[0002] With the rapid development of marine resource exploration, environmental monitoring, maritime transport, and national defense security, the importance of unmanned surface vessel (USV) intelligent cruise technology is becoming increasingly prominent. USVs can perform long-duration, wide-area autonomous missions in complex and even dangerous marine environments, effectively reducing manpower costs and risks. Their intelligent cruise capability is a core guarantee for achieving efficient and precise operations, and is of great significance for promoting marine economic development and safeguarding maritime rights.

[0003] Traditional unmanned surface vessel (USV) path planning methods typically rely solely on real-time sensor data from onboard radar, sonar, and other sensors for local obstacle avoidance planning. This approach has significant drawbacks: firstly, its limited perception range prevents proactive obstacle avoidance along long-distance routes, leading to insufficient global path optimization; secondly, it lacks the ability to effectively distinguish between static phantoms or dynamic non-threatening targets detected by sensors, easily resulting in unnecessary conservative detours and thus increasing travel distance and time. Summary of the Invention

[0004] This invention provides an intelligent cruise method and system for unmanned vessels based on multimodal perception. Its main purpose is to improve the globality and efficiency of unmanned vessel path planning and reduce fuel consumption and travel time during unmanned vessel cruise.

[0005] To achieve the above objectives, the present invention provides an intelligent cruise method for unmanned vessels based on multimodal perception, comprising:

[0006] The unmanned vessel is identified and navigation mission instructions are received. The unmanned vessel includes a radar system, a sonar system, and a positioning system.

[0007] The target mission coordinates are obtained based on the navigation mission instructions, the current coordinates of the unmanned vessel are determined using the positioning system, and the current navigation direction is determined based on the current coordinates of the unmanned vessel and the target mission coordinates.

[0008] Based on the current coordinates of the unmanned vessel, the current navigation direction, and the coordinates of the target mission, remote sensing satellite images are acquired, and edge recognition is performed on the remote sensing satellite images to obtain obstacle outline images;

[0009] The obstacle contour image is smoothed to obtain the obstacle boundary image;

[0010] Based on obstacle boundary images, radar system, and sonar system, path feasibility planning is performed to obtain a coordinate navigation dataset. The coordinate navigation dataset includes multiple coordinate navigation data, which include: obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be navigated, and direction adjustment angle.

[0011] The optimal route is selected based on the coordinate navigation dataset, and the optimal navigation route is obtained. The intelligent cruise of the unmanned vessel based on multimodal perception is completed based on the optimal navigation route.

[0012] Optionally, the step of smoothing the contour image of the obstacle to obtain an obstacle boundary image includes:

[0013] Identify multiple contour boundaries in the obstacle contour image;

[0014] Extract contour boundaries sequentially from multiple contour boundaries, and record the extracted contour boundaries as the boundaries to be smoothed;

[0015] Obtain the remote sensing resolution of the obstacle contour image;

[0016] The tortuosity of the smooth boundary is calculated to obtain the boundary tortuosity. The minimum tortuosity distance is then calculated based on the remote sensing resolution and the boundary tortuosity.

[0017] The smoothed boundary is obtained by performing a smoothing operation on the boundary to be smoothed based on the minimum tortuosity distance;

[0018] By summing up the smoothed boundaries corresponding to each contour boundary in multiple contour boundaries, multiple smoothed boundaries are obtained.

[0019] The obstacle contour image is updated using multiple smoothed boundaries to obtain the obstacle boundary image.

[0020] Optionally, the step of smoothing the boundary to be smoothed based on the minimum tortuosity distance to obtain the smoothed boundary includes:

[0021] Identify the set of boundary pixels in the boundary to be smoothed, wherein the set of boundary pixels includes multiple boundary pixels;

[0022] Identify the start and end points of the boundary in the set of boundary pixels;

[0023] Construct boundary line segments based on the boundary start and boundary end points;

[0024] Calculate the pixel distance from each boundary pixel in the boundary pixel set to the boundary line segment to obtain the pixel distance set;

[0025] Identify the maximum distance in the pixel distance set and record the boundary pixel point corresponding to the maximum distance as the far-end pixel point;

[0026] Determine if the maximum distance is less than the minimum tortuous distance;

[0027] If the maximum distance is not less than the minimum tortuous distance, then the boundary to be smoothed is segmented based on the far-end pixels to obtain multiple repeated non-smooth boundaries;

[0028] Extract repeated non-smooth boundaries sequentially from multiple repeated non-smooth boundaries, and use the extracted repeated non-smooth boundaries as boundaries to be smoothed, and return to the step of confirming the set of boundary pixels in the boundaries to be smoothed, until the maximum distance is less than the minimum tortuosity distance;

[0029] If the maximum distance is less than the minimum tortuous distance, then the boundary line segments are summarized to obtain multiple boundary line segments. Connecting these multiple boundary line segments yields a smoothed boundary.

[0030] Optionally, the step of performing path feasibility planning based on obstacle boundary images, radar systems, and sonar systems to obtain a coordinate navigation dataset includes:

[0031] Identify the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current coordinates of the unmanned vessel;

[0032] The nearest obstacle is assessed for passability using radar and sonar systems, and the assessment results are obtained, including whether it is passable or impassable.

[0033] If the evaluation result is passable, the nearest obstacle is removed from the obstacle boundary image to obtain an updated boundary image. The updated boundary image is used as the obstacle boundary image, and the process of identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current UAV coordinates is returned until the evaluation result is impassable.

[0034] If the assessment result is that the road is impassable, then the nearest obstacle will be marked as the obstacle to be bypassed.

[0035] Based on the identification of navigable feature points in the obstacle boundary image, multiple navigable feature coordinates are obtained;

[0036] The navigable feature coordinates are extracted sequentially from multiple navigable feature coordinates, and the extracted navigable coordinates are used as the target navigation coordinates;

[0037] The target's navigation direction is determined based on the target's navigation coordinates and the current coordinates of the unmanned vessel;

[0038] Coordinate navigation data is constructed based on the target navigation coordinates. The coordinate navigation data includes: obstacle detour number, target navigation coordinates, starting navigation coordinates, distance to be navigated, and direction adjustment angle.

[0039] The target navigation coordinates and target navigation direction in the coordinate navigation data are respectively used as the current unmanned vessel coordinates and current navigation direction, and the step of identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and current unmanned vessel coordinates is returned until the navigable feature coordinates among multiple navigable feature coordinates are extracted.

[0040] The coordinate navigation data corresponding to the navigable feature coordinates are summarized to obtain the coordinate navigation dataset, which includes multiple coordinate navigation data.

[0041] Optionally, the construction of coordinate navigation data based on the target navigation coordinates includes:

[0042] Set a detour sequence number for each obstacle to be bypassed, and obtain the obstacle detour sequence number;

[0043] The directional adjustment angle is calculated based on the target navigation direction and the current navigation direction, and the distance to be traveled between the target navigation coordinates and the current unmanned vessel coordinates is obtained;

[0044] The current unmanned vessel coordinates are set as the starting navigation coordinates;

[0045] The obstacle detour number, target navigation coordinates, starting navigation coordinates, distance to be traversed, and direction adjustment angle are combined to obtain coordinate navigation data.

[0046] Optionally, the step of selecting the optimal route based on the coordinate navigation dataset to obtain the optimal navigation route includes:

[0047] A parrot population is generated based on preset constraints. The parrot population includes multiple parrot individuals, and each parrot individual corresponds to a candidate solution vector. The candidate solution vector includes multiple candidate solution coordinates, and the number of candidate solution coordinates is the same as the maximum value of the obstacle detour sequence number.

[0048] The parrot population was iterated based on the coordinate navigation dataset to obtain the single optimal solution vector;

[0049] Apply local optimal suppression perturbation to the single optimal solution vector to obtain the updated optimal solution vector;

[0050] Based on the preset maximum number of iterations and the updated optimal solution vector, multiple iterations are performed to obtain the global optimal solution vector;

[0051] Extract the coordinates of multiple optimal solutions from the global optimal solution vector, and formulate the optimal navigation path based on the coordinates of multiple optimal solutions.

[0052] Optionally, the step of iterating the parrot population based on the coordinate navigation dataset to obtain a single optimal solution vector includes:

[0053] For each candidate solution vector in the parrot population, the following operation is performed:

[0054] Extract the coordinates of candidate solutions sequentially from multiple candidate solution coordinates in the candidate solution vector, and mark the extracted candidate solution coordinates as the target solution coordinates;

[0055] Identify the target navigation data corresponding to the target solution coordinates in multiple coordinate navigation data sets of the coordinate navigation dataset;

[0056] By summing the target navigation data corresponding to the candidate solution coordinates, multiple target navigation data are obtained;

[0057] Fitness is calculated based on navigation data from multiple targets to obtain the fitness of candidate solutions.

[0058] The fitness of candidate solutions is summarized to obtain multiple candidate solution fitness values. Based on the fitness values ​​of multiple candidate solutions, the single optimal solution vector is identified in the parrot population.

[0059] Optionally, the fitness calculation based on multiple target navigation data to obtain the fitness of candidate solutions includes:

[0060] Target navigation data is extracted sequentially from multiple target navigation data sets, and the extracted target navigation data is recorded as navigation data to be evaluated.

[0061] Based on the preset navigation acceleration strategy, the navigation fuel consumption and navigation time are evaluated for the navigation distance in the navigation data to be evaluated, and the navigation fuel consumption value and navigation time are obtained.

[0062] Based on the preset steering strategy, the steering fuel consumption and steering time are evaluated respectively in the directional adjustment angle of the navigation data to be evaluated, and the steering fuel consumption value and steering time are obtained.

[0063] Calculate the total fuel consumption and total travel time based on the navigation fuel consumption value, turning fuel consumption value, travel time and turning time;

[0064] The fitness of candidate solutions is obtained by weighting the total fuel consumption and total travel time.

[0065] Optionally, the step of applying local optimal suppression perturbation to the single-time optimal solution vector to obtain an updated optimal solution vector includes:

[0066] Get the current iteration number;

[0067] The optimal solution vector for a single iteration is perturbed based on the current iteration number and the maximum iteration number to obtain the updated optimal solution vector, which is expressed as:

[0068]

[0069] in, This indicates updating the optimal solution vector. This represents the vector of the single optimal solution. Indicates the current iteration number. Indicates the maximum number of iterations. This represents the pre-defined Lorentz distribution function. This represents the predefined standard normal distribution function.

[0070] To achieve the above objectives, the present invention also provides an intelligent cruise system for unmanned vessels based on multimodal perception, comprising:

[0071] The navigation direction determination module is used to identify the unmanned vessel and receive navigation mission instructions. The unmanned vessel includes a radar system, a sonar system, and a positioning system. It obtains the target mission coordinates based on the navigation mission instructions, uses the positioning system to determine the current coordinates of the unmanned vessel, and determines the current navigation direction based on the current coordinates of the unmanned vessel and the target mission coordinates.

[0072] The boundary image acquisition module is used to acquire remote sensing satellite images based on the current coordinates of the unmanned vessel, the current navigation direction, and the coordinates of the target mission; to perform edge recognition on the remote sensing satellite images to obtain obstacle contour images; and to perform contour boundary smoothing on the obstacle contour images to obtain obstacle boundary images.

[0073] The navigation data construction module is used to perform path feasibility planning based on obstacle boundary images, radar systems, and sonar systems to obtain a coordinate navigation dataset. The coordinate navigation dataset includes multiple coordinate navigation data, which include: obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be navigated, and direction adjustment angle.

[0074] The optimal route selection module is used to select the optimal route based on the coordinate navigation dataset to obtain the optimal navigation route.

[0075] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0076] Memory, storing at least one instruction;

[0077] The processor executes the instructions stored in the memory to implement the above-described intelligent cruise method for unmanned vessels based on multimodal perception.

[0078] To address the aforementioned issues, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned intelligent cruise method for unmanned vessels based on multimodal perception.

[0079] To address the problems described in the background section, this invention first acquires remote sensing satellite images based on the current coordinates of the unmanned surface vessel, its current navigation direction, and the target mission coordinates. Edge recognition is then performed on these satellite images to obtain obstacle contour images. This step, by incorporating remote sensing satellite images, allows for the early acquisition of obstacle information over a large area of ​​water and enables edge recognition, thereby enhancing the foresight of path planning, reducing reliance on real-time perception, and improving overall cruise efficiency. Traditional methods primarily rely on radar or sonar for real-time obstacle detection, which has a limited range and is often delayed. Furthermore, the obstacle contour images undergo contour boundary smoothing to obtain obstacle boundary images. This step, through contour boundary smoothing, adaptively eliminates invalid details based on boundary curvature and remote sensing resolution, simplifying obstacle boundaries and significantly reducing the complexity of path planning, thus improving computational efficiency. In contrast, traditional edge recognition methods... Obstacle outlines often contain subtle bumps and depressions caused by noise, leading to numerous unnecessary detours in path planning. This solution then performs path feasibility planning based on obstacle boundary images, radar systems, and sonar systems, generating a coordinate navigation dataset. This step integrates radar and sonar systems for accessibility assessment, dynamically filtering non-threatening obstacles and generating the coordinate navigation dataset to ensure path feasibility and improve the accuracy and safety of path planning. Traditional path planning often statically processes obstacles, ignoring the risk of misjudgment of dynamic objects or phantoms. Finally, this solution selects the optimal path based on the coordinate navigation dataset, obtaining the optimal navigation path. This step employs the Parrot optimization algorithm combined with a local optimum disturbance suppression strategy, adaptively balancing global exploration and local search to achieve multi-objective optimization, thereby generating an optimal path with low fuel consumption, short time, and safety, improving cruise performance. Therefore, this invention can improve the globality and efficiency of unmanned surface vessel (USV) path planning, reducing fuel consumption and travel time during USV cruise. Attached Figure Description

[0080] Figure 1 This is a flowchart illustrating an embodiment of the intelligent cruise method for unmanned vessels based on multimodal perception provided by the present invention.

[0081] Figure 2 A functional block diagram of an unmanned vessel intelligent cruise system based on multimodal perception provided in an embodiment of the present invention;

[0082] Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the multimodal perception-based intelligent cruise method for unmanned vessels, according to an embodiment of the present invention.

[0083] Explanation of reference numerals in the attached figures:

[0084] 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0085] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0086] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0087] This application provides a method for intelligent navigation of unmanned surface vessels (USVs) based on multimodal perception. The executing entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software may be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0088] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent cruise method for unmanned surface vessels based on multimodal perception, according to an embodiment of the present invention. In this embodiment, the intelligent cruise method for unmanned surface vessels based on multimodal perception includes:

[0089] S1. Confirm the location of the unmanned vessel and receive navigation mission instructions. The unmanned vessel includes a radar system, a sonar system, and a positioning system.

[0090] It is clear that the unmanned vessel refers to an unmanned surface vessel with autonomous navigation capabilities. The unmanned vessel includes a radar system, a sonar system, and a positioning system. The radar system is a radio detection system used to detect obstacles on the water surface and other vessels and measure their distance, orientation, and speed. The sonar system is a detection system that uses sound waves to detect underwater terrain, reefs, and other potential obstacles. The positioning system is a system used to determine the precise geographical coordinates of the unmanned vessel in real time (such as GPS or BeiDou satellite navigation system). In addition to the above, the unmanned vessel may also be equipped with visual sensors (such as cameras), a passage module, and a main control computer.

[0091] Furthermore, the navigation mission instruction refers to an instruction initiated by a remote terminal to guide the unmanned vessel to a designated location. For example, the instruction content may include the latitude and longitude coordinates of the target point (such as "sail to 30.5 degrees north latitude, 122.8 degrees east longitude"), and may be accompanied by navigation speed requirements or time limits.

[0092] S2. Obtain the target mission coordinates based on the navigation mission instructions, determine the current coordinates of the unmanned vessel using the positioning system, and determine the current navigation direction based on the current unmanned vessel coordinates and the target mission coordinates.

[0093] It should be explained that the target mission coordinates refer to the location that the unmanned vessel needs to reach, as included in the navigation mission command. The current unmanned vessel coordinates refer to the specific location of the unmanned vessel when it receives the navigation mission command. The current navigation direction refers to the position of the current unmanned vessel coordinates pointing towards the target mission coordinates, which represents the approximate direction of the unmanned vessel's subsequent travel.

[0094] S3. Obtain remote sensing satellite images based on the current coordinates of the unmanned vessel, the current navigation direction, and the coordinates of the target mission. Perform edge recognition on the remote sensing satellite images to obtain obstacle contour images.

[0095] Understandably, the remote sensing satellite image refers to a remote sensing image obtained through remote sensing satellites that includes the coordinates of the target mission and the current coordinates of the unmanned surface vessel (USV). This image contains both the current USV coordinates and the target mission coordinates. The specific method for obtaining the remote sensing satellite image based on the current USV coordinates, current navigation direction, and target mission coordinates is as follows: the USV initiates an image request to the satellite ground station or cloud service platform via its communication module. This request includes parameters such as the current USV coordinates, the target mission coordinates, and the required image resolution and coverage area. Subsequently, the remote sensing image data for the corresponding area returned by the service platform is received and parsed. If the current USV coordinates are far from the target mission coordinates (i.e., exceeding the maximum effective coverage area of ​​a single remote sensing satellite image), an intermediate navigation coordinate needs to be selected within the current navigation direction. This allows the USV to first reach the central navigation coordinate, and then continue the above steps at the central navigation coordinate until the target mission coordinates are reached.

[0096] Furthermore, the obstacle contour image refers to an image containing clear outlines of obstacles such as islands, shorelines, and other vessels in the water. The specific steps for obtaining the obstacle contour image by edge recognition of remote sensing satellite images are as follows: First, the obstacle contours are preprocessed by grayscale conversion, binarization, and denoising to obtain a denoised obstacle image. The relevant steps of this preprocessing are all existing technologies and will not be elaborated here. Next, the denoised obstacle image is subjected to contour extraction to obtain the obstacle contour image. Contour extraction can be achieved through algorithms such as Canny edge detection and contour connection. This contour extraction can effectively identify the boundaries between different obstacles and water areas in the image, connecting continuous pixels into complete contour lines, providing accurate obstacle spatial information for subsequent path planning.

[0097] S4. Perform contour boundary smoothing on the obstacle contour image to obtain the obstacle boundary image.

[0098] It is clear that the obstacle boundary image refers to the obstacle contour image after contour boundary smoothing. Although edge recognition has been performed, subtle gradient changes still exist in the obstacle contour image. These gradient changes manifest as concave or convex regions in the image contour. These concave or convex regions can make the generated obstacle boundary overly complex and fragmented, resulting in a large number of unnecessary and overly conservative detour points in subsequent path planning, increasing path length and computational complexity. Therefore, it is necessary to eliminate these concave or convex regions caused by image noise, remote sensing imaging errors, or edge recognition algorithm sensitivity, so that subsequent path feasibility planning can be based on a simpler obstacle boundary, improving the efficiency of path planning.

[0099] Specifically, the process of smoothing the contour boundaries of the obstacle image to obtain an obstacle boundary image includes:

[0100] Identify multiple contour boundaries in the obstacle contour image;

[0101] Extract contour boundaries sequentially from multiple contour boundaries, and record the extracted contour boundaries as the boundaries to be smoothed;

[0102] Obtain the remote sensing resolution of the obstacle contour image;

[0103] The tortuosity of the smooth boundary is calculated to obtain the boundary tortuosity. The minimum tortuosity distance is then calculated based on the remote sensing resolution and the boundary tortuosity.

[0104] The smoothed boundary is obtained by performing a smoothing operation on the boundary to be smoothed based on the minimum tortuosity distance;

[0105] By summing up the smoothed boundaries corresponding to each contour boundary in multiple contour boundaries, multiple smoothed boundaries are obtained.

[0106] The obstacle contour image is updated using multiple smoothed boundaries to obtain the obstacle boundary image.

[0107] Understandably, the contour boundary refers to the curve formed by the continuous outer pixels of a single independent obstacle in the image, extracted by the edge recognition algorithm. The remote sensing resolution refers to the actual ground distance corresponding to a single pixel in the remote sensing satellite image (e.g., 1 pixel represents 1 meter). The boundary inflection degree refers to a numerical value that quantifies the degree of inflection of the boundary to be smoothed. The greater the boundary inflection degree, the more inflected the boundary to be smoothed. The boundary inflection degree is calculated as follows: multiple straight line segments in the boundary to be smoothed are identified, and the angle between every two adjacent straight line segments is calculated to obtain multiple angles, where all angles are less than 180 degrees. These angles represent the degree of inflection between two adjacent straight line segments. The larger the angle, the more inflected the connection between the two straight line segments corresponding to that angle. Then, the multiple angles are summed to obtain the total angle. This total angle can represent the degree to which the overall boundary to be smoothed deviates from a straight state. Finally, the total angle is divided by pi to normalize the angle value from radians to a dimensionless scale, thus obtaining the boundary inflection degree.

[0108] Furthermore, the minimum tortuosity distance refers to the product of remote sensing resolution and boundary tortuosity. This minimum tortuosity distance represents the smallest scale of concavity or convexity that can be considered as image noise or minor undulations in actual geographic space and does not need to be considered in path planning. The larger the minimum tortuosity distance, the larger the concavity or convexity that can be smoothed out. The smoothed boundary refers to the boundary to be smoothed after a smoothing operation, which will be explained later. Updating the obstacle contour image using multiple smoothed boundaries means replacing multiple contour boundaries in the obstacle contour image with multiple smoothed boundaries. The resulting obstacle contour image is the obstacle boundary image.

[0109] In detail, the smoothing operation on the boundary to be smoothed based on the minimum tortuosity distance to obtain the smoothed boundary includes:

[0110] Identify the set of boundary pixels in the boundary to be smoothed, wherein the set of boundary pixels includes multiple boundary pixels;

[0111] Identify the start and end points of the boundary in the set of boundary pixels;

[0112] Construct boundary line segments based on the boundary start and boundary end points;

[0113] Calculate the pixel distance from each boundary pixel in the boundary pixel set to the boundary line segment to obtain the pixel distance set;

[0114] Identify the maximum distance in the pixel distance set and record the boundary pixel point corresponding to the maximum distance as the far-end pixel point;

[0115] Determine if the maximum distance is less than the minimum tortuous distance;

[0116] If the maximum distance is not less than the minimum tortuous distance, then the boundary to be smoothed is segmented based on the far-end pixels to obtain multiple repeated non-smooth boundaries;

[0117] Extract repeated non-smooth boundaries sequentially from multiple repeated non-smooth boundaries, and use the extracted repeated non-smooth boundaries as boundaries to be smoothed, and return to the step of confirming the set of boundary pixels in the boundaries to be smoothed, until the maximum distance is less than the minimum tortuosity distance;

[0118] If the maximum distance is less than the minimum tortuous distance, then the boundary line segments are summarized to obtain multiple boundary line segments. Connecting these multiple boundary line segments yields a smoothed boundary.

[0119] It is clear that the boundary pixel set refers to the set of all pixels on the boundary to be smoothed. In the contour tracking algorithm, the coordinates of the first pixel recorded along the boundary to be smoothed are defined as follows: the boundary endpoint refers to the coordinates of the last pixel recorded along the boundary to be smoothed in the contour tracking algorithm. The boundary line segment refers to the direct line segment connecting the boundary start point and the boundary endpoint. If the boundary to be smoothed is a closed contour, then the boundary start point and the boundary endpoint are the same pixel or adjacent pixels. The pixel distance refers to the Euclidean distance from the boundary pixel to the boundary line segment. This pixel distance represents the degree of offset of the boundary pixel relative to the line connecting the boundary start point and the boundary endpoint. The larger the pixel distance, the more obvious the curvature at the boundary pixel.

[0120] Furthermore, the maximum distance refers to the pixel distance with the largest value in the pixel distance set. If the maximum distance is not less than the minimum tortuous distance mentioned above, it indicates that the boundary straight line segment here cannot adequately approximate the original boundary curve (i.e., the boundary to be smoothed). There are significant uneven parts on the boundary to be smoothed that need to be smoothed out. In this case, the far-end pixel point needs to be used as the new segmentation point to recursively smooth the current boundary segment. The segmentation of the boundary to be smoothed based on the far-end pixel point means: using the far-end pixel point as the new key point, dividing the original boundary to be smoothed into two sub-boundaries: one from the boundary start point to the far-end pixel point, and the other from the far-end pixel point to the boundary end point. This divides the boundary to be smoothed into two curves, i.e., multiple repeated non-smooth boundaries. The repeated non-smooth boundaries refer to the boundaries that still need to be smoothed after segmentation. The connection of multiple boundary line segments refers to the following: when all sub-boundaries (i.e. boundary line segments) satisfy the condition that the maximum distance is less than the minimum tortuous distance, all boundary line segments generated during the recursive process and used to approximate each sub-boundary segment are connected end to end in the order on the boundary to be smoothed, forming a new boundary composed of line segments that approximates the boundary to be smoothed.

[0121] S5. Based on the obstacle boundary image, radar system and sonar system, perform path feasibility planning to obtain a coordinate navigation dataset. The coordinate navigation dataset includes multiple coordinate navigation data, and the coordinate navigation data includes: obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be navigated and direction adjustment angle.

[0122] Understandably, the coordinate navigation dataset refers to a collection of multiple coordinate navigation data sets, where each coordinate navigation data set represents the relevant data of a certain coordinate that the unmanned vessel can reach. The obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be traveled, and direction adjustment angle in the coordinate navigation data will be explained in subsequent steps.

[0123] In detail, the path feasibility planning based on obstacle boundary images, radar systems, and sonar systems to obtain a coordinate navigation dataset includes:

[0124] Identify the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current coordinates of the unmanned vessel;

[0125] The nearest obstacle is assessed for passability using radar and sonar systems, and the assessment results are obtained, including whether it is passable or impassable.

[0126] If the evaluation result is passable, the nearest obstacle is removed from the obstacle boundary image to obtain an updated boundary image. The updated boundary image is used as the obstacle boundary image, and the process of identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current UAV coordinates is returned until the evaluation result is impassable.

[0127] If the assessment result is that the road is impassable, then the nearest obstacle will be marked as the obstacle to be bypassed.

[0128] Based on the identification of navigable feature points in the obstacle boundary image, multiple navigable feature coordinates are obtained;

[0129] The navigable feature coordinates are extracted sequentially from multiple navigable feature coordinates, and the extracted navigable coordinates are used as the target navigation coordinates;

[0130] The target's navigation direction is determined based on the target's navigation coordinates and the current coordinates of the unmanned vessel;

[0131] Coordinate navigation data is constructed based on the target navigation coordinates. The coordinate navigation data includes: obstacle detour number, target navigation coordinates, starting navigation coordinates, distance to be navigated, and direction adjustment angle.

[0132] The target navigation coordinates and target navigation direction in the coordinate navigation data are respectively used as the current unmanned vessel coordinates and current navigation direction, and the step of identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and current unmanned vessel coordinates is returned until the navigable feature coordinates among multiple navigable feature coordinates are extracted.

[0133] The coordinate navigation data corresponding to the navigable feature coordinates are summarized to obtain the coordinate navigation dataset, which includes multiple coordinate navigation data.

[0134] It should be explained that the nearest obstacle refers to the first obstacle encountered in the obstacle boundary image along the current navigation direction from the current coordinates of the unmanned vessel. The assessment of the passability of the nearest obstacle using radar and sonar systems involves: using radar to determine whether the nearest obstacle is a dynamic object, such as a fishing boat, floating containers, or flocks of birds; if it is a dynamic object, or the radar echo characteristics are not significant, further identification is performed using a visual system (such as a high-definition camera) or sonar system to determine whether there is an actual static obstacle that poses a threat to navigation, such as small driftwood, dense aquatic plant areas, or a phantom image caused by wave reflection. If the nearest obstacle is a dynamic object or there is no actual obstacle at the location of the nearest obstacle, it means that the obstacle does not pose a substantial threat to the current route of the unmanned vessel, and the assessment result is recorded as passable; otherwise, it means that detour is necessary, and the assessment result is recorded as impassable. It should be noted that if the unmanned vessel is not at the current unmanned vessel coordinates (which may be simulated coordinates as the recursion progresses), the evaluation result will be assumed to be passable. The judgment will be made at this point when the unmanned vessel travels to the actual coordinates of the current unmanned vessel coordinates.

[0135] Furthermore, the updated boundary image refers to the obstacle boundary image after removing the nearest obstacle. The navigable feature coordinates refer to the coordinates around the obstacle to be bypassed that allow for bypassing, such as the tangent point of the obstacle's outline convex point, the midpoint of the safe passage on both sides of the obstacle, etc. Optionally, the specific steps for identifying navigable feature points based on the obstacle to be bypassed in the obstacle boundary image can be as follows: First, based on the outline of the obstacle to be bypassed, extend outward by a preset safe distance to generate a buffer boundary. Then, on this buffer boundary, using the line connecting the obstacle's center of gravity and the current UAV coordinates as a reference, symmetrically select several points on the left and right sides of the obstacle (relative to the current navigation direction) as candidate feature points. Finally, if relevant radar and sonar data are available, real-time radar and sonar detection data can be combined to filter out candidate points that still pose a collision risk or have insufficient water depth, ultimately determining multiple safe navigable feature coordinates.

[0136] Specifically, the construction of coordinate navigation data based on the target navigation coordinates includes:

[0137] Set a detour sequence number for each obstacle to be bypassed, and obtain the obstacle detour sequence number;

[0138] The directional adjustment angle is calculated based on the target navigation direction and the current navigation direction, and the distance to be traveled between the target navigation coordinates and the current unmanned vessel coordinates is obtained;

[0139] The current unmanned vessel coordinates are set as the starting navigation coordinates;

[0140] The obstacle detour number, target navigation coordinates, starting navigation coordinates, distance to be traversed, and direction adjustment angle are combined to obtain coordinate navigation data.

[0141] As you can understand, the obstacle detour sequence number refers to a unique number used to identify the order in which obstacles to be detoured are processed in this navigation mission. For example, if an obstacle to be detoured is the second obstacle that the unmanned vessel needs to detour, then the obstacle detour sequence number corresponding to that obstacle is recorded as 2. This obstacle detour sequence number is used to constrain the parrot position in the subsequent parrot algorithm, that is, to ensure that the algorithm can access the coordinates of each obstacle according to the correct physical detour order when optimizing the path. The details will be given in subsequent steps. The target navigation direction refers to the vector direction formed by the current unmanned vessel coordinates pointing to the target navigation coordinates, usually expressed as the angle (azimuth) with true north. The direction adjustment angle refers to the clockwise deviation angle between the target navigation direction and the current heading direction, and its value can be calculated by the vector cross product or angle difference formula. The distance to be traversed refers to the Euclidean distance between the target navigation coordinates and the current unmanned vessel coordinates.

[0142] S6. Select the optimal route based on the coordinate navigation dataset to obtain the optimal navigation route, and complete the intelligent cruise of the unmanned vessel based on multimodal perception based on the optimal navigation route.

[0143] Understandably, the optimal navigation path refers to the path obtained after the optimal path selection for the unmanned vessel to reach the target mission coordinates. This optimal navigation path has the comprehensive optimal characteristics of short total travel distance, low total fuel consumption, short total travel time, and high safety (i.e., maintaining a sufficient safe distance from obstacles). Moreover, the optimal navigation path will be continuously updated as the unmanned vessel moves forward and in combination with real-time perception data from radar, sonar, and other systems until the unmanned vessel reaches the target mission coordinates.

[0144] In detail, the optimal route selection based on the coordinate navigation dataset to obtain the optimal navigation route includes:

[0145] A parrot population is generated based on preset constraints. The parrot population includes multiple parrot individuals, and each parrot individual corresponds to a candidate solution vector. The candidate solution vector includes multiple candidate solution coordinates, and the number of candidate solution coordinates is the same as the maximum value of the obstacle detour sequence number.

[0146] The parrot population was iterated based on the coordinate navigation dataset to obtain the single optimal solution vector;

[0147] Apply local optimal suppression perturbation to the single optimal solution vector to obtain the updated optimal solution vector;

[0148] Based on the preset maximum number of iterations and the updated optimal solution vector, multiple iterations are performed to obtain the global optimal solution vector;

[0149] Extract the coordinates of multiple optimal solutions from the global optimal solution vector, and formulate the optimal navigation path based on the coordinates of multiple optimal solutions.

[0150] It is clear that the constraints refer to the conditions that constrain the position and order of the coordinates of each candidate solution in the candidate solution vector. For example, the constraint is that the order number of the candidate solution coordinate in the candidate solution vector is the same as the obstacle detour order in the coordinate navigation data corresponding to the candidate solution coordinate, which ensures that the physical order of bypassing obstacles in the unmanned surface vessel's path planning is correct. Two adjacent candidate solution coordinates in the candidate solution vector must satisfy the following condition: the target navigation coordinate in the coordinate navigation data corresponding to the preceding candidate solution coordinate is the same as the starting navigation coordinate in the coordinate navigation data corresponding to the following candidate solution coordinate, which ensures that the generated path is continuous and reachable, avoiding breakpoints between path segments. The "parrot population" refers to the initial solution set used in the parrot optimization algorithm to simulate the search behavior of a parrot swarm. The candidate solution vector refers to a sequence of multiple path point coordinates, representing a possible path for the unmanned surface vessel to reach the target task coordinate from its current position. The candidate solution vector includes multiple candidate solution coordinates, each of which is a coordinate point on the path corresponding to the candidate solution vector, and this coordinate point exists within the coordinate navigation dataset. The process of generating a parrot population includes standard steps such as initializing location and initializing fitness, which will not be repeated here.

[0151] Furthermore, the single-step optimal solution vector refers to the candidate solution vector with the best fitness value in the current parrot population obtained after one iteration. The updated optimal solution vector refers to the single-step optimal solution vector after local optimum suppression perturbation, where local optimum suppression perturbation refers to applying a random perturbation to the current optimal solution during the algorithm iteration process to prevent it from prematurely falling into a local optimum. The maximum number of iterations refers to the maximum number of times the algorithm search process will terminate, which is set manually. The global optimal solution vector refers to the candidate solution vector with the best fitness value found in the entire search process after reaching the maximum number of iterations. The execution of multiple iterations based on the preset maximum number of iterations and the updated optimal solution vector means that the algorithm updates the position of individuals in the parrot population according to the updated optimal solution vector. This update process simulates the parrots' foraging, resting, and communication behaviors, and repeats the iteration steps until the number of iterations reaches the maximum number of iterations. The specific content is existing technology and will not be elaborated here. The multiple optimal solution coordinates refer to the coordinates of multiple candidate solutions in the global optimal solution vector. Optimal navigation path formulation based on multiple optimal solution coordinates refers to connecting the current unmanned vessel coordinates, the coordinates of each optimal solution, and the target mission coordinates in sequence according to the order of the candidate solution coordinates in the global optimal solution vector, to form a complete and continuous navigation path.

[0152] In detail, the iterative process of analyzing the parrot population based on the coordinate navigation dataset to obtain a single optimal solution vector includes:

[0153] For each candidate solution vector in the parrot population, the following operation is performed:

[0154] Extract the coordinates of candidate solutions sequentially from multiple candidate solution coordinates in the candidate solution vector, and mark the extracted candidate solution coordinates as the target solution coordinates;

[0155] Identify the target navigation data corresponding to the target solution coordinates in multiple coordinate navigation data sets of the coordinate navigation dataset;

[0156] By summing the target navigation data corresponding to the candidate solution coordinates, multiple target navigation data are obtained;

[0157] Fitness is calculated based on navigation data from multiple targets to obtain the fitness of candidate solutions.

[0158] The fitness of candidate solutions is summarized to obtain multiple candidate solution fitness values. Based on the fitness values ​​of multiple candidate solutions, the single optimal solution vector is identified in the parrot population.

[0159] It is clear that the target navigation data refers to the coordinate navigation data in the coordinate navigation dataset that can connect two adjacent path points (candidate solution coordinates) in the candidate solution vector. The target navigation data is identified as follows: if the sequence number of the target solution coordinate in the candidate solution vector is 1, then the lagging coordinate (i.e., the second candidate solution coordinate in the candidate solution vector) that is adjacent to and lags behind the target solution coordinate is queried. The coordinate navigation data whose starting navigation coordinate and target navigation coordinate are the target solution coordinate and the lagging coordinate, respectively, is recorded as the target navigation data. If the sequence number of the target solution coordinate in the candidate solution vector is not 1, then the preceding coordinate that is adjacent to and precedes the target solution coordinate is queried. The coordinate navigation data whose starting navigation coordinate and target navigation coordinate are the preceding coordinate and the target solution coordinate, respectively, is recorded as the target navigation data. The candidate solution fitness refers to a comprehensive evaluation index obtained by weighted summarization of the total fuel consumption and total travel time of the complete path corresponding to the candidate solution vector. The smaller the candidate solution fitness, the lower the cost (fuel and time) of traveling according to the candidate solution vector, and the better the path. The method of identifying the single optimal solution vector in a parrot population based on the fitness of multiple candidate solutions refers to recording the minimum value among the fitness of multiple candidate solutions as the single optimal solution vector.

[0160] In detail, the fitness calculation based on multiple target navigation data to obtain the fitness of candidate solutions includes:

[0161] Target navigation data is extracted sequentially from multiple target navigation data sets, and the extracted target navigation data is recorded as navigation data to be evaluated.

[0162] Based on the preset navigation acceleration strategy, the navigation fuel consumption and navigation time are evaluated for the navigation distance in the navigation data to be evaluated, and the navigation fuel consumption value and navigation time are obtained.

[0163] Based on the preset steering strategy, the steering fuel consumption and steering time are evaluated respectively in the directional adjustment angle of the navigation data to be evaluated, and the steering fuel consumption value and steering time are obtained.

[0164] Calculate the total fuel consumption and total travel time based on the navigation fuel consumption value, turning fuel consumption value, travel time and turning time;

[0165] The fitness of candidate solutions is obtained by weighting the total fuel consumption and total travel time.

[0166] Understandably, the navigation acceleration strategy refers to the speed control rules adopted by the unmanned surface vessel (USV) during straight-line navigation, such as a constant-speed cruise strategy or an "acceleration-constant-deceleration" strategy depending on the distance. Specific driving methods are determined by the relevant engineers. The navigation fuel consumption value refers to the total amount of fuel consumed by the USV after traveling the required distance according to the navigation acceleration strategy. The navigation duration refers to the time required for the USV to travel the required distance according to the navigation acceleration strategy. The turning strategy refers to the rudder angle and speed control rules adopted by the USV when changing course, such as a minimum radius turning strategy or a slow, constant-speed turning strategy for smoother navigation. Specific methods are determined by the relevant engineers. The turning fuel consumption value refers to the total amount of fuel consumed by the USV after adjusting its direction according to the turning strategy. The turning duration refers to the time required for the USV to adjust its direction according to the turning strategy. The total navigation fuel consumption value is the sum of the navigation fuel consumption value and the turning fuel consumption value. The total navigation duration is the sum of the navigation duration and the turning duration. The weighted calculation of total fuel consumption and total travel time refers to the weighted summation of total fuel consumption and total travel time according to the weight of fuel consumption and the weight of travel time. The sum of the weight of fuel consumption and the weight of travel time is 1, and their specific values ​​are set manually.

[0167] In detail, the step of performing local optimal suppression perturbation on the single-time optimal solution vector to obtain the updated optimal solution vector includes:

[0168] Get the current iteration number;

[0169] The optimal solution vector for a single iteration is perturbed based on the current iteration number and the maximum iteration number to obtain the updated optimal solution vector, which is expressed as:

[0170]

[0171] in, This indicates updating the optimal solution vector. This represents the vector of the single optimal solution. Indicates the current iteration number. Indicates the maximum number of iterations. This represents the pre-defined Lorentz distribution function. This represents the predefined standard normal distribution function.

[0172] It should be explained that the purpose of introducing the Lorentz distribution function and the standard normal distribution function in the above formula is to implement an adaptive perturbation strategy. This strategy uses a weighting coefficient that varies with time. and To balance global exploration and local search: In the early stages of algorithm iteration, at the current iteration number... When the value is small, the coefficient corresponding to the Lorentz distribution The coefficients are relatively large, and their distribution has the characteristic of generating large perturbations, which helps the algorithm to conduct extensive global exploration in the solution space and avoid getting trapped in local optima too early. As the iteration progresses, the coefficients... As the value increases, the effect of the standard normal distribution strengthens, and the resulting perturbations become more concentrated and milder. This helps the algorithm to perform fine-grained local searches in the later stages, while retaining a certain degree of randomness to help escape possible local optima, thereby increasing the likelihood of finding the global optimum.

[0173] To address the problems described in the background section, this invention first acquires remote sensing satellite images based on the current coordinates of the unmanned surface vessel, its current navigation direction, and the target mission coordinates. Edge recognition is then performed on these satellite images to obtain obstacle contour images. This step, by incorporating remote sensing satellite images, allows for the early acquisition of obstacle information over a large area of ​​water and enables edge recognition, thereby enhancing the foresight of path planning, reducing reliance on real-time perception, and improving overall cruise efficiency. Traditional methods primarily rely on radar or sonar for real-time obstacle detection, which has a limited range and is often delayed. Furthermore, the obstacle contour images undergo contour boundary smoothing to obtain obstacle boundary images. This step, through contour boundary smoothing, adaptively eliminates invalid details based on boundary curvature and remote sensing resolution, simplifying obstacle boundaries and significantly reducing the complexity of path planning, thus improving computational efficiency. In contrast, traditional edge recognition methods... Obstacle outlines often contain subtle bumps and depressions caused by noise, leading to numerous unnecessary detours in path planning. This solution then performs path feasibility planning based on obstacle boundary images, radar systems, and sonar systems, generating a coordinate navigation dataset. This step integrates radar and sonar systems for accessibility assessment, dynamically filtering non-threatening obstacles and generating the coordinate navigation dataset to ensure path feasibility and improve the accuracy and safety of path planning. Traditional path planning often statically processes obstacles, ignoring the risk of misjudgment of dynamic objects or phantoms. Finally, this solution selects the optimal path based on the coordinate navigation dataset, obtaining the optimal navigation path. This step employs the Parrot optimization algorithm combined with a local optimum disturbance suppression strategy, adaptively balancing global exploration and local search to achieve multi-objective optimization, thereby generating an optimal path with low fuel consumption, short time, and safety, improving cruise performance. Therefore, this invention can improve the globality and efficiency of unmanned surface vessel (USV) path planning, reducing fuel consumption and travel time during USV cruise.

[0174] like Figure 2 The diagram shown is a functional block diagram of an unmanned vessel intelligent cruise system based on multimodal perception provided in an embodiment of the present invention.

[0175] The multimodal perception-based unmanned vessel intelligent navigation system 100 described in this invention can be installed in an electronic device. Depending on the functions implemented, the multimodal perception-based unmanned vessel intelligent navigation system 100 may include a navigation direction determination module 101, a boundary image acquisition module 102, a navigation data construction module 103, and an optimal path selection module 104. The module described in this invention can also be called a unit, referring to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0176] The navigation direction determination module 101 is used to identify the unmanned vessel and receive navigation mission instructions. The unmanned vessel includes a radar system, a sonar system and a positioning system. It obtains the target mission coordinates based on the navigation mission instructions, uses the positioning system to determine the current coordinates of the unmanned vessel, and determines the current navigation direction based on the current coordinates of the unmanned vessel and the target mission coordinates.

[0177] The boundary image acquisition module 102 is used to acquire remote sensing satellite images based on the current coordinates of the unmanned vessel, the current navigation direction and the target mission coordinates, perform edge recognition on the remote sensing satellite images to obtain obstacle contour images, and perform contour boundary smoothing processing on the obstacle contour images to obtain obstacle boundary images.

[0178] The navigation data construction module 103 is used to perform path feasibility planning based on obstacle boundary images, radar systems and sonar systems to obtain a coordinate navigation dataset. The coordinate navigation dataset includes multiple coordinate navigation data, and the coordinate navigation data includes: obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be navigated and direction adjustment angle.

[0179] The optimal path selection module 104 is used to select the optimal path based on the coordinate navigation dataset to obtain the optimal navigation path.

[0180] In detail, the modules in the multimodal perception-based unmanned vessel intelligent cruise system 100 described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The method uses the same technical means as the multimodal perception-based intelligent cruise method for unmanned vessels described in the article, and can produce the same technical effects, so it will not be repeated here.

[0181] like Figure 3 The diagram shown is a structural schematic of an electronic device for implementing a multimodal perception-based intelligent cruise method for unmanned vessels, according to an embodiment of the present invention.

[0182] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a method program for intelligent cruise of unmanned ships based on multimodal perception.

[0183] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of an unmanned vessel intelligent navigation method program based on multimodal perception, but also to temporarily store data that has been output or will be output.

[0184] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a multimodal perception-based intelligent cruise method program for unmanned vessels) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0185] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0186] Figure 3 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0187] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management system, thereby enabling functions such as charging management, discharging management, and power consumption management through the power management system. The power supply may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0188] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0189] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0190] The unmanned vessel intelligent navigation method program based on multimodal perception, stored in the memory 11 of the electronic device 1, is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0191] The unmanned vessel is identified and navigation mission instructions are received. The unmanned vessel includes a radar system, a sonar system, and a positioning system.

[0192] The target mission coordinates are obtained based on the navigation mission instructions, the current coordinates of the unmanned vessel are determined using the positioning system, and the current navigation direction is determined based on the current coordinates of the unmanned vessel and the target mission coordinates.

[0193] Based on the current coordinates of the unmanned vessel, the current navigation direction, and the coordinates of the target mission, remote sensing satellite images are acquired, and edge recognition is performed on the remote sensing satellite images to obtain obstacle outline images;

[0194] The obstacle contour image is smoothed to obtain the obstacle boundary image;

[0195] Based on obstacle boundary images, radar system, and sonar system, path feasibility planning is performed to obtain a coordinate navigation dataset. The coordinate navigation dataset includes multiple coordinate navigation data, which include: obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be navigated, and direction adjustment angle.

[0196] The optimal route is selected based on the coordinate navigation dataset, and the optimal navigation route is obtained. The intelligent cruise of the unmanned vessel based on multimodal perception is completed based on the optimal navigation route.

[0197] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0198] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0199] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0200] The unmanned vessel is identified and navigation mission instructions are received. The unmanned vessel includes a radar system, a sonar system, and a positioning system.

[0201] The target mission coordinates are obtained based on the navigation mission instructions, the current coordinates of the unmanned vessel are determined using the positioning system, and the current navigation direction is determined based on the current coordinates of the unmanned vessel and the target mission coordinates.

[0202] Based on the current coordinates of the unmanned vessel, the current navigation direction, and the coordinates of the target mission, remote sensing satellite images are acquired, and edge recognition is performed on the remote sensing satellite images to obtain obstacle outline images;

[0203] The obstacle contour image is smoothed to obtain the obstacle boundary image;

[0204] Based on obstacle boundary images, radar system, and sonar system, path feasibility planning is performed to obtain a coordinate navigation dataset. The coordinate navigation dataset includes multiple coordinate navigation data, which include: obstacle detour sequence number, target navigation coordinates, starting navigation coordinates, distance to be navigated, and direction adjustment angle.

[0205] The optimal route is selected based on the coordinate navigation dataset, and the optimal navigation route is obtained. The intelligent cruise of the unmanned vessel based on multimodal perception is completed based on the optimal navigation route.

[0206] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0207] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0208] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0209] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multi-modal perception based intelligent cruise method for unmanned ship, characterized in that, The method comprises: Confirming an unmanned ship and receiving a navigation task instruction, wherein the unmanned ship comprises a radar system, a sonar system and a positioning system; Obtaining target task coordinates based on the navigation task instruction, determining current unmanned ship coordinates of the unmanned ship by using the positioning system, and confirming a current navigation direction according to the current unmanned ship coordinates and the target task coordinates; Obtaining a remote sensing satellite image according to the current unmanned ship coordinates, the current navigation direction and the target task coordinates, performing edge recognition on the remote sensing satellite image, and obtaining an obstacle contour image; Performing contour boundary smoothing processing on the obstacle contour image to obtain an obstacle boundary image; Performing path feasibility planning according to the obstacle boundary image, the radar system and the sonar system to obtain a coordinate navigation data set, wherein the coordinate navigation data set comprises a plurality of coordinate navigation data, and each coordinate navigation data comprises an obstacle bypassing serial number, target navigation coordinates, starting navigation coordinates, a distance to be navigated and a direction adjustment angle; Performing optimal path selection based on the coordinate navigation data set to obtain an optimal navigation path, and completing intelligent cruising of the unmanned ship based on multi-modal perception based on the optimal navigation path; The optimal path selection based on the coordinate navigation data set to obtain the optimal navigation path comprises: Generating a parrot population based on a preset constraint condition, wherein the parrot population comprises a plurality of parrot individuals, each parrot individual corresponds to a candidate solution vector, the candidate solution vector comprises a plurality of candidate solution coordinates, and the number of candidate solution coordinates is the same as the maximum value of the obstacle bypassing serial number; Iterating the parrot population based on the coordinate navigation data set to obtain a single optimal solution vector; Performing local optimal suppression disturbance on the single optimal solution vector to obtain an updated optimal solution vector; Performing multiple iterations based on the preset maximum iteration number and the updated optimal solution vector to obtain a global optimal solution vector; Extracting a plurality of optimal solution coordinates in the global optimal solution vector, and formulating an optimal navigation path based on the plurality of optimal solution coordinates.

2. The unmanned ship intelligent cruising method based on multi-modal perception of claim 1, wherein, The contour boundary smoothing processing on the obstacle contour image to obtain the obstacle boundary image comprises: Confirming a plurality of contour boundaries in the obstacle contour image; Extracting contour boundaries in the plurality of contour boundaries in sequence, and recording the extracted contour boundaries as to-be-smoothed boundaries; Obtaining a remote sensing resolution of the obstacle contour image; Calculating the tortuosity of the to-be-smoothed boundaries to obtain boundary tortuosity, and calculating a minimum tortuosity distance based on the remote sensing resolution and the boundary tortuosity; Performing smoothing operation on the to-be-smoothed boundaries according to the minimum tortuosity distance to obtain smoothed boundaries; Summarizing the smoothed boundaries corresponding to each contour boundary in the plurality of contour boundaries to obtain a plurality of smoothed boundaries; Updating the obstacle contour image by using the plurality of smoothed boundaries to obtain the obstacle boundary image. 3.The unmanned ship intelligent cruising method based on multi-modal perception of claim 2, wherein, The smoothing operation on the to-be-smoothed boundaries according to the minimum tortuosity distance to obtain the smoothed boundaries comprises: Confirming a set of boundary pixel points in the to-be-smoothed boundaries, wherein the set of boundary pixel points comprises a plurality of boundary pixel points; Identifying a boundary starting point and a boundary ending point in the set of boundary pixel points; Constructing a boundary straight line segment based on the boundary starting point and the boundary ending point; Calculating the pixel distance of each boundary pixel point in the set of boundary pixel points to the boundary straight line segment to obtain a set of pixel distances; Identify the maximum distance in the pixel distance set, and record the boundary pixel point corresponding to the maximum distance as a far-end pixel point; Determine whether the maximum distance is less than the minimum tortuosity distance; If the maximum distance is not less than the minimum tortuosity distance, segment the to-be-smoothed boundary based on the far-end pixel point to obtain a plurality of repeated non-smoothed boundaries; In the plurality of repeated non-smoothed boundaries, extract the repeated non-smoothed boundaries in turn, take the extracted repeated non-smoothed boundary as the to-be-smoothed boundary, and return to the step of confirming the set of boundary pixel points in the to-be-smoothed boundary until the maximum distance is less than the minimum tortuosity distance; If the maximum distance is less than the minimum tortuosity distance, aggregate the boundary straight line segments to obtain a plurality of boundary straight line segments, and connect the plurality of boundary straight line segments to obtain a smoothed boundary.

4. The unmanned ship intelligent cruising method based on multi-modal perception of claim 3, wherein, The path feasibility planning based on the obstacle boundary image, the radar system and the sonar system obtains a coordinate navigation data set, including: Identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current unmanned ship coordinates; Using the radar system and the sonar system to perform passable evaluation on the nearest obstacle to obtain an evaluation result, wherein the evaluation result includes: passable and impassable; If the evaluation result is passable, the nearest obstacle is excluded from the obstacle boundary image to obtain an updated boundary image, the updated boundary image is taken as the obstacle boundary image, and the step of identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current unmanned ship coordinates is returned until the evaluation result is impassable; If the evaluation result is impassable, the nearest obstacle is recorded as a to-be-avoided obstacle; Based on the to-be-avoided obstacle, perform passable feature point identification in the obstacle boundary image to obtain a plurality of passable feature coordinates; In the plurality of passable feature coordinates, extract the passable feature coordinates in turn, and mark the extracted passable coordinates as target navigation coordinates; Confirm the target navigation direction based on the target navigation coordinates and the current unmanned ship coordinates; Based on the target navigation coordinates, construct coordinate navigation data, wherein the coordinate navigation data includes: obstacle avoidance sequence number, target navigation coordinates, starting navigation coordinates, to-be-navigated distance, and direction adjustment angle; Take the target navigation coordinates and the target navigation direction in the coordinate navigation data as the current unmanned ship coordinates and the current navigation direction respectively, and return to the step of identifying the nearest obstacle in the obstacle boundary image based on the current navigation direction and the current unmanned ship coordinates until the passable feature coordinates in the plurality of passable feature coordinates are extracted completely; Aggregate the coordinate navigation data corresponding to the passable feature coordinates to obtain a coordinate navigation data set, wherein the coordinate navigation data set includes a plurality of coordinate navigation data.

5. The unmanned ship intelligent cruising method based on multi-modal perception of claim 4, wherein, The step of constructing coordinate navigation data based on the target navigation coordinates includes: Set the to-be-avoided obstacle to obtain an obstacle avoidance sequence number; Calculate the direction adjustment angle according to the target navigation direction and the current navigation direction, and obtain the to-be-navigated distance between the target navigation coordinates and the current unmanned ship coordinates; Mark the current unmanned ship coordinates as starting navigation coordinates; Merge the obstacle avoidance sequence number, the target navigation coordinates, the starting navigation coordinates, the to-be-navigated distance, and the direction adjustment angle to obtain the coordinate navigation data. 6.The unmanned ship intelligent cruising method based on multi-modal perception of claim 5, wherein, The parrot population is iterated based on the coordinate navigation data set to obtain a single optimal solution vector, including: Each candidate solution vector in the parrot population performs the following operations: Extract the candidate solution coordinates in the candidate solution vector in turn, and mark the extracted candidate solution coordinates as target solution coordinates; Identify the target navigation data corresponding to the target solution coordinates in the multiple coordinate navigation data of the coordinate navigation data set; Aggregate the target navigation data corresponding to the candidate solution coordinates to obtain multiple target navigation data; Based on the multiple target navigation data, the fitness is calculated to obtain the candidate solution fitness; Aggregate the candidate solution fitness to obtain multiple candidate solution fitness, and confirm the single optimal solution vector in the parrot population based on the multiple candidate solution fitness.

7. The unmanned ship intelligent cruising method based on multi-modal perception of claim 6, wherein, The fitness is calculated based on the multiple target navigation data to obtain the candidate solution fitness, including: Extract the target navigation data in the multiple target navigation data in turn, and mark the extracted target navigation data as the to-be-evaluated navigation data; According to the preset navigation acceleration strategy, the to-be-traveled distance in the to-be-evaluated navigation data is respectively evaluated for navigation fuel consumption and navigation time length to obtain the navigation fuel consumption value and the navigation time length; According to the preset steering strategy, the direction adjustment angle in the to-be-evaluated navigation data is respectively evaluated for steering fuel consumption and steering time length to obtain the steering fuel consumption value and the steering time length; Based on the navigation fuel consumption value, the steering fuel consumption value, the navigation time length and the steering time length, the total navigation fuel consumption value and the total navigation time length are calculated; The total navigation fuel consumption value and the total navigation time length are weighted to obtain the candidate solution fitness. 8.The unmanned ship intelligent cruising method based on multi-modal perception of claim 7, wherein, The single optimal solution vector is subjected to local optimal suppression disturbance to obtain an updated optimal solution vector, including: Obtain the current iteration number; Based on the current iteration number and the maximum iteration number, the single optimal solution vector is disturbed to obtain the updated optimal solution vector, wherein the updated optimal solution vector is represented as: wherein, denotes updating the optimal solution vector, denotes a single optimal solution vector, denotes the current iteration number, denotes the maximum iteration number, denotes a preset Lorentz distribution function, denotes a preset standard normal distribution function.

9. A system using the unmanned ship intelligent cruising method based on multi-modal perception according to any one of claims 1 to 8, characterized in that, The system includes: A navigation direction determination module for confirming an unmanned ship and receiving a navigation task instruction, wherein the unmanned ship includes a radar system, a sonar system and a positioning system, obtaining target task coordinates based on the navigation task instruction, determining the current unmanned ship coordinates of the unmanned ship using the positioning system, and confirming the current navigation direction according to the current unmanned ship coordinates and the target task coordinates; A boundary image acquisition module for acquiring a remote sensing satellite image according to the current unmanned ship coordinates, the current navigation direction and the target task coordinates, performing edge recognition on the remote sensing satellite image to obtain an obstacle contour image, and performing contour boundary smoothing processing on the obstacle contour image to obtain an obstacle boundary image; A navigation data construction module for performing path feasibility planning based on the obstacle boundary image, the radar system and the sonar system to obtain a coordinate navigation data set, wherein the coordinate navigation data set includes multiple coordinate navigation data, and the coordinate navigation data includes: obstacle bypass serial number, target navigation coordinates, starting navigation coordinates, to-be-traveled distance and direction adjustment angle; An optimal path selection module for selecting an optimal path based on the coordinate navigation data set to obtain an optimal navigation path.

Images