Ship collision early warning method based on radar detection and electronic equipment

Abstract

The invention relates to a radar detection-based ship collision early warning method and electronic equipment. The method comprises the steps of obtaining multi-source information of a target ship; carrying out information fusion processing on the multi-source information, constructing a marine environment multi-dimensional three-dimensional situation map, and identifying a specified target; determining a to-be-collided threat obstacle according to the target recognition result, and creating a bounding box model corresponding to the to-be-collided threat obstacle; based on a global nearest neighbor data association algorithm and the established bounding box model, associating the barrier data with the global nearest neighbor data association algorithm based on the model; according to the correlation result of the obstacle data, calculating speed parameters of the obstacle objects of the continuous frames, and constructing an obstacle motion feature model based on the speed parameters and a bounding box model; and verifying the constructed obstacle motion feature model by adopting a TEB algorithm so as to complete ship obstacle detection and obstacle avoidance. According to the invention, the ship collision detection efficiency and accuracy are improved, so that early warning can be carried out in time.

Description

Technical Field

[0001] The present application relates to the technical field of ship collision warning, and in particular to a ship collision warning method and electronic equipment based on radar detection. Background Art

[0002] Shipping is an important part of international trade, and the shipping industry is the foundation of global trade development. Coastal port waterways are the main arteries of shipping traffic, and their traffic capacity directly affects the operational efficiency and economic benefits of ports. When ships enter and leave the port, ensuring a safe distance is the most important task, which is related to the safety of ships and crew members, and also involves the normal operation and service quality of the port. Therefore, how to improve the collision avoidance decision-making ability of ships and reduce the risk of accidents and the losses caused by accidents has become a key and difficult issue in current research.

[0003] In addition, the navigation control process is very complex. How to enable ships to clearly and accurately recognize their navigation status, accurately define the safe area with other navigation entities, quantify the risk of collision, and reasonably judge whether collision avoidance is necessary and when to avoid collision requires the coordination of multiple technologies to build a safe and effective autonomous navigation decision-making system. For this, the International Maritime Organization has formulated the International Regulations for Preventing Collisions at Sea (COLREGs), which is an important maritime rule or navigation regulation. It provides a systematic reference for the existing autonomous navigation decision-making system.

[0004] like Figure 1 As shown, the autonomous navigation decision system in the prior art is divided into four stages:

[0005] 1) Navigation environment perception stage: Use multiple types of sensors (such as radar, AIS, lidar, infrared camera, visible light camera, etc.) to obtain the environment around the ship, including dynamic and static information such as the ship's position, speed, and heading, as well as environmental information (such as waterways, obstacles, etc.).

[0006] 2) Route planning stage: Based on the perceived information of surrounding ships and the environment, multiple navigation paths for the ship are formulated to avoid collisions with other ships and obstacles.

[0007] 3) Decision-making and control execution stage: Select the optimal path from multiple navigation paths of the ship, convert the optimal path into control instructions, and control the ship's heading, speed and navigation trajectory through the control system to achieve autonomous navigation and collision avoidance.

[0008] 4) Feedback and adjustment stage. During autonomous navigation, it is necessary to continuously monitor the surrounding ships and environment so as to adjust the navigation strategy of the ship in a timely manner. At the same time, the effect of autonomous navigation is evaluated to verify the performance and accuracy of the collision avoidance decision algorithm and control system.

[0009] Although the existing autonomous navigation decision-making system relies on auxiliary equipment such as automatic radar plotters, ship traffic services, and integrated bridge systems to obtain the navigation environment more accurately, there are still many problems. For example, most decision-making systems do not take into account the impact of environmental factors and distance conditions on the equipment, resulting in a high risk of collision accidents and poor intelligent collision avoidance decision-making capabilities of ships. Summary of the invention

[0010] The present application provides a ship collision warning method and electronic equipment based on radar detection, which are used to improve the efficiency and accuracy of ship collision detection and provide timely warning.

[0011] In a first aspect, the present application provides a ship collision warning method based on radar detection, comprising:

[0012] S1: Acquire multi-source information of the target ship, wherein the multi-source information includes motion elements, environmental elements, ship static elements and surrounding ship influence elements;

[0013] S2: performing information fusion processing on the multi-source information, constructing a multi-dimensional three-dimensional situation map of the ocean environment, and performing identification of the designated target;

[0014] S3: Determine the threatening obstacle to be collided with according to the target recognition result, and create a bounding box model corresponding to the threatening obstacle to be collided with;

[0015] S4: Based on the global nearest neighbor data association algorithm and the established bounding box model, the model-based global nearest neighbor data association algorithm is used to associate the obstacle data;

[0016] S5: calculating the speed parameters in the motion features of the obstacle objects in the continuous frames through the association results of the obstacle data, and constructing the obstacle motion feature model based on the speed parameters and the bounding box model;

[0017] S6: The TEB algorithm in the local path planning algorithm is used to verify the constructed obstacle motion feature model to complete ship obstacle detection and avoidance.

[0018] In a possible implementation, the multi-source information is fused and processed to construct a multi-dimensional situation map of the ocean environment, and the identification of the designated target includes:

[0019] First, feature extraction is performed on SAR and HRRP respectively, and then the features are concatenated and fused before the classification layer. AIS is trained separately, and the feature extraction and classification networks of HRRP and SAR are trained synchronously and the network parameters are updated, and finally they are fused with AIS at the decision level.

[0020] In a possible implementation, determining the threatening obstacle to be collided with according to the target recognition result and creating a bounding box model corresponding to the threatening obstacle to be collided with includes:

[0021] After segmenting the point cloud using the connected component clustering algorithm, we obtain multiple different point cloud clusters generated by obstacle data. Each cluster represents the relative position of an obstacle and the radar and displays the contour features of the object. We use the bounding box to extract its geometric features and construct a geometric feature model of the obstacle.

[0022] In a possible implementation, determining the threatening obstacle to be collided with according to the target recognition result, and creating a bounding box model corresponding to the threatening obstacle to be collided with further includes:

[0023] OBB (oriented bounding box) and OBEC (oriented elliptical cylinder bounding box) are used to describe the point cloud features of obstacles. OBB is used to geometrically describe ship obstacles, while OBEC is used to geometrically describe the obstacle point cloud for other obstacles.

[0024] In a possible implementation, the geometric description of the ship obstacle using OBB includes:

[0025] The obstacle point cloud is projected onto a two-dimensional plane, and then the principal component direction of each obstacle point cloud cluster is obtained through PCA. The geometric features of the obstacle are described using the OBB bounding box. Each data point is set as P i =(x i ,y i ), then the point cloud cluster can be regarded as {P1, P2, P3, ..., P n}, a data set consisting of n points, first calculate the mean m:

[0026]

[0027] Construct the covariance matrix of the point cloud in the x and y directions:

[0028]

[0029] The covariance matrix is ​​a symmetric matrix consisting of the following three elements:

[0030]

[0031] Find the maximum eigenvalues ​​λ1 and λ2 of the covariance matrix C and their corresponding eigenvectors ξ1 and ξ2, and then perform orthogonal normalization on each eigenvector; the eigenvector λ1 corresponding to the largest eigenvalue is the main direction of the OBB bounding box, and the maximum and minimum values ​​of the point cloud in the main direction are obtained, and then the length, width and center coordinates of the bounding box are obtained as follows:

[0032] L = x′ max -x′ min (6)

[0033] W=y′ max -y′ min (7)

[0034]

[0035] The difference between the highest point and the lowest point of the obstacle point cloud is used as the Z-axis height of the OBB bounding box to establish a geometric feature model that fits the obstacle point cloud.

[0036] In one possible implementation, the geometric description of the obstacle point cloud using OBEC includes:

[0037] First, initialize, set the initial weight vector w, calculate the weighted mean center c and weighted covariance matrix C:

[0038]

[0039] Calculate the Mahalanobis distance d from each point to the center c i :

[0040]

[0041] The shape and size of the ellipse are changed by iteratively adjusting the weights until the smallest ellipse is found, which contains all the points. The major axis of the ellipse is the rotation direction of the obstacle. Then, this smallest circumscribed ellipse is stretched into an elliptical cylinder in the Z-axis direction with the maximum height difference of the point cloud to obtain the OBEC bounding box.

[0042] In a possible implementation, based on the global nearest neighbor data association algorithm and the established bounding box model, the model-based global nearest neighbor data association algorithm is used to associate obstacle data, including:

[0043] Let σ be a one-to-one mapping from the current frame to the set of objects in the previous frame, and σ(i) represents the current frame object associated with target i. Then the association problem is formalized as an assignment problem, that is, finding a mapping from a target to the current frame object to minimize the total cost:

[0044]

[0045] For each possible data association, calculate its Mahalanobis distance and establish a cost matrix; suppose there are m obstacles in the previous frame point cloud data after clustering and segmentation and establishing the obstacle geometric feature model, and there are n obstacles in the current frame point cloud data, establish the association matrix:

[0046]

[0047] According to the above matrix, data association can be achieved by obtaining the minimum cost.

[0048] In a possible implementation, calculating the speed parameters in the motion features of the obstacle objects in the consecutive frames through the association results of the obstacle data, and constructing the obstacle motion feature model based on the speed parameters and the bounding box model includes:

[0049] Based on the above-mentioned association results of obstacle data between frames, the speed parameters in the motion characteristics of obstacle objects in consecutive frames are calculated. Suppose the obstacle speed parameter of the current frame is

[0050]

[0051] Where T is the working cycle of the laser radar, Δd represents the relative distance difference of the obstacle object in the coordinate system between adjacent frames; according to the obstacle data between consecutive frames, the speed of the obstacle in the coordinate system can be obtained, and then according to the current speed of the moving ship, the actual speed of the obstacle can be obtained:

[0052]

[0053] Speed ​​of passing obstacles in world coordinates To determine whether the obstacle is a static obstacle, as shown below:

[0054]

[0055] μ velo is the speed threshold of the obstacle. When the speed of the obstacle at time T is less than the speed threshold, the obstacle is regarded as a static obstacle and is not added to the dynamic obstacle list. Otherwise, it is regarded as a dynamic obstacle and the dynamic obstacle list is updated.

[0056] In a conventional navigation environment, the movement of obstacles detected by the ship can be regarded as the movement in the plane coordinate system composed of the X-axis and the Y-axis. According to the established obstacle geometric feature model, the obstacle plane expression is established; for the OBEC bounding box model, the major axis a′ of the ellipse obstacle With minor axis b′ obstacle Can be expressed as:

[0057]

[0058] Among them, θ′ obstacle Indicates the angle between the obstacle and the Y axis in OBEC mode. (x obstacle ,y obstacle ) represents the coordinates of the obstacle center, X obstacle With Y obstacle Two variables representing obstacles;

[0059] For the OBB bounding box model, the length and width of the obstacle are expressed as: 2a″ obstacle With 2b″ obstacle ; In this case, the obstacle can be expressed as four linear equations:

[0060] X obstacle cosθ″ obstacle -Y obstacle sinθ″ obstacle =-a″ obstacle (18)

[0061] X obstacle cosθ″ obstacle -Y obstacle sinθ″ obstacle =a″ obstacle (19)

[0062] X obstacle sinθ″ obstacle +Y obstacle cosθ″ obstacle =-b″ obstacle (20)

[0063] X obstacle sinθ″ obstacle +Y obstacle cosθ″ obstacle =b″ obstacle (twenty one)

[0064] θ″ obstacle Indicates the angle between the obstacle and the Y axis of the OBB model. X obstacle With Y obstacle Two variables representing obstacles;

[0065] Thus, a mathematical model can be established to characterize the motion characteristics of ships and dynamic obstacles. The dynamic characteristics generally include the position, velocity, acceleration and other information of the dynamic obstacles. Therefore, the dynamic obstacle motion characteristic model is proposed to be represented by the following function:

[0066]

[0067] Where (xobstacle (t), y obstacle (t)) represents the coordinate position of the obstacle center, and Respectively represent the velocity vector and acceleration vector of the dynamic obstacle, θ obstacle (t) represents the angle between the obstacle and the Y axis.

[0068] In a possible implementation, the TEB algorithm in the local path planning algorithm is used to verify the construction of the obstacle motion feature model to complete the ship obstacle detection and avoidance, including:

[0069] The initial path from the starting point to the target point is obtained through the global path planning algorithm, and a series of control points are inserted on the initial path to form an elastic band; each control point contains the position of the ship at that point and the estimated time of arrival at that point; secondly, the elastic band is optimized with multiple objectives, including minimizing the path length, avoiding obstacles, meeting speed and acceleration constraints, and satisfying non-holonomic kinematic constraints; the optimization process is achieved by adjusting the position and time information of the control points; speed, acceleration restrictions and obstacle avoidance requirements, as well as non-holonomic kinematic constraints, are handled by introducing constraints in the form of penalty functions;

[0070] The position and angle of each control point of the ship in the algorithm are as follows:

[0071]

[0072] Where (x i ,y i ) represents the coordinates of the i-th control point of the mobile ship, β i represents the heading angle of the i-th control point of the mobile ship. The time interval between adjacent control points is ΔT i Represents the time required for a mobile vessel to move from one control point to the next;

[0073] The optimization process aims to find the optimal pose sequence Q = {s0, s1, ..., s n} and the time interval sequence τ = {ΔT0, ΔT1, ..., ΔT n-1}, where B represents the combination of path and time series, where B = (Q, τ);

[0074] The optimization of the ship position and time variables by TEB is achieved through multiple constraints. For the ship path planning of this embodiment, two constraints are mainly considered: global path following and obstacle avoidance constraints and speed and acceleration constraints;

[0075] The global path following and obstacle avoidance constraints include: Let d min，j is the shortest distance between the jth control point and the ship, rpmax is the maximum distance from the global planning path, d min，k is the shortest distance between the kth obstacle and the ship, r omin is the shortest safe distance between the obstacle and the ship, then the objective function can be expressed as:

[0076] f path =f(d min，j , r p max ) (twenty four)

[0077] f obstacle =f(-d min，k , r o min ) (25)

[0078] The velocity and acceleration constraints include: between two adjacent control points, there is a time interval ΔT i The translation speed of the ship can be obtained:

[0079]

[0080] The constraint function of translation velocity is:

[0081] f v =f v，i (X i , X i+1 , ΔT i ) (27)

[0082] The acceleration of the ship is:

[0083]

[0084] The constraint function of acceleration is:

[0085] f a =f a，i (X i , X i+1 , X i+2 , ΔT i , ΔT i+1 ) (29).

[0086] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the method described in the first aspect and any possible implementation method of the first aspect.

[0087] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0088] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0089] This application fuses multi-source information, constructs a multi-dimensional three-dimensional situation map of the marine environment, and performs identification of designated targets, which can obtain more comprehensive information, improve accuracy and reliability, increase the credibility of decision-making, promote innovation and discovery, and make up for the limitations of a single information source. At the same time, this application optimizes the multi-source sensor data preprocessing method, constructs a multi-feature fused dynamic obstacle model based on point cloud, and tracks and predicts the position of dynamic obstacles based on the established model to achieve obstacle avoidance of dynamic obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a schematic diagram of the structure of the autonomous navigation decision-making system in the prior art;

[0091] Figure 2 is a flow chart of a ship collision warning method based on radar detection provided in an embodiment of the present application;

[0092] Figure 3 It is a flowchart of using HRRP and SAR feature-level fusion + AIS decision-level fusion in the embodiment of the present application. DETAILED DESCRIPTION

[0093] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0094] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0095] like Figure 2FIG. 1 is a flow chart of a ship collision warning method based on radar detection provided by an embodiment of the present invention. The method may specifically include:

[0096] Obtain multi-source information of the target ship including motion elements, environmental elements, ship static elements and surrounding ship influence elements;

[0097] Regarding the acquisition of multi-source information, in the face of the complexity and variability of the actual marine environment, using different sensors to obtain target information is the primary condition for the perception system to fully understand the marine environment. Commonly used sensor equipment for marine environment perception is radar, optoelectronics, and sonar. In this application, the optoelectronics, radar, and sonar sensors carried by ships and feature fusion technology are used to obtain image feature information, original point cloud information, and target sound wave information, so as to realize the comprehensive cognition of scene information by the intelligent perception system and the accurate identification of designated targets. However, the collection of marine information often has problems such as lack of data and low quality; on the other hand, the collected data also needs to meet the perception tasks such as detection and segmentation. Therefore, it is necessary to process the collected marine environment information. Specifically, this application uses the unique properties of sensors to perform information fusion processing on designated targets, construct a multi-dimensional three-dimensional situation map of the marine environment, and perform detection, identification, and tracking tasks on designated targets.

[0098] like Figure 3 As shown in FIG. 1 , this embodiment uses HRRP (high resolution range Profile, HRRP), AIS (automatic identification system, AIS) and SAR (synthetic aperture radar, SAR) to perform information fusion, wherein HRRP and SAR adopt a feature-level fusion strategy and then perform decision-level fusion with AIS. Since SAR and HRRP correspond one to one, firstly, SAR and HRRP are respectively subjected to feature extraction, and then the features are spliced ​​and fused before the classification layer, and AIS is trained separately, and the feature extraction and classification networks of HRRP and SAR are trained synchronously, and the network parameters are updated, and finally, decision-level fusion is performed with AIS.

[0099] The target recognition based on tri-modal data fusion not only has the advantages of strong fusion capability, good model generalization ability, and high target recognition accuracy, but also can seamlessly handle the problem of missing modalities. For embedded application environments, the model deployment is simple.

[0100] Further, the threatening obstacle to be collided is determined according to the target recognition result, and a bounding box model corresponding to the threatening obstacle to be collided is created;

[0101] After segmenting the point cloud using the connected component clustering algorithm, multiple different point cloud clusters generated by obstacle data are obtained. Each cluster represents the relative position of an obstacle and the radar, and displays the contour features of the object. In order to obtain geometric feature information such as the size, center point position, and movement direction of the obstacle, the bounding box is used to extract its geometric features and construct a geometric feature model of the obstacle.

[0102] A bounding box is a simple geometric body used to surround an object or geometric shape so that it can be quickly represented, calculated and processed. The shape of this box or bounding body can be a cube, a cuboid or other geometric shapes, depending on the needs of the application. The bounding box of a point cloud is usually a minimum axis-aligned cube or cuboid that completely surrounds the entire point cloud data or a subset of the point cloud. The six faces of the bounding box are parallel to the coordinate axes. At present, the commonly used bounding box for obstacles is a cubic bounding box, which consists of the minimum and maximum boundaries of the object in three directions. In this embodiment, OBB (directed bounding box) and OBEC (directed elliptical cylinder bounding box) are used to describe the obstacle point cloud features. Among them, since common ship obstacles are usually other ships, which are similar to square obstacles, OBB is used to geometrically describe the obstacle point cloud; for other obstacles, OBEC is used to geometrically describe the obstacle point cloud.

[0103] Specifically, the collision threat obstacles usually move on the horizontal plane, so the obstacle point cloud is projected onto the two-dimensional plane, and then the principal component direction of each obstacle point cloud cluster is obtained through PCA, and the OBB bounding box is used to describe the geometric characteristics of the obstacle. Each data point is set as P i =(x i ,y i ), then the point cloud cluster can be regarded as {P1, P2, P3, ..., P n}, a data set consisting of n points, first calculate the mean m:

[0104]

[0105] Construct the covariance matrix of the point cloud in the x and y directions:

[0106]

[0107] The covariance matrix is ​​a symmetric matrix consisting of the following three elements:

[0108]

[0109]

[0110] Find the maximum eigenvalues ​​λ1 and λ2 of the covariance matrix C and their corresponding eigenvectors ξ1 and ξ2, and then perform orthogonal normalization on each eigenvector. The eigenvector λ1 corresponding to the largest eigenvalue is the main direction of the OBB bounding box, and the maximum and minimum values ​​of the point cloud in the main direction are obtained, and then the length, width and center coordinates of the bounding box are obtained as follows:

[0111] L = x′ max -x′ min (6)

[0112] W=y′ max -y′ min (7)

[0113]

[0114] The difference between the highest point and the lowest point of the obstacle point cloud is used as the Z-axis height of the OBB bounding box to establish a geometric feature model that is more consistent with the obstacle point cloud.

[0115] OBEC is based on the Khachiyan algorithm, which solves the minimum circumscribed ellipse of the obstacle's two-dimensional plane projection point set, and then uses the height difference of the point cloud cluster as the height of the elliptical cylinder to establish the minimum circumscribed elliptical cylinder bounding box of the obstacle point cloud cluster. The Khachiyan algorithm, also known as the minimum volume enclosing ellipsoid algorithm, is an effective method for solving the minimum circumscribed ellipse. The algorithm approximates the minimum circumscribed ellipse through iterative optimization, so that all given points are contained in the ellipse and the area of ​​the ellipse is as small as possible. First, initialize, set the initial weight vector w, and calculate the weighted mean center c and weighted covariance matrix C:

[0116]

[0117] Calculate the Mahalanobis distance d from each point to the center c i :

[0118]

[0119] The shape and size of the ellipse are changed by iteratively adjusting the weights until the smallest ellipse is found, which contains all the points. The major axis of the ellipse is the rotation direction of the obstacle. Then, this smallest circumscribed ellipse is stretched to an elliptical cylinder in the Z-axis direction with the maximum height difference of the point cloud to obtain the OBEC bounding box.

[0120] In the process of establishing the bounding box, filtering parameters are set to filter out irrelevant target objects. In this embodiment, in addition to the three standard dimensions of length, width, and height of the obstacle, the ratio of the length to the width of the obstacle must also be considered to eliminate objects that are too slender and do not fall within the range of obstacles that need to be considered, such as floating water plants.

[0121] Furthermore, in order to improve the accuracy of dynamic target tracking, this embodiment associates obstacle data with a model-based global nearest neighbor data association algorithm based on a global nearest neighbor data association algorithm and an established obstacle feature model. A conventional nearest neighbor association algorithm associates the measurement of the current frame closest to the previous frame object with it. This distance generally refers to the Euclidean distance or the Mahalanobis distance. The basic principle of the global nearest neighbor association algorithm is similar to that of the nearest neighbor association algorithm, but it improves the association process. Let σ be a one-to-one mapping from the current frame to the previous frame object set, and σ(i) represents the current frame object associated with target i. Then the association problem is formalized into an allocation problem, that is, finding a mapping from a target to the current frame object to minimize the total cost:

[0122]

[0123] For each possible data association, calculate its Mahalanobis distance and establish a cost matrix. Assume that there are m obstacles in the previous frame point cloud data after clustering and segmentation and establishing the obstacle geometric feature model, and there are n obstacles in the current frame point cloud data, and establish the association matrix:

[0124]

[0125] According to the above matrix, data association can be achieved by obtaining the minimum cost.

[0126] The above methods have obtained the geometric characteristic parameters of obstacles and established a geometric characteristic model for them, and the acquisition of the motion characteristic parameters of obstacles is also crucial. Obtaining the motion characteristic parameters of obstacles helps to perceive and understand the environment more accurately and helps the system to predict behavior and plan paths.

[0127] Based on the above-mentioned association results of obstacle data between frames, the speed parameters in the motion characteristics of obstacle objects in consecutive frames are calculated. Suppose the obstacle speed parameter of the current frame is

[0128]

[0129] Where T is the working cycle of the laser radar, and Δd represents the relative distance difference of the obstacle object in the coordinate system between adjacent frames. According to the obstacle data between consecutive frames, the speed of the obstacle in the coordinate system can be obtained, and then according to the current speed of the moving ship, the actual speed of the obstacle can be obtained:

[0130]

[0131] Speed ​​of passing obstacles in world coordinates To determine whether the obstacle is a static obstacle, as shown below:

[0132]

[0133] μ velo is the speed threshold of the obstacle. When the speed of the obstacle at time T is less than the speed threshold, the obstacle is regarded as a static obstacle and is not added to the dynamic obstacle list. Otherwise, it is regarded as a dynamic obstacle and the dynamic obstacle list is updated.

[0134] In a conventional navigation environment, the movement of obstacles detected by the ship can only be regarded as the movement in the plane coordinate system composed of the X-axis and the Y-axis. According to the obstacle geometric feature model established in the previous article, the obstacle plane expression is established. For the OBEC bounding box model, the major axis a′ of the ellipse obstacle With minor axis b′ obstacle Can be expressed as:

[0135]

[0136] Among them, θ′ obstacle Indicates the angle between the obstacle and the Y axis in OBEC mode. (x obstacle ,y obstacle ) represents the obstacle center coordinates, X obstacle With Y obstacle Two variables representing obstacles.

[0137] For the OBB bounding box model, the length and width of the obstacle are expressed as: 2a″ obstacle With 2b″ obstacle ; In this case, the obstacle can be expressed as four linear equations:

[0138] X obstacle cosθ″ obstacle -Y obstacle sinθ″ obstacle =-a″ obstacle (18)

[0139] X obstacle cosθ″ obstacle -Y obstacle sinθ″ obstacle =a″ obstacle (19)

[0140] X obstacle sinθ″ obstacle +Y obstacle cosθ″ obstacle =-b″ obstacle (20)

[0141] X obstacle sinθ″ obstacle +Yobstacle cosθ″ obstacle =b″ obstacle (twenty one)

[0142] θ″ obstacle Indicates the angle between the obstacle and the Y axis of the OBB model. X obstacle With Y obstacle Two variables representing obstacles.

[0143] Thus, a mathematical model can be established to characterize the motion characteristics of ships and dynamic obstacles. The dynamic characteristics generally include the position, velocity, acceleration and other information of the dynamic obstacles. Therefore, the dynamic obstacle motion characteristic model is proposed to be represented by the following function:

[0144]

[0145] Where (x obstacle (t), y obstacle (t)) represents the coordinate position of the obstacle center, and Respectively represent the velocity vector and acceleration vector of the dynamic obstacle, θ obstacle (t) represents the angle between the obstacle and the Y axis.

[0146] This embodiment uses the TEB algorithm in the local path planning algorithm to verify it to complete the ship obstacle detection and avoidance. The algorithm first obtains the initial path from the starting point to the target point through the global path planning algorithm, and inserts a series of control points on the initial path to form an elastic band. Each control point contains the position of the ship at that point and the estimated time of arrival at that point. Secondly, the elastic band is optimized with multiple objectives, including minimizing the path length, avoiding obstacles, meeting speed and acceleration constraints, and satisfying non-complete kinematic constraints. The optimization process is achieved by adjusting the position and time information of the control points. Constraints in the form of penalty functions are introduced to handle speed, acceleration restrictions and obstacle avoidance requirements, as well as non-complete kinematic constraints.

[0147] The position and posture of each control point of the ship in the algorithm are as follows:

[0148]

[0149] Where (x i ,y i ) represents the coordinates of the i-th control point of the mobile ship, β i represents the heading angle of the i-th control point of the mobile ship. The time interval between adjacent control points is ΔT i Represents the time required for a mobile ship to move from one control point to the next.

[0150] The goal of the optimization process is to find the optimal pose sequence Q = {s0, s1, ..., s n} and the time interval sequence τ = {ΔT0, ΔT1, ..., ΔT n-1}to minimize the total cost function F(B), where B represents the combination of path and time series, B=(Q,τ).

[0151] TEB optimizes the ship position and time variables through multiple constraints. The ship path planning in this embodiment mainly considers two constraints: global path following and obstacle avoidance constraints and speed and acceleration constraints.

[0152] The global path following and obstacle avoidance constraints include: Let d min，j is the shortest distance between the jth control point and the ship, r p max is the maximum distance from the global planning path, d min，k is the shortest distance between the kth obstacle and the ship, r omin is the shortest safe distance between the obstacle and the ship, then the objective function can be expressed as:

[0153] f path =f(d min，j , r p max ) (twenty four)

[0154] f obstacle =f(-d min，k , r o min ) (25).

[0155] The velocity and acceleration constraints include: between two adjacent control points, there is a time interval ΔT i The translation speed of the ship can be obtained:

[0156]

[0157] The constraint function of translation velocity is:

[0158] f v =f v，i (X i , X i+1 , ΔT i ) (27)

[0159] The acceleration of the ship is:

[0160]

[0161] The constraint function of acceleration is:

[0162] f a =f a，i (X i , Xi+1 , X i+2 , ΔT i , ΔT i+1 ) (29).

[0163] In the process of iterating each control point, the ship's orientation information is dynamically updated and optimized in a timely manner to ensure that the spatial and temporal resolution matches the length and range of the remaining planned trajectory. For the ship's motion control, the planning results, including acceleration, angular velocity, and specific path information, are accurately transmitted to the ship's controller to achieve accurate obstacle avoidance and navigation functions.

[0164] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0165] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0166] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0167] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A ship collision warning method based on radar detection, characterized in that: The following steps are involved: S1: Acquire multi-source information of the target ship, wherein the multi-source information includes motion elements, environmental elements, ship static elements and surrounding ship influence elements; S2: performing information fusion processing on the multi-source information, constructing a multi-dimensional three-dimensional situation map of the ocean environment, and performing identification of the designated target; S3: Determine the threatening obstacle to be collided with according to the target recognition result, and create a bounding box model corresponding to the threatening obstacle to be collided with; S4: Based on the global nearest neighbor data association algorithm and the established bounding box model, the model-based global nearest neighbor data association algorithm is used to associate the obstacle data; S5: calculating the speed parameters in the motion features of the obstacle objects in the continuous frames through the association results of the obstacle data, and constructing the obstacle motion feature model based on the speed parameters and the bounding box model; S6: The TEB algorithm in the local path planning algorithm is used to verify the constructed obstacle motion feature model to complete ship obstacle detection and avoidance.

2. The ship collision warning method based on radar detection according to claim 1 is characterized in that: The multi-source information is subjected to information fusion processing to construct a multi-dimensional three-dimensional situation map of the ocean environment, and the identification of the designated target is performed, including: First, feature extraction is performed on SAR and HRRP respectively, and then the features are concatenated and fused before the classification layer. AIS is trained separately, and the feature extraction and classification networks of HRRP and SAR are trained synchronously and the network parameters are updated, and finally they are fused with AIS at the decision level.

3. The ship collision warning method based on radar detection according to claim 1 is characterized in that: Determine the threatening obstacle to be collided according to the target recognition result, and create the bounding box model corresponding to the threatening obstacle to be collided, including: After segmenting the point cloud using the connected component clustering algorithm, multiple different point cloud clusters generated by obstacle data are obtained. Each cluster represents the relative position of an obstacle and the radar, and displays the contour features of the object. The geometric features of the obstacle are extracted using the bounding box and a geometric feature model of the obstacle is constructed.

4. The ship collision warning method based on radar detection according to claim 3 is characterized in that: Determining the threatening obstacle to be collided with based on the target recognition result and creating a bounding box model corresponding to the threatening obstacle to be collided with also includes: OBB and OBEC are used to describe obstacle point cloud features; Among them, OBB is used to geometrically describe the ship obstacle; for other obstacles, OBEC is used to geometrically describe the obstacle point cloud.

5. The ship collision warning method based on radar detection according to claim 4 is characterized in that: The geometric description of the ship obstacle by using OBB includes: The obstacle point cloud is projected onto a two-dimensional plane, and then the principal component direction of each obstacle point cloud cluster is obtained through PCA. The geometric features of the obstacle are described using the OBB bounding box. Each data point is set as P i =(x i ,y i ), then the point cloud cluster {P1, P2, P3, ..., P n } is a data set consisting of n points; First calculate the mean m: Construct the covariance matrix of the point cloud in the x and y directions: The covariance matrix is ​​a symmetric matrix consisting of the following three elements: Find the maximum eigenvalues ​​λ1 and λ2 of the covariance matrix C and their corresponding eigenvectors ξ1 and ξ2, and then perform orthogonal normalization on each eigenvector; the eigenvector λ1 corresponding to the largest eigenvalue is the main direction of the OBB bounding box, and the maximum and minimum values ​​of the point cloud in the main direction are obtained, and then the length, width and center coordinates of the bounding box are obtained as follows: L=x′ max -x′ min (6) W=y′ max -y′ min (7) The difference between the highest point and the lowest point of the obstacle point cloud is used as the Z-axis height of the OBB bounding box to establish a geometric feature model that fits the obstacle point cloud.

6. The ship collision warning method based on radar detection according to claim 5 is characterized in that: The geometric description of the obstacle point cloud using OBEC includes: First, initialize, set the initial weight vector w, calculate the weighted mean center c and weighted covariance matrix C: Calculate the Mahalanobis distance d from each point to the center c i : The shape and size of the ellipse are changed by iteratively adjusting the weights until the smallest ellipse is found, which contains all the points. The major axis of the ellipse is the rotation direction of the obstacle. Then the smallest circumscribed ellipse is stretched into an elliptical cylinder in the Z-axis direction with the maximum height difference of the point cloud to obtain the OBEC bounding box.

7. The ship collision warning method based on radar detection according to claim 6 is characterized in that: The method of associating obstacle data with the model-based global nearest neighbor data association algorithm based on the global nearest neighbor data association algorithm and the established bounding box model includes: Let σ be a one-to-one mapping from the current frame to the set of objects in the previous frame, and σ(i) represents the current frame object associated with target i. Then the association problem is formalized as an assignment problem, that is, finding a mapping from a target to the current frame object to minimize the total cost: For each possible data association, calculate its Mahalanobis distance and establish a cost matrix; suppose there are m obstacles in the previous frame point cloud data after clustering and segmentation and establishing the obstacle geometric feature model, and there are n obstacles in the current frame point cloud data, establish the association matrix: According to the above matrix, data association can be achieved by obtaining the minimum cost.

8. The ship collision warning method based on radar detection according to claim 7 is characterized in that: The step of calculating the speed parameters in the motion features of the obstacle objects in the continuous frames through the association results of the obstacle data, and constructing the obstacle motion feature model based on the speed parameters and the bounding box model includes: By correlating the obstacle data between frames, the speed parameters of the obstacle objects in the continuous frames are calculated. The obstacle speed parameter of the current frame is set to Where T is the working cycle of the laser radar, Δd represents the relative distance difference of the obstacle object in the coordinate system between adjacent frames; the speed of the obstacle in the coordinate system is obtained according to the obstacle data between consecutive frames, and then the actual speed of the obstacle is obtained according to the current speed of the moving ship: Speed ​​of passing obstacles in world coordinates To determine whether the obstacle is a static obstacle, as follows: Among them, μ velo is the speed threshold of the obstacle. When the speed of the obstacle at time T is less than the speed threshold, the obstacle is regarded as a static obstacle and is not added to the dynamic obstacle list. Otherwise, it is regarded as a dynamic obstacle and the dynamic obstacle list is updated. In a conventional navigation environment, the movement of obstacles detected by the ship is only regarded as the movement in the plane coordinate system composed of the X-axis and the Y-axis. Based on the established obstacle geometric feature model, the obstacle plane expression is established; for the OBEC bounding box model, the major axis a′ of the ellipse obstacle With minor axis b′ obstacle Respectively expressed as: Among them, θ′ obstacle Indicates the angle between the obstacle and the Y axis in OBEC mode. (x obstacle ,y obstacle ) represents the obstacle center coordinates, X obstacle With Y obstacle Two variables representing obstacles; For the OBB bounding box model, the length and width of the obstacle are expressed as: 2a″ obstacle With 2b″ obstacle ; In this case, the obstacle is expressed as four linear equations: X obstacle cosθ″ obstacle -Y obstacle sinθ″ obstacle =-a″ obstacle (18) X obstacle cosθ″ obstacle -Y obsacle sinθ″ obstacle =a″ obstacle (19) X obstacle sinθ″ obstacle +Y obstacle cosθ″ obstcacle =-b″ obstacle (20) X obstacle sinθ" obstacle +Y obstacle cosθ″ obstacle =b" obstacle (21) θ″ obstacle Indicates the angle between the obstacle and the Y axis of the OBB model. X obstacle With Y obstacle Two variables representing obstacles; A mathematical model is established to characterize the motion characteristics of ships and dynamic obstacles. For the dynamic characteristics, including the position, velocity, and acceleration of the dynamic obstacles, the dynamic obstacle motion characteristic model is proposed to be represented by the following function: Where (x obstacle (t), y obstacle (t)) represents the coordinate position of the obstacle center, and Respectively represent the velocity vector and acceleration vector of the dynamic obstacle, θ obstacle (t) represents the angle between the obstacle and the Y axis.

9. The ship collision warning method based on radar detection according to claim 8, characterized in that: The TEB algorithm in the local path planning algorithm is used to verify the construction of the obstacle motion feature model to complete the ship obstacle detection and obstacle avoidance, including: The initial path from the starting point to the target point is obtained through the global path planning algorithm, and a series of control points are inserted on the initial path to form an elastic band; each control point contains the position of the ship at that point and the estimated time of arrival at that point; secondly, the elastic band is optimized with multiple objectives, including minimizing the path length, avoiding obstacles, meeting speed and acceleration constraints, and satisfying non-holonomic kinematic constraints; the optimization process is achieved by adjusting the position and time information of the control points; speed, acceleration restrictions and obstacle avoidance requirements, as well as non-holonomic kinematic constraints, are handled by introducing constraints in the form of penalty functions; Among them, the position and angle of each control point of the ship are as follows: Where (x i ,y i ) represents the coordinates of the i-th control point of the mobile ship, β i represents the heading angle of the i-th control point of the mobile ship; the time interval between adjacent control points is ΔT i Represents the time required for a mobile vessel to move from one control point to the next; The goal of the optimization process is to find the optimal pose sequence Q = {s0, s1, ..., s n } and the time interval sequence τ = {ΔT0, ΔT1, ..., ΔT n-1 }, where B represents the combination of path and time series, B = (Q, τ); TEB’s optimization of ship position and time variables is achieved through multiple constraints. Ship path planning considers two constraints: global path following and obstacle avoidance constraints and speed and acceleration constraints; The global path following and obstacle avoidance constraints include: Let d min，j is the shortest distance between the jth control point and the ship, r p max is the maximum distance from the global planning path, d min，k is the shortest distance between the kth obstacle and the ship, r o min is the shortest safe distance between the obstacle and the ship, then the objective function is expressed as: f path =f(d min，j ,r pmax ) (24) f obstacle =f(-d min，k ,r o min ) (25) The velocity and acceleration constraints include: between two adjacent control points, there is a time interval ΔT i Get the translation speed of the ship: The constraint function of translation velocity is: f v =f v，i (X i ,X i+1 ,ΔT i ) (27) The acceleration of the ship is: The constraint function of acceleration is: f a =f a，i (X i ,X i+1 ,X i+2 ,ΔT i ,ΔT i+1 ) (29)。 10. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the electronic device executes the method as described in any one of claims 1-9.

Images