Ship berthing collision warning method, system and storage medium

Abstract

The present invention discloses a ship berthing collision warning method, system and storage medium, which are applied to the field of ship berthing technology, can accurately predict the ship's movement, realize early warning monitoring of ship berthing, and reduce the occurrence of ship berthing collision accidents. The method includes: acquiring ship automatic identification system data, and preprocessing to obtain ship preprocessing data; predicting the ship's speed prediction value and heading prediction value through a preset neural network according to the ship preprocessing data; inputting the speed prediction value and heading prediction value into the ship trajectory dynamics equation model to solve and obtain the ship trajectory data; inputting the ship trajectory data into a preset iteration block to iterate to obtain a ship coordinate prediction sequence; fusing the ship radar data and the video monitoring data to obtain the video target image data; fusing the ship coordinate prediction sequence and the video target image data to obtain the ship prediction trajectory image; performing ship collision warning according to the ship prediction trajectory image to obtain collision warning information.

Description

Technical Field

[0001] The present invention relates to the technical field of ship berthing, and in particular to a ship berthing collision warning method, system and storage medium. Background Art

[0002] With the continuous growth of global shipping volume and the development of large-scale and high-speed ships, the density of navigation in waterways is increasing, and marine accidents are frequent, which seriously affects the efficiency of terminal loading and unloading and also causes certain economic losses. During the berthing process, the tall hull affects the observation of the operating crew, and the superstructure or side of the ship is prone to collision with the onshore loading and unloading equipment. At the same time, the huge inertia of the overweight hull will cause delays in ship operation. In related technologies, due to the inaccurate grasp of the ship's status and the surrounding spatial orientation by the operating crew, the ship itself is prone to collision with onshore equipment, and ship berthing collision accidents occur frequently. Summary of the invention

[0003] In order to solve at least one of the above technical problems, the present invention proposes a ship berthing collision warning method, system and storage medium, which can more accurately predict the ship's movement, realize early warning monitoring of ship berthing, and effectively reduce the occurrence of ship berthing collision accidents.

[0004] On the one hand, an embodiment of the present invention provides a ship berthing collision warning method, comprising the following steps:

[0005] Obtaining ship automatic identification system data;

[0006] Preprocessing the ship automatic identification system data to obtain ship preprocessing data;

[0007] Obtaining a predicted speed value and a predicted heading value of the ship through a preset neural network prediction according to the preprocessed ship data;

[0008] Inputting the predicted speed value and the predicted heading value into a ship trajectory dynamics equation model for solving to obtain ship trajectory data;

[0009] Input the ship trajectory data into a preset iteration block for iteration to obtain a ship coordinate prediction sequence; wherein the preset iteration block is constructed according to the ship trajectory dynamics equation model and the double hidden layer radial basis function network model;

[0010] The video target image data is obtained by fusing the ship radar data and the video surveillance data;

[0011] The ship coordinate prediction sequence and the video target image data are merged to obtain a ship prediction trajectory image;

[0012] A ship collision warning is performed according to the ship predicted trajectory image to obtain collision warning information.

[0013] A ship berthing collision warning method according to an embodiment of the present invention has at least the following beneficial effects: This embodiment first obtains the ship automatic identification system data, and pre-processes the ship automatic identification system data to obtain the ship pre-processing data. Next, this embodiment predicts the ship's speed prediction value and heading prediction value through a preset neural network based on the ship pre-processing data. Then, this embodiment inputs the speed prediction value and heading prediction value into the ship trajectory dynamics equation model to solve the ship trajectory data, and constructs a preset iteration block based on the ship dynamics equation model and the double hidden layer radial basis function network model, so as to iteratively obtain the ship coordinate prediction sequence by inputting the ship trajectory data into the preset iteration block. This embodiment can more accurately predict the ship's movement by predicting the ship's trajectory and the ship's coordinates, so that sufficient time can be reserved to adjust the ship and alleviate the collision problem of the ship berthing. Furthermore, this embodiment obtains video target image data by fusing ship radar data and video surveillance data, and obtains ship prediction trajectory image by fusing ship coordinate prediction sequence and video target image data, thereby performing ship collision warning according to the prediction trajectory image, obtaining collision warning information, realizing early warning monitoring of ship berthing, and effectively reducing the occurrence of ship berthing collision accidents. At the same time, this embodiment can effectively identify the position and movement of the ship by fusing ship radar data and video surveillance data, greatly reducing the probability of collision accidents during ship berthing.

[0014] According to some embodiments of the present invention, the preprocessing of the ship automatic identification system data includes:

[0015] Screening the ship automatic identification system data to obtain abnormal data; wherein the abnormal data includes erroneous data and missing data;

[0016] Classifying the abnormal data to obtain a data classification result;

[0017] Performing a removal operation on the erroneous data according to the data classification result;

[0018] A repair operation is performed on the missing data according to the data classification result.

[0019] According to some embodiments of the present invention, the preset neural network includes a convolutional layer, a pooling layer, and a fully connected layer;

[0020] The method of obtaining the predicted speed value and the predicted heading value of the ship by predicting the ship preprocessed data through a preset neural network includes:

[0021] Inputting the ship pre-processed data into the convolution layer for first processing to obtain local area feature data;

[0022] Inputting the local area feature data into the pooling layer for second processing to obtain feature mapping data;

[0023] The feature mapping data is input into the fully connected layer for third processing to obtain the predicted speed value and the predicted heading value; wherein the third processing includes connection, combination and operation.

[0024] According to some embodiments of the present invention, fusing the ship radar data and the video surveillance data to obtain the video target image data includes:

[0025] Calculating the ship spatial data using the Euclidean distance algorithm and the clustering algorithm according to the ship radar data;

[0026] The video surveillance data is identified by a preset target detection algorithm to obtain a ship target;

[0027] Obtaining ship position data according to the ship target through the video image space and geographic space mapping relationship;

[0028] The video target image data is obtained by fusing the ship space data and the ship position data.

[0029] According to some embodiments of the present invention, the calculating the ship spatial data according to the ship radar data by using the Euclidean distance algorithm and the clustering algorithm includes:

[0030] According to the ship radar data, a point cloud cluster is constructed by using a tree data structure;

[0031] Dividing the point cloud cluster according to the space vector to obtain subset cloud clusters according to the Euclidean distance algorithm;

[0032] Calculating the distance between each cloud point of the first cloud cluster in the subset of cloud clusters and a preset cloud point to obtain point distance data; wherein the preset cloud point includes the cloud point in the first cloud cluster and each cloud point in a second cloud cluster adjacent to the first cloud cluster;

[0033] Performing cluster analysis on the cloud points in the subset cloud cluster according to the point distance data to obtain a clustering result;

[0034] Estimating the geometric center of the clustering result to obtain geometric center data;

[0035] The ship spatial data of the clustering result is obtained by calculating the geometric center data.

[0036] According to some embodiments of the present invention, the fusing the ship spatial data and the ship position data to obtain the video target image data includes:

[0037] Performing a time synchronization operation on the ship spatial data and the ship position data to obtain time synchronization data;

[0038] A coordinate system transformation operation is performed according to the time synchronization data to obtain preset coordinate synchronization data; wherein the preset coordinate synchronization data includes ship space coordinate synchronization data and ship position coordinate synchronization data;

[0039] The video target image data is obtained by performing association matching based on the ship space coordinate synchronization data and the ship position coordinate synchronization data.

[0040] According to some embodiments of the present invention, performing a ship collision warning according to the ship predicted trajectory image to obtain collision warning information includes:

[0041] Construct a ship collision warning model based on collision time;

[0042] The ship prediction trajectory image is input into the ship collision warning model for prediction to obtain the collision warning information.

[0043] On the other hand, an embodiment of the present invention further provides a ship berthing collision warning system, comprising:

[0044] An acquisition module is used to acquire ship automatic identification system data;

[0045] A preprocessing module, used for preprocessing the ship automatic identification system data to obtain ship preprocessing data;

[0046] A prediction module, used to obtain a predicted speed value and a predicted heading value of the ship through a preset neural network prediction based on the pre-processed ship data;

[0047] A solution module, used for inputting the predicted speed value and the predicted heading value into a ship trajectory dynamics equation model for solution to obtain ship trajectory data;

[0048] An iteration module, used for inputting the ship trajectory data into a preset iteration block for iteration to obtain a ship coordinate prediction sequence; wherein the preset iteration block is constructed according to the ship trajectory dynamics equation model and the double hidden layer radial basis function network model;

[0049] The first fusion module is used to fuse the ship radar data and the video surveillance data to obtain the video target image data;

[0050] A second fusion module is used to fuse the ship coordinate prediction sequence and the video target image data to obtain a ship prediction trajectory image;

[0051] The warning module is used to perform ship collision warning according to the ship predicted trajectory image to obtain collision warning information.

[0052] On the other hand, an embodiment of the present invention further provides a ship berthing collision warning system, comprising:

[0053] at least one processor;

[0054] at least one memory for storing at least one program;

[0055] When the at least one program is executed by the at least one processor, the at least one processor implements the ship berthing collision warning method as described in the above embodiment.

[0056] On the other hand, an embodiment of the present invention further provides a computer storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the ship berthing collision warning method as described in the above embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of a ship berthing collision warning method provided by an embodiment of the present invention;

[0058] Figure 2 This is a principle block diagram of a ship berthing collision warning system provided by an embodiment of the present invention;

[0059] Figure 3 It is a principle block diagram of a preset iteration block provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The embodiments described in the embodiments of this application should not be regarded as limitations of this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0061] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0063] Before introducing the embodiments of the present application, the relevant terms involved in the present application are first explained.

[0064] Automatic Identification System (AIS): It is composed of shore-based (base station) facilities and ship-borne equipment. It is a new type of digital navigation system and equipment that integrates network technology, modern communication technology, computer technology, and electronic information display technology.

[0065] Radial basis function network (RBF network) is an artificial neural network that uses radial basis function as activation function. The output of the radial basis function network is a linear combination of the input radial basis function and neuron parameters. The radial basis function network has many uses, including function approximation, time series prediction, classification, and system control.

[0066] With the continuous growth of global shipping volume and the development of large-scale and high-speed ships, the density of navigation in waterways is increasing, and marine accidents are frequent, which seriously affects the efficiency of terminal loading and unloading and also causes certain economic losses. During the berthing process, the tall hull affects the observation of the operating crew, and the superstructure or side of the ship is prone to collision with the onshore loading and unloading equipment. At the same time, the huge inertia of the overweight hull will cause delays in ship operation. In related technologies, due to the inaccurate grasp of the ship's status and the surrounding spatial orientation by the operating crew, the ship itself is prone to collision with onshore equipment, and ship berthing collision accidents occur frequently.

[0067] An embodiment of the present invention provides a ship berthing collision warning method, system and storage medium, which can accurately predict the ship's movement, realize early warning monitoring of ship berthing, and effectively reduce the occurrence of ship berthing collision accidents. Figure 1 The method of the embodiment of the present invention includes but is not limited to step S110, step S120, step S130, step S140, step S150, step S160, step S170 and step S180.

[0068] Specifically, the method application process of the embodiment of the present invention includes but is not limited to the following steps:

[0069] S110: Acquire ship automatic identification system data.

[0070] S120: Preprocessing the ship automatic identification system data to obtain ship preprocessing data.

[0071] S130: Obtaining a predicted speed value and a predicted heading value of the ship through a preset neural network prediction based on the ship preprocessing data.

[0072] S140: Inputting the predicted speed value and the predicted heading value into the ship trajectory dynamics equation model for solving to obtain ship trajectory data.

[0073] S150: Inputting the ship trajectory data into a preset iteration block for iteration to obtain a ship coordinate prediction sequence, wherein the preset iteration block is constructed according to the ship trajectory dynamics equation model and the double hidden layer radial basis function network model.

[0074] S160: Fusing the ship radar data and the video surveillance data to obtain video target image data.

[0075] S170: Fusing the ship coordinate prediction sequence and the video target image data to obtain a ship prediction trajectory image.

[0076] S180: Performing a ship collision warning according to the ship predicted trajectory image to obtain collision warning information.

[0077] During the working process of this specific embodiment, this embodiment first obtains the ship automatic identification system data. Specifically, this embodiment automatically identifies the ship target through the ship automatic identification (AIS) system to obtain the ship AIS data, that is, the ship automatic identification system data. Among them, the ship automatic identification system data in this embodiment includes ship dynamic information and static information, the dynamic information includes the ship's longitude, latitude, ground speed, ground heading, and static information includes identification code, water mobile communication service identification code, call sign, ship name, ship length, ship width, tonnage, and ship type. At the same time, this embodiment predicts the trajectory of the original AIS sentence received from the AIS receiver forwarded by the serial port server. This embodiment first pre-processes the ship automatic identification system data to obtain ship pre-processed data. For example, there may be damaged data or erroneous data in the acquired ship automatic identification system data. This embodiment improves the reliability of the data by pre-processing the ship AIS data. Next, this embodiment predicts the speed prediction value and heading prediction value of the ship through a preset neural network according to the ship pre-processed data. In this embodiment, the ship pre-processed data is input into a preset neural network to predict the ship's speed and heading at the next moment, and the corresponding speed prediction value and heading prediction value are obtained. Furthermore, in this embodiment, the ship trajectory data is obtained by inputting the speed prediction value and heading prediction value into the ship trajectory dynamics equation model for solving. Specifically, the ship trajectory dynamics equation model in this embodiment is shown in the following equations (1) to (5):

[0078]

[0079] β=4.635×10 -6 (4)

[0080]

[0081] wherein, in the formula, l on0 is the initial longitude value, and l at0 is the initial latitude value. If the speed function speed(t), the course function course(t), and the initial longitude l on0 and the initial latitude l at0 are known, the position of the ship at any time can be calculated.

[0082] Furthermore, in this embodiment, the ship trajectory data is input into a preset iteration block for iteration to obtain a ship coordinate prediction sequence. Specifically, referring to Figure 3 , in this embodiment, a preset iteration block 310 is constructed according to the ship trajectory dynamics equation model 311 and the double-hidden-layer radial basis function network model 312. Among them, the formula of the double-hidden-layer radial basis function network model 312 in this embodiment is shown as the following formulas (6) to (8):

[0083] h1 = K(x; C, σ) (6)

[0084] h2 = sigmoid(W2h1 + b2) (7)

[0085] y = W3h2 + b3 (8)

[0086] wherein, in the formula, x is a vector composed of longitude and latitude, y is the speed or course, C is the center of the basis function, σ is the width of the basis function; h1 is the output value of the first hidden layer, h2 is the output value of the second hidden layer, W2 is the weight of the second hidden layer, b2 is the bias of the second hidden layer, W3 is the weight of the output layer, b3 is the bias of the output layer, and K() is the Gaussian basis function as the basis function, as shown in the following formula (9):

[0087]

[0088] Further, the present embodiment fuses the ship radar data and the video surveillance data to obtain the video target image data. Specifically, in the present embodiment, the ship radar data is obtained by the radar subsystem, including the millimeter wave radar and the point cloud signal processing of the millimeter wave radar. At the same time, the present embodiment performs video surveillance on the dock warning area through the 4K visible light and infrared dual-light extremely low illumination camera to obtain the video surveillance data, and displays the video screen on the monitoring interface. Then, the present embodiment fuses the ship radar data and the video surveillance data to obtain the video target image data. Next, the present embodiment fuses the ship coordinate prediction sequence and the video target image data to obtain the ship prediction trajectory image. The present embodiment fuses the predicted ship longitude and latitude prediction sequence, that is, the ship coordinate prediction sequence, with the video target image data to obtain the ship prediction trajectory image, and combines the ship trajectory with the ship space image, so as to accurately predict the ship's movement. Further, the present embodiment performs a ship collision warning according to the ship prediction trajectory image to obtain collision warning information. The present embodiment performs a ship collision warning by using the ship prediction trajectory image that fuses multi-source information, realizes the early warning monitoring of the ship berthing, and effectively reduces the occurrence of ship berthing collision accidents.

[0089] It should be noted that in some embodiments of the present invention, the present embodiment performs multi-source information fusion processing, collision warning and directional broadcasting through the ship display alarm subsystem. At the same time, the present embodiment also performs sound and light alarm through the ship display alarm subsystem, and can perform electronic chart display module, satellite image display, and AIS ship target display function to support daily dock work.

[0090] In some embodiments of the present invention, the ship automatic identification system data is pre-processed, including but not limited to:

[0091] By screening the data of the ship automatic identification system, abnormal data is obtained, which includes wrong data and missing data.

[0092] Classify the abnormal data and obtain the data classification result.

[0093] The erroneous data is eliminated according to the data classification results.

[0094] Repair the missing data based on the data classification results.

[0095] In this specific embodiment, this embodiment first screens the data of the ship automatic identification system to obtain abnormal data. Then, this embodiment classifies the abnormal data to obtain data classification results, so as to eliminate the erroneous data according to the data classification results, and repair the missing data according to the data classification results, thereby obtaining ship pre-processing data. Specifically, the abnormal data in this embodiment includes erroneous data and missing data. This embodiment divides the identified abnormal data into an error state and a missing state, that is, erroneous data and missing data. Then, this embodiment classifies the types of erroneous data and missing data to obtain corresponding data classification results. Further, this embodiment performs corresponding elimination operations or repair operations according to different types of abnormal data, thereby constructing and obtaining ship pre-processing data.

[0096] In some embodiments of the present invention, the preset neural network includes a convolution layer, a pooling layer, and a fully connected layer. Accordingly, in this embodiment, the speed prediction value and heading prediction value of the ship are obtained by predicting the ship preprocessed data through the preset neural network, including but not limited to:

[0097] The ship preprocessed data is input into the convolution layer for the first processing to obtain local area feature data.

[0098] The local area feature data is input into the pooling layer for a second process to obtain feature mapping data.

[0099] The feature map data is input into the fully connected layer for the third processing to obtain the speed prediction value and the heading prediction value. The third processing includes connection, combination and operation.

[0100] In this specific embodiment, the preset neural network is composed of a convolution layer, a pooling layer and a fully connected layer. In this embodiment, the first processing is performed on the ship pre-processing data by the convolution layer to obtain local area feature data, and then the local area feature data is input into the pooling layer for the second processing to obtain feature mapping data. Next, this implementation performs a third processing on the feature mapping data by the fully connected layer to obtain the speed prediction value and the heading prediction value. At the same time, the third processing in this embodiment includes connection, combination and operation processing. Specifically, the convolution layer in this embodiment includes a convolution kernel and an activation function. In this embodiment, the convolution layer performs a convolution operation on the ship pre-processing data to extract the corresponding local area feature data. Among them, each convolution kernel contains multiple neurons, and different convolution kernel sizes are equivalent to different feature extractors. In this embodiment, the pooling layer is arranged after the convolution layer, and the output of the convolution layer is connected to the input of the pooling layer. In this embodiment, the local area feature data is pooled, such as feature selection, reducing the number of features and reducing the number of parameters, so as to obtain feature mapping data. Among them, the second processing in this embodiment includes maximum pooling processing and average pooling processing. Furthermore, in this embodiment, the fully connected layer is located at the top layer of the preset neural network, and the input end of the fully connected layer is connected to the output end of the pooling layer. In this embodiment, the feature mapping data obtained by pooling is connected, combined and calculated through the fully connected layer, so as to output the final result, namely the speed prediction value and the heading prediction value. It is easy to understand that in this embodiment, the speed prediction sequence and the heading prediction sequence at the future moment are obtained by inputting the compacted prediction value and the heading prediction value into the preset neural network again and performing multiple iterations, that is, the preset speed prediction value and the heading prediction value are obtained.

[0101] In some embodiments of the present invention, the video target image data is obtained by fusing the ship radar data and the video surveillance data, including but not limited to:

[0102] The ship spatial data is obtained by calculating the Euclidean distance algorithm and clustering algorithm based on the ship radar data.

[0103] The video surveillance data is identified through a preset target detection algorithm to obtain the ship target.

[0104] According to the ship target, the ship position data is obtained through the mapping relationship between the video image space and the geographic space.

[0105] The video target image data is obtained by fusing the ship spatial data and the ship position data.

[0106] In this specific embodiment, this embodiment first calculates the ship spatial data according to the ship radar data through the Euclidean distance algorithm and the clustering algorithm, and then identifies the video surveillance data through the preset target detection algorithm to obtain the ship target. Then, this embodiment obtains the ship position data according to the ship target through the video image space and the geographic space mapping relationship, so as to obtain the video target image data by fusing the ship spatial data and the ship position data. Specifically, this embodiment calculates the point cloud data collected by the radar subsystem, that is, the ship radar data, through the Euclidean distance algorithm and the clustering algorithm to obtain the ship spatial data, such as the volume, length, width, height, etc. of the ship. Then, this embodiment identifies the video surveillance data through the preset target detection algorithm to obtain the ship target in the video surveillance data. Exemplarily, after the video surveillance data is obtained by recording the camera, the ship target is identified from the video surveillance data through the yolov3 algorithm. Among them, the yolov3 algorithm is composed of different network layers. In this embodiment, the network layer includes a coding layer and a decoding layer. The coding layer is composed of several convolutional layers, activation functions and pooling layers, which are used to propose high-order abstract features in the image. At the same time, the decoding layer includes a convolution layer, a pooling layer and a fully connected layer. The decoding layer outputs a prediction result by learning the characteristics of the data transmitted by the encoding layer, that is, outputs the identified ship target. Furthermore, this embodiment obtains the specific location information of the ship, that is, the ship location data, based on the mapping relationship between the video image space and the geographic space of the video surveillance data according to the identified ship target. Then, this embodiment fuses the ship spatial data and the ship location data to obtain video target image data. It is easy to understand that the video target image data in this embodiment combines the spatial data and the location information of the ship, which can effectively improve the accuracy of the ship berthing warning.

[0107] In some embodiments of the present invention, the ship spatial data is calculated based on the ship radar data by using the Euclidean distance algorithm and the clustering algorithm, including but not limited to:

[0108] Point cloud clusters are obtained by constructing a tree data structure based on ship radar data.

[0109] According to the Euclidean distance algorithm, the point cloud clusters are divided according to the space vectors to obtain subset cloud clusters;

[0110] The distance between each cloud point of the first cloud cluster in the subset cloud cluster and the preset cloud point is calculated to obtain point distance data, wherein the preset cloud point includes the cloud points in the first cloud cluster and each cloud point in the second cloud cluster adjacent to the first cloud cluster.

[0111] The cloud points in the subset cloud cluster are clustered according to the point distance data to obtain the clustering results.

[0112] Estimate the geometric center of the clustering results to obtain the geometric center data.

[0113] The ship spatial data of clustering results is calculated through geometric center data.

[0114] In this specific embodiment, this embodiment first constructs a point cloud cluster through a tree data structure based on the ship radar data. Specifically, after acquiring the ship radar data through the millimeter wave radar, this embodiment inputs the ship radar data into a tree data structure (Kdtree structure) to establish a corresponding topological relationship between discrete point cloud data points to obtain a point cloud cluster M, so that a fast field point set search can be performed. Among them, Kdtree is a multidimensional binary tree structure that can organize and store high-dimensional data. Kdtree can divide the data space into multiple subspaces, each subspace does not intersect with each other, and each node on the binary tree divides its data space into two subspaces. If the data contained in each node is less than the pre-designed numerical lower limit, the smaller data space is no longer divided. At the same time, since the point cloud signal is three-dimensional spatial data, it is usually processed for its three-dimensional information. Then, this embodiment divides the point cloud cluster according to the spatial vector according to the Euclidean distance algorithm to obtain a subset cloud cluster. For example, this embodiment divides the point cloud cluster M according to the spatial vector d={d1,d2,…,d n} is divided into different subset point cloud clusters M={M1,M2,…,M n}. Furthermore, this embodiment obtains point distance data by calculating the distance between each cloud point of the first cloud cluster in the subset cloud cluster and the preset cloud point. The preset cloud point includes the cloud points in the first cloud cluster and each cloud point in the second cloud cluster adjacent to the first cloud cluster, that is, the distance between each cloud point of the first cloud cluster in the subset cloud cluster and other cloud points in the first cloud cluster except the cloud point is calculated, as well as the distance between each cloud point in the second cloud cluster adjacent to the first cloud cluster. Exemplarily, this embodiment first creates an empty cluster list R to store the result index of clustering. Then, this embodiment calculates the distance between each point cloud cluster M according to the number of points in the first cloud cluster. i = {P1,P2,…,P i}, that is, the first cloud cluster, each cloud point P i The coordinate data (x pi ,y pi ) to solve its i And with M i The distance D between adjacent point cloud clusters, that is, the common cloud points in the second cloud cluster, is as follows:

[0115]

[0116] Furthermore, this embodiment performs cluster analysis on the cloud points in the subset cloud cluster according to the point distance data to obtain a clustering result. Specifically, in this embodiment, if there is a relationship D≤d i Then we can judge that the two points belong to the same feature cluster and mark them as feature cluster r. i In addition, when the point sets in each neighborhood have completed the above process, the influence of the error points should be eliminated. min , where r <P min is the error point set, if the number of points p in r satisfies the condition p ≥ P min , then r i Add to R. Repeat this process for each M i ∈{P1,P2,…,P i} are traversed once, and the categories are merged according to the Euclidean distances of all clusters in R until the Euclidean distances between all clusters in list R are greater than the distance threshold d i Among them, d i It is calculated by the following formula (11):

[0117]

[0118] Among them, D i ∈{D1,D2,…,D n} is the value of the distance division area, ζ is the relaxation coefficient, and its size is related to the downsampling value size in the data preprocessing process. is the horizontal resolution angle of the laser radar, which is related to the physical structure of the laser radar. When the rotation frequency is constant, its value is a constant. So far, the final clustering result list R is obtained in this embodiment.

[0119] Furthermore, in this embodiment, the geometric center of the clustering result is estimated to obtain geometric center data, and the ship space data of the clustering result is calculated by the geometric center data. Specifically, the clustering result R obtained in this embodiment is i ∈R. This embodiment calculates the geometric center of each clustering result obtained by each clustering to obtain the geometric center data of each clustering result. Next, this embodiment calculates the size of each clustering result after clustering by the following formula (12) to obtain the ship space data.

[0120]

[0121] Among them, V n , l n ,d n 、h n , respectively represent the volume, length, width, and height of the nth clustering result, are the maximum and minimum x-coordinates of the nth clustering result, respectively. are the maximum and minimum y coordinates of the nth clustering result, respectively. They are respectively the maximum value and the minimum value of the z coordinate in the nth clustering result.

[0122] In some embodiments of the present invention, the video target image data is obtained by fusing the ship space data and the ship position data, including but not limited to:

[0123] Time synchronization operation is performed on the ship spatial data and the ship position data to obtain time synchronization data.

[0124] A coordinate system transformation operation is performed according to the time synchronization data to obtain preset coordinate synchronization data, wherein the preset coordinate synchronization data includes ship space coordinate synchronization data and ship position coordinate synchronization data.

[0125] The video target image data is obtained by performing association matching based on the ship space coordinate synchronization data and the ship position coordinate synchronization data.

[0126] In this specific embodiment, this embodiment first performs time synchronization operation on the ship space data and the ship position data to obtain time synchronization data. Specifically, in the timestamp alignment strategy of this embodiment, time series 1 and time series 2 are used to represent the time process of data processing of two different sensors. Among them, this embodiment uses t1 and t2 to represent the average time of sensor a algorithm processing and the average time of sensor b algorithm processing in time series 1. When the system starts working from T0, when the time reaches T1, the data processing of sensor a is completed, but according to the previous design of the system, the processing result is not uploaded at this time. Then, when the time reaches T2, the algorithm processing of sensor b is completed, and the results of the two parts of the processing should be uploaded at the same time. At the same time, before the time reaches T3, the two parts should be suspended respectively, waiting for the next cycle to start, and during this period the system should give up the CPU usage right to ensure the real-time performance of the system. When the time reaches T3, a new cycle starts, and the above process is repeated, thereby ensuring the time synchronization of the data. Further, this embodiment performs a coordinate system transformation operation according to the time synchronization data to obtain preset coordinate synchronization data. Specifically, in this embodiment, the preset coordinate synchronization data includes ship space coordinate synchronization data and ship position coordinate synchronization data. Exemplarily, in the coordinate system space transformation of this embodiment, the radar coordinate system and the hull coordinate system are firstly rigidly transformed, and then the internal parameter matrix of the image coordinate system is unified. After passing through the rigid body, it can be unified with the point cloud coordinate system to the hull coordinate system and matched. In addition, for image targets in this embodiment, it is generally based on the pinhole imaging principle, which can realize the conversion of geodetic coordinates to pixel coordinates. In order to obtain a larger viewing angle, this embodiment adopts a four-eye camera, and the image is cropped and spliced ​​to form a whole picture. In this embodiment, the one-dimensional coordinate axis based on the azimuth is used for fusion, and the distortion correction of the image is also ignored. Based on the pinhole imaging principle, for an image, a picture contains all image information within the viewing angle range, and different pixel positions are fixed relative to the camera in the viewing angle. Therefore, the target azimuth coordinates can be obtained as shown in the following formula (13):

[0127]

[0128] Furthermore, the present embodiment performs association matching based on the ship space coordinate synchronization data and the ship position coordinate synchronization data to obtain video target image data. Specifically, in the feature association matching of the present embodiment, it is necessary to extract the detection frame in the image and the detection frame in the point cloud, and project the detection frame of the point cloud through the main view, calculate the relative distance value between each target frame in the point cloud and each detection frame in the image, and then retain the minimum value from all the distances obtained. If the minimum value is less than the set minimum error value, an association match is performed to obtain the video target image data. It is easy to understand that the present embodiment can display the predicted trajectory image of the ship, that is, the video target image data, in the CCTV (closed-circuit television monitoring system) video by fusing the video tracking target information with the AIS target prediction data information and associating the predicted continuous longitude and latitude sequence with the azimuth coordinates in the image.

[0129] In some embodiments of the present invention, a ship collision warning is performed based on the ship predicted trajectory image to obtain collision warning information, including but not limited to:

[0130] Construct a ship collision warning model based on the collision time.

[0131] The ship prediction trajectory image is input into the ship collision warning model for prediction to obtain collision warning information.

[0132] In this specific embodiment, this embodiment first constructs a ship collision warning model based on the collision time. Specifically, this embodiment constructs a ship collision warning model based on a collision time (TTC) model. Among them, the TTC model is a method for warning based on the relationship between the distance and speed ratio between the ship and the dock target, and its expression is shown in the following formula (14):

[0133]

[0134] Wherein, t1 and t2 represent two different moments, and d1 and d2 represent the relative distances between the ship and the dock at moments t1 and t2, respectively.

[0135] Furthermore, in this embodiment, the ship prediction trajectory image is input into the ship collision warning model for prediction to obtain collision warning information. Specifically, in this embodiment, in the collision warning, according to t TTC Different thresholds are set for the value of V, and the warning is divided into three levels. d >0, it means that the distance between the ship and the dock is increasing. In this case, t TTC >0 is always true, so no warning is needed. d ≤0, according to t TTC The value of is different, the warning level is divided into three levels, and a graded warning is performed. For example, when tTTC When it is greater than 0, the warning system outputs 0 and no warning is issued. <t TTC When ≤0, the first level warning is output; when -300 <t TTC When t ≤ -120, the second level warning is output. TTC ≤-300, a third-level warning is output. Accordingly, in this embodiment, the first-level warning includes broadcasting a call at a normal speaking speed and sending a first-level alarm information via AIS text message, wherein the broadcast is non-directional. In this embodiment, the second-level alarm includes broadcasting a call at a faster speaking speed, and sending a second-level alarm information via AIS text message. At the same time, within a certain area, the broadcast tracks the target ship. In this embodiment, the third-level alarm includes broadcasting a call at the fastest speaking speed, sending an AIS text message, and sending a third-level alarm information via VHF voice. At the same time, within a certain area, the broadcast tracks the target ship. In addition, it is easy to understand that in this embodiment, at night, the searchlight is aimed at the water surface in the direction of the ship's advance and flashes (do not directly illuminate the ship) to draw a line on the water surface to indicate the route of travel.

[0136] It should be noted that in some embodiments of the present invention, when this embodiment performs directional broadcasting for ships with berthing risks, it supports the manual broadcasting of voice information and other information on any client through a microphone and a broadcasting system, and only one client can speak at the same time, supports the preemption mechanism and the maximum duration limit, and automatically releases the voice control when the duration limit is exceeded to give other clients the opportunity to preempt. In addition, this embodiment supports manual adjustment of the broadcast pan-tilt head to point to a specific direction or target. Among them, the broadcast can transmit sound to at least 1000 meters in a windless state. In this embodiment, the sound and light alarm searchlight can be remotely controlled to be turned on, and can be pointed to a specific direction or target as the pan-tilt head rotates. Similarly, in this embodiment, the alarm can be remotely controlled to be turned on to flash the alarm light and sound the alarm. This embodiment supports listening to the communication information of channel 16 or other channels on any client through VHF communication equipment, and supports manual calling of ships for communication or broadcasting of warning information through the licensed channels on the licensed clients, and supports the maximum speaking time limit. When the time limit is exceeded, the control is automatically released to allow other clients to preempt the speaking, and automatically broadcast warning expulsion information or other information through the VHF licensed channel.

[0137] It should be noted that in some embodiments of the present invention, an electronic chart display module is provided in this embodiment. Among them, the electronic chart display module is used to support the loading and updating of standard electronic charts in IHOS57 format and IHOS-63 format and electronic navigation charts in CJ-57 format. At the same time, the electronic chart display module of this embodiment supports the display of charts in vector mode according to the requirements of IHOS-52611. In addition, the electronic chart display module of this embodiment includes but is not limited to the switching of daytime, dusk, and night color modes, the switching of basic, standard, and all display groups, and the custom display group, and supports the basic operations of zooming in, zooming out, translating, and rotating charts. This embodiment supports the automatic display of charts with appropriate editing scales when the display scale changes, and can query the attribute information of any object on the chart. In addition, this embodiment can also plot symbols such as points, lines, surfaces, rectangles, and ellipses on the chart, support the plotting of chart-specific symbols, and can perform measurements on the chart, including measurements of distance, azimuth, area, etc.

[0138] It should be noted that in some embodiments of the present invention, the present embodiment displays the monitored waters in the form of satellite images. The satellite images and electronic charts are aligned and have consistent coordinates. At the same time, the present embodiment can perform multi-level zooming and supports translation and rotation. The satellite images and electronic charts share a set of monitoring area settings, and can switch between the chart display mode and the satellite image display mode.

[0139] It should be noted that in some embodiments of the present invention, the AIS ship target display module receives the AIS signal of ships within a radius of at least 2 kilometers with the midpoint of the south edge of the dock as the center, and displays the AIS ship target on the electronic chart display module or the satellite map. At the same time, the AIS ship target display module supports displaying AIS ship target information in a list format. By clicking on the AIS target, the AIS static information and real-time dynamic information of the ship can be viewed in the information window. In addition, in this embodiment, you can choose to mark the AIS target on the chart with Chinese name, English name, call sign, MMSI number, etc., and choose whether to display the wake, speed vector line, heading, turning prompt, etc. of the AIS ship target.

[0140] It should be noted that in some embodiments of the present invention, this embodiment is also provided with an information management subsystem for adding users, deleting users, modifying user permissions, editing the information of ships in the ship library, including Chinese names, etc., and entering and editing a white list of ships. Among them, ships in the white list will not alarm when entering the warning area.

[0141] It should be noted that in some embodiments of the present invention, a recording and playback unit is provided in the present embodiment for selecting a playback time range, and the system will read the saved historical information of AIS and radar targets, as well as the historical video data, and completely replay the situation at that time on the same interface as the monitoring interface of the system, including the trajectory of the ship's navigation, the video recorded by the camera, and the alarm information generated. Among them, the corresponding alarm information does not need to be sent through broadcasting, AIS or VHF, nor does it need to light up the searchlight and trigger the alarm, but the playback device has an alarm sound, including the synthesized voice alarm played.

[0142] It should be noted that in some embodiments of the present invention, this embodiment provides AIS information services to the client through the central service unit. At the same time, this embodiment provides a warning zone query service to the client by responding to the warning zone setting request and saving the warning zone setting. Accordingly, this embodiment determines in real time whether the radar target and AIS target have entered the warning zone, and provides a graded alarm information service to the client. Then, the central service unit of this embodiment can also respond to historical playback requests, query radar target and AIS target historical records, broadcast radar target and AIS target data to the client according to speed, and respond to requests such as authority management and information management, save data, and save and back up data.

[0143] An embodiment of the present invention further provides a ship berthing collision warning system, comprising:

[0144] The acquisition module is used to obtain the ship automatic identification system data.

[0145] The preprocessing module is used to preprocess the ship automatic identification system data to obtain ship preprocessing data.

[0146] The prediction module is used to obtain the ship's speed prediction value and heading prediction value through a preset neural network according to the ship's preprocessed data.

[0147] The solution module is used to input the predicted speed value and the predicted heading value into the ship trajectory dynamics equation model for solution to obtain the ship trajectory data.

[0148] The iteration module is used to input the ship trajectory data into the preset iteration block for iteration to obtain the ship coordinate prediction sequence. The preset iteration block is constructed according to the ship trajectory dynamics equation model and the double hidden layer radial basis function network model.

[0149] The first fusion module is used to fuse the ship radar data and the video surveillance data to obtain video target image data.

[0150] The second fusion module is used to fuse the ship coordinate prediction sequence and the video target image data to obtain the ship prediction trajectory image.

[0151] The early warning module is used to issue a ship collision warning based on the ship's predicted trajectory image to obtain collision warning information.

[0152] Reference Figure 2 An embodiment of the present invention further provides a ship berthing collision warning system, comprising:

[0153] At least one processor 210 .

[0154] At least one memory 220 is used to store at least one program.

[0155] When at least one program is executed by at least one processor 210, the at least one processor 210 implements the ship berthing collision warning method as described in the above embodiment.

[0156] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by one or more control processors, for example, to execute the steps described in the above embodiment.

[0157] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0158] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A ship berthing collision warning method, characterized in that: The following steps are involved: Obtaining ship automatic identification system data; Preprocessing the ship automatic identification system data to obtain ship preprocessing data; Obtaining a predicted speed value and a predicted heading value of the ship through a preset neural network prediction according to the preprocessed ship data; Inputting the predicted speed value and the predicted heading value into a ship trajectory dynamics equation model for solving to obtain ship trajectory data; Input the ship trajectory data into a preset iteration block for iteration to obtain a ship coordinate prediction sequence; wherein the preset iteration block is constructed according to the ship trajectory dynamics equation model and the double hidden layer radial basis function network model; The video target image data is obtained by fusing the ship radar data and the video surveillance data; The ship coordinate prediction sequence and the video target image data are merged to obtain a ship prediction trajectory image; A ship collision warning is performed according to the ship predicted trajectory image to obtain collision warning information.

2. The ship berthing collision warning method according to claim 1, characterized in that: The preprocessing of the ship automatic identification system data includes: Screening the ship automatic identification system data to obtain abnormal data; wherein the abnormal data includes erroneous data and missing data; Classifying the abnormal data to obtain a data classification result; Performing a removal operation on the erroneous data according to the data classification result; A repair operation is performed on the missing data according to the data classification result.

3. The ship berthing collision warning method according to claim 1, characterized in that: The preset neural network includes a convolutional layer, a pooling layer and a fully connected layer; The method of obtaining the predicted speed value and the predicted heading value of the ship by predicting the ship preprocessed data through a preset neural network includes: Inputting the ship pre-processed data into the convolution layer for first processing to obtain local area feature data; Inputting the local area feature data into the pooling layer for second processing to obtain feature mapping data; The feature mapping data is input into the fully connected layer for third processing to obtain the predicted speed value and the predicted heading value; wherein the third processing includes connection, combination and operation.

4. The ship berthing collision warning method according to claim 1, characterized in that: The method of fusing the ship radar data and the video surveillance data to obtain the video target image data includes: Calculating the ship spatial data using the Euclidean distance algorithm and the clustering algorithm according to the ship radar data; The video surveillance data is identified by a preset target detection algorithm to obtain a ship target; Obtaining ship position data according to the ship target through the video image space and geographic space mapping relationship; The video target image data is obtained by fusing the ship space data and the ship position data.

5. The ship berthing collision warning method according to claim 4, characterized in that: The ship spatial data is obtained by calculating the ship radar data through the Euclidean distance algorithm and the clustering algorithm, including: According to the ship radar data, a point cloud cluster is constructed by using a tree data structure; Dividing the point cloud cluster according to the space vector to obtain subset cloud clusters according to the Euclidean distance algorithm; Calculating the distance between each cloud point of the first cloud cluster in the subset of cloud clusters and a preset cloud point to obtain point distance data; wherein the preset cloud point includes the cloud point in the first cloud cluster and each cloud point in a second cloud cluster adjacent to the first cloud cluster; Performing cluster analysis on the cloud points in the subset cloud cluster according to the point distance data to obtain a clustering result; Estimating the geometric center of the clustering result to obtain geometric center data; The ship spatial data of the clustering result is obtained by calculating the geometric center data.

6. The ship berthing collision warning method according to claim 4, characterized in that: The step of fusing the ship spatial data and the ship position data to obtain the video target image data comprises: Performing a time synchronization operation on the ship spatial data and the ship position data to obtain time synchronization data; A coordinate system transformation operation is performed according to the time synchronization data to obtain preset coordinate synchronization data; wherein the preset coordinate synchronization data includes ship space coordinate synchronization data and ship position coordinate synchronization data; The video target image data is obtained by performing association matching based on the ship space coordinate synchronization data and the ship position coordinate synchronization data.

7. The ship berthing collision warning method according to claim 1, characterized in that: The performing of ship collision warning according to the ship predicted trajectory image to obtain collision warning information includes: Construct a ship collision warning model based on collision time; The ship prediction trajectory image is input into the ship collision warning model for prediction to obtain the collision warning information.

8. A ship berthing collision warning system, characterized in that: include: An acquisition module is used to acquire ship automatic identification system data; A preprocessing module, used for preprocessing the ship automatic identification system data to obtain ship preprocessing data; A prediction module, used to obtain a predicted speed value and a predicted heading value of the ship through a preset neural network prediction based on the pre-processed ship data; A solution module, used for inputting the predicted speed value and the predicted heading value into a ship trajectory dynamics equation model for solution to obtain ship trajectory data; An iteration module, used for inputting the ship trajectory data into a preset iteration block for iteration to obtain a ship coordinate prediction sequence; wherein the preset iteration block is constructed according to the ship trajectory dynamics equation model and the double hidden layer radial basis function network model; The first fusion module is used to fuse the ship radar data and the video surveillance data to obtain the video target image data; A second fusion module is used to fuse the ship coordinate prediction sequence and the video target image data to obtain a ship prediction trajectory image; The warning module is used to perform ship collision warning according to the ship predicted trajectory image to obtain collision warning information.

9. A ship berthing collision warning system, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the ship berthing collision warning method as described in any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the ship berthing collision warning method as described in any one of claims 1 to 7 when executed by the processor.

Images