An unmanned ship inspection and early warning method based on a hybrid algorithm

Abstract

The application discloses an unmanned ship inspection and early warning method based on a hybrid algorithm, a water surface image feature is obtained through an unmanned ship vision system to analyze a water surface target, feature extraction is carried out on the water surface target, a target feature library is formed after the feature extraction, and an initial screening water surface target picture set is obtained through convolutional neural network training; target images in the initial screening water surface target picture set are detected through multi-dimensional evaluation indexes, and the water surface target picture set is accurately screened; indexes and parameters obtained through image detection are brought into an unmanned ship autonomous navigation image target detection system based on a YOLO algorithm, a maritime target prediction result is obtained through execution of a maritime target hybrid algorithm intelligent detection system test process, finally, an early warning model is constructed, a collision state anomaly or normality possibly occurring with the maritime target prediction result is predicted, and unmanned ship inspection and early warning are realized. The hybrid algorithm is used to quickly and accurately detect image targets, and unmanned ship inspection and early warning are realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned ship inspection, and particularly relates to an unmanned ship inspection and early warning method based on a hybrid algorithm. BACKGROUND

[0002] In 2022, the new generation of Beidou will provide more accurate and reliable services for global users, and through inter-satellite links, it will realize star-star networking and interconnection. In the same year, China Telecom's "Tiantong-1" is the world's first 5G SA commercial network with the largest scale, which also provides assistance for China Telecom's air base station satellite precise positioning. With the commercialization of air base stations and Beidou satellite navigation, air satellite commercial data is becoming increasingly closely related to the ground and the ocean.

[0003] Countries are increasingly attaching importance to maritime rights and interests. As an artificial intelligence platform, unmanned ships can monitor the marine environment, survey hydrographic geography, and perform military tasks, which has attracted the attention of domestic and foreign intelligent research institutions. To enable unmanned ships to autonomously navigate in complex and variable marine environments, i.e., to navigate without human control based on the information obtained by the unmanned ship, the unmanned ship must have good environmental perception and intelligent decision-making and control capabilities.

[0004] Good environmental perception is the premise and key to the autonomous navigation of unmanned ships. There are currently four main ways to obtain information about marine targets: radar images, satellite images, infrared images, and video images. Regardless of the method used to obtain information about marine targets, image detection is required to obtain target information. Radar images have strong detection capabilities, but are easily disturbed in military applications. Satellite remote sensing, especially SAR imaging, has good penetration and observation capabilities, but cannot achieve all-weather, real-time monitoring. Infrared imaging is only suitable for night or insufficient light and is energy-consuming. Video images have the advantages of all-weather, low cost, and strong real-time performance. SUMMARY

[0005] The technical problem to be solved by the present application is to provide an unmanned ship inspection and early warning method based on a hybrid algorithm to address the deficiencies of the prior art.

[0006] To achieve the above technical purpose, the technical solution adopted by the present application is as follows:

[0007] An unmanned ship inspection and early warning method based on a hybrid algorithm, comprising:

[0008] Step one, analyze the water surface target by obtaining the water surface image features through the unmanned ship vision system, extract the features, form a target feature library after feature extraction, and obtain the preliminary screening water surface target picture set through convolutional neural network training;

[0009] Step two, detecting the target images in the preliminary screening water surface target picture set through multi-dimensional evaluation indexes, and accurately screening the water surface target picture set;

[0010] Step three, bringing the indexes obtained by the image detection in step two and the parameters into the unmanned ship autonomous navigation image target detection system based on the YOLO algorithm, obtaining the marine target prediction result through the execution of the marine target mixed algorithm intelligent detection system test process, finally constructing a warning model to predict the abnormal or normal collision state with the marine target prediction result, and realizing the unmanned ship patrol and early warning.

[0011] To optimize the above technical solutions, the specific measures taken also include:

[0012] The water surface target described above includes a ship, an island, and a reef.

[0013] The step one extracts the texture energy, texture entropy, and texture gradient features of the water surface target.

[0014] The texture energy calculation formula is:

[0015]

[0016] The texture entropy calculation formula is:

[0017]

[0018] The texture gradient calculation formula is:

[0019]

[0020] Wherein, p(g1, g2) is a pixel point in the water surface target image, and k is a coefficient.

[0021] The convolutional neural network training in step one includes two stages of forward propagation and backward propagation.

[0022] The forward propagation includes:

[0023] The image data is input into the convolutional neural network, and the parameters learned by each layer are output through convolutional pooling operations layer by layer. Finally, more significant features are obtained through the full connection layer and the output layer.

[0024] The backward propagation includes:

[0025] (1) calculating the residual error and the activation value of the last layer through the network;

[0026] (2) passing the residual error and the activation value of the last layer to the previous layer layer by layer through the backward propagation, so that the previous neuron updates its weight value according to the error;

[0027] (3) According to the residual, further calculate the gradient of the weight parameter, and adjust the convolutional neural network parameter again;

[0028] (4) Continue step (3) until convergence or the maximum number of iterations has been reached.

[0029] The multi-dimensional evaluation index of step two includes:

[0030] (1) Precision, the proportion of correct targets among all detected targets;

[0031] (2) Recall, the proportion of correctly identified targets among the total number of targets

[0032] (3) F1 score

[0033]

[0034] P and R correspond to Precision and Recal, respectively;

[0035] (4) Precision-Recal curve;

[0036] (5) Average precision, the average of the average precision of each class.

[0037] The test process of the offshore target mixed algorithm intelligent detection system described in step three includes:

[0038] Obtain the initial screening water surface target picture set through step one;

[0039] Through the F1 score and Precision-Recal curve of step two, compare the labels of the result test set one by one. If the input category and the framed object region are consistent with the label, the detection image is successful, otherwise the detection fails;

[0040] Through the average precision of step two, calculate the detection rate of each class, take the average value, and obtain the average detection rate mAP.

[0041] The early warning model described in step three is, set two states, one is abnormal R1, the other is normal R2, and the extracted offshore detection target result feature value is represented as:

[0042] T={t1, t2,..., t 12}, represents the average detection rate mAP value;

[0043] According to the conditional probability formula, under the condition of meeting the feature value T, the probability of suspected abnormality being judged as R1 is:

[0044]

[0045] Where P(T|R1) represents the probability of the feature value T appearing in the abnormal subset of the sample set, and P(R1) represents the proportion of the abnormal subset in the entire sample set.

[0046] Given an eigenvalue T, the probability of a suspected anomaly being classified as R2 is expressed as follows:

[0047]

[0048] Where P(T|R2) represents the probability of the feature value T appearing in the normal subset of the sample set, and P(R2) represents the proportion of the normal subset in the entire sample set;

[0049] If P(R1|T) > P(R2|T), the monitored average detection rate mAP value is considered abnormal; otherwise, it is considered normal.

[0050] The present invention has the following beneficial effects:

[0051] The surface image features obtained through the unmanned surface vessel (USV) vision system are analyzed to identify surface targets such as ships, islands, and reefs. Therefore, feature extraction and analysis are performed on both the outer contour and surface properties of the targets, resulting in a target feature library. A preliminary set of surface target images is then obtained through a convolutional training process. Next, target images are detected using multi-dimensional evaluation metrics. This process further refines the surface target image set. Then, steps 1-5 of the image detection process (step 2) along with the parameters are fed into an unmanned surface vessel autonomous navigation image target detection system based on the YOLO algorithm. The system's maritime target prediction results are obtained by executing a test procedure for a maritime target hybrid algorithm intelligent detection system. Finally, an early warning model is constructed to predict whether collisions with the predicted maritime targets will occur in an abnormal or normal state. This system enables rapid and accurate target detection based on a hybrid algorithm, achieving unmanned surface vessel inspection and early warning capabilities. Attached Figure Description

[0052] Figure 1 This is a flowchart of the unmanned surface vessel inspection and early warning method based on a hybrid algorithm according to the present invention;

[0053] Figure 2 This illustrates the changes in Precision and Recal under different threshold conditions according to the present invention;

[0054] Figure 3 The YOLO algorithm flow of this invention is described below. Detailed Implementation

[0055] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0056] like Figure 1 As shown, an unmanned surface vessel (USV) inspection and early warning method based on a hybrid algorithm includes:

[0057] Step one, the water surface image features obtained by the unmanned ship vision system are analyzed to analyze the water surface targets including ships, islands, reefs, etc. Therefore, feature extraction and analysis are performed from two aspects of the peripheral contour and surface properties of the water surface target, and a target feature library is formed after feature extraction. Then, an initial screening water surface target picture set is obtained through a convolution application training process.

[0058] 1. Water surface target texture features

[0059] 1-1, Texture energy: it is the sum of squares of elements in the gray level co-occurrence matrix, which reflects the uniformity of the image texture, and the calculation formula is:

[0060]

[0061] 1-2, Texture entropy: it is a measure of the amount of information possessed by the image, indicating the complexity of the image. When the complexity is high, the entropy value is large, and vice versa. The calculation formula is:

[0062]

[0063] 1-3, Texture gradient: it represents the ability to inverse difference of small details of the image, reflecting the hierarchical content and clarity of the image. The calculation formula of the average gradient is:

[0064]

[0065] The texture feature extraction results are shown in Table 1:

[0066] Table 1 Texture feature extraction results

[0067]

[0068] 2. Training process of convolutional neural network

[0069] D = max (V)

[0070]

[0071] The training process of CNN can be divided into two stages of forward propagation and back propagation

[0072] First stage, forward propagation:

[0073] 1. Input image data into the convolutional neural network.

[0074] 2. Pass through the convolution and pooling operations layer by layer, output the parameters learned by each layer, and the output of the n-1 layer is the input of the layer. The relationship between the input and output x l of the last layer is

[0075] x l = f(W l x l-1 ) + b l )

[0076] where j is the number of layers; W is the weight; b is a bias; and f is an activation function.

[0077] 3Finally, more significant features are obtained through the full connection layer and the output layer.

[0078] The second stage, back propagation:

[0079] (1) Calculate the residual and activation value of the last layer through the network.

[0080] (2) The residual and activation value of the last layer are passed forward layer by layer through the back propagation method, so that the previous neurons update their weights according to the error.

[0081] (3) According to the residual, further calculate the gradient of the weight parameter, and adjust the convolutional neural network parameters again.

[0082] (4) Continue step (3) until convergence or the maximum number of iterations has been reached.

[0083] For unsupervised learning of CNN, the essence is the mode of "pre-training + supervised fine-tuning". The form of pre-training is to train each layer separately using input and output. The training sample set is unlabeled, and then the labeled data is used to fine-tune the weight parameters. Compared with the cost of obtaining labeled data, it is easy to obtain a large amount of unlabeled data. The self-learning method can obtain the best initial weight of all layers by using a large amount of unlabeled data, that is, a better model is obtained. Compared with supervised learning, self-learning uses a large amount of data to learn and discover the patterns existing in the data.

[0084] Step two, detect the target image through multi-dimensional evaluation index. Secondary accurate screening of water surface target picture set, including:

[0085] 1. Precision (Precision): The proportion of correct targets in all detected targets.

[0086]

[0087] 2. Recall (Recall): The proportion of correctly positioned and identified targets in the total number of targets, that is,

[0088]

[0089] True positives is 3, false negatives is 2, so the Recall value is = 0.6

[0090] 3、F1 score: Usually we use Precision and Recal to measure the goodness of the model, but at the same time, the two quantities are balanced, which will affect the speed of decision-making, and F1 score can be used to combine the two quantities (also known as F score, F measure, the name F has no meaning), F1 score [0,1].

[0091]

[0092] P and R correspond to Precision and Recal, respectively, and the larger the value, the better, so F1 score is also larger.

[0093] 4、Precision-Recal curve. By adjusting the threshold, we can choose how many pictures the system can recognize, and thus change the value of Precision or Recal. The changes of Precision and Recal under different threshold conditions are shown in Figure 2 .

[0094] 5、Average precision (AP): the average precision of each class; mean average precision (mAP): calculate the average precision for each class, and then calculate the mean value of all classes.

[0095] Step three, bring the image detection steps 1-5 in step two and the parameters into the unmanned ship autonomous navigation image target detection system based on YOLO algorithm. Through the execution of the sea target hybrid algorithm intelligent detection system test process, the sea target prediction result is obtained. Finally, the

early warning model

[0096] The autonomous navigation of unmanned ships depends on their ability to perceive the environment, so fast and accurate detection of image targets is the fundamental guarantee for the autonomous navigation of unmanned ships. Among several convolutional neural network algorithms, YOLO algorithm has the fastest detection speed, and its processing speed meets the real-time requirement of the environmental perception ability of unmanned ships. The YOLO algorithm process is shown in Figure 3 . The sea target hybrid algorithm intelligent detection system test process includes:

[0097] (1) Extract the target image of the target feature library in step one and run the related instructions for target detection, and the detection rate is 74%

[0098] (2) Input the 74% target images into the grid one by one, and after forward propagation and backward propagation, the preliminary screening of water surface target picture set is obtained.

[0099] (3) Through step two (F1 score, Precision-Recal curve), the labels of the result test set are compared one by one, if the input category and the framed object region are consistent with the label, the detection image is successful, otherwise the detection fails.

[0100] (4) Through step two average precision (Average precision, AP), the detection rate of each class is counted, the average value is obtained, and the average detection rate mAP is obtained.

[0101]

Early warning model

[0102] The state of the method has only two, one is abnormal R1, and the other is normal R2, and the extracted offshore detection target result characteristic value is represented as

[0103] T={t1,t2,...,t 12}Average detection rate mAP value

[0104] According to the conditional probability formula, under the condition of meeting the characteristic value T, the probability of suspected abnormality being judged as R1 is

[0105]

[0106] Where, P(T|R1) represents the probability of the characteristic value T appearing in the abnormal subset in the sample set, and P(R1) represents the proportion of the abnormal subset in the entire sample set.

[0107] Similarly, the probability of suspected abnormality being judged as R2 under the condition of the characteristic value T appearing is

[0108]

[0109] Where, P(T|R2) represents the probability of the characteristic value T appearing in the normal subset in the sample set, and P(R2) represents the proportion of the normal subset in the entire sample set.

[0110] According to the idea of Bayes, each characteristic value t i is independent of each other, after calculating the probability of suspected abnormality being judged as R1 and R2 according to the relative independent event probability, the size of the two probabilities is compared, if P(R1|T)>P(R2|T), it is considered that the monitored average detection rate mAP value is abnormal; otherwise, it is normal.

[0111] The above is only the preferred embodiment of the present application, the protection scope of the present application is not limited to the above-mentioned examples, any technical scheme belonging to the idea of the present application is within the protection scope of the present application. It should be pointed out that for ordinary skilled in the art, some improvements and decorations without departing from the principle of the present application should be regarded as the protection scope of the present application.

Claims

1. A hybrid algorithm-based unmanned surface vehicle inspection and early warning method, characterized in that, The application relates to an unmanned ship target detection method based on a YOLO algorithm. Step one: analyzing a water surface target through water surface image features obtained by an unmanned ship vision system, performing feature extraction on the water surface target, forming a target feature library after the feature extraction, and obtaining an initial screening water surface target picture set through convolutional neural network training; Step two: detecting target images in the initial screening water surface target picture set through multi-dimensional evaluation indexes, and accurately screening the water surface target picture set; The multi-dimensional evaluation indexes include: precision, which is the proportion of correct targets in all detected targets; and recall, which is the proportion of correctly positioned and identified targets in the total number of targets; : , P and R correspond to Precision and Recal, respectively; Precision-Recal curve; average precision, the mean of the average precision for each class; Step three: bringing the indexes obtained through the image detection in step two and parameters into an unmanned ship autonomous navigation image target detection system based on the YOLO algorithm, obtaining a marine target prediction result through an intelligent detection system test process of a marine target hybrid algorithm, finally constructing a warning model to predict a collision state of the marine target prediction result, realizing unmanned ship patrol and early warning; the intelligent detection system test process of the marine target hybrid algorithm includes: obtaining the initial screening water surface target picture set through step one; comparing the labels of the result test set one by one through the F1 score and the Precision-Recal curve in step two, if the input category and the framed object region are consistent with the labels, the detection image is successful, otherwise the detection fails; calculating the detection rate of each class through the average precision in step two, obtaining the average detection rate mAP by averaging; The pre-warning model sets two states, one is abnormal , and the other is normal , and the extracted offshore detection target result characteristic value is represented as: T = { , ,..., }, which represents the average detection rate mAP value; according to the conditional probability formula, the probability that the suspected abnormality is judged as R1 under the condition of the characteristic value T is: , wherein P(T| ) represents the probability of the occurrence of the characteristic value T in the abnormal subset in the sample set, and P( ) represents the proportion of the abnormal subset in the entire sample set; and the probability that the suspected abnormality is judged as R2 under the condition of the characteristic value T is represented as: where P(T|N) represents the probability of the feature value T appearing in the normal subset in the sample set, P(N) represents the proportion of the normal subset in the entire sample set; if the monitored average detection rate mAP value is considered abnormal; otherwise, it is normal.​​​ 2. The unmanned surface vehicle inspection and early warning method based on hybrid algorithm according to claim 1, characterized in that, The water surface target includes a ship, an island and a reef.

3. The unmanned surface vehicle inspection and early warning method based on hybrid algorithm according to claim 1, characterized in that, Step one: performing texture energy, texture entropy and texture gradient feature extraction on the water surface target; The texture energy calculation formula is: The texture entropy calculation formula is: The texture gradient calculation formula is: wherein, is a pixel point in the water surface target image, and k is a coefficient.

4. The unmanned surface vehicle inspection and early warning method based on hybrid algorithm according to claim 1, characterized in that, The convolutional neural network training in step one includes two stages of forward propagation and backward propagation; The forward propagation includes: inputting image data into the convolutional neural network; performing convolution and pooling operations layer by layer, outputting learned parameters of each layer; and finally obtaining more significant features through a full connection layer and an output layer; The backward propagation includes: (1) calculating the residual error and the activation value of the last layer through the network; (2) layer by layer forward transmission of the residual error and the activation value of the last layer through the backward transmission mode, so that the last neuron updates its weight value according to the error; (3) further calculating the gradient of the weight parameter according to the residual error, and adjusting the convolutional neural network parameters again; (4) continuing step (3) until convergence or the maximum iteration number is reached.

Images