YOLOv3-based ship target detection and identification method and system

Abstract

The invention provides a YOLOv3-based ship target detection and identification method and system. The method comprises the steps of generating an anchoring frame of a ship target on a ship image data set by adopting a K Means clustering method; calculating a bounding box of the ship target based on the anchoring frame; the method comprises the following steps: dividing an input image into grids with three scales, namely low resolution, middle resolution and high resolution; a Darknet53 network is used as a backbone network, a marked ship image data set is used for training a YOLOv3 network, and hyper-parameters are adjusted to optimize model performance; and inputting a to-be-detected ship image into the trained YOLOv3 network, and outputting a detected ship target position and category information thereof. The anchoring frame is generated by adopting the clustering method, the bounding box closer to the actual demand can be generated according to the actual size and the length-width ratio characteristic of the ship target, and the detection accuracy is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship detection, and in particular, to a ship target detection and recognition method and system based on YOLOv3. Background Art

[0002] In the fields of modern shipping, maritime monitoring, and marine resource management, ship target detection and recognition is a key technology. Traditional ship target detection methods usually rely on manually designed feature extraction algorithms (such as HOG, SIFT, etc.). These methods have limited detection effects in complex backgrounds and are difficult to meet the real-time requirements. In recent years, with the development of deep learning technology, object detection algorithms based on convolutional neural networks (CNNs) have gradually become the mainstream. Among them, the YOLO (You Only Look Once) series of algorithms are widely used in various object detection tasks due to their high efficiency and accuracy.

[0003] However, in the field of ship target detection, due to the diverse ship shape sizes, large aspect ratio variations, and complex background interference, traditional object detection algorithms often perform poorly. To solve these problems, the present invention proposes a ship target detection and recognition method based on the optimization of the YOLOv3 network, which significantly improves the accuracy and efficiency of ship target detection by improving the anchor box design and network structure. Summary of the Invention

[0004] The present invention aims to at least solve one of the technical problems existing in the prior art, and provides a ship target detection and recognition method and system based on YOLOv3.

[0005] In a first aspect, the present invention provides a ship target detection and recognition method based on YOLOv3, and the method includes:

[0006] Anchor box generation: Generating anchor boxes for ship targets of three scales by using the K Means clustering method on a ship image dataset;

[0007] Bounding box generation: Calculating the bounding boxes of ship targets based on the anchor boxes of the ship targets;

[0008] YOLOv3 Network Design: The input image is divided into grids at three scales, namely low resolution, medium resolution, and high resolution. Each grid is responsible for predicting the objects whose center points fall into it, and performs bounding box regression in combination with the generated anchor boxes. The Darknet53 network is used as the backbone network to extract the low-level features of the ship image. The outputs of the last three residual blocks of Darknet53 are used as inputs, and the three scales of inputs correspond to three scales of multi-resolution ship target anchor boxes. The loss function includes the center point loss LDC and the size offset value loss LDS. The loss function LC in the recognition part uses multiple sets of logistic regression to support multi-label classification tasks.

[0009] Model Training and Detection: Use the labeled ship image dataset to train the YOLOv3 network, and adjust the hyperparameters to optimize the model performance. Input the ship image to be detected into the trained YOLOv3 network, and output the detected ship target position and its category information.

[0010] In some possible embodiments, the method of generating anchor boxes by using K Means clustering on the ship image dataset includes:

[0011] Step 1: Given the number of ship anchor box categories as 9, read the set of ship bounding boxes in the training set, and convert the length and width of the bounding boxes into relative values of the pixel size of the image.

[0012] Step 2: Randomly select 9 bounding boxes from the set of bounding boxes as centroids, and each centroid represents an anchor box category.

[0013] Step 3: Set the category for each centroid, that is, label them as 1-9 in sequence, and set the category of each bounding box except the centroid to 0.

[0014] Step 4: Traverse each element in the set of bounding boxes, calculate its Euclidean norm with each centroid, and set its category to the category corresponding to the centroid with the smallest norm.

[0015] Step 5: Calculate the average value of all bounding boxes in each category, use this average value as the virtual centroid, replace the original centroid with the new virtual centroid, and save the original centroid.

[0016] Step 6: Calculate the Euclidean norm of the original centroid and the new virtual centroid in each category. If the centroid norm value in each category is less than the given convergence threshold of 1.5, output all centroids as ship target anchor boxes and the algorithm ends; otherwise, execute Step 3.

[0017] In some possible embodiments, after obtaining the clustering output, the method further includes:

[0018] Arrange the clustering results of the 9 ship anchor boxes according to Sort from largest to smallest, and take the first 3 as the anchor boxes of ship targets with low resolution, the middle 3 as the anchor boxes of ship targets with medium resolution, and the last 3 as the anchor boxes of ship targets with high resolution.

[0019] In some possible embodiments, calculating the bounding box of the ship target based on the anchor box of the ship target includes:

[0020] The position and size information of the bounding box uses the proportional information relative to the pixel size of the original ship image. For the relative abscissa of the center point of the i-th bounding box The relative ordinate of the center point The relative width offset value of the bounding box And the relative height offset value of the bounding box Given the known pixel size x i , y i , w i And h i of the true bounding box, the calculation method is:

[0021]

[0022] In a second aspect, an embodiment of the present invention provides a ship target detection and recognition system based on YOLOv3. The system includes:

[0023] An anchor box generation module, configured to generate anchor boxes of ship targets at three scales by using the K Means clustering method on a ship image dataset;

[0024] A bounding box generation module, configured to: calculate the bounding box of the ship target based on the anchor box of the ship target;

[0025] A YOLOv3 network design module, configured to divide the input image into grids at three scales, namely low resolution, medium resolution, and high resolution. Each grid is responsible for predicting the targets whose center points fall into it, and perform bounding box regression in combination with the above-generated anchor boxes; use the Darknet53 network as the backbone network to extract the low-level features of the ship image, use the outputs of the last three residual blocks of Darknet53 as the input, and the inputs at these three scales correspond to multi-resolution ship target anchor boxes at three scales; the loss function includes a center point loss LDC and a size offset value loss LDS, and the loss function LC of the recognition part uses multiple sets of logistic regression to support multi-label classification tasks;

[0026] The training and detection module is used to train the YOLOv3 network using the labeled ship image dataset, adjust the hyperparameters to optimize the model performance, input the ship image to be detected into the trained YOLOv3 network, and output the detected ship target position and its category information.

[0027] In some possible embodiments, the anchor box generation module is further specifically configured to:

[0028] Step 1: Given the number N of ship anchor box categories, read the set of ship bounding boxes in the training set, and convert the length and width of the bounding boxes into relative values of the pixel size of the image.

[0029] Step 2: Randomly select N bounding boxes from the set of bounding boxes as centroids, and each centroid represents an anchor box category.

[0030] Step 3: Set a category for each centroid, that is, label them as 1 - N in sequence, and set the category of each bounding box except the centroid to 0.

[0031] Step 4: Traverse each element in the set of bounding boxes, calculate its Euclidean norm with each centroid, and set its category to the category corresponding to the centroid with the minimum norm.

[0032] Step 5: Calculate the average value of all bounding boxes of each category, use this average value as the virtual centroid, replace the original centroid with the new virtual centroid, and save the original centroid.

[0033] Step 6: Calculate the Euclidean norm of the original centroid and the new virtual centroid of each category. If the centroid norm value of each category is less than the given convergence threshold of 1.5, output all centroids as the ship target anchor boxes and end the algorithm; otherwise, execute Step 3.

[0034] In some possible embodiments, the anchor box generation module is further specifically configured to:

[0035] After obtaining the clustering output, sort the clustering results of 9 ship anchor boxes in descending order, take the first 3 as the low - resolution ship target anchor boxes, the middle 3 as the medium - resolution ship target anchor boxes, and the last 3 as the high - resolution ship target anchor boxes.

[0036] In some possible embodiments, the bounding box generation module is further specifically configured to:

[0037] The position and size information of the bounding box adopts the proportional information relative to the pixel size of the original ship image. For the relative abscissa of the center point of the i - th bounding box The relative ordinate of the center point The relative width offset value of the bounding box Relative offset value with respect to the bounding box height The known true pixel dimensions x of the bounding box i 、y i 、w i and h i ,which is calculated as follows:

[0038]

[0039] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0040] One or more processors;

[0041] A storage unit for storing one or more programs, which when executed by the one or more processors, can enable the one or more processors to implement the YOLOv3-based ship target detection and recognition method described above.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it can implement the YOLOv3-based ship target detection and recognition method described above.

[0043] The beneficial effects of the present invention are as follows:

[0044] The YOLOv3-based ship target detection and recognition method and system according to the embodiments of the present invention can generate anchor boxes by using the K-Means clustering method, and can generate bounding boxes closer to the actual needs according to the actual size and aspect ratio characteristics of the ship target, significantly improving the detection accuracy. In addition, the multi-resolution mechanism combined with grid partitioning at three scales (low resolution, medium resolution, and high resolution) enhances the adaptability of the model to ship targets of different sizes, thereby improving the overall detection accuracy. In addition, the YOLOv3 network is an end-to-end object detection algorithm, avoiding the candidate region generation step in traditional two-stage detection methods, greatly improving the detection speed, and meeting the real-time requirements. Moreover, using Darknet53 as the backbone network for transfer learning reduces the training difficulty while extracting low-level features, further accelerating the model convergence speed and inference efficiency. For the ship target detection problem in complex backgrounds, through the multi-resolution mechanism and optimized loss function design, the adaptability of the model to background interference and target diversity is enhanced. The design of the loss function supporting multi-label classification tasks (using multiple sets of logistic regression instead of softmax regression) enables the model to better handle the multiple category attributes that ship targets may have (such as "general cargo ship" belonging to the "cargo ship" category at the same time), improving the flexibility and accuracy of the classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of an example electronic device for implementing a ship target detection and recognition method based on YOLOv3 according to an embodiment of the present invention;

[0046] Figure 2 Flowchart of a ship target detection and recognition method based on YOLOv3 according to another embodiment of the present invention;

[0047] Figure 3 Cluster result diagram of ship target anchor boxes according to another embodiment of the present invention;

[0048] Figure 4 Curve graph of the maximum norm decay of ship target anchor box clustering according to another embodiment of the present invention;

[0049] Figure 5 Network structure diagram of ship target detection and recognition according to another embodiment of the present invention;

[0050] Figure 6 Schematic diagram of the structure of a ship target detection and recognition system based on YOLOv3 according to another embodiment of the present invention. Detailed implementation manners

[0051] The technical solutions of the present invention will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the protection scope of the present invention.

[0052] Figure 1 Schematic diagram of an example electronic device for implementing a ship target detection and recognition method based on YOLOv3 according to an embodiment of the present invention. As Figure 1 shown, the electronic device 100 includes one or more processors 110, one or more storage devices 120, one or more input devices 130, one or more output devices 140, etc., and these components are interconnected through a bus system 150 and / or other forms of connection mechanisms. It should be noted that Figure 1 the components and structure of the shown electronic device are exemplary and not restrictive. According to needs, the electronic device may also have other components and structures.

[0053] The processor 110 may be a central processing unit (CPU), or may be composed of multiple processing cores, or may be other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0054] The storage device 120 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor may run the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present disclosure described below and / or other desired functions. Various application programs and various data may also be stored in the computer-readable storage media. For example, various data used and / or generated by the application programs, etc.

[0055] The input device 130 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, etc.

[0056] The output device 140 may output various information (such as images or sounds) to the outside (such as the user), and may include one or more of a display, a speaker, etc.

[0057] Figure 2 The flowchart of the ship target detection and recognition method based on YOLOv3 according to another embodiment of the present invention is as Figure 2 shown. A ship target detection and recognition method based on YOLOv3, the method includes steps S210 to step S240:

[0058] Step S210, anchor box generation: Generate anchor boxes for ship targets of three scales by using the K Means clustering method on the ship image dataset.

[0059] In the traditional ship target detection and recognition network, the target detection and recognition system generally adopts a direct bounding box prediction method, that is, directly regresses and predicts the specific position and size information of the bounding box. The most representative network using this method is the ship target detection network based on YOLOv1, and the performance of this network in experiments is not good.

[0060] The inventor of the present invention found through analysis that since the convolutional neural network is a network structure designed based on the idea of spatial filters, it performs not well in numerical regression problems, and the bounding box prediction in the ship target detection and recognition system is exactly a typical numerical regression problem. In order to reduce the difficulty of the bounding box regression of the convolutional neural network, one idea is to reduce the value range of this numerical value, and the anchor box mechanism starts from this idea.

[0061] In the problem of ship target detection and recognition, the size of the bounding box is not a very flexible value. Targets of the same type often have relatively similar shape sizes and aspect ratios. Specifically, ships usually have a flat and long rectangular boundary contour. The bounding box contours of ships of the same type are similar, while the main differences in ships of different types are reflected in the aspect ratio and size of the bounding box. For example, all container ships tend to have a flat and long shape, while all passenger ships tend to have a thick and short shape.

[0062] Therefore, it is possible to consider counting the common bounding boxes of several typical ship types. Here, this common bounding box is defined as the anchor box, and the offset value of the size is calculated using the actual bounding box and the anchor box that is most similar in shape to it. These offset values are generally small compared to the size of the ship's bounding box, which can reduce the difficulty of parameter regression in the regression problem of the convolutional layer.

[0063] Since the ship target anchor box only represents the shape contours of several typical ship types, the anchor box only has two attributes, length and width, and does not have the position information of the positioning point. First, define the relative width of the ship target anchor box and the relative height The relative size values are not the actual pixel size information, but the ratio information of the actual pixel sizes w and h of the anchor box relative to the pixel size of the original ship image. For a ship image with a resolution of W×H, and The calculation methods are:

[0064]

[0065] In the embodiment of the present invention, the method based on K Means clustering is used to generate anchor boxes on the ship image dataset. Its basic idea is: starting from the Euclidean norm of the input ship bounding box, find several groups with the smallest norm values among each other, and take the average value of all feature vectors of each group as the clustering result output. It should be noted that the number of ship target anchor boxes needs to be specified in advance in KMeans clustering. Since the YOLOv3 network uses outputs of 3 resolutions and predicts 3 anchor boxes at each resolution, 3×3 = 9 types of ship target anchor boxes need to be generated.

[0066] The algorithm flow for generating anchor boxes based on K Means clustering is as follows:

[0067] (1) Given that the number of ship anchor box categories is 9, read the set of ship bounding boxes in the training set, and convert the length and width of the bounding boxes into relative values of the pixel size of the image;

[0068] (2) Randomly select 9 bounding boxes from the set of bounding boxes as centroids, and each centroid represents an anchor box category;

[0069] (3) Set a category for each centroid, that is, label them as 1 - 9 in sequence, and set the category of each bounding box except the centroid to 0;

[0070] (4) Traverse each element in the set of bounding boxes, calculate its Euclidean norm with each centroid, and set its category to the category corresponding to the centroid with the minimum norm;

[0071] (5) Calculate the average value of all bounding boxes of each category, and use this average value as the virtual centroid. Replace the original centroid with the new virtual centroid, and save the original centroid;

[0072] (6) Calculate the Euclidean norm between the original centroid and the new virtual centroid of each category. If the centroid norm value of each category is less than the given convergence threshold of 1.5, output all centroids as the ship target anchor boxes and end the algorithm; otherwise, execute step (3).

[0073] Refer to Figure 3 as well. During the clustering experiment, the migration process of the centroid of the anchor boxes of 9 categories of ships is shown in the following figure, and the clustering results of the anchor boxes of each category are as Figure 3 shown. From the clustering process, the migration of the centroid conforms to the expected multi - resolution mechanism, that is, the convergence regions of each centroid have relatively obvious scale divisions, which conforms to the basic idea of the multi - resolution mechanism.

[0074] Refer to Figure 4 as well. Figure 4 shows the maximum norm decay curve of the ship target anchor box clustering. It can be seen that the algorithm basically converges at the 5th iteration and finally converges at the 59th iteration. The overall running time of the algorithm is relatively fast, and the convergence effect is ideal.

[0075] After obtaining the clustering output, sort the clustering results of 9 ship anchor boxes from largest to smallest, take the first 3 as the ship target anchor boxes with low resolution, the middle 3 as the ship target anchor boxes with medium resolution, and the last 3 as the ship target anchor boxes with high resolution. The results are shown in Table 1 below.

[0076] Table 1 Generation results of ship target anchor boxes

[0077]

[0078] Step S220, Bounding box generation: Calculate the bounding box of the ship target based on the anchor box of the ship target.

[0079] Specifically, in this step, for the convenience of designing the network loss function, the position and size information of the bounding box also adopts the proportional information relative to the pixel size of the original ship image. For the relative abscissa of the center point of the i - th bounding box Relative ordinate of the center point Relative width offset value of the bounding box And relative height offset value of the bounding box Known true pixel size x of the bounding box i 、y i 、w i And h i , and its calculation method is as follows:

[0080]

[0081] Through the above method, after obtaining the anchor box of the ship target, the label information of the ship target can be calculated, and this information will be applied to the design of the loss function and the gradient descent algorithm of the network.

[0082] Step S230, YOLOv3 network design: Divide the input image into grids of three scales, namely low resolution, medium resolution and high resolution. Each grid is responsible for predicting the target whose center point falls into it, and combines the generated anchor box for bounding box regression; Use the Darknet53 network as the backbone network to extract the low-level features of the ship image, use the outputs of the last three residual blocks of Darknet53 as the input, and the three scales of input correspond to three scales of multi-resolution ship target anchor boxes; The loss function includes the center point loss LDC and the size offset value loss LDS, and the loss function LC of the recognition part uses multiple groups of logistic regression to support multi-label classification tasks.

[0083] Specifically, in this step, first give the ship target and recognition network based on YOLOv3, and the network structure is as Figure 5 shown. The YOLOv3 network contains two special structures: the residual block (Res) and the basic convolutional block (DBL). Among them, ResN represents that the residual block contains N residual units (ResU), and the basic convolutional block is composed of a convolutional layer, a max pooling layer and a Leaky Relu activation function.

[0084] The essential idea of YOLOv3 is grid division, and each grid is only responsible for predicting the target whose center point falls into it. Restricted by the structure of the current convolutional neural network and the calculation method of the GPU, the input image size and output feature tensor size of the network are fixed, which results in the network can only predict a fixed number of targets. Therefore, the inventor of the present invention integrates the idea of multi-resolution on the basis of YOLOv3's grid division, so that the grid can be divided and the prediction box can be output from three scales respectively, greatly increasing the number of detectable targets.

[0085] Specifically, in this step, similar to the design of the ship target anchor boxes, three resolutions of outputs are adopted, namely low resolution, medium resolution, and high resolution. At the low resolution, the grid is divided into 13×13 grids, at the medium resolution into 26×26 grids, and at the high resolution into 52×52 grids. The grid division mentioned here is not an intuitive division method, but rather the original ship image is compressed into a feature map through convolutional operations and each unit of the feature map is used to represent a grid. In other words, each unit in the output layer feature map corresponds to a region in the original ship image, and this region can be regarded as a grid. Therefore, the number of divided grids is equal to the dimension of the output layer feature map. Since nine different ship target anchor boxes are calculated in this paper, that is, each resolution contains three different shapes of ship target anchor boxes, each grid needs to be responsible for predicting the outputs of all three ship target anchor boxes corresponding to the corresponding resolution, that is, one grid can predict 3 ship targets with different outer dimensions. In summary, this network structure can predict at most (13×13 + 26×26 + 52×52)×3 = 10647 ship targets, and this target quantity is sufficient to meet the requirements of most ship target detection and recognition tasks.

[0086] Furthermore, the YOLOv3 network of the embodiment of the present invention adopts a transfer learning method. The so-called transfer learning is a method of constructing a new network structure based on other mature and already trained basic networks. This basic network is also called the backbone network. In YOLOv3, the Darknet53 network is adopted. Transfer learning generally does not directly transplant the entire backbone network. Instead, a new network is often designed on the basis of removing the output layer or the last few sub-networks of the network. Generally, it is considered that the first few layers of a convolutional neural network are a low-level image feature extractor, and the work of this low-level feature extraction is similar in different networks. The backbone network, as a trained network structure, can be regarded as a perfect ship image feature extractor. Designing a ship target detection network based on Darknet53 can avoid retraining the network's ability to extract low-level ship image features, greatly reducing the training difficulty of the network. Therefore, the parameters of the backbone network generally do not need to participate in the gradient descent update during the training process. YOLOv3 uses the outputs of the last three residual blocks of Darknet53 as inputs, and the three-scale inputs correspond to three-scale multi-resolution ship target anchor boxes.

[0087] The loss function of the ship target detection and recognition network takes into account the multi-label problem of multiple samples for future function expansion. In ship target detection and recognition, often a target can be divided into multiple labels. For example, a general cargo ship can be classified as "general cargo ship" or "cargo ship". Therefore, multiple sets of logistic regression are considered to replace softmax regression. Each ship label is predicted separately, and the one with the highest confidence is taken as the category of the current target. For each scale, the loss function L can be divided into two parts: the loss function LD of ship target detection and the loss function LC of ship target recognition. First, define the grid division at this scale as S×S, and each grid predicts B ship targets, where the value here is 3. The loss function LD of ship target detection includes two parts: the center point loss LDC and the size offset value loss LDS, that is:

[0088]

[0089] L D = L DC + L DS

[0090] where λcoord is a hyperparameter used to adjust the proportion of ship target detection in the loss function. is used to judge whether there is a target in the grid i,j. It is 1 when there is a target and 0 when there is no target.

[0091] The following gives the loss function LC of ship target recognition at a certain scale. LC is divided into three parts: the confidence loss LCO with target, the confidence loss LCNO without target, and the category loss LCC. Let the true label of the sample be C, the true confidence be P, and the predicted label be The predicted confidence is where P has only two values, 0 and 1. It is 1 when there is a target and 0 when there is no target. The loss function of target recognition can be described as:

[0092]

[0093]

[0094] L C = L CO + L CNO + L CC

[0095] where λ noobj is a hyperparameter used to reduce the contribution of the loss in the non-target area to the training process. is exactly the opposite of . It is 0 when there is a target and 1 when there is no target.

[0096] In summary, the overall loss function of YOLOv3 can be expressed as:

[0097] L=L D +L C

[0098] Step S240, model training and detection: Use the labeled ship image dataset to train the YOLOv3 network, adjust the hyperparameters to optimize the model performance; input the ship image to be detected into the trained YOLOv3 network, and output the detected ship target position and its category information.

[0099] The ship target detection and recognition method based on YOLOv3 of the embodiment of the present invention generates an anchor box by adopting the K-Means clustering method, and can generate a bounding box that is closer to the actual needs according to the actual size and aspect ratio characteristics of the ship target, thereby significantly improving the detection accuracy. In addition, the multi-resolution mechanism combines the grid division of three scales (low resolution, medium resolution and high resolution) to enhance the adaptability of the model to ship targets of different sizes, thereby improving the overall detection accuracy. In addition, the YOLOv3 network is an end-to-end target detection algorithm that avoids the candidate region generation step in the traditional two-stage detection method, greatly improves the detection speed, and can meet the real-time requirements. In addition, Darknet53 is used as the backbone network for transfer learning, which extracts low-level features while reducing the difficulty of training, and further accelerates the model convergence speed and reasoning efficiency. For the problem of ship target detection under complex backgrounds, the model's adaptability to background interference and target diversity is enhanced through the multi-resolution mechanism and optimized loss function design. The loss function design that supports multi-label classification tasks (using multiple groups of logistic regression instead of softmax regression) enables the model to better handle the multiple category attributes that ship targets may have (such as "general cargo ship" also belongs to the "cargo ship" category), thereby improving the flexibility and accuracy of the classification results.

[0100] In summary, the present invention performs well in the field of ship target detection and recognition. It not only improves the detection accuracy and real-time performance, but also enhances the robustness and applicability of the model, providing strong technical support for maritime monitoring, port management, and marine resource development.

[0101] Based on the same inventive concept, Figure 6 As shown, an embodiment of the present invention further provides a ship target detection and recognition system based on YOLOv3, the system comprising: an anchor box generation module 610, a bounding box generation module 620, a YOLOv3 network design module 630 and a training and detection module 640.

[0102] The anchor box generation module 610 is used to generate anchor boxes for ship targets at three scales by using the K Means clustering method on the ship image dataset; the bounding box generation module 620 is used to: calculate the bounding box of the ship target based on the anchor box of the ship target.

[0103] The YOLOv3 network design module 630 is used to divide the input image into grids at three scales, namely low resolution, medium resolution and high resolution. Each grid is responsible for predicting the targets whose center points fall into it, and combines the generated anchor boxes above for bounding box regression; uses the Darknet53 network as the backbone network to extract the low-level features of the ship image, uses the outputs of the last three residual blocks of Darknet53 as the input, and the inputs at these three scales correspond to multi-resolution ship target anchor boxes at three scales; the loss function includes the center point loss LDS and the size offset value loss LDS, and the loss function LC of the recognition part uses multiple sets of logistic regression to support multi-label classification tasks.

[0104] The training and detection module 640 is used to train the YOLOv3 network using the labeled ship image dataset, adjust the hyperparameters to optimize the model performance; input the ship image to be detected into the trained YOLOv3 network, and output the detected ship target position and its category information.

[0105] In some possible embodiments, the anchor box generation module 610 is specifically further used for:

[0106] Step 1: Given the number N of ship anchor box categories, read the set of ship bounding boxes in the training set, and convert the length and width of the bounding boxes into relative values of the pixel size of the picture;

[0107] Step 2: Randomly select N bounding boxes from the set of bounding boxes as centroids, and each centroid represents an anchor box category;

[0108] Step 3: Set the category for each centroid, that is, label them as 1-N in sequence, and set the category of each bounding box except the centroid to 0;

[0109] Step 4: Traverse each element in the set of bounding boxes, calculate its Euclidean norm with each centroid, and set its category to the category corresponding to the centroid with the smallest norm;

[0110] Step 5: Calculate the average value of all bounding boxes of each category, use this average value as the virtual centroid, replace the original centroid with the new virtual centroid, and save the original centroid;

[0111] Step 6: Calculate the Euclidean norms of the original centroids and the new virtual centroids of each category. If the centroid norm values of each category are less than the given convergence threshold of 1.5, output all centroids as the ship target anchor boxes and end the algorithm; otherwise, execute Step 3.

[0112] In some possible embodiments, the anchor box generation module 610 is further specifically configured to:

[0113] After obtaining the clustering output, sort the clustering results of the 9 ship anchor boxes in descending order, and take the first 3 as the ship target anchor boxes with low resolution, the middle 3 as the ship target anchor boxes with medium resolution, and the last 3 as the ship target anchor boxes with high resolution.

[0114] In some possible embodiments, the bounding box generation module 620 is further specifically configured to:

[0115] The position and size information of the bounding box uses the proportional information relative to the pixel size of the original ship image. For the relative abscissa of the center point of the i-th bounding box the relative ordinate of the center point the relative width offset value of the bounding box and the relative height offset value of the bounding box Given the known true pixel size x of the bounding box i 、y i 、w i and h i , the calculation method is:

[0116]

[0117] Based on the same inventive concept, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement the YOLOv3-based ship target detection and recognition method described above.

[0118] Among them, the computer-readable medium can be included in the device, equipment, and system of the present disclosure, or can exist independently.

[0119] Among them, the computer-readable storage medium can be any tangible medium that contains or stores a program, and it can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment. More specific examples include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, an optical fiber, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0120] Among them, the computer-readable storage medium may also include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Specific examples thereof include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination thereof.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A ship target detection and recognition method based on YOLOv3, characterized in that: The method comprises: Anchor box generation: The K Means clustering method is used on the ship image dataset to generate anchor boxes of ship targets at three scales; Bounding box generation: obtaining a bounding box of the ship target based on the anchor box of the ship target; YOLOv3 network design: The input image is divided into three scales of grids, namely low resolution, medium resolution and high resolution. Each grid is responsible for predicting the target whose center point falls into it, and performing bounding box regression in combination with the generated anchor box; the Darknet53 network is used as the backbone network to extract low-level features of the ship image, and the output of the last three residual blocks of Darknet53 is used as input. The input of these three scales corresponds to the multi-resolution ship target anchor box of three scales; the loss function includes the center point loss LDC and the size bias value loss LDS. The loss function LC of the recognition part adopts multi-group logistic regression to support multi-label classification tasks; Model training and detection: Use the labeled ship image dataset to train the YOLOv3 network and adjust the hyperparameters to optimize the model performance. Input the ship image to be detected into the trained YOLOv3 network and output the detected ship target location and its category information.

2. The ship target detection and recognition method based on YOLOv3 according to claim 1, characterized in that: The method of using K Means clustering to generate anchor frames on the ship image dataset includes: Step 1: Given the number of ship anchor box categories, 9, and reading the ship bounding box set of the training set, convert the length and width of the bounding box into relative values ​​of the image pixel size; Step 2: Randomly select 9 bounding boxes from the bounding box set as centroids, each centroid represents an anchor box category; Step 3: Set a category for each centroid, i.e., mark them as 1-9 in sequence, and set the category of each bounding box except the centroid to 0; Step 4: traverse each element in the bounding box set, calculate its Euclidean norm with each centroid, and set its category to the category corresponding to the centroid with the smallest norm; Step 5: Calculate the average value of all bounding boxes of each category, use the average value as the virtual centroid, replace the original centroid with the new virtual centroid, and save the original centroid; Step 6: Calculate the Euclidean norm of the original centroid and the new virtual centroid of each category. If the centroid norm value of each category is less than the given convergence threshold 1.5, then output all centroids as the ship target anchor box and the algorithm ends; otherwise, execute step 3.

3. The ship target detection and recognition method based on YOLOv3 according to claim 2, characterized in that: After obtaining the clustering output, the method further includes: The clustering results of the 9 ship anchor frames are classified according to Sort them from large to small, take the first 3 as low-resolution ship target anchor boxes, the middle 3 as medium-resolution ship target anchor boxes, and the last 3 as high-resolution ship target anchor boxes.

4. The ship target detection and recognition method based on YOLOv3 according to any one of claims 1 to 3, characterized in that: The step of calculating a bounding box of the ship target based on the anchor box of the ship target includes: The position and size information of the bounding box adopts the ratio information of the pixel size of the original ship image. For the center point of the i-th bounding box, the relative horizontal coordinate Center point relative vertical coordinate Bounding box relative width offset Relative offset to the bounding box height Known true bounding box pixel size x i ,y i 、w i and h i , which is calculated as:

5. A ship target detection and recognition system based on YOLOv3, characterized in that: The system comprises: Anchor frame generation module, used to generate anchor frames of ship targets of three scales using K Means clustering method on the ship image dataset; A bounding box generation module is used to: calculate the bounding box of the ship target based on the anchor box of the ship target; The YOLOv3 network design module is used to divide the input image into three scales of grids, namely low resolution, medium resolution and high resolution. Each grid is responsible for predicting the target whose center point falls into it, and performing bounding box regression in combination with the anchor box generated above; the Darknet53 network is used as the backbone network to extract the low-level features of the ship image, and the output of the last three residual blocks of Darknet53 is used as input, and the multi-resolution ship target anchor boxes of three scales are corresponded through the input of these three scales; the loss function includes the center point loss LDC and the size bias value loss LDS, and the loss function LC of the recognition part adopts multiple groups of logistic regression to support multi-label classification tasks; The training and detection module is used to train the YOLOv3 network using the labeled ship image dataset and adjust the hyperparameters to optimize the model performance; the ship image to be detected is input into the trained YOLOv3 network, and the detected ship target position and its category information are output.

6. The ship target detection and recognition system based on YOLOv3 according to claim 5, characterized in that: The anchor frame generation module is further specifically used for: Step 1: Given the number of ship anchor box categories N, read the ship bounding box set of the training set, and convert the length and width of the bounding box into relative values ​​of the image pixel size; Step 2: Randomly select N bounding boxes from the bounding box set as centroids, each centroid represents an anchor box category; Step 3: Set a category for each centroid, i.e. mark them as 1-N in sequence, and set the category of each bounding box except the centroid to 0; Step 4: traverse each element in the bounding box set, calculate its Euclidean norm with each centroid, and set its category to the category corresponding to the centroid with the smallest norm; Step 5: Calculate the average value of all bounding boxes of each category, use the average value as the virtual centroid, replace the original centroid with the new virtual centroid, and save the original centroid; Step 6: Calculate the Euclidean norm of the original centroid and the new virtual centroid of each category. If the centroid norm value of each category is less than the given convergence threshold 1.5, then output all centroids as the ship target anchor box and the algorithm ends; otherwise, execute step 3.

7. The ship target detection and recognition system based on YOLOv3 according to claim 6, characterized in that: The anchor frame generation module is further specifically used for: After obtaining the clustering output, the clustering results of the 9 ship anchor frames are classified according to Sort them from large to small, take the first 3 as low-resolution ship target anchor boxes, the middle 3 as medium-resolution ship target anchor boxes, and the last 3 as high-resolution ship target anchor boxes.

8. The ship target detection and recognition system based on YOLOv3 according to any one of claims 5 to 7, characterized in that: The bounding box generation module is further specifically used for: The position and size information of the bounding box adopts the ratio information of the pixel size of the original ship image. For the center point of the i-th bounding box, the relative horizontal coordinate Center point relative vertical coordinate Bounding box relative width offset Relative offset to the bounding box height Known true bounding box pixel size x i ,y i 、w i and h i , which is calculated as:

9. An electronic device, characterized in that: include: one or more processors; A storage unit, used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the ship target detection and recognition method based on YOLOv3 according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can implement the ship target detection and recognition method based on YOLOv3 according to any one of claims 1 to 4.

Images