A target detection method based on underwater optical images

Abstract

The present invention provides a target detection method based on underwater optical images. The method comprises: step 1: obtaining an underwater optical image target data set, and dividing it into a training set, a validation set and a test set; step 2: improving the YOLOv7 network to construct a target detection model based on underwater optical images, including constructing a new backbone network, a new detection head module and incorporating a new attention mechanism; step 3: using the training set and the validation set to train the target detection model; step 4: using the trained target detection model to detect the test set. The target detection model constructed by the present invention greatly improves the average detection accuracy of underwater target detection, and effectively reduces the scale of model parameters, making the detection model more lightweight, which is conducive to deployment on low-cost underwater detection equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and artificial intelligence, and in particular to a target detection method based on underwater optical images. Background Art

[0002] Underwater target detection technology plays an important role in marine environment monitoring and early warning. Underwater robots equipped with target detection equipment can replace divers to accurately locate and identify targets of interest in underwater environments. The following three factors make underwater target detection tasks quite challenging: (1) Factors such as weather and ocean currents make the underwater environment complex and changeable, and the captured images have low saturation and blurred edges due to environmental noise; (2) There are many types of underwater targets, some of which are small and hidden; (3) The limited storage and computing power of underwater target detection equipment requires a lightweight detection model. Traditional target detection methods based on underwater optical images are not very effective.

[0003] With the development of computer vision and deep learning technology, using deep learning methods to detect targets of interest from underwater optical images has become a research hotspot. Jian Zhang et al. proposed a marine life detection method based on an improved YOLOv5 framework. In addition, Hao Wang et al. improved the Faster RCNN two-stage algorithm for detecting sea cucumbers, sticklebacks, scallops, starfish, and water plants. Although they have achieved considerable results, there are still obvious defects. The detection accuracy of the improved YOLOv5 is still unsatisfactory, and the real-time performance of the improved Faster RCNN cannot meet the requirements. Therefore, improving and innovating underwater target detection technology is an important issue that needs to be solved urgently. Summary of the invention

[0004] In order to take into account the detection accuracy and real-time performance required for target detection tasks based on underwater optical images, the present invention proposes an underwater target detection method. By deeply improving YOLOv7, a high-precision and lightweight YOLOv7-CHS target detection model is constructed, which can better adapt to the special requirements of the underwater environment.

[0005] The present invention provides a target detection method based on underwater optical images, comprising:

[0006] Step 1: Obtain an underwater optical image target dataset and divide it into a training set, a validation set, and a test set;

[0007] Step 2: Improve the YOLOv7 network to build an underwater optical image-based target detection model, including building a new backbone network, a new detection head module, and incorporating a new attention mechanism;

[0008] Step 3: Using the training set and the validation set to train the target detection model;

[0009] Step 4: Use the trained object detection model to detect the test set.

[0010] Furthermore, in step 2, building a new backbone network specifically includes:

[0011] The two ELAN modules in the backbone network of YOLOv7 are replaced by C3HB modules, and a downsampling convolution layer is added after the first C3HB module, and a feature extraction convolution layer is added after the second C3HB module.

[0012] Furthermore, the construction process of the C3HB module specifically includes:

[0013] Use recursive gated convolution RGConv to build a high-order spatial interaction HOSI module;

[0014] The C3HB module is obtained by replacing one convolutional layer in the C3 module with the HOSI module.

[0015] Furthermore, in step 2, constructing a new detection head module specifically includes:

[0016] The ELAN module in the detection head of YOLOv7 is replaced by a CT3 module; wherein the CT3 module includes a first branch, a second branch, a connection layer and a first CBS module; wherein the output of the first branch and the output of the second branch are first spliced ​​through the connection layer, and the spliced ​​feature map is then processed by the first CBS module; the first branch includes a second CBS module and a CT bottleneck module connected thereto, and the second branch includes a third CBS module;

[0017] The CT bottleneck module includes a fourth CBS module and a CT module; wherein the input original feature map is processed by the fourth CBS module and the CT module in sequence and then merged with the input original feature map.

[0018] Furthermore, the incorporation of a new attention mechanism includes:

[0019] The 1×1 convolution block in the MP-2 module in the detection head of YOLOv7 is replaced by the SPFA module to obtain the MP-SPFA module; wherein the SPFA module includes: first generating 3-D weights from features, and then expanding to generate three-dimensional weights that include both channel attention and spatial attention.

[0020] Furthermore, step 3 specifically includes:

[0021] Pre-training the object detection model using the COCO benchmark dataset;

[0022] The pre-trained target detection model is adjusted using the training set and the validation set to obtain a trained target detection model.

[0023] Beneficial effects of the present invention:

[0024] (1) The C3HB module is proposed to construct a new backbone network of the detection model, which can better extract target features in underwater optical images. At the same time, it overcomes the defects of huge number of detection model parameters and high computational complexity, making the improved detection model more lightweight and more conducive to deploying the detection model in the target detection equipment of underwater robots.

[0025] (2) The CT3 module is proposed to replace the ELAN module in the YOLOv7 detection head, which improves the detection effect of the detection model on small underwater targets and improves the detection accuracy of the model to adapt to the harsh underwater detection environment;

[0026] (3) The SPFA attention mechanism is introduced to improve the MP-2 module, which further enhances the visual representation ability of the detection model to more accurately identify blurred underwater images and improves the detection accuracy of the model.

[0027] (4) When training an object detection model based on underwater optical images, based on the transfer learning method, the model weights on the COCO benchmark dataset are first used as the pre-trained weights of the object detection model, and then re-trained on the underwater optical image dataset to accelerate the convergence of the model and enhance the generalization ability;

[0028] (5) The target detection model constructed by the present invention greatly improves the average detection accuracy of underwater target detection and effectively reduces the scale of model parameters, making the detection model more lightweight and conducive to deployment on low-cost underwater target detection equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a flow chart of a target detection method based on underwater optical images provided by an embodiment of the present invention;

[0030] Figure 2 Schematic diagrams of underwater optical images in different environments provided by embodiments of the present invention: (a) dim light environment; (b) fuzzy imaging environment; (c) tiny target environment;

[0031] Figure 3 The overall structure of the YOLOv7-CHS target detection model provided by the embodiment of the present invention;

[0032] Figure 4 Four detection target distributions of underwater optical images provided by the embodiment of the present invention;

[0033] Figure 5 A structural diagram of the HOSI module provided in an embodiment of the present invention;

[0034] Figure 6 A structural diagram of a C3HB module provided in an embodiment of the present invention;

[0035] Figure 7 A structural diagram of a CT module provided in an embodiment of the present invention;

[0036] Figure 8 (a) CT bottleneck module structure diagram provided by the embodiment of the present invention; (b) CT3 module structure diagram;

[0037] Fig. 9 The SPFA module structure diagram provided by the embodiment of the present invention;

[0038] Fig.10 The MP-SPFA module structure diagram provided by the embodiment of the present invention;

[0039] Fig.11 The detection results of the YOLOv7-CHS target detection model provided by the embodiment of the present invention on the test set. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] In order to achieve high-precision and lightweight underwater target detection tasks with excellent performance, this paper proposes a method for underwater target detection based on YOLOv7-CHS, which solves the problems faced by traditional underwater target detection tasks, such as poor underwater imaging quality, tiny targets, and excessively large detection model parameter scale. Figure 1 As shown, the specific implementation is as follows:

[0042] S101: Acquire an underwater optical image target dataset and divide it into a training set, a validation set, and a test set;

[0043] In this embodiment, the underwater optical image target data set collected includes four types of targets to be detected: sea cucumbers, sea urchins, scallops, and starfish. The total number of the four types of targets is 74,515, of which sea cucumbers, sea urchins, scallops, and starfish are 7,887, 50,156, 1,924, and 14,548 respectively (e.g. Figure 4The dataset contains a total of 7782 underwater optical images of different resolutions and corresponding annotation information, which are the categories of the targets in the images and the coordinate values ​​of the bounding box surrounding the targets.

[0044] The collected underwater optical image dataset is made into a YOLO format dataset, which mainly includes two folders: image and label. The image folder contains three subfolders, which are used to store the underwater optical images in jpg format of the training set, validation set and test set respectively. The label folder also contains three subfolders, which are used to store the label information in txt format corresponding to the three subfolders in the image folder respectively. The normalized label information format is [class, x_center, y_center, w, h]. Among them, class represents the target category; (x_center, y_center) represents the center point coordinates of the bounding box; w represents the width of the bounding box; h represents the height of the bounding box.

[0045] S102: Improve the YOLOv7 network to build the YOLOv7-CHS target detection model, including building a new backbone network, a new detection head module, and incorporating a new attention mechanism;

[0046] In order to complete the underwater target detection task more efficiently, the embodiment of the present invention makes in-depth improvements to the YOLOv7 network. The main improvements include building a new backbone network, a new detection head module, and incorporating a new attention mechanism. This embodiment uses Python 3.7 environment on the Windows 10 operating system and builds a target detection model based on the Pytorch deep learning framework. The overall structure of the target detection model proposed in the embodiment of the present invention is as follows: Figure 3 shown.

[0047] (1) Construct a new backbone network to enhance the effective feature extraction of underwater targets

[0048] The working environment of underwater imaging equipment is extremely harsh. Light intensity, temperature, salinity, ocean current speed and direction, and the working state of underwater robots will have a huge impact on the image capture of the underwater environment. Figure 2 The quality of the images (shown) cannot reach the level of optical images in the atmosphere.

[0049] In order to better extract target features in underwater optical images and make the model backbone network more lightweight, the two ELAN modules in the YOLOv7 backbone network are replaced by C3HB modules, and a downsampling convolution layer is added after the first C3HB module, and a feature extraction convolution layer is added after the second C3HB module, so that the backbone network can achieve high-order spatial interaction and the network is more lightweight.

[0050] The construction process of the C3HB module specifically includes: using recursive gated convolution RGConv to construct a high-order spatial interaction HOSI module, such as Figure 5 As shown; the HOSI module is used to replace a convolutional layer in the C3 module to obtain the C3HB module, as shown Figure 6 As shown in the figure, this module enables the detection model to extract underwater optical image features more powerfully and with lighter parameters.

[0051] (2) Propose a new detection head module to improve the detection effect of tiny underwater targets

[0052] Most of the targets in the environmental optical images captured by underwater equipment are small targets smaller than 30×30 pixels and are easily missed. In order to improve their detection rate, the ELAN module is replaced with the CT3 module that can detect small targets in the detection head, and the model's visual modeling ability for local image features is improved by fusing static and dynamic contexts.

[0053] The CT3 module is mainly designed to improve the detection rate of small targets in underwater images. The CT bottleneck module is first constructed using the CT module, and then the CT3 module is constructed from the CT bottleneck module. Regarding the CT module, the input key is first encoded and convolved through the context to represent the static context of the input. Then, the encoded key is connected to the input query through two consecutive convolutions, and then the learned attention matrix is ​​multiplied by the input value to achieve dynamic context representation of the input. The fusion of static and dynamic context representations is finally used as the output. The structure of the CT module is as follows: Figure 7 shown. Figure 8 (a) is the CT bottleneck module, and Figure 8 (b) represents the CT3 module. The CT3 module effectively improves the accuracy of the model in detecting small targets.

[0054] (3) Incorporating a new attention mechanism to improve the detection accuracy and real-time performance of underwater targets

[0055] The cost and real-time performance of underwater robot target detection equipment require that the detection model further improves accuracy without increasing the size of network parameters. To achieve this goal, a new attention mechanism SPFA is proposed to replace a 1×1 convolution operation in the MP-2 module to obtain the MP-SPFA module, with the following structure: Fig.10As shown in Figure 1, the MP-SPFA module consists of two branches: the first branch consists of a maximum pooling layer and a CBS module with a convolution kernel of 1×1 and a stride of 1; the second branch consists of an SPFA module and a CBS module with a convolution kernel of 3×3 and a stride of 2. Compared with the existing channel attention and spatial attention mechanisms, the SPFA module can directly infer the three-dimensional attention weights without adding any parameters, improve the representation ability of the convolution operation, and further effectively improve the detection ability of the model. The structure is shown in Figure 1. Fig. 9 As shown in Figure 2. The SPFA module includes: first generating 3-D weights from features, and then expanding to generate 3D weights that include both channel attention and spatial attention. The same color indicates that a single scalar is used for each point on each feature.

[0056] S103: training the target detection model using a training set and a validation set;

[0057] In order to ensure the reliability and robustness of the experimental results, it is necessary to reasonably select the hyperparameters of model training. The size of the underwater optical image input to the model is 640×640 pixels, the training is performed for 300 iterations, and the training batch size is 8. In the comparative experiment, the Adam optimizer is used to adjust the model parameters, and the learning rate is set to 0.001. In the ablation experiment, the SGD optimizer is used to adjust the model parameters, and the learning rate is set to 0.01.

[0058] When training the model, the transfer learning method is used. A weight file is first trained on the COCO benchmark dataset to obtain a weight file, and then the weight file is used as the pre-trained weight of the YOLOv7-CHS model to train on the target detection dataset based on underwater optical images (training set and validation set), and finally the target detection model is obtained.

[0059] S104: Use the trained target detection model to detect the test set;

[0060] After the detection model is built and trained, the test set is used to evaluate the performance of the YOLOv7-CHS detection model. Under a given confidence threshold, the final detection result is as follows: Fig.11 As shown, it contains the predicted target category and the coordinate information of the predicted bounding box.

[0061] Table 1 compares the detection accuracy and real-time performance of different target detection models

[0062]

[0063] Table 2 Ablation experiment

[0064]

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method based on underwater optical images, characterized in that: include: Step 1: Obtain an underwater optical image target dataset and divide it into a training set, a validation set, and a test set; Step 2: Improve the YOLOv7 network to construct a target detection model based on underwater optical images, including constructing a new backbone network, a new detection head module and incorporating a new attention mechanism; wherein, constructing the new backbone network specifically includes: replacing the first and fourth ELAN modules in the backbone network of YOLOv7 with C3HB modules, and adding a downsampling convolution layer after the first C3HB module, and adding a feature extraction convolution layer after the second C3HB module; the construction process of the C3HB module specifically includes: using recursive gated convolution RGConv to construct a high-order spatial interaction HOSI module; using the HOSI module to replace a convolution layer in the C3 module to obtain the C3HB module; wherein the HOSI module includes a first Layer Norm layer, an RGConv layer, a second Layer Norm layer and an MLP layer connected in sequence, and has a first skip connection before the first Layer Norm layer and after the RGConv layer, and a first skip connection before the second Layer Norm layer. There is a second jump connection before the Norm layer and after the MLP layer; building a new detection head module specifically includes: replacing the ELAN module in the detection head of YOLOv7 with a CT3 module; wherein the CT3 module includes a first branch, a second branch, a connection layer and a first CBS module; wherein the output of the first branch and the output of the second branch are first spliced ​​through the connection layer, and the spliced ​​feature map is then processed by the first CBS module; the first branch includes a connected second CBS module and a CT bottleneck module, and the second branch includes a third CBS module; the CT bottleneck module includes a fourth CBS module and a CT module; wherein the input original feature map is processed by the fourth CBS module and the CT module in sequence, and then merged with the input original feature map; the incorporation of the new attention mechanism includes: replacing the 1×1 convolution block in the MP-2 module in the detection head of YOLOv7 with an SPFA module to obtain an MP-SPFA module; wherein the SPFA module includes: first generating 3-D weights from features, and then expanding to generate three-dimensional weights that include both channel attention and spatial attention; Step 3: Using the training set and the validation set to train the target detection model; Step 4: Use the trained object detection model to detect the test set.

2. The target detection method based on underwater optical images according to claim 1, characterized in that: Step 3 specifically includes: Pre-training the object detection model using the COCO benchmark dataset; The pre-trained target detection model is adjusted using the training set and the validation set to obtain a trained target detection model.

Images