Underwater target detection method and device based on comparative learning, and medium

Abstract

The invention relates to an underwater target detection method and device based on comparative learning and a medium, and the method comprises the following steps: obtaining an underwater image, inputting a trained underwater target detection model based on comparative learning, and obtaining an underwater target detection result; wherein the underwater target detection model based on comparative learning comprises a backbone network, a feature pyramid network and a classification regression network which are connected in sequence, and a target detection precision evaluation module and a comparative feature enhancement module which are connected with the backbone network; wherein the target detection precision evaluation module is used for calculating detection scores of different image enhancement methods, and generating a positive sample and a negative sample corresponding to an original image according to the detection scores; and the comparative feature enhancement module is used for guiding the backbone network to extract features close to the features of the positive samples through a comparative learning method according to the positive samples and the negative samples. Compared with the prior art, the underwater target detection precision can be further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater target detection, and in particular relates to an underwater target detection method, device and medium based on contrast learning. Background Art

[0002] Underwater object detection (UOD) is a technology that uses high-definition cameras and other devices to acquire underwater images and detect and identify targets of interest such as marine life through image recognition algorithms. Due to problems such as uneven background light, jitter, blur, and low visibility, the quality of underwater images varies greatly, which seriously affects the accuracy of underwater object detection. Existing underwater object detection methods often use underwater image enhancement algorithms to alleviate image quality degradation. However, image enhancement is guided by the visual perception quality of the human eye, and it is difficult to determine the accuracy improvement of target detection. How to use existing underwater image enhancement methods to help underwater object detection tasks achieve better performance is a problem worth studying.

[0003] The existing underwater target detection methods use underwater image enhancement in two ways: preprocessing mode and cascade mode. Among them, the preprocessing mode is to use underwater image enhancement as an initial step for underwater target detection. For example, Chinese patent application CN115410078A discloses a low-quality underwater image fish target detection method, in which the image to be identified is input into the input module for image enhancement. However, the underwater image enhancement method used in this mode is pre-trained and the network weights are fixed in advance. Since these enhancement methods are designed for the visual perception quality of the human eye, if they are directly used in the preprocessing step of underwater target detection, in most cases, the improvement of detection accuracy is limited, and even the detection accuracy may decrease; the cascade mode is to cascade the underwater image enhancement network before the underwater target detection network, and regard the two as a jointly optimized whole. However, in the process of extracting features from underwater images and reconstructing enhanced images, this mode is likely to cause the loss of useful information in the original image, and the improvement of target detection accuracy is also limited. Therefore, it is necessary to redesign an underwater target detection method to further improve the accuracy of underwater target detection. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an underwater target detection method, device and medium based on contrast learning, which can further improve the accuracy of underwater target detection.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides an underwater target detection method based on contrast learning, comprising the following steps:

[0007] Acquire underwater images, input the trained underwater target detection model based on contrastive learning, and obtain underwater target detection results;

[0008] The underwater target detection model based on contrastive learning includes a backbone network, a feature pyramid network and a classification regression network connected in sequence, and a target detection accuracy evaluation module and a contrast feature enhancement module connected to the backbone network. The backbone network is used to extract features of multiple scales from underwater images, the feature pyramid network is used to fuse the multi-scale features extracted by the backbone network, and the classification regression network is used to perform underwater target detection according to the fused features.

[0009] The target detection accuracy evaluation module is used to calculate the detection scores of different image enhancement methods, and generate positive samples and negative samples corresponding to the original image based on the detection scores. The detection scores are used to evaluate the impact of different image enhancement methods on the underwater target detection accuracy. The positive sample is the enhanced image with the highest detection score among the enhanced images with higher detection scores than the original image, and the negative sample is the enhanced image with the lowest detection score among the enhanced images with lower detection scores than the original image.

[0010] The contrast feature enhancement module is used to guide the backbone network to extract features close to the positive sample features through a contrast learning method based on the positive samples and negative samples generated by the target detection accuracy evaluation module.

[0011] Furthermore, the image enhancement methods evaluated in the target detection accuracy evaluation module include STSC, TUDA, TACL and TOPAL.

[0012] Furthermore, in the target detection accuracy evaluation module, the detection scores of different image enhancement methods are comprehensively calculated based on multiple target detection models. The specific process is as follows:

[0013] Obtain an original underwater image dataset, and process it using N image enhancement methods respectively to obtain N enhanced underwater image datasets;

[0014] Each enhanced underwater image dataset is used to train and test M target detection models respectively, and the average precision index of each target detection model under each enhanced underwater image dataset is calculated. The average value of the M average precision indexes is taken as the detection score of the image enhancement method.

[0015] Furthermore, the target detection models used in the target detection accuracy evaluation module include Faster-RCNN, Cascade-RCNN, YOLOX, Co-DETR and ATSS.

[0016] Furthermore, in the contrast feature enhancement module, the process of guiding the backbone network through the contrast learning method is specifically as follows: constructing contrast loss, and updating the weights and parameters of the backbone network through back propagation.

[0017] Furthermore, the contrast loss includes a loss term for features at each scale, and the loss term for features at each scale is constructed based on a ratio of the mean absolute errors of original features extracted from the original image and positive sample features and negative sample features extracted from positive samples and negative samples corresponding to the original image.

[0018] Furthermore, the underwater target detection model based on contrastive learning is trained based on a final loss, wherein the final loss includes a total target detection loss and the contrast loss, and the total target detection loss includes target detection losses of the original image and the corresponding positive and negative samples.

[0019] Furthermore, the backbone network is a Resnet50 network.

[0020] The invention also provides an electronic device, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the above method when executing the program.

[0021] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The present invention provides an underwater target detection method based on contrastive learning, and obtains underwater target detection results through an underwater target detection model based on contrastive learning. Specifically, the present invention designs a target detection accuracy evaluation module and a contrast feature enhancement module connected to a backbone network in the model, wherein the target detection accuracy evaluation module is used to calculate the detection scores of different image enhancement methods. The detection scores can be used to evaluate the influence of different image enhancement methods on the underwater target detection accuracy, determine the relationship between the original image and its enhanced image in terms of detection accuracy, and use the enhanced image with the highest detection score among the enhanced images with higher detection scores than the original image as the positive sample corresponding to the original image, and use the enhanced image with the lowest detection score among the enhanced images with lower detection scores than the original image as the negative sample corresponding to the original image. The contrast feature enhancement module guides the backbone network to extract features close to the positive sample features through a contrastive learning method according to the positive and negative samples generated by the target detection accuracy evaluation module. The above method is guided by target detection, can select a suitable image enhancement method for the original image, and guide the backbone network to extract detection-friendly features, which can further improve the accuracy of underwater target detection.

[0024] 2. In the total target detection loss, in addition to the target detection loss of the original image, the target detection loss of the positive and negative samples corresponding to the original image is also considered, which can ensure that the positive and negative samples can always guide the backbone network to extract detection-friendly features in the feature space. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the structure of the underwater target detection model based on contrastive learning.

[0026] Among them, I represents the original image, I E represents the enhanced image, I + and I - Represent positive samples and negative samples respectively, f is the original feature, f + and f - They are positive sample features and negative sample features respectively;

[0027] Figure 2 This is a schematic diagram of the structure of the target detection accuracy evaluation module.

[0028] Among them, mAP is the mean average precision;

[0029] Figure 3 This is a schematic diagram of the structure of the contrast feature enhancement module.

[0030] Among them, f1, f2 and f3 are original features, and is the positive sample feature, and is a negative sample feature. DETAILED DESCRIPTION

[0031] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0032] Example:

[0033] This embodiment provides an underwater target detection method based on contrastive learning, and designs an underwater target detection model based on contrastive learning, such as Figure 1 As shown in Figure 1, the model is guided by target detection and guides the backbone network to extract detection-friendly robust features. After the model is trained, it takes underwater images as input and outputs underwater target detection results.

[0034] like Figure 3As shown, the underwater target detection model based on contrastive learning includes a backbone network, a feature pyramid network and a classification regression network connected in sequence, wherein the backbone network is used to extract features of multiple scales from underwater images, the feature pyramid network is used to fuse the multi-scale features extracted by the backbone network, and the classification regression network is used to detect targets based on the fused features. It should be particularly noted that this embodiment designs a target detection accuracy evaluation module and a contrast feature enhancement module connected to the backbone network, wherein the target detection accuracy evaluation module is used to calculate the detection scores of different image enhancement methods, and generate positive samples and negative samples corresponding to the original image accordingly, and the contrast feature enhancement module is used to guide the backbone network to extract features close to the positive sample features through a contrastive learning method based on the positive and negative samples generated by the target detection accuracy evaluation module. The specific designs of the target detection accuracy evaluation module and the contrast feature enhancement module are as follows:

[0035] A. Object Detection Accuracy Evaluation Module

[0036] In the target detection accuracy evaluation module, the detection scores of different image enhancement methods are calculated comprehensively based on multiple target detection models. The specific process is as follows: obtain the original underwater image dataset, process it separately through N image enhancement methods, and obtain N enhanced underwater image datasets; use each enhanced underwater image dataset to train and test M target detection models, calculate the mean average precision (mAP) of each target detection model under each enhanced underwater image dataset, and take the average of the M mean average precision (mAP) as the detection score of the image enhancement method. The detection score can evaluate the impact of different image enhancement methods on the underwater target detection accuracy. The enhanced image with the highest detection score among the enhanced images with higher detection scores than the original image is taken as the positive sample, and the enhanced image with the lowest detection score among the enhanced images with lower detection scores than the original image is taken as the negative sample.

[0037] In this embodiment, four underwater image enhancement methods that are beneficial to detection performance are selected to generate enhanced images, including STSC, TUDA, TACL and TOPAL. The original data set is processed by these enhancement methods to obtain four enhanced data sets. In order to evaluate the impact of different enhancement methods on detection accuracy, this embodiment selects five typical target detection (UOD) methods, including Faster-RCNN, Cascade-RCNN, YOLOX, Co-DETR and ATSS. Five data sets (including four enhanced data sets and one original data set) are used to train these five UOD methods respectively. Figure 2 As shown in the upper part of , “Model Set - Enhancement n” represents the 5 UOD detection models corresponding to the nth enhancement method. These 25 trained detection models are then used to evaluate the detection accuracy of different enhanced versions (including the original version) of the same image, as shown in Figure 2As shown in the lower part of , for example, for a certain enhanced image processed by enhancement method 2, in order to avoid the domain difference between training and testing, the detection accuracy of the image is tested by 5 detection models trained with the corresponding enhanced image data set, and 5 mAP indicators are obtained, and then the average of these 5 indicators is used as the detection score of the input image. Therefore, for each original image and its different enhanced versions, the detection score can be obtained according to the above process. In other words, the relationship between any given original image and its enhanced image in terms of detection accuracy can be determined. Therefore, the detection performance of the originally uncertain enhanced image can be evaluated, and then the corresponding positive and negative samples of the original image can be determined for subsequent comparative learning.

[0038] It should be noted that, for the original training set of the underwater target detection data set, this embodiment divides it into a new training set and a test set according to a certain ratio, wherein the new training set is used to train 5 typical detection networks in the target accuracy evaluation module, and the detection accuracy is evaluated in the new test set. Therefore, the new test set will be used as a training set to train the final proposed detection network. For ease of explanation, in the target detection accuracy evaluation module, the combination of the above new training set and test set is called a data set.

[0039] B. Contrast Feature Enhancement Module

[0040] Since underwater target detection is to detect targets in underwater images based on high-level semantic information extracted by the backbone network, it is more reasonable to guide in the feature space. Figure 3 As shown in the figure, the original underwater image and a pair of positive and negative samples are simultaneously input into the shared backbone network, that is, they are all sent to the same backbone network. Here, the commonly used Resnet50 network is used as the backbone network. The network extracts three scale features from the input image, which become multi-scale features in the target detection task. For the original image, these three features are f1, f2, and f3, respectively. For the positive sample image and the negative sample image, these three features are and The goal of this embodiment is to make the original features f1, f2, and f3 close to the positive sample features at all scales. Stay away from negative sample features To this end, this embodiment constructs the contrast loss L con , update the weights and parameters of the backbone network through back propagation. Contrastive loss L con The specific expression is as follows:

[0041]

[0042] In the above formula, the three items correspond to the features of three scales respectively. L1(·) is the mean absolute error, which is used to measure the distance between two variables. The numerator is the distance between the original feature and the positive sample feature, and the denominator is the distance between the original feature and the negative sample feature. e = 1×10 -8 , which is used to solve the problem that the denominator may be zero during the training process. Using the above contrast loss, the three-scale features extracted by the backbone network from the original image are guided to be close to the features of the positive sample and away from the features of the negative sample, thereby achieving the purpose of extracting detection-friendly features.

[0043] In the training phase, in order to ensure that positive and negative samples can always guide the backbone network to extract detection-friendly features in the feature space, and It will still go through the feature pyramid network and classification regression sub-network and calculate the target detection loss L det :

[0044] L det =L cls +L reg

[0045] In the above formula, L cls represents the classification loss, L reg Indicates position loss.

[0046] Object detection loss L det is the target detection loss of a certain sample (image). In this embodiment, L det The configuration is the same as that in the Retinanet detection network. In summary, the final loss function is:

[0047]

[0048] In the above formula, and are the object detection losses for original images, positive samples, and negative samples, respectively.

[0049] In order to verify the effectiveness of the above method, this example uses mAP@0.5 and mAP@0.5:0.95 as indicators for evaluating detection accuracy, and compares the performance of the method of the present invention with that of various underwater target detection methods on the RUOD dataset. The results are shown in Table 1.

[0050] The RUOD dataset is a challenging dataset covering a variety of underwater scenes. The target categories included in the dataset include: fish, divers, starfish, corals, turtles, sea urchins, sea cucumbers, scallops, squids, and jellyfish. Its training set contains 9,800 underwater images and the test set contains 4,200 underwater images.

[0051] The comparison methods include 4 general object detection models: RetinaNet, Faster-RCNN, Cascade-RCNN and CO-DETR, and 4 dedicated underwater object detection models: ELR-Net, Booting-RCNN, ROIMIX, GCC-Net.

[0052] Table 1 Performance of various underwater target detection methods on the RUOD dataset

[0053]

[0054] It can be seen from the results that most general target detection methods are difficult to cope with complex underwater target detection tasks. Even though CO-DETR has achieved good accuracy, its Transformer-based network structure makes its detection speed very slow. Compared with the dedicated underwater target detection model, the method of the present invention (Ours) has obvious improvements in both mAP@0.5 and mAP@0.5:0.95 evaluation indicators. For example, the indicators of the ELR-Net algorithm on the RUOD dataset are 0.767 (mAP@0.5) and 0.486 (mAP@0.5:0.95), while the indicators of the method of the present invention (Ours) on the RUOD dataset are 0.783 (mAP@0.5) and 0.489 (mAP@0.5:0.95), which fully proves the superior effect of the method of the present invention.

[0055] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0056] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. An underwater target detection method based on contrastive learning, characterized in that: The following steps are involved: Acquire underwater images, input the trained underwater target detection model based on contrastive learning, and obtain underwater target detection results; The underwater target detection model based on contrastive learning includes a backbone network, a feature pyramid network and a classification regression network connected in sequence, and a target detection accuracy evaluation module and a contrast feature enhancement module connected to the backbone network. The backbone network is used to extract features of multiple scales from underwater images, the feature pyramid network is used to fuse the multi-scale features extracted by the backbone network, and the classification regression network is used to perform underwater target detection according to the fused features. The target detection accuracy evaluation module is used to calculate the detection scores of different image enhancement methods, and generate positive samples and negative samples corresponding to the original image based on the detection scores. The detection scores are used to evaluate the impact of different image enhancement methods on the underwater target detection accuracy. The positive sample is the enhanced image with the highest detection score among the enhanced images with higher detection scores than the original image, and the negative sample is the enhanced image with the lowest detection score among the enhanced images with lower detection scores than the original image. The contrast feature enhancement module is used to guide the backbone network to extract features close to the positive sample features through a contrast learning method based on the positive samples and negative samples generated by the target detection accuracy evaluation module.

2. The underwater target detection method based on contrastive learning according to claim 1, characterized in that: The image enhancement methods evaluated in the target detection accuracy evaluation module include STSC, TUDA, TACL and TOPAL.

3. The underwater target detection method based on contrastive learning according to claim 1, characterized in that: In the target detection accuracy evaluation module, the detection scores of different image enhancement methods are comprehensively calculated based on multiple target detection models. The specific process is as follows: Obtain an original underwater image dataset, and process it using N image enhancement methods respectively to obtain N enhanced underwater image datasets; Each enhanced underwater image dataset is used to train and test M target detection models respectively, and the average precision index of each target detection model under each enhanced underwater image dataset is calculated. The average value of the M average precision indexes is taken as the detection score of the image enhancement method.

4. The underwater target detection method based on contrastive learning according to claim 3 is characterized in that: The target detection models used in the target detection accuracy evaluation module include Faster-RCNN, Cascade-RCNN, YOLOX, Co-DETR and ATSS.

5. The underwater target detection method based on contrastive learning according to claim 1, characterized in that: In the contrast feature enhancement module, the process of guiding the backbone network by contrast learning method is as follows: construct contrast loss, and update the weights and parameters of the backbone network by back propagation.

6. The underwater target detection method based on contrastive learning according to claim 5, characterized in that: The contrast loss includes a loss term for features at each scale, and the loss term for features at each scale is constructed based on a ratio of the mean absolute errors of original features extracted from the original image and positive sample features and negative sample features extracted from positive samples and negative samples corresponding to the original image.

7. The underwater target detection method based on contrastive learning according to claim 5, characterized in that: The underwater target detection model based on contrastive learning is trained based on a final loss, wherein the final loss includes a total target detection loss and the contrast loss, and the total target detection loss includes target detection losses of an original image and corresponding positive and negative samples.

8. The underwater target detection method based on contrastive learning according to claim 1, characterized in that: The backbone network is a Resnet50 network.

9. An electronic device comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images