A fish body detection method and system based on a decoupled cross-domain fish detection model

Abstract

The application discloses a fish body detection method and system based on a decoupling cross-domain fish detection model, relates to the technical field of fish body detection, and comprises the following steps: based on a multiscale feature enhancement detector in the decoupling cross-domain fish detection model, image with source domain region suggestion or image with target domain region suggestion in multiscale fish body data is extracted; the extracted image is input into a category adaptive module, the foreground region prediction suggestion of the image with the target domain region suggestion in the fish body image and the category pseudo label are determined through category adaptive adversarial learning; the foreground region prediction suggestion is input into a regression adaptive module, and the bounding box regression pseudo label of the image with the target domain region suggestion in the fish body image is determined through regression adaptive adversarial learning. The application combines the field adaptive technology and the target detection in aquaculture, so that the training of the detection model does not depend on a large amount of data labeling, the workload of data labeling is reduced, and the fish body detection efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of fish detection technology, and in particular to a fish detection method and system based on a decoupled cross-domain fish detection model. Background Technology

[0002] Intelligent aquaculture, as an important industry in China, represents a major development trend in modern fisheries. Automated monitoring of fish growth status is the foundation of precision aquaculture, while fish detection is the core of automated growth status monitoring. Therefore, low-cost, efficient, and reliable fish target detection in aquaculture can lay the foundation for fish behavior analysis and growth status monitoring.

[0003] Currently, deep learning-based fish detection algorithms still face some challenges in practical applications of factory farming. In existing detection algorithms, the training of the detection model depends entirely on the labeling information in the dataset. Before training the model, a large amount of data labeling is required. Furthermore, once the aquaculture scenario changes or is replaced, the corresponding training dataset needs to be reacquired and labeled. Therefore, the fish detection efficiency of existing detection algorithms is low. Summary of the Invention

[0004] The purpose of this invention is to provide a fish detection method and system based on a decoupled cross-domain detection model, which can detect fish in different aquaculture scenarios based on a small amount of cross-domain fish data.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] This invention provides a fish detection method based on a decoupled cross-domain fish detection model, comprising:

[0007] Obtain a cross-domain fish dataset; the cross-domain fish dataset includes multi-scale fish data under different aquaculture scenarios, and the multi-scale fish data consists of images of fish at different scales taken due to the different relative distances between the fish and the underwater camera.

[0008] Based on the multi-scale feature enhancement detector in the decoupled cross-domain fish detection model, feature extraction is performed on the multi-scale fish body data to determine fish body images at each scale; the fish body images are images with source domain region proposals or images with target domain region proposals.

[0009] The image with source domain region proposal and the image with target domain region proposal are input into the category adaptation module in the decoupled cross-domain fish detection model. Through category adaptive adversarial learning, the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image are determined.

[0010] The foreground region prediction suggestion is input into the regression adaptive module in the decoupled cross-domain fish detection model. Through regression adaptive adversarial learning, the bounding box regression pseudo-label of the target domain region suggestion image in the fish image is determined.

[0011] Based on the fish image, the category pseudo-label, and the bounding box regression pseudo-label, the decoupled cross-domain fish detection model is iteratively trained.

[0012] Multi-scale fish data from the target cross-domain fish dataset are input into a trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data.

[0013] Optionally, the step of inputting the image with source domain region proposal and the image with target domain region proposal into the category adaptation module in the decoupled cross-domain fish detection model, and determining the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image through category adaptive adversarial learning, specifically includes:

[0014] Based on the domain discriminator, the source domain region proposal and the target domain region proposal in each of the fish body images are distinguished;

[0015] Based on the feature extractor, regional features of the source domain region proposal and the target domain region proposal are extracted to obfuscate the discrimination results of the domain discriminator.

[0016] When the adversarial loss of the domain discriminator and the feature extractor is less than the loss threshold, the foreground region prediction proposal of the source domain and the category pseudo-label of the target domain are determined.

[0017] Optionally, the adversarial loss function of the category adaptation module is as follows:

[0018]

[0019]

[0020] in, Let be the cross-entropy loss during supervised pre-training in the source domain, and λ be the parameter for balancing class-adaptive adversarial loss. Adaptive weights for the source domain For adaptive weights of the target domain, , Let be the reweighted feature representations of the source and target domains, respectively, and D be the domain discriminator. Suggested distribution of source regions Suggested distribution for the target area. Indicates feature extractor, , These represent the expected distributions of samples belonging to the source region proposal and the target region proposal, respectively. To counteract the loss function.

[0021] Optionally, the regression adaptive module includes: a feature generator, an adversarial regressor, and a regressor;

[0022] The feature generator aims to reduce the distance between two predicted boxes and minimize the difference in the regression target.

[0023] The regressor aims to increase the distance between the two prediction boxes, maximizing the difference in the regression target.

[0024] Optionally, the step of inputting the foreground region prediction proposal into the regression adaptive module in the decoupled cross-domain fish detection model, and determining the bounding box regression pseudo-label of the target domain region proposal image in the fish image through regression adaptive adversarial learning, specifically includes:

[0025] Based on the minimum regression target difference generated by the feature generator and the maximum regression target difference generated by the regressor, the regression adaptive module is subjected to adversarial training to obtain a trained regression adaptive module.

[0026] Based on the trained regression adaptive module, the regression pseudo-labels of the bounding box of the target domain are determined.

[0027] Optionally, the adversarial loss function of the regression adaptive model is as follows:

[0028]

[0029] in, This is a commonly used regression loss in Faster R-CNN, and the symbol ◦ represents the Hadamard product. and The distributions of foreground suggestions for the source and target domains are respectively, x s and x t The source and target domain samples are input respectively. To counter the regressor, For feature extractor, , These represent the expected distributions of the prospective suggestions for samples belonging to the source and target domains, respectively.

[0030] Optionally, the different breeding environments include different light intensities, breeding densities, and background complexity in the breeding environment; the background complexity is increased by adding shading objects or structures.

[0031] This invention also provides a fish detection system based on a decoupled cross-domain fish detection model, comprising:

[0032] The data acquisition module is used to acquire cross-domain fish datasets; the cross-domain fish datasets include multi-scale fish data under different aquaculture scenarios, and the multi-scale fish data are images of fish at different scales taken due to the different relative distances between the fish and the underwater camera.

[0033] The feature extraction module is used to extract features from the multi-scale fish body data based on the multi-scale feature enhancement detector in the decoupled cross-domain fish detection model, and determine the fish body images at each scale; the fish body images are images with source domain region proposals or images with target domain region proposals.

[0034] The category pseudo-label module is used to input the image with source domain region proposal and the image with target domain region proposal into the category adaptation module in the decoupled cross-domain fish detection model, and determine the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image through category adaptive adversarial learning;

[0035] The bounding box regression pseudo-label module is used to input the foreground region prediction proposal into the regression adaptive module in the decoupled cross-domain fish detection model, and determine the bounding box regression pseudo-label of the target domain region proposal image in the fish image through regression adaptive adversarial learning;

[0036] The training module is used to iteratively train the decoupled cross-domain fish detection model based on the fish image, the category pseudo-label, and the bounding box regression pseudo-label.

[0037] The detection module is used to input multi-scale fish data from the target cross-domain fish dataset into the trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data.

[0038] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] This invention provides a fish detection method and system based on a decoupled cross-domain fish detection model. The method includes: acquiring a cross-domain fish dataset; the cross-domain fish dataset includes multi-scale fish data from different aquaculture scenarios, wherein the multi-scale fish data consists of images of fish at different scales captured due to varying relative distances between the fish and an underwater camera; extracting features from the multi-scale fish data using a multi-scale feature enhancement detector in the decoupled cross-domain fish detection model to determine fish images at each scale; the fish images are either images with source domain region proposals or images with target domain region proposals; and inputting the images with source domain region proposals and the images with target domain region proposals into the decoupled cross-domain fish detection model. The category adaptation module in the cross-domain fish detection model determines the foreground region prediction proposal and category pseudo-label in the fish image with the target domain region proposal image through category adaptive adversarial learning. The foreground region prediction proposal is input into the regression adaptation module in the decoupled cross-domain fish detection model, and the bounding box regression pseudo-label in the fish image with the target domain region proposal image is determined through regression adaptive adversarial learning. Based on the fish image, category pseudo-label, and bounding box regression pseudo-label, the decoupled cross-domain fish detection model is trained iteratively. Multi-scale fish data from the target cross-domain fish dataset is input into the trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data. This invention introduces a detector, a category adaptive module, and a regression adaptive module, combining domain adaptive technology with target detection in aquaculture. It applies region proposals from the source domain to the target domain data, making the training of the detection model less reliant on extensive data labeling, reducing the workload of data labeling, and improving the efficiency of fish detection. In addition, this invention embeds a multi-scale enhanced feature pyramid into the feature extraction backbone network, improving the detection capability of non-rigid objects with multi-scale changes in aquaculture and accelerating the efficiency of fish detection. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart of fish body detection provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the model structure of DMDnet provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of feature fusion for residual feature enhancement provided in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the category adaptive module structure provided in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the regression adaptive module structure provided in an embodiment of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] The purpose of this invention is to provide a fish detection method and system with high detection efficiency and good transferability.

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] like Figure 1 As shown, a fish detection method based on a decoupled cross-domain fish detection model includes:

[0050] Step 1: Obtain cross-domain fish body dataset; the cross-domain fish body dataset includes multi-scale fish body data under different aquaculture scenarios, and the multi-scale fish body data consists of images of fish body at different scales taken due to the different relative distances between the fish body and the underwater camera.

[0051] Step 2: Based on the multi-scale feature enhancement detector in the decoupled cross-domain fish detection model, feature extraction is performed on the multi-scale fish body data to determine the fish body images at each scale; the fish body images are images with source domain region proposals or images with target domain region proposals.

[0052] Step 3: Input the image with source domain region proposal and the image with target domain region proposal into the category adaptation module in the decoupled cross-domain fish detection model. Through category adaptive adversarial learning, determine the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image.

[0053] Step 4: Input the foreground region prediction proposal into the regression adaptive module in the decoupled cross-domain fish detection model, and determine the bounding box regression pseudo-label of the target domain region proposal image in the fish image through regression adaptive adversarial learning.

[0054] Step 5: Based on the fish image, the category pseudo-label, and the bounding box regression pseudo-label, iteratively train the decoupled cross-domain fish detection model.

[0055] Step 6: Input the multi-scale fish data from the target cross-domain fish dataset into the trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data.

[0056] Among them, the multi-scale fish images (data) consist of a portion of source domain data with region prior labels and another portion of target domain data without region prior labels. The region prior labels include category and location bounding boxes.

[0057] In some embodiments, the training process of decoupling the cross-domain fish detection model can be as follows:

[0058] The cross-domain fish dataset is input into a multi-scale feature enhancement detector, which outputs region suggestions for each fish image.

[0059] The region suggestions for each fish image are input into the category adaptation module. After category adaptation adversarial training by the category adaptation module, the category pseudo-labels and foreground region prediction suggestions for each fish image are output; the categories include foreground and background.

[0060] The foreground region prediction suggestions for each fish image are input into the regression adaptive module. After regression adaptive adversarial training by the regression adaptive module, the bounding box regression pseudo-labels for each fish image are output.

[0061] Based on source domain data, the region prior labels corresponding to the source domain data, target domain data, the category pseudo-labels corresponding to the target domain data, and the bounding box regression pseudo-labels, the decoupled cross-domain fish detection model is iteratively trained.

[0062] In the iterative training, the weights in the decoupled cross-domain fish detection model are adjusted based on the prior region labels corresponding to the source domain data and the category pseudo-labels and bounding box regression pseudo-labels obtained in the previous iteration of the target domain data.

[0063] Specifically, during the iterative training of the decoupled cross-domain fish detection model, in this iteration, the source domain data and target domain data are used as inputs. The output of the decoupled cross-domain fish detection model in this iteration is compared with the region prior label corresponding to the source data, the category pseudo-label output by the category adaptive module, and the bounding box regression pseudo-label output by the regression adaptive module. The weights in the decoupled cross-domain fish detection model are adjusted according to the comparison results.

[0064] In some embodiments, the residual feature enhancement module compensates for the semantic information loss during the pyramid feature fusion process, thereby enhancing the feature extraction capability of the backbone detection network and improving the discriminability of the entire model.

[0065] Specifically, the residual feature enhancement module enhances the features of the shared convolutional layers by weighted summation before feature fusion, and then enhances these features through residual connections. Feature extraction is a crucial step in object detection. For objects at different scales, the network needs strong feature extraction capabilities to better distinguish between the target and the background. However, since the size and shape of targets in an image may vary, feature extraction at different feature layers may yield different results. Therefore, multi-scale feature pyramids are introduced into object detection to improve the model's ability to detect objects at different scales. However, this feature fusion process may lead to the loss of some semantic information, thus reducing model performance. The residual feature enhancement module preserves the information of the shared convolutional layers through weighted summation and re-injects the enhanced features into the backbone detection network through residual connections. This enhances the feature extraction capability of the backbone detection network while preserving semantic information, improving the overall discriminativeness of the model.

[0066] Specifically, the details of the horizontal connectivity and top-down structure of the multi-scale augmented feature pyramid are as follows: Figure 2 As shown, {C2, C3, C4, C5} represent feature maps at different scales, which are the outputs of the convolutional layers {block1, block2, block3, block4}. Deep convolutions have a larger receptive field, lower resolution, and better semantic feature extraction capabilities. To combine high-resolution features (lower layers) with strong semantic features (higher layers), top-down and horizontal connections are needed to effectively utilize features from different levels. For example, C5 and C4 are convolved with a 1×1 filter to obtain M5 and M4. Then, M4 is convolved with a 3×3 filter and connected to the upsampled P5 to obtain P4, where P4 simultaneously possesses the semantic information of both C5 and C4. This top-down structure allows high-level semantic information to continue to be passed down, ultimately obtaining the feature pyramid output {P2, P3, P4, P5}. The top-down feature fusion process combines high-level features from the top layer with low-level features from the bottom layer, thereby enhancing the low-level features with the high-level features from the top layer.

[0067] However, due to the reduction in the number of feature channels after 1×1 convolution, the highest pyramid level feature M5 suffers from information loss, and since this layer lacks input from the previous layer, it only contains single-scale contextual information. To alleviate this problem, this invention introduces a residual feature enhancement module into the traditional feature pyramid. By using residual branches, spatial contextual information is injected into the original branches to improve the feature representation of M5. Detailed structure is as follows... Figure 3 As shown, the goal of this module is to reduce information loss in the M5 channel and improve the discriminative performance of the feature pyramid by incorporating spatial context information. To this end, adaptive pooling is first performed at the C5 scale to generate multiple context features at different scales. Then, each context feature is independently convolved using a 1×1 convolution to reduce the dimension of the feature channels to 256. Finally, they are upsampled to the same scale using bilinear interpolation. Considering the aliasing effect caused by interpolation, an adaptive spatial fusion (ASF) module is used to combine these context features instead of simply summing them adaptively. The structure of ASF is as follows: Figure 3 As shown at the bottom, specifically, ASF uses the sampled individual features as input to generate a spatial weight map for each feature. Contextual features are aggregated into M6 according to their weights, at which point M6 is endowed with multi-scale contextual information. After ASF generates M6, it is summed and combined with M5. This strategy compensates for the information loss in M5 and enhances the overall discriminative ability of the detector.

[0068] In some embodiments, such as Figure 2 As shown, the category adaptation module is independent of the detector branch. Based on the source domain prior knowledge (source domain region proposal), the category adaptation module distinguishes the unlabeled multi-scale fish foreground features and multi-scale fish background features in the target domain, which enables the model to distinguish the foreground and background features in the target domain more accurately.

[0069] Specifically, the category-adaptive module (i.e. Figure 2The Category adaptor (also known as the class adapter) in this paper adapts to the category in the target domain by introducing labeled "region proposals" from the source domain. It does not rely on any object detector, but only utilizes labeled source domain data for adaptive category learning, avoiding the problems of insufficient labeled data and target class imbalance. Furthermore, it can maintain the foreground and background discrimination in the target domain as much as possible during model adaptation. Generally, category adaptation methods achieve adaptation by calculating the difference between the source and target regions. However, the method described in this paper achieves adaptation by utilizing labeled region proposals from the source domain. These region proposals provide useful prior information, enabling the model to more accurately distinguish between foreground and background, thereby improving detection accuracy.

[0070] Specifically, the category adaptation module uses the label information from the source domain for model pre-training to generate pseudo-labels for unlabeled target domain images (i.e., Figure 2 The category labels (pseudo-labels are generated based on source domain region proposals) are then generated using adversarial learning to achieve adaptive alignment of features between domains. Through pseudo-labels and adversarial training, the model fully exploits the domain-invariant common features of the source and target domains, achieving good detection performance in the unlabeled target domain. The category label refers to a clear category identifier for each data sample, indicating which category or set of categories the sample belongs to.

[0071] Specifically, the image with source domain region proposal and the image with target domain region proposal are input into the category adaptation module of the decoupled cross-domain fish detection model. Through category adaptive adversarial learning, the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image are determined. Specifically, it can be as follows:

[0072] Based on the domain discriminator, the source domain region proposal and the target domain region proposal are distinguished in each of the fish body images.

[0073] Based on the feature extractor, regional features of the source domain region proposal and the target domain region proposal are extracted, and the discrimination results of the domain discriminator are obfuscated.

[0074] When the adversarial loss of the domain discriminator and the feature extractor is less than the loss threshold, the foreground region prediction proposal of the source domain and the category pseudo-label of the target domain are determined.

[0075] Specifically, in the category adaptation phase, the category adapter uses ResNet50 as the backbone, and the domain discriminator D consists of a three-layer fully connected network designed to distinguish between the source domain proposal and the target region proposal for each target individual. Conversely, the feature extractor... The aim is to fool the domain discriminator D. When D struggles to distinguish which domain a feature comes from, this indicates a problem with the feature extractor. Sufficient domain-invariant features were extracted. This adversarial process is implemented through a gradient inversion layer (GRL). The class-adaptive adversarial loss is defined as follows:

[0076] .

[0077] .

[0078] in, Let be the cross-entropy loss during supervised pre-training in the source domain, and λ be the parameter for balancing class-adaptive adversarial loss. Adaptive weights for the source domain For adaptive weights of the target domain, , Let be the reweighted feature representations of the source and target domains, respectively, and D be the domain discriminator. Suggested distribution of source regions To suggest a distribution for the target region, G cls Indicates feature extractor, , These represent the expected distributions of samples belonging to the source region proposal and the target region proposal, respectively. To counteract the loss function.

[0079] Category-adaptive adversarial training, such as Figure 4 As shown, the domain discriminator D aims to distinguish between the source domain proposal and the target domain proposal for each fish body. Conversely, the feature extractor G... cls The intention is to fool the domain discriminator D; when D has difficulty distinguishing which domain the proposed features for a region come from, this indicates that the feature extractor G... cls Being able to extract more domain-invariant features also means that the learned feature extraction capabilities are also applicable to unlabeled target domains.

[0080] In some embodiments, the regression adaptive module (i.e. Figure 2 The regression adaptor (also known as the regression adapter) uses labeled source domain foreground candidate boxes to obtain bounding box labels (regression labels) for unlabeled target domain candidate boxes.

[0081] Specifically, the regression-adaptive module is a technique in object detection used to learn bounding box regression targets in the target domain. This module utilizes labeled foreground candidate boxes in the source domain as prior information and learns from unlabeled foreground candidate boxes in the target domain during the adaptation process. In this process, using the labeled information in the source domain, the regression-adaptive module can effectively infer the bounding box labels of the neighboring ground truth foreground boxes in the target domain, achieving bounding box regression of foreground candidate boxes in the target domain. The introduction of the regression-adaptive module can solve the domain shift problem caused by different data distributions in the target domain. By utilizing labeled information in the source domain to propose bounding box regression targets for candidate boxes, the regression task is learned in the unlabeled target domain. Therefore, this technique can effectively utilize prior knowledge of the source domain data, improving the model's generalization performance and detection accuracy in the target domain.

[0082] As another important subtask of object detection, this invention uses bounding box regression to locate specific positions of fish. Due to the coupling between features, features suitable for classification may not be suitable for regression. To avoid mutual interference between classification and regression, this invention decouples the two subtasks. The goal of the regression adapter is to locate, as accurately as possible, suggestions in the unlabeled target domain using annotated suggestions from the source domain.

[0083] Among them, such as Figure 5 As shown, the bounding box adapter consists of three parts: a feature generator, a feature generator, and a feature adapter. Regressor and the adversarial regressor Regressor The feature generator learns to maximize the target difference by moving two predicted boxes further apart from each other. The target difference is minimized by making the two prediction boxes overlap as much as possible.

[0084] Specifically, the adversarial loss function of the regression adaptive model is as follows:

[0085] .

[0086] in, This is a commonly used regression loss in Faster R-CNN, and the symbol ◦ represents the Hadamard product. and The distributions of foreground suggestions for the source and target domains are respectively, x s and x t The source and target domain samples are input respectively. To counter the regressor, For feature extractor, , Let $\mathbf$ be the expected distributions of the foreground suggestions belonging to the source and target domains, respectively. Then the objective of the feature extractor is as follows:

[0087] .

[0088] in, The loss generated when training the source regressor based on the true bounding box labels of each proposal in the source domain, β, is a balancing parameter between the source loss and the adversarial loss. After obtaining the adaptive regressor, this invention can obtain pseudo-labels for each target domain proposal.

[0089] In some embodiments, the implementation of the model can be divided into four stages: First, pre-training is performed on existing labeled source domain fish data, and Faster R-CNN with multi-scale feature enhancement is used as the detection framework, trained with a learning rate of 0.005; Second, when new domain data with different distributions appear, the category adapter uses ResNet50 with a classification head as the backbone feature extractor and a domain discriminator D composed of three fully connected layers to implement the category adaptation process, achieving category alignment between the fish and the background; In the regression adaptation stage, ResNet50 is used as the backbone extractor, followed by a regression head composed of two convolutional layers to locate region proposals in the unlabeled target domain as accurately as possible; The last stage is the training stage for the obtained target domain pseudo-labels, which overcomes the problems of multi-scale fish and data distribution differences through continuous iterative optimization.

[0090] This invention also provides a fish detection system based on a decoupled cross-domain fish detection model, comprising:

[0091] The data acquisition module is used to acquire cross-domain fish datasets; the cross-domain fish datasets include multi-scale fish data under different aquaculture scenarios, and the multi-scale fish data are images of fish at different scales taken due to the different relative distances between the fish and the underwater camera.

[0092] The feature extraction module is used to extract features from the multi-scale fish body data based on the multi-scale feature enhancement detector in the decoupled cross-domain fish detection model, and determine the fish body images at each scale; the fish body images are images with source domain region proposals or images with target domain region proposals.

[0093] The category pseudo-label module is used to input the image with source domain region proposal and the image with target domain region proposal into the category adaptation module in the decoupled cross-domain fish detection model, and determine the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image through category adaptive adversarial learning.

[0094] The bounding box regression pseudo-label module is used to input the foreground region prediction proposal into the regression adaptive module in the decoupled cross-domain fish detection model, and determine the bounding box regression pseudo-label of the target domain region proposal image in the fish image through regression adaptive adversarial learning.

[0095] The training module is used to iteratively train the decoupled cross-domain fish detection model based on the fish image, the category pseudo-label, and the bounding box regression pseudo-label.

[0096] The detection module is used to input multi-scale fish data from the target cross-domain fish dataset into the trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data.

[0097] In some embodiments, the fish detection system further includes an adversarial training module, which is used to perform adversarial training on the regression adaptive module based on the minimum regression target difference generated by the feature generator and the maximum regression target difference generated by the regressor, so as to obtain a trained regression adaptive module.

[0098] In summary, the present invention has the following advantages:

[0099] This invention provides a fish detection method and system based on a decoupled cross-domain fish detection model. It integrates a multi-scale enhanced feature pyramid into the feature extraction backbone network to improve the feature extraction capability for non-rigid fish with multi-scale variations in aquaculture and enhance the network's discriminability. The method introduces two branch modules with independent parameters: category adaptation and regression adaptation, to address the domain transfer problem caused by different data distributions in aquaculture. These two adapters are independent of the detector and obtain domain-invariant features between different regions through adversarial training, thereby improving the network's transferability. Therefore, this method can achieve good detection performance in new target domains without requiring any additional labeled data.

[0100] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fish detection method based on a decoupled cross-domain fish detection model, characterized in that, include: Obtain a cross-domain fish dataset; the cross-domain fish dataset includes multi-scale fish data under different aquaculture scenarios, and the multi-scale fish data consists of images of fish at different scales taken due to the different relative distances between the fish and the underwater camera. Based on the multi-scale feature enhancement detector in the decoupled cross-domain fish detection model, feature extraction is performed on the multi-scale fish body data to determine the fish body images at each scale. The fish image is either an image with source region proposal or an image with target region proposal; The image with source domain region proposal and the image with target domain region proposal are input into the category adaptation module in the decoupled cross-domain fish detection model. Through category adaptive adversarial learning, the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image are determined. The foreground region prediction suggestion is input into the regression adaptive module in the decoupled cross-domain fish detection model. Through regression adaptive adversarial learning, the bounding box regression pseudo-label of the target domain region suggestion image in the fish image is determined. Based on the fish image, the category pseudo-label, and the bounding box regression pseudo-label, the decoupled cross-domain fish detection model is iteratively trained. The multi-scale fish data in the target cross-domain fish dataset is input into the trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data; The step of inputting the image with source domain region proposal and the image with target domain region proposal into the category adaptation module in the decoupled cross-domain fish detection model, and determining the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image through category adaptive adversarial learning, specifically includes: Based on the domain discriminator, the source domain region proposal and the target domain region proposal in each of the fish body images are distinguished; Based on the feature extractor, regional features of the source domain region proposal and the target domain region proposal are extracted to obfuscate the discrimination results of the domain discriminator. When the adversarial loss of the domain discriminator and the feature extractor is less than the loss threshold, the foreground region prediction proposal of the source domain and the category pseudo-label of the target domain are determined. The adversarial loss function of the category adaptation module is as follows: in, Let be the cross-entropy loss during supervised pre-training in the source domain, and λ be the parameter for balancing class-adaptive adversarial loss. Adaptive weights for the source domain For adaptive weights of the target domain, , These are the reweighted feature representations of the source and target domains, respectively. D For the domain discriminator, Suggested distribution of source regions Suggested distribution for the target area. Indicates feature extractor, , These represent the expected distributions of samples belonging to the source region proposal and the target region proposal, respectively. To counteract the loss function.

2. The fish body detection method according to claim 1, characterized in that, The regression adaptive module includes: a feature generator, an adversarial regressor, and a regressor; The feature generator aims to reduce the distance between two predicted boxes and minimize the difference in the regression target. The regressor aims to increase the distance between the two prediction boxes, maximizing the difference in the regression target.

3. The fish body detection method according to claim 1, characterized in that, The step of inputting the foreground region prediction proposal into the regression adaptive module of the decoupled cross-domain fish detection model, and determining the bounding box regression pseudo-label of the target domain region proposal image in the fish image through regression adaptive adversarial learning, specifically includes: Based on the minimum regression target difference generated by the feature generator and the maximum regression target difference generated by the regressor, the regression adaptive module is subjected to adversarial training to obtain a trained regression adaptive module. Based on the trained regression adaptive module, the regression pseudo-labels of the bounding box of the target domain are determined.

4. The fish body detection method according to claim 3, characterized in that, The adversarial loss function of the regression adaptive model is as follows: in, In order to be in Faster RCNN The commonly used regression loss in regression, denoted by ◦, is the Hadamard product. and The distributions of foreground suggestions for the source and target domains are respectively. x s and x t The source and target domain samples are input respectively. To counter the regressor, For feature extractor, , These represent the expected distributions of the prospective suggestions for samples belonging to the source and target domains, respectively.

5. The fish body detection method according to claim 1, characterized in that, The different breeding environments include different light intensities, breeding densities, and background complexity; the background complexity is increased by adding shading objects or structures.

6. A fish detection system based on a decoupled cross-domain fish detection model, used to implement the fish detection method based on a decoupled cross-domain fish detection model as described in claim 1, characterized in that, include: The data acquisition module is used to acquire cross-domain fish datasets; the cross-domain fish datasets include multi-scale fish data under different aquaculture scenarios, and the multi-scale fish data are images of fish at different scales taken due to the different relative distances between the fish and the underwater camera. The feature extraction module is used to extract features from the multi-scale fish body data based on the multi-scale feature enhancement detector in the decoupled cross-domain fish detection model, and determine the fish body images at each scale; the fish body images are images with source domain region proposals or images with target domain region proposals. The category pseudo-label module is used to input the image with source domain region proposal and the image with target domain region proposal into the category adaptation module in the decoupled cross-domain fish detection model, and determine the foreground region prediction proposal and category pseudo-label of the image with target domain region proposal in the fish image through category adaptive adversarial learning; The bounding box regression pseudo-label module is used to input the foreground region prediction proposal into the regression adaptive module in the decoupled cross-domain fish detection model, and determine the bounding box regression pseudo-label of the target domain region proposal image in the fish image through regression adaptive adversarial learning; The training module is used to iteratively train the decoupled cross-domain fish detection model based on the fish image, the category pseudo-label, and the bounding box regression pseudo-label. The detection module is used to input multi-scale fish data from the target cross-domain fish dataset into the trained decoupled cross-domain fish detection model to determine the category and location of each fish in the multi-scale fish data.

Images