Method and device for detecting underwater fish targets

Abstract

The application provides an underwater fish target detection method and device, and belongs to the technical field of target detection. The method comprises the following steps: inputting an underwater image into a target recognition model to obtain a fish species recognition result output by the target recognition model and position information of the fish in the underwater image; wherein the target recognition model comprises a backbone network and a fusion module; the backbone network comprises a plurality of serially connected feature extraction and restoration networks; each feature extraction and restoration network is used for performing feature extraction on the underwater image to obtain a fish feature map, and performing low-frequency information restoration on the underwater image to obtain a low-frequency texture feature map; the fish feature map and the low-frequency texture feature map are fused to obtain a restored feature map; and the fusion module is used for fusing the restored feature maps of the feature extraction and restoration networks. The underwater fish target detection method and device provided by the application can realize reliable and accurate detection of underwater fish targets.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to an underwater fish target detection method and device. BACKGROUND

[0002] Underwater fish target detection is an important and challenging problem in marine ecological monitoring and underwater robot research. However, the complex and changeable underwater environment (such as the complexity of the seabed background, the diversity of fish posture, the large variation of target size, and the limited image resolution) and the interference of background source domain information will lead to the loss of low-frequency feature details of the target, and with the deepening of the network depth, the target feature information becomes chaotic, thereby affecting the detection performance.

[0003] Although the existing underwater fish target detection improves the extraction ability of the model for target feature information, the underwater fish target detection image is essentially an unknown degraded image, and when facing unknown multiple degraded underwater images, the adaptability is insufficient, and the demand of the real world environment cannot be met.

[0004] On the other hand, unknown degraded underwater background source domain information will seriously interfere with the learning of target feature information, and the interference will be more serious with the deepening of the model network depth, and further more chaotic in high-dimensional space feature visualization, resulting in serious loss of target feature information details. SUMMARY

[0005] The present application provides an underwater fish target detection method and device to solve the defect that the existing technology only focuses on the extraction of fish target feature information, resulting in insufficient adaptability of underwater fish target detection, and realizes reliable and accurate detection of underwater fish targets.

[0006] In a first aspect, the present application provides an underwater fish target detection method, comprising:

[0007] inputting an underwater image into a target recognition model to obtain a fish species recognition result and position information of the fish in the underwater image output by the target recognition model;

[0008] The target recognition model comprises a backbone network and a fusion module.

[0009] The backbone network comprises a plurality of serially connected feature extraction and restoration networks, each feature extraction and restoration network being configured to:

[0010] extracting fish feature maps from the underwater image and restoring low-frequency texture feature maps from the underwater image;

[0011] fuse the fish feature maps and the low-frequency texture feature maps to obtain restored feature maps;

[0012] The fusion module is configured to fuse the feature maps restored by the feature extraction and restoration networks to obtain the fish species identification result and the position information of the fish in the underwater image.

[0013] In one embodiment, the plurality of serially connected feature extraction and restoration networks are configured to:

[0014] An input of a current feature extraction and restoration network is an output of a previous feature extraction and restoration network, and an input of a first feature extraction and restoration network is the underwater image.

[0015] The feature maps restored by the feature extraction and restoration networks have different sizes.

[0016] In one embodiment, the feature extraction and restoration network comprises a visual spatial state (VSS) block.

[0017] The VSS block is configured to:

[0018] decompose an input image into a plurality of image blocks, and expand the plurality of image blocks into sequences along different traversal paths;

[0019] process each sequence in parallel based on the S6 block, and reshape and combine the processing results of each sequence to obtain the fish feature map.

[0020] In one embodiment, the feature extraction and restoration network comprises a content-driven prompt (CPB) block.

[0021] The CPB is configured to:

[0022] strengthen spatial feature information of an input image to obtain a first image;

[0023] fuse the input image, the first image, and a chain-of-thought (COT) prompt feature to obtain a second image; the COT prompt feature is obtained by a COT module based on background source domain information learning of the input image;

[0024] strengthen spatial channels of the second image to obtain the low-frequency texture feature map.

[0025] In one embodiment, the backbone network further comprises an enhancement block, and the enhancement block is the VSS block or the CPB.

[0026] The enhancement block is located between a last feature extraction and restoration network and the fusion module.

[0027] The fusion module is further configured to fuse the feature maps restored by the feature extraction and restoration networks and an output image of the enhancement block to obtain the fish species identification result and the position information of the fish in the underwater image.

[0028] In an embodiment, the method further comprises:

[0029] The underwater images in different scenes are taken as samples, and corresponding fish species and positions of the fish in the underwater images are taken as labels to train the target recognition model.

[0030] In a second aspect, the present application provides an underwater fish target detection device, comprising:

[0031] The detection module is configured to input the underwater image into the target recognition model to obtain fish species recognition results and position information of the fish in the underwater image output by the target recognition model.

[0032] The target recognition model comprises a backbone network and a fusion module.

[0033] The backbone network comprises a plurality of serially connected feature extraction and restoration networks, and each feature extraction and restoration network is configured to:

[0034] extract fish feature maps from the underwater image and restore low-frequency texture feature maps from the underwater image;

[0035] fuse the fish feature maps and the low-frequency texture feature maps to obtain restored feature maps;

[0036] The fusion module is configured to fuse the restored feature maps of each feature extraction and restoration network to obtain the fish species recognition results and the position information of the fish in the underwater image.

[0037] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the underwater fish target detection method of the first aspect.

[0038] In a fourth aspect, the present application provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the underwater fish target detection method of the first aspect.

[0039] In a fifth aspect, the present application further provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the underwater fish target detection method of the first aspect.

[0040] The underwater fish target detection method and device provided by the application can process the feature information loss caused by degradation at the feature map level, thereby improving the details of high-dimensional features. In addition, by fusing the recovered feature maps of each feature extraction and recovery network, the feature information can be dynamically recovered layer by layer, the feature loss is significantly reduced, and further degradation of the features in a deeper layer is avoided. Therefore, compared with the prior art which only focuses on the extraction of fish target feature information, the underwater fish target detection method provided by the application can realize reliable and accurate detection of underwater fish targets. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is a flowchart of the underwater fish target detection method provided by the application.

[0043] Figure 2 is one of the structural schematic diagrams of the target recognition model provided by the application.

[0044] Figure 3 is the structural schematic diagram of the feature extraction and recovery network provided by the application.

[0045] Figure 4 is the structural schematic diagram of the COT module provided by the application.

[0046] Figure 5 is the second structural schematic diagram of the target recognition model provided by the application.

[0047] Figure 6 is the structural schematic diagram of the underwater fish target detection system provided by the application.

[0048] Figure 7 is the structural schematic diagram of the light intensity transmitter provided by the application.

[0049] Figure 8 is the structural schematic diagram of the underwater fish target detection device provided by the application.

[0050] Figure 9 is the structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0051] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0052] Figure 1 is a flowchart of the underwater fish target detection method provided by the present application, as shown in Figure 1 The method can include the following steps:

[0053] Step 110, inputting the underwater image into a target recognition model to obtain a fish species recognition result and position information of the fish in the underwater image output by the target recognition model;

[0054] The target recognition model includes a backbone network and a fusion module.

[0055] The backbone network includes a plurality of serially connected feature extraction and restoration networks, and each feature extraction and restoration network is configured to:

[0056] extract fish feature maps from the underwater image and restore low-frequency texture feature maps from the underwater image;

[0057] fuse the fish feature maps and the low-frequency texture feature maps to obtain restored feature maps;

[0058] The fusion module is configured to fuse the restored feature maps of each feature extraction and restoration network to obtain the fish species recognition result and the position information of the fish in the underwater image.

[0059] It should be noted that the execution subject of the above-mentioned underwater fish target detection method can be a computer device, such as a mobile phone, a tablet computer, a notebook computer, a palm computer, etc.

[0060] In step 110, the underwater image can be an image of an underwater scene containing fish. Specifically, the underwater image can be an underwater image of the same place or different places at different time periods, angles, and light attenuation, or an image of different underwater environments such as deep sea, shallow sea, and breeding farm. The present application does not limit the specific type of underwater image.

[0061] After acquiring underwater images, certain preprocessing operations can be performed on them. For example, the Stem layer can be used to perform preprocessing operations such as low-level feature extraction, spatial dimensionality reduction, multi-scale fusion, and modular adaptation on the underwater images. Depending on the type of target recognition model, the structure and connection method of the Stem layer can be adjusted to balance computational efficiency and feature representation ability, thereby providing optimized input features for subsequent deep networks.

[0062] The target recognition model can be any model with image recognition capabilities, such as Transformer, Mamba, RNN, CNN, etc.

[0063] The technical solution of this invention will be described below using the Mamba network as an example for target recognition model. In the following text, the target recognition model may be referred to as "Fish Mamba".

[0064] After preprocessing, the underwater image can be input into Fish Mamba to obtain the fish species identification results and the location information of the fish in the underwater image.

[0065] Specifically, such as Figure 2 As shown, Fish Mamba may include a backbone network 210 and a convergence module 220.

[0066] The backbone network 210 may include several cascaded feature extraction and recovery networks 211 ( Figure 2 The three feature extraction and recovery networks 211 shown are for illustrative purposes only and do not limit the specific number of feature extraction and recovery networks 211. Each feature extraction and recovery network 211 is used for:

[0067] Fish feature maps are obtained by extracting features from underwater images, and low-frequency texture feature maps are obtained by restoring low-frequency information from underwater images.

[0068] Specifically, the feature extraction and recovery network 211 can use various networks (functional modules) such as VSS (Visual Spatial State), CNN, Transformer, ResNet, YOLO and their combinations to extract features from underwater images to obtain fish feature maps.

[0069] The feature extraction and restoration network 211 can use various networks (functional modules) such as CPB (Content-driven Prompt Block), WDMamba, FocalNet, HLNet, MSB, SwinIR and their combinations to restore low-frequency information of underwater images to obtain low-frequency texture feature maps.

[0070] After obtaining the fish feature map and the low-frequency texture feature map, the feature extraction and restoration network 211 can fuse the two to obtain a restored feature map.

[0071] The fusion module 220 can use various networks (functional modules) such as DINO network, TSJNet, EfficientViT, CoCoNet, and combinations thereof to fuse the restored feature maps of each feature extraction and restoration network 211 to obtain fish species identification results and fish position information in underwater images.

[0072] The underwater fish target detection method provided by the present application restores low-frequency information from underwater images to obtain a low-frequency texture feature map, and fuses the low-frequency texture feature map with a fish feature map, which can process feature information loss caused by degradation at the feature map level, thereby improving the details of high-dimensional features. In addition, by fusing the restored feature maps of each feature extraction and restoration network, the feature information can be dynamically restored layer by layer, significantly reducing feature loss and avoiding further degradation of features at a deeper level. Therefore, compared with the prior art which only focuses on the extraction of fish target feature information, the underwater fish target detection method provided by the present application can achieve reliable and accurate detection of underwater fish targets.

[0073] In one embodiment, in a plurality of serially connected feature extraction and restoration networks 211:

[0074] The input of the current feature extraction and restoration network 211 is the output of the previous feature extraction and restoration network 211, and the input of the first feature extraction and restoration network 211 is an underwater image;

[0075] The sizes of the restored feature maps of each feature extraction and restoration network 211 are different.

[0076] As shown in Figure 2 For the first feature extraction and restoration network 211, the input is an underwater image, for the second feature extraction and restoration network 211, the input is the output of the first feature extraction and restoration network 211, and for the third feature extraction and restoration network 211, the input is the output of the second feature extraction and restoration network 211.

[0077] Assuming that the size of the underwater image is W H C, where W is the image width, H is the image height, and C is the number of image channels (for example, a visible light image is usually composed of three colors of blue, red, and yellow, so the value of C is 3), the size of the restored feature map output by the first feature extraction and restoration network 211 can be W / 2 H / 2 2C, the size of the restored feature map output by the second feature extraction and restoration network 211 can be W / 4. H / 4 4C. The size of the restored feature map output by the third feature extraction and restoration network 211 can be W / 8. H / 8 8C.

[0078] It is understandable that by concatenating the inputs and outputs of each feature extraction and recovery network 211, and by making the recovered feature maps of each feature extraction and recovery network have different sizes, the orderly execution of feature extraction and recovery can be achieved, which helps to improve the learning ability of Fish Mamba.

[0079] In one embodiment, the feature extraction and recovery network 211 may include a VSS block;

[0080] VSS blocks are used for:

[0081] The input image is decomposed into multiple image blocks, and these multiple image blocks are expanded into sequences along different traversal paths;

[0082] Each sequence is processed in parallel using S6 blocks, and the processing results of each sequence are reshaped and merged to obtain the fish feature map.

[0083] Figure 3 This is a schematic diagram of the feature extraction and recovery network provided by the present invention. Figure 3 This corresponds to the first feature extraction and recovery network 211.

[0084] like Figure 3 As shown, the VSS block first processes the input image. Decomposed into multiple image patches Figure 2 The example shown is of 9 types (for illustrative purposes only), and it is expanded into a sequence along 4 different traversal paths (selective scan, i.e., Cross-Scan).

[0085] Next, the VSS block processes each image block sequence in parallel using separate S6 blocks, and reshapes and merges the processing results of each sequence to obtain the fish feature map. (i.e., Cross-Merge).

[0086] Furthermore, by employing complementary one-dimensional traversal paths in the S6 block, the VSS block can use SS2D to enable each pixel in the image to effectively integrate information from all other pixels in different directions, thereby facilitating the establishment of fish feature information (i.e., fish feature maps) in a global receptive field in two-dimensional space. ).

[0087] Wherein, the S6 block is a space state model, used for describing a linear time invariant (LTI) system, in the present application, specifically:

[0088] (1)

[0089] Wherein, is an image block with a time sequence, is feature information after state updating, represents a state vector or hidden state of a system, y represents an output variable, L represents the number of output variables, and the matrix A represents how the current state evolves over time, B , C and D the matrices respectively record each batch and sequence position under different conditions, represents a step size.

[0090] The underwater fish target detection method provided by the present application can effectively integrate fish feature information from different directions by using the VSS block to extract features of underwater images to obtain fish feature maps, so that the fish feature acquisition is accurate and comprehensive, and the performance of Fish Mamba is improved.

[0091] In one embodiment, the feature extraction and restoration network 211 includes a CPB;

[0092] The CPB is used for:

[0093] strengthening spatial feature information of the input image to obtain a first image;

[0094] fusing the input image, the first image and a COT (Chain of Thought, thought chain) prompt feature to obtain a second image; the COT prompt feature is obtained by learning background source domain information of the input image by a COT module;

[0095] strengthening a spatial channel of the second image to obtain a low-frequency texture feature map.

[0096] Specifically, as shown in Figure 3 , the CPB first strengthens the input image through a spatial feature information enhancement module in a spatial feature layer to obtain a first image . At the same time, the CPB fuses the first image , a COT prompt feature and the input image to obtain a second image .

[0097] Then, the CPB inputs the second image into a spatial channel enhancement module for spatial channel enhancement. Specifically, the spatial channel enhancement module divides the second image into n groups, each group ( j = 1, 2, …, n) is enhanced by an independent Transformer block, and finally the outputs of each Transformer block are fused to obtain a low-frequency texture feature map . Wherein, the fish feature map and the low-frequency texture feature map are fused to obtain the restored feature map .

[0098] It should be noted that, unlike the conventional underwater image enhancement which needs pre-processing or image visual level information recovery, the above image low-frequency information recovery is only at the feature level, thereby enhancing the target feature information and being end-to-end.

[0099] The above image low-frequency information recovery process can be represented as follows:

[0100] (2)

[0101] Wherein, represents the feature information after attention learning, represents the feature information after adding prompt learning, represents the low-frequency information containing image texture features, represents the attention mechanism module function at the feature level (i.e. image width W and height H), represents the spatial (i.e. channel C) attention mechanism module function.

[0102] For the CPB provided by the present application, it is designed to promote the interaction between the input image and the COT prompt feature , so that Fish Mamba can adjust its enhancement strategy according to the degradation type.

[0103] In order to better utilize the input content, the CPB calculates the spatial importance map of each channel, and fully mixes the channel dimension attention weight Figure 3 in and the spatial dimension attention weight Figure 3 in to fully capture the features.

[0104] Next, Each part is fed into an independent Transformer block, and each Transformer block learns with the degradation information encoded in the COT cues.

[0105] Finally, all output results are connected along the channel dimension.

[0106] The CPB provided by the present application at least has the following advantages:

[0107] 1) Each part can focus on the correlation of different channels and features, thereby increasing the expression ability of the model;

[0108] 2) Reduce the number of parameters and computational complexity;

[0109] 3) Each part is calculated independently, which significantly reduces the training time.

[0110] Further, in the CPB of the present application, the COT module is used to encode the context information of the specific degradation.

[0111] Specifically, the COT can sample and learn the background source domain information of the input image by itself. Since it is unknown, it is initially a random image feature vector.

[0112] As shown in Figure 4 , the COT module generates multi-scale cues through a series of transpose convolution layers, and applies a Hardswish activation function after all transpose convolutions (TConv) to control the COT cue features input to the next stage.

[0113] The size of the COT cue feature generated by each stage is the same as the size of the corresponding first image , and the size of the COT cue feature of each stage is different, which not only helps Fish Mamba better understand the degradation type in a coherent and step-by-step manner, but also helps Fish Mamba learn hierarchical representation.

[0114] The CPB provided by the present application can drive the CPB to recover more rich low-frequency feature information of fish targets by integrating the COT module to adaptively learn different background source domain information and generate corresponding cues, especially in the case of insufficient light, color attenuation, blur and other degradation phenomena commonly seen in underwater environments. Combined with different background source domain information cue learning, feature low-frequency information learning and recovery compensate for the loss of low-frequency features due to light and environmental degradation, and realize high-precision underwater fish target detection of Fish Mamba under multiple degradation conditions.

[0115] In summary, the feature extraction and recovery network 211 provided by this invention, combined with the CPB-VSS architecture proposed by the VSS and COT modules, can generate lost low-frequency image feature information based on underwater source domain information cues and drive the CPB. Simultaneously, it utilizes the correlation of elements in the sequence to selectively retain or filter information, thereby enhancing the capture and recovery of target features.

[0116] In one embodiment, the backbone network 210 further includes an enhancement block 212, which may be a VSS block or a CPB;

[0117] Enhancement block 212 is located between the last feature extraction and recovery network 211 and the fusion module 220;

[0118] The fusion module 220 is also used to fuse the restored feature maps of each feature extraction and restoration network 211 and the output image of the enhancement block 212 to obtain the fish species identification results and the location information of the fish in the underwater image.

[0119] Specifically, such as Figure 5 As shown, after connecting multiple feature extraction and recovery networks 211 in series, an independent enhancement block 212 can be set between the last feature extraction and recovery network 211 and the fusion module 220.

[0120] When enhancement block 212 is a VSS block, enhancement block 212 can further extract fish features from the recovered feature map of the last feature extraction and recovery network 211, thereby enhancing Fish Mamba's recognition of fish features.

[0121] When enhancement block 212 is a CPB block, enhancement block 212 can further restore low-frequency texture feature maps of the restored feature map of the last feature extraction and restoration network 211, thereby enhancing Fish Mamba's restoration of low-frequency texture features.

[0122] It is understandable that we assume the size of the underwater image is W. H C, then in Figure 5 The size of the restored feature map output by the first feature extraction and restoration network 211 can be W / 2. H / 2 2C, the size of the restored feature map output by the second feature extraction and restoration network 211 can be W / 4. H / 4 4C, the size of the restored feature map output by the third feature extraction and restoration network 211 can be W / 8. H / 8 The output image size of the 8C enhancement block 212 can be W / 16. H / 16 16C.

[0123] The underwater fish target detection method provided by the present application can strengthen the recognition ability of Fish Mamba for fish characteristics or improve the recovery ability of low-frequency texture characteristics by setting an enhancement block 212 between the last feature extraction and recovery network 211 and the fusion module 220, so that Fish Mamba can meet various needs and improve the practicability.

[0124] In one embodiment, the underwater fish target detection method provided by the present application can further include:

[0125] The target recognition model is trained by taking underwater images in different scenes as samples and corresponding fish species and positions of fish in underwater images as labels.

[0126] For example, a camera device can be used to shoot underwater images containing fish targets in various underwater scenes, and the corresponding fish species and positions of fish in underwater images are taken as labels to construct an underwater image training sample set.

[0127] Then, the underwater image training sample set can be taken as the input of Fish Mamba, the initial parameters of the network are set, and the Fish Mamba is trained using the SGD optimizer.

[0128] Specifically, the shot underwater images can be converted into pictures, and the converted pictures can be labeled with fish species and positions of fish in underwater images.

[0129] Subsequently, the underwater image training sample set is re-allocated into a training set and a test set at a certain ratio, for example, 0.85:0.15.

[0130] Finally, the short side of the image in the training set is adjusted to W, the long side is limited to H, and image flipping enhancement is performed, and after the Fish Mamba training is completed, the test set is used to verify the recognition effect of Fish Mamba.

[0131] The underwater fish target detection method provided by the present application can effectively improve the generalization ability of the target recognition model and improve the recognition accuracy of the target recognition model by training the target recognition model using underwater images in various scenes.

[0132] Figure 6 is a structural schematic diagram of the underwater fish target detection system provided by the present application. Figure 6As shown, in order to realize the above-mentioned underwater fish target detection method, the underwater fish target detection system provided by the present application can comprise: a light source 1, an underwater shooting device 2, a light intensity transmitter 3, a control processor 4 and an oxygen generating device 5.

[0133] The control processor 4 is in communication connection with the underwater shooting device 2, the light source 1 and the light intensity transmitter 3 respectively.

[0134] The underwater shooting device 2 can collect fish target motion video in real time under the control of the control processor 4, and extract the fish target video into a video stream.

[0135] The light source 1 is used for light compensation for the underwater shooting device 2, the light intensity transmitter 3 can sense the light intensity of the environment and transmit the light intensity information to the control processor 4, the control processor 4 controls the light source 1 switch and the light intensity according to the light intensity information, and the control processor 4 can detect and identify the fish target according to the trained Fish Mmaba.

[0136] Further, as shown, Figure 7 The light intensity transmitter 3 comprises a light intensity sensor 31, a microcontroller 32 and a communication interface 33 which are connected in sequence, the microcontroller 32 is connected with the light intensity sensor 31 and the communication interface 33 respectively, the microcontroller 32 can control the light intensity sensor 31 to collect data, and transmit the data collected by the light intensity sensor to the control processor 4 through the communication interface 33.

[0137] The underwater fish target detection system provided by the present application has refined structure and strong implementability, and can be effectively applied in aquaculture environment, and provides a reliable and accurate technical means for underwater fish target detection in unknown degradation domain.

[0138] The underwater fish target detection device provided by the present application is described below, the underwater fish target detection device described below can be correspondingly referred to the underwater fish target detection method described above, and the same technical effects can be achieved, which will not be described here.

[0139] Figure 8 is a structural schematic diagram of the underwater fish target detection device provided by the present application, as shown, Figure 8 The device can comprise:

[0140] The detection module 810 is used for inputting the underwater image into the target recognition model to obtain the fish species recognition result and the position information of the fish in the underwater image output by the target recognition model.

[0141] The target recognition model comprises a backbone network and a fusion module.

[0142] The backbone network comprises a plurality of serial feature extraction and restoration networks, each of which is used for:

[0143] The fish feature map is obtained by performing feature extraction on the underwater image, and the low-frequency texture feature map is obtained by performing low-frequency information restoration on the underwater image.

[0144] The fish feature map and the low-frequency texture feature map are fused to obtain a restored feature map.

[0145] The fusion module is configured to fuse the restored feature maps of the feature extraction and restoration networks to obtain the fish species recognition result and the position information of the fish in the underwater image.

[0146] In one embodiment, the plurality of serial feature extraction and restoration networks comprise:

[0147] The input of the current feature extraction and restoration network is the output of the previous feature extraction and restoration network, and the input of the first feature extraction and restoration network is the underwater image.

[0148] The sizes of the restored feature maps of the feature extraction and restoration networks are different.

[0149] In one embodiment, the feature extraction and restoration network comprises a visual spatial state (VSS) block.

[0150] The VSS block is configured to:

[0151] The input image is decomposed into a plurality of image blocks, and the plurality of image blocks are unfolded into sequences along different traversal paths.

[0152] Each sequence is processed in parallel based on the S6 block, and the processing results of each sequence are reshaped and merged to obtain the fish feature map.

[0153] In one embodiment, the feature extraction and restoration network comprises a content-driven prompt block (CPB).

[0154] The CPB is configured to:

[0155] The input image is spatially enhanced to obtain a first image.

[0156] The input image, the first image, and a chain-of-thought (COT) prompt feature are fused to obtain a second image; the COT prompt feature is obtained by a COT module learning background source domain information of the input image.

[0157] The second image is spatially enhanced to obtain the low-frequency texture feature map.

[0158] In one embodiment, the backbone network further includes an enhancement block, which is either the VSS block or the CPB;

[0159] The enhancement block is located between the last feature extraction and recovery network and the fusion module;

[0160] The fusion module is also used to fuse the restored feature maps of each feature extraction and restoration network with the output image of the enhancement block to obtain the fish species identification result and the location information of the fish in the underwater image.

[0161] In one embodiment, the method further includes a training module (not shown in the figure) for:

[0162] The target recognition model is trained by using underwater images from different scenarios as samples, and the corresponding fish species and their positions in the underwater images as labels.

[0163] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communications interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute the underwater fish target detection method described in any of the above embodiments, for example including:

[0164] The underwater image is input into the target recognition model to obtain the fish species recognition result and the location information of the fish in the underwater image.

[0165] The target recognition model includes a backbone network and a fusion module.

[0166] The backbone network comprises multiple cascaded feature extraction and recovery networks, each of which is used for:

[0167] Fish feature maps are obtained by extracting features from the underwater images, and low-frequency texture feature maps are obtained by restoring low-frequency information from the underwater images.

[0168] The fish feature map is fused with the low-frequency texture feature map to obtain the restored feature map;

[0169] The fusion module is used to fuse the restored feature maps of each feature extraction and restoration network to obtain the fish species identification result and the location information of the fish in the underwater image.

[0170] Moreover, the logic instructions in the memory 930 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the prior art that contributes essentially or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0171] On the other hand, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the underwater fish target detection method described in any of the above embodiments, for example, including:

[0172] inputting the underwater image into the target recognition model to obtain a fish species recognition result output by the target recognition model and position information of the fish in the underwater image;

[0173] wherein the target recognition model comprises a backbone network and a fusion module;

[0174] the backbone network comprises a plurality of serially connected feature extraction and restoration networks, each feature extraction and restoration network being configured to:

[0175] extracting fish features from the underwater image and restoring low-frequency texture features from the underwater image;

[0176] fusing the fish features and the low-frequency texture features to obtain restored features;

[0177] the fusion module is configured to fuse the restored features of each feature extraction and restoration network to obtain the fish species recognition result and the position information of the fish in the underwater image.

[0178] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the underwater fish target detection method described in any of the above embodiments, for example, including:

[0179] input the underwater image into a target recognition model to obtain a fish species recognition result and position information of the fish in the underwater image output by the target recognition model;

[0180] The target recognition model comprises a backbone network and a fusion module.

[0181] The backbone network comprises a plurality of serially connected feature extraction and restoration networks, each feature extraction and restoration network being configured to:

[0182] extract fish feature maps from the underwater image and restore low-frequency texture feature maps from the underwater image;

[0183] fuse the fish feature maps and the low-frequency texture feature maps to obtain restored feature maps;

[0184] The fusion module is configured to fuse the restored feature maps of each feature extraction and restoration network to obtain the fish species recognition result and the position information of the fish in the underwater image.

[0185] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0186] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course, they can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0187] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of underwater fish target detection, characterized by, The method comprises the following steps: inputting the underwater image into a target recognition model to obtain fish species recognition results and fish position information in the underwater image output by the target recognition model; wherein the target recognition model comprises a backbone network and a fusion module; the backbone network comprises a plurality of serially connected feature extraction and restoration networks, each feature extraction and restoration network being configured to: extract fish features from the underwater image and restore low-frequency information of the underwater image to obtain a low-frequency texture feature map; fuse the fish features and the low-frequency texture feature map to obtain a restored feature map; the fusion module is configured to fuse the restored feature maps of the feature extraction and restoration networks to obtain the fish species recognition results and the fish position information in the underwater image.

2. The underwater fish target detection method according to claim 1, characterized in that, In the plurality of serially connected feature extraction and restoration networks: the input of a current feature extraction and restoration network is the output of a previous feature extraction and restoration network, and the input of a first feature extraction and restoration network is the underwater image; the sizes of the restored feature maps of the feature extraction and restoration networks are different.

3. The underwater fish target detection method according to claim 2, characterized in that, The feature extraction and restoration network comprises a visual space state (VSS) block; the VSS block is configured to: divide the input image into a plurality of image blocks and expand the plurality of image blocks into sequences along different traversal paths; process each sequence in parallel based on the S6 block, and reshape and combine the processing results of each sequence to obtain the fish feature map.

4. The underwater fish target detection method according to claim 3, characterized in that, The feature extraction and restoration network comprises a content-driven prompt block (CPB); the CPB is configured to: strengthen spatial feature information of the input image to obtain a first image; fuse the input image, the first image, and a chain of thought (COT) prompt feature to obtain a second image; the COT prompt feature is obtained by learning background source domain information of the input image by a COT module; strengthen the spatial channel of the second image to obtain the low-frequency texture feature map.

5. The underwater fish target detection method according to claim 4, characterized in that, The backbone network further comprises an enhancement block, and the enhancement block is the VSS block or the CPB; the enhancement block is located between the last feature extraction and restoration network and the fusion module; the fusion module is further configured to fuse the restored feature maps of the feature extraction and restoration networks and the output image of the enhancement block to obtain the fish species recognition results and the fish position information in the underwater image.

6. The underwater fish target detection method according to any one of claims 1 to 5, characterized in that, The method further comprises: training the target recognition model by taking underwater images in different scenes as samples and corresponding fish species and fish positions in the underwater images as labels.

7. An underwater fish target detection apparatus characterized by comprising: The device comprises: a detection module configured to input an underwater image into a target recognition model to obtain fish species recognition results and fish position information in the underwater image output by the target recognition model; wherein the target recognition model comprises a backbone network and a fusion module; the backbone network comprises a plurality of serially connected feature extraction and restoration networks, each feature extraction and restoration network being configured to: extract fish features from the underwater image and restore low-frequency information of the underwater image to obtain a low-frequency texture feature map; fuse the fish features and the low-frequency texture feature map to obtain a restored feature map; the fusion module is configured to fuse the restored feature maps of the feature extraction and restoration networks to obtain the fish species recognition results and the fish position information in the underwater image. The fish feature map is fused with the low-frequency texture feature map to obtain a restored feature map; The fusion module is configured to fuse the restored feature maps of the feature extraction and restoration networks to obtain the fish species recognition result and the position information of the fish in the underwater image.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the underwater fish target detection method according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the underwater fish target detection method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the underwater fish target detection method according to any one of claims 1 to 6.

Images