Fish identification method, system, terminal and computer-readable storage medium

Abstract

The present invention discloses a fish identification method, system, terminal and computer-readable storage medium, wherein the fish identification method comprises: constructing a backbone network, inputting a target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting a multi-layer feature map; constructing and training a neck network using a learning data set to obtain a weight branch, upsampling the multi-layer feature map in the neck network using the weight branch and then performing splicing processing, and outputting hierarchical information; constructing an initial detection head, and introducing a weight coefficient into the initial detection head for training to obtain a detection head, inputting hierarchical information into the detection head, and outputting the position information and fish information of all fish targets in the target image. The present invention effectively improves the accuracy of fish detection, significantly improves the robustness of the model, and optimizes the detection process through multi-module design and deep optimization.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a fish recognition method, system, terminal and computer-readable storage medium. Background Art

[0002] With the rapid development of marine ranches, the marine aquaculture industry is gradually moving towards intelligence and automation. In order to ensure the sustainable development and efficient management of marine ranches, real-time monitoring and accurate identification of the aquaculture environment, fish growth conditions and fish numbers are necessary.

[0003] However, traditional convolutional neural networks (CNN) usually extract features through fixed convolution kernels. This method is prone to losing some key detail information when dealing with multi-scale targets, thus affecting the accuracy of the detection results. In addition, when faced with complex backgrounds, it is easily affected by background noise, leading to misidentification or missed identification. Especially in marine ranches, fish are often highly integrated with the surrounding environment, and traditional models find it difficult to effectively distinguish fish from the background.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a fish identification method, system, terminal and computer-readable storage medium, aiming to solve the problems of low accuracy of identification results, low model robustness and insufficient real-time performance of the inspection process when identifying fish in the prior art.

[0006] To achieve the above object, the present invention provides a fish identification method, which comprises the following steps: Constructing a backbone network, inputting the target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting multiple multi-layer feature maps; A neck network is constructed and trained using a learning data set, and all the multi-layer feature maps are input into the neck network. The neck network uses weight branches to upsample a plurality of the multi-layer feature maps and then performs splicing processing to output a plurality of hierarchical information; An initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output.

[0007] Optionally, the modular fish identification method, wherein the step of constructing a backbone network, inputting a target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting a plurality of multi-layer feature maps, specifically includes: Constructing a backbone network, wherein the backbone network includes a convolution unit, a coupled feedback unit, and a spatial pyramid pooling unit; Acquire a target image input by a user, input the target image into the convolution unit, and the convolution unit preprocesses and extracts features of the target image to obtain multiple multi-scale features of multiple fish targets in the target image; Inputting each of the multi-scale features into the coupling feedback unit, the coupling feedback unit iteratively optimizes each of the multi-scale features to obtain a multi-level detail feature corresponding to each of the multi-scale features; Each of the multi-level detail features is input into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features and then splices them to output a plurality of multi-layer feature maps.

[0008] Optionally, the module fish identification method, wherein the step of obtaining a target image input by a user, inputting the target image into the convolution unit, and the convolution unit performing preprocessing and feature extraction on the target image to obtain multiple multi-scale features of multiple fish targets in the target image, specifically includes: Acquire a target image input by a user, input the target image into the convolution unit, and the convolution unit performs grayscale processing, denoising processing, and intensity enhancement processing on the target image to obtain a preprocessed image; The convolution unit performs edge detection on the preprocessed image by an edge detection algorithm, performs texture analysis on the preprocessed image, obtains edge features of each fish target and texture features of each fish target, and quantizes the edge features of each fish target and the texture features of each fish target; The convolution unit fuses each quantized edge feature and each quantized texture feature to obtain multiple multi-scale features of each fish target.

[0009] Optionally, the modular fish identification method, wherein each of the multi-scale features is input into the coupling feedback unit, and the coupling feedback unit iteratively optimizes each of the multi-scale features to obtain a multi-level detail feature corresponding to each of the multi-scale features, specifically includes: Input each of the multi-scale features in each fish target into the coupling feedback unit, the coupling feedback unit identifies the exposure of each of the multi-scale features, and fuses the low-exposure multi-scale features and the high-exposure multi-scale features in each fish target to obtain a fusion feature result, wherein the low exposure and the high exposure are both determined according to a preset exposure value; The coupled feedback unit obtains feedback information input by the user, and performs iterative fusion processing on all multi-scale features of each fish target according to the feedback information and the fusion feature result, so as to obtain multiple fused multi-scale features of each fish target; The coupled feedback unit continuously samples multiple fused multi-scale features of each fish target to optimize each fused multi-scale feature and obtain multiple multi-level detail features of each fish target.

[0010] Optionally, the modular fish identification method, wherein each of the multi-level detail features is input into the spatial pyramid pooling unit, the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features and then splices them to output multiple multi-layer feature maps, specifically includes: Input each of the multi-level detail features into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features of each fish target passing through the initial convolutional layer to obtain a pooling result corresponding to each of the multi-level detail features; The spatial pyramid pooling unit splices all multi-level detail features of each fish target according to all the pooling results, and outputs multiple multi-layer feature maps of each fish target through the final convolution layer.

[0011] Optionally, the modular fish identification method, wherein the neck network is constructed and trained using a learning data set, all the multi-layer feature maps are input into the neck network, the neck network uses weight branches to upsample multiple multi-layer feature maps and then performs splicing processing to output multiple hierarchical information, specifically including: Constructing an initial neck network, and inputting a pre-stored learning data set into the initial neck network for training, to obtain a neck network and a weight branch, wherein the weight branch is used to weight different types of fish targets; Inputting all the multi-layer feature maps into the neck network, the neck network weights all the multi-layer feature maps of each fish target according to the weight branch, and extracts a plurality of target key features of each fish target; The neck network integrates all key features of each fish target through an attention mechanism to obtain detailed features and global information of each fish target; The neck network divides all the multi-layer feature maps of each fish target into a plurality of first multi-layer feature maps and a plurality of second multi-layer feature maps, and amplifies the resolution of each of the first multi-layer feature maps to a preset resolution to obtain a corresponding third multi-layer feature map, wherein the resolution of the first multi-layer feature map is lower than a preset value, and the resolutions of the second multi-layer feature map and the third multi-layer feature map are higher than the preset value; The neck network upsamples all the second multi-layer feature maps according to each of the detail features and each of the global information, and concatenates all the second multi-layer feature maps and the third multi-layer feature maps in each of the multi-layer feature maps to obtain and output multiple hierarchical information of each of the fish targets.

[0012] Optionally, the modular fish identification method, wherein the initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output, specifically includes: Constructing an initial detection head, inputting a preset weight coefficient into the initial detection head, adding a weighting term according to the weight coefficient, adjusting the weight coefficient by the weighting term to obtain a loss calculation value, and performing adaptive optimization using the loss calculation value to obtain a detection head; Inputting all the hierarchical information of each fish target into the detection head, the detection head analyzing all the hierarchical information to obtain the boundary overlapping area of ​​multiple fish targets in the target image; The detection head marks the position and type of each fish target according to each hierarchical information, and obtains the position information and fish information of each fish target.

[0013] In addition, to achieve the above-mentioned purpose, the present invention also provides a modular fish identification system, wherein the modular fish identification system comprises: A feature extraction module is used to construct a backbone network, input the target image into the backbone network for feature extraction, iteratively optimize and fuse the extracted features, and output multiple multi-layer feature maps; A feature splicing module is used to construct and train a neck network using a learning data set, input all the multi-layer feature maps into the neck network, and the neck network uses weight branches to upsample multiple multi-layer feature maps and then splices them to output multiple hierarchical information; The result generation module is used to construct an initial detection head, introduce weight coefficients into the initial detection head for training, obtain a detection head, input all the hierarchical information into the detection head, and output the position information and fish information of all fish targets in the target image.

[0014] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a fish identification program stored in the memory and executable on the processor, and when the fish identification program is executed by the processor, the steps of the fish identification method described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a fish identification program, and when the fish identification program is executed by a processor, the steps of the fish identification method as described above are implemented.

[0016] In the present invention, a backbone network is constructed, the target image is input into the backbone network for feature extraction, the extracted features are iteratively optimized and feature fusion processed, and multiple multi-layer feature maps are output; a neck network is constructed and trained using a learning data set, all the multi-layer feature maps are input into the neck network, the neck network uses weight branches to upsample multiple multi-layer feature maps and then performs splicing processing, and outputs multiple hierarchical information; an initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output. The present invention effectively improves the accuracy of fish detection, significantly improves the robustness of the model, and optimizes the detection process through multi-module design and deep optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a preferred embodiment of the fish identification method of the present invention; Figure 2 It is a schematic diagram of the module design of a preferred embodiment of the fish identification method of the present invention; Figure 3 Schematic diagram of a first detection module of a preferred embodiment of the fish identification method of the present invention; Figure 4 Schematic diagram of a second detection module of a preferred embodiment of the fish identification method of the present invention; Figure 5 is a structural diagram of a preferred embodiment of the fish identification system of the present invention; Figure 6 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] Existing fish recognition technologies are mainly based on traditional convolutional neural networks (CNNs) and improved convolutional neural network architectures (such as residual networks), and fish recognition is performed through image classification, target detection, etc. However, these methods are difficult to effectively process targets of different scales in the same image when facing different types of targets. In marine ranches, the size of fish varies greatly, and there are obvious size differences between different types of fish. Therefore, the use of CNN methods is prone to losing key details when processing multi-scale targets, and is easily affected by background noise when facing complex backgrounds, resulting in misidentification or missed identification.

[0020] The fish identification method described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the fish identification method comprises the following steps: Step S10: construct a backbone network, input the target image into the backbone network for feature extraction, perform iterative optimization and feature fusion processing on the extracted features, and output multiple multi-layer feature maps.

[0021] Among them, Figure 2 As shown in the figure, the constructed backbone network includes a convolutional layer (Conv, Convolutional layer, convolution unit), a feature conversion module, a coupled feedback block (CFB, Coupled feedback block, coupled feedback module) and a spatial pyramid pooling fast module (SPPF, Spatial Pyramid Pooling - Fast, spatial pyramid pooling unit). The backbone network constructed by these modules is mainly responsible for extracting multi-scale features of the original image (i.e., the target image), enhancing the model's adaptability to the complex morphology and multi-scale structure of fish, and improving the accuracy of fish detection, especially in the case of multiple fish targets and complex backgrounds.

[0022] Among them, the CFB module can also use attention mechanisms (such as multi-head attention or Self-Attention, self-attention mechanism) to dynamically adjust feature weights and improve the expressiveness of multi-scale target detection.

[0023] Specifically, a backbone network is constructed, wherein the backbone network includes a convolution unit, a coupling feedback unit and a spatial pyramid pooling unit; a target image input by a user is obtained, and the target image is input into the convolution unit, and the convolution unit performs preprocessing and feature extraction on the target image to obtain multiple multi-scale features of multiple fish targets in the target image; each of the multi-scale features is input into the coupling feedback unit, and the coupling feedback unit performs iterative optimization on each of the multi-scale features to obtain multi-level detail features corresponding to each of the multi-scale features; each of the multi-level detail features is input into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features and then splices them to output multiple multi-layer feature maps.

[0024] Furthermore, a target image input by a user is obtained, and the target image is input into the convolution unit, and the convolution unit performs grayscale processing, denoising processing and intensity enhancement processing on the target image to obtain a preprocessed image; the convolution unit performs edge detection on the preprocessed image through an edge detection algorithm, performs texture analysis on the preprocessed image to obtain edge features of each fish target and texture features of each fish target, and quantizes the edge features of each fish target and the texture features of each fish target; the convolution unit fuses each quantized edge feature and each quantized texture feature to obtain multiple multi-scale features of each fish target.

[0025] Among them, the convolution unit extracts features of the target image based on the standard convolutional neural network: the target image is preprocessed by grayscale, denoising and contrast enhancement operations, and then the edge information in the preprocessed target image is detected using methods such as Sobel (Sobel operator) and Canny (a multi-level edge detection algorithm), and the texture features of the image are analyzed through methods such as Gray-level Co-occurrence Matrix (GLCM) or Local Binary Patterns (LBP). The extracted edge features and texture features are then quantized and combined into two feature vectors (i.e., the multi-scale features mentioned above), thereby realizing the feature extraction of the target image by the convolution unit.

[0026] Furthermore, each of the multi-scale features in each fish target is input into the coupled feedback unit, the coupled feedback unit identifies the exposure of each of the multi-scale features, and fuses the low-exposure multi-scale features and the high-exposure multi-scale features in each fish target to obtain a fused feature result, wherein the low exposure and the high exposure are both determined according to a preset exposure value; the coupled feedback unit obtains feedback information input by the user, and iteratively fuses all multi-scale features of each fish target according to the feedback information and the fused feature result to obtain multiple fused multi-scale features of each fish target; the coupled feedback unit continuously samples the multiple fused multi-scale features of each fish target to optimize each fused multi-scale feature to obtain multiple multi-level detail features of each fish target.

[0027] Among them, a coupling feedback mechanism is introduced in the backbone network. Through a unique coupling structure, the multi-scale features of underexposure or overexposure are fused and iterated multiple times. The feedback of user input and the fused multi-scale features can be combined to gradually improve the quality of the fusion result; the coupling feedback unit then continuously up- and down-samples the multi-scale features at different scales (after the underexposed or overexposed multi-scale features have been fused) to optimize the multi-scale features to improve the quality of feature expression and the resolution of the image. Through the operation of multi-scale feature fusion, these multi-scale features are not only transmitted between different blocks of the same sub-network (i.e., convolutional units, coupled feedback units, and spatial pyramid pooling units), but also flow between the two sub-networks; in the process of optimizing multi-scale features, embedding strategies and end-to-end training can also be used to enhance the capture capability of multi-scale features.

[0028] For example, before enhancement, that is, in a traditional image segmentation model without the use of a coupled feedback unit, only the rough structure in the image may be captured, and the recognition of details and edges is not accurate enough, resulting in rough segmentation results. However, after enhancement, it is possible to better capture the multi-level details in the image, including multi-scale features such as edges and textures, through multiple iterations and feedback mechanisms, thereby obtaining more refined and accurate segmentation results. At the same time, in the target detection model that did not use a coupled feedback unit before enhancement, inconsistent performance may occur when detecting targets of different sizes. The coupled feedback unit, by coupling feature maps of different scales, can ensure that both large and small targets can be well processed, and achieve balanced detection of multi-scale targets. Therefore, the coupled feedback unit effectively improves the ability to express details of fish targets, especially in the detection of targets with significant size changes.

[0029] Furthermore, each of the multi-level detail features is input into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features of each fish target that passes through the initial convolutional layer to obtain a pooling result corresponding to each of the multi-level detail features; the spatial pyramid pooling unit splices all the multi-level detail features of each fish target according to all the pooling results, and outputs multiple multi-layer feature maps of each fish target through the final convolutional layer.

[0030] Among them, the spatial pyramid pooling unit uses four steps, namely the initial convolution layer, multiple serial maximum pooling, feature map splicing, and the final convolution layer, to unify the processing of features of different scales to ensure efficient multi-scale information fusion.

[0031] Furthermore, the backbone network described in this embodiment can also be replaced with a more efficient lightweight model (for example, MobileNet, Mobile Neural Network, a lightweight deep learning model) to achieve efficient operation when the device computing resources are limited.

[0032] Step S20, constructing and training a neck network using a learning data set, inputting all the multi-layer feature maps into the neck network, and the neck network upsampling multiple multi-layer feature maps using weight branches and then performing splicing processing to output multiple hierarchical information.

[0033] Among them, the main function of the neck network is to further integrate and enhance multi-layer feature information to achieve accurate positioning and classification of fish targets of different scales. It mainly includes adaptive vision unit (AVBlock, Adaptive VisionBlock), multi-level feature fusion network (MultiFPN_SDI, Multi-level Feature Pyramid Network with Scale-Dependent Integration) and upsample-concat (Upsample-Concat) units, which significantly improve the robustness of the model and can maintain efficient detection performance under conditions of lighting, dynamic water flow and complex background interference.

[0034] Among them, AVBlock can be replaced by a feature aggregation module based on the Graph Convolution Network (GCN). GCN can learn the relationship between features through the graph structure and achieve efficient context information fusion.

[0035] Specifically, an initial neck network is constructed, and a pre-stored learning data set is input into the initial neck network for training to obtain a neck network and a weight branch, wherein the weight branch is used to weight different types of fish targets; all the multi-layer feature maps are input into the neck network, and the neck network weights all the multi-layer feature maps of each fish target according to the weight branch, and extracts multiple target key features of each fish target; the neck network integrates all the target key features of each fish target through an attention mechanism to obtain detail features and global information of each fish target; the neck network divides all the multi-layer feature maps of each fish target into multiple first multi-layer feature maps and multiple second multi-layer feature maps, and enlarges the resolution of each of the first multi-layer feature maps to a preset resolution to obtain a corresponding third multi-layer feature map, wherein the resolution of the first multi-layer feature map is lower than a preset value, and the resolutions of the second multi-layer feature map and the third multi-layer feature map are higher than a preset value; the neck network upsamples all the second multi-layer feature maps according to each of the detail features and each of the global information, and splices all the second multi-layer feature maps and the third multi-layer feature maps in each of the multi-layer feature maps to obtain and output multiple hierarchical information of each of the fish targets.

[0036] Among them, after the multi-layer feature map is input into the trained neck network, the adaptive visual unit in the neck network uses a dynamic weight adjustment mechanism (i.e., the weight branch obtained when training the neck network) to adaptively optimize the input multi-layer feature map; according to the contextual information of the fish target, the weight of each branch is adaptively adjusted to achieve adaptive adjustment of the weight of the feature channel, so as to better capture the key target features of the fish target.

[0037] Furthermore, the multi-level feature fusion network provides multi-level feature fusion, especially for complex ocean backgrounds and fish diversity scenes, adopts spatial and channel attention mechanisms to integrate local spatial information and global channel information, effectively integrates detail features and global information, and enhances the detection performance of small targets.

[0038] Furthermore, the multi-layer feature maps of different resolutions are upsampled and concatenated: the low-resolution multi-layer feature map is enlarged to high resolution through CARAFE (Content-Aware ReAssembly of FEatures, a lightweight and efficient operator for feature upsampling), and then the upsampled feature map is merged with the high-resolution feature map in the channel dimension to realize hierarchical processing of information and finally output multiple hierarchical information.

[0039] Among them, by combining adaptive visual units and multi-level feature fusion networks, the adaptive fusion capability of multi-level features is optimized, and the robustness and feature expression ability of the model in complex marine environments are improved, especially under conditions of lighting changes and dynamic water flow interference, while maintaining high accuracy of fish identification.

[0040] Step S30, construct an initial detection head, and introduce weight coefficients into the initial detection head for training to obtain a detection head, input all the hierarchical information into the detection head, and output the position information and fish information of all fish targets in the target image.

[0041] Among them, the detection head part is used for the final fish target positioning and classification. Its training process is mainly trained through the WIOU loss function (Weighted Intersection over Union, weighted intersection loss function), which replaces the traditional IOU (Intersection over Union, a loss function used in target detection tasks) loss function, which can optimize the positioning accuracy of the fish target box. The WIOU loss function can introduce a weight mechanism for the boundary overlapping area, which solves the shortcomings of the IOU loss function in dealing with dense targets and overlapping targets, optimizes the calculation efficiency, and ensures the application of the system in real-time monitoring.

[0042] In addition to the WIOU loss function, similar effects can be achieved by using the GIOU (Generalized Intersection over Union Loss, an improved version of the IOU loss function) or the DIOU (Distance Intersection over Union Loss, a loss function commonly used in object detection tasks to optimize the position of the bounding box) loss function, or by adjusting the weight mechanism.

[0043] Specifically, an initial detection head is constructed, and a preset weight coefficient is input into the initial detection head. The initial detection head adds a weighting item according to the weight coefficient, and the weight coefficient is adjusted by the weighting item to obtain a loss calculation value. The loss calculation value is used for adaptive optimization to obtain a detection head; all the hierarchical information of each fish target is input into the detection head, and the detection head analyzes all the hierarchical information to obtain the boundary overlapping area of ​​multiple fish targets in the target image; the detection head marks the position and type of each fish target according to each hierarchical information to obtain the position information and fish information of each fish target.

[0044] Among them, Figure 3 and Figure 4 As shown, Figure 3The activation function in can be the Sigmoid activation function (SiLU, SigmoidLinear Unit, Sigmoid linear unit), Figure 4 The loss function in is WIOU. By introducing a weight mechanism for the boundary overlap area output by the detection head: that is, the advanced basic IOU loss function, and then introducing a weight coefficient, adjusting the IOU value through the weighted term, and adaptively adjusting the weight coefficient, and finally performing the final loss calculation, the detection frame positioning accuracy is improved, especially in scenes with overlapping or densely distributed fish.

[0045] Furthermore, in the classification and regression module of the detection head, the two major tasks of regression and classification are combined to achieve fish species recognition and position labeling through precise feature matching: that is, the labeling of the location of the fish in the picture and the labeling of the fish species, which significantly improves the accuracy of bounding box positioning and effectively reduces the detection error in target overlapping scenarios; by organically integrating CFB, AVBlock, MultiFPN_SDI and WIOU loss functions, a multi-scale detection network suitable for marine ranch fish identification is constructed, which significantly improves the adaptability of the system in a variety of complex environments, while reducing the computational complexity of the model to meet real-time monitoring needs.

[0046] Furthermore, in this embodiment, if there are special scenes with extremely poor lighting conditions or large water flow effects, the number of feedbacks of the coupled feedback unit (CFB) can be further enhanced, or the weight dynamic range of the adaptive vision module (AVBlock) can be adjusted to adapt to extreme conditions.

[0047] Furthermore, this embodiment can also use a model different from CNN, for example, based on the Transformer model architecture (converter model, including four main parts: input processing, encoder, decoder and output processing), which has a strong global feature modeling capability, can replace CNN for feature extraction, and provide a multi-scale feature learning effect similar to the present invention. As for the model pre-training part, task-specific fine-tuning can also be performed based on general pre-training models (such as CLIP, Contrastive Language-Image Pre-Training, SAM, Segment Anything Model, segmentation everything model). Through the rich feature space and cross-domain adaptability of these models, it is possible to achieve an efficient solution without building a network architecture from scratch.

[0048] Furthermore, in this embodiment, the network architecture can be simplified by removing specific modules (such as AVBlock or CFB modules), and additional computing resources or optimization algorithms can be used to make up for the performance gap. Although the simplified architecture may reduce the detection accuracy, it can still meet the requirements in some applications.

[0049] Furthermore, in this embodiment, a non-deep learning fish detection system can also be constructed based on traditional computer vision algorithms (such as SIFT, Scale-Invariant Feature Transform, SURF, Speeded Up Robust Features, accelerated robust feature algorithms, etc.), combined with optimizers or machine learning models. Although these traditional methods have limited performance in complex environments, they may be used as a design solution to avoid deep learning methods.

[0050] Furthermore, in this embodiment, target detection can also be achieved by changing the data source, for example, using other types of sensor data (such as sonar imaging, laser scanning) for target detection.

[0051] Furthermore, in this embodiment, the CFB module can be replaced by a hierarchical attention module, or grouped convolution and lightweight architecture optimization can be used to achieve an effect similar to that of this embodiment.

[0052] Furthermore, in this embodiment, multimodal data (such as sonar imaging, temperature sensing, and optical and non-optical data fusion) can be combined to build a more efficient and accurate fish identification system through in-depth analysis and fusion of multimodal data. The introduction of multimodal data will significantly improve the robustness and adaptability of the model in complex environments, thereby expanding the application scenarios of the technology. In addition, in order to adapt to the limited computing resources that may exist in marine ranches, the technology of the present invention can also explore lightweight design and edge computing solutions for deep learning models. For example, the model architecture is optimized through model pruning, quantization or distillation technology to reduce computing and storage overhead, and some core computing tasks are migrated to edge devices (such as underwater camera equipment or buoy systems) to achieve localized processing and real-time feedback, reducing dependence on cloud computing resources.

[0053] Furthermore, in this embodiment, data enhancement and active learning techniques can be combined to further improve the adaptability and recognition ability of the model. In view of the special lighting conditions and changes in fish postures in underwater environments, image enhancement strategies that conform to actual scenarios can be designed, such as rotation, blurring, and color adjustment. In addition, through active learning methods, the most informative samples can be selected from unlabeled data for labeling and training, and the model performance can be continuously optimized in practical applications.

[0054] Furthermore, in this embodiment, in addition to realizing the fish identification function, comprehensive ecological environment management can also be carried out in combination with the detection of environmental parameters, so that users can intuitively understand the real-time status of the pasture and conduct efficient and convenient remote monitoring and management through wireless communication technology.

[0055] Through multi-module design and deep optimization, the present invention effectively improves the accuracy of fish detection, significantly improves the robustness of the model, and optimizes the detection process. It can not only be used for real-time monitoring of marine ranches, but can also be expanded to fields such as marine ecological monitoring, aquaculture management, and illegal fishing monitoring. In marine ecological monitoring, it is possible to provide accurate data support for studying the dynamic changes of marine ecosystems by analyzing the distribution and quantity changes of fish populations. In aquaculture management, the health status, stocking density, and abnormal behavior of fish can be monitored in real time to improve breeding efficiency and management effects. When combined with target detection and tracking algorithms, the fishing situation can also be monitored and recorded in real time, which helps to protect marine resources.

[0056] Furthermore, if Figure 5 As shown, based on the above fish identification method, the present invention also provides a fish identification system accordingly, wherein the fish identification system comprises: A feature extraction module 51 is used to construct a backbone network, input a target image into the backbone network for feature extraction, perform iterative optimization and feature fusion processing on the extracted features, and output multiple multi-layer feature maps; A feature splicing module 52 is used to construct and train a neck network using a learning data set, input all the multi-layer feature maps into the neck network, and the neck network uses weight branches to upsample multiple multi-layer feature maps and then splices them to output multiple hierarchical information; The result generation module 53 is used to construct an initial detection head, and introduce weight coefficients into the initial detection head for training to obtain a detection head, input all the hierarchical information into the detection head, and output the position information and fish information of all fish targets in the target image.

[0057] Furthermore, if Figure 6 As shown, based on the above-mentioned fish identification method and system, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 6 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0058] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit and an external storage device of the terminal. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a fish identification program 40 is stored on the memory 20, and the fish identification program 40 can be executed by the processor 10, thereby realizing the fish identification method in the present application.

[0059] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, which is used to run the program code or process data stored in the memory 20, such as executing the fish identification method.

[0060] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0061] In one embodiment, when the processor 10 executes the fish identification program 40 in the memory 20, the following steps are implemented: Constructing a backbone network, inputting the target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting multiple multi-layer feature maps; A neck network is constructed and trained using a learning data set, and all the multi-layer feature maps are input into the neck network. The neck network uses weight branches to upsample a plurality of the multi-layer feature maps and then performs splicing processing to output a plurality of hierarchical information; An initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output.

[0062] The step of constructing a backbone network, inputting a target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting multiple multi-layer feature maps specifically includes: Constructing a backbone network, wherein the backbone network includes a convolution unit, a coupled feedback unit, and a spatial pyramid pooling unit; Acquire a target image input by a user, input the target image into the convolution unit, and the convolution unit preprocesses and extracts features of the target image to obtain multiple multi-scale features of multiple fish targets in the target image; Inputting each of the multi-scale features into the coupling feedback unit, the coupling feedback unit iteratively optimizes each of the multi-scale features to obtain a multi-level detail feature corresponding to each of the multi-scale features; Each of the multi-level detail features is input into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features and then splices them to output a plurality of multi-layer feature maps.

[0063] The step of obtaining a target image input by a user, inputting the target image into the convolution unit, and performing preprocessing and feature extraction on the target image to obtain multiple multi-scale features of multiple fish targets in the target image specifically includes: Acquire a target image input by a user, input the target image into the convolution unit, and the convolution unit performs grayscale processing, denoising processing, and intensity enhancement processing on the target image to obtain a preprocessed image; The convolution unit performs edge detection on the preprocessed image by an edge detection algorithm, performs texture analysis on the preprocessed image, obtains edge features of each fish target and texture features of each fish target, and quantizes the edge features of each fish target and the texture features of each fish target; The convolution unit fuses each quantized edge feature and each quantized texture feature to obtain multiple multi-scale features of each fish target.

[0064] The step of inputting each of the multi-scale features into the coupling feedback unit, and the coupling feedback unit iteratively optimizing each of the multi-scale features to obtain a multi-level detail feature corresponding to each of the multi-scale features, specifically includes: Input each of the multi-scale features in each fish target into the coupling feedback unit, the coupling feedback unit identifies the exposure of each of the multi-scale features, and fuses the low-exposure multi-scale features and the high-exposure multi-scale features in each fish target to obtain a fusion feature result, wherein the low exposure and the high exposure are both determined according to a preset exposure value; The coupled feedback unit obtains feedback information input by the user, and performs iterative fusion processing on all multi-scale features of each fish target according to the feedback information and the fusion feature result, so as to obtain multiple fused multi-scale features of each fish target; The coupled feedback unit continuously samples multiple fused multi-scale features of each fish target to optimize each fused multi-scale feature and obtain multiple multi-level detail features of each fish target.

[0065] The step of inputting each of the multi-level detail features into the spatial pyramid pooling unit, performing maximum pooling processing on each of the multi-level detail features, and then splicing the features to output multiple multi-layer feature maps specifically includes: Input each of the multi-level detail features into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features of each fish target passing through the initial convolutional layer to obtain a pooling result corresponding to each of the multi-level detail features; The spatial pyramid pooling unit splices all multi-level detail features of each fish target according to all the pooling results, and outputs multiple multi-layer feature maps of each fish target through the final convolution layer.

[0066] The method of constructing and training a neck network using a learning data set, inputting all the multi-layer feature maps into the neck network, and performing splicing processing on the neck network after upsampling the multi-layer feature maps using weight branches to output multiple hierarchical information specifically includes: Constructing an initial neck network, and inputting a pre-stored learning data set into the initial neck network for training, to obtain a neck network and a weight branch, wherein the weight branch is used to weight different types of fish targets; Inputting all the multi-layer feature maps into the neck network, the neck network weights all the multi-layer feature maps of each fish target according to the weight branch, and extracts a plurality of target key features of each fish target; The neck network integrates all key features of each fish target through an attention mechanism to obtain detailed features and global information of each fish target; The neck network divides all the multi-layer feature maps of each fish target into a plurality of first multi-layer feature maps and a plurality of second multi-layer feature maps, and amplifies the resolution of each of the first multi-layer feature maps to a preset resolution to obtain a corresponding third multi-layer feature map, wherein the resolution of the first multi-layer feature map is lower than a preset value, and the resolutions of the second multi-layer feature map and the third multi-layer feature map are higher than the preset value; The neck network upsamples all the second multi-layer feature maps according to each of the detail features and each of the global information, and concatenates all the second multi-layer feature maps and the third multi-layer feature maps in each of the multi-layer feature maps to obtain and output multiple hierarchical information of each of the fish targets.

[0067] The initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output, specifically including: Constructing an initial detection head, inputting a preset weight coefficient into the initial detection head, adding a weighting term according to the weight coefficient, adjusting the weight coefficient by the weighting term to obtain a loss calculation value, and performing adaptive optimization using the loss calculation value to obtain a detection head; Inputting all the hierarchical information of each fish target into the detection head, the detection head analyzing all the hierarchical information to obtain the boundary overlapping area of ​​multiple fish targets in the target image; The detection head marks the position and type of each fish target according to each hierarchical information, and obtains the position information and fish information of each fish target.

[0068] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a fish identification program, and when the fish identification program is executed by a processor, the steps of the fish identification method described above are implemented.

[0069] In summary, the present invention provides a fish identification method and related equipment, the method comprising: constructing a backbone network, inputting a target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting multiple multi-layer feature maps; constructing and training a neck network using a learning data set, inputting all the multi-layer feature maps into the neck network, the neck network up-sampling multiple multi-layer feature maps using weight branches and then performing splicing processing, and outputting multiple hierarchical information; constructing an initial detection head, and introducing a weight coefficient into the initial detection head for training to obtain a detection head, inputting all the hierarchical information into the detection head, and outputting the position information and fish information of all fish targets in the target image. The present invention effectively improves the accuracy of fish detection, significantly improves the robustness of the model, and optimizes the detection process through multi-module design and deep optimization.

[0070] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0071] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0072] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A fish identification method, characterized in that: The fish identification method specifically comprises: Constructing a backbone network, inputting the target image into the backbone network for feature extraction, iteratively optimizing and fusion processing the extracted features, and outputting multiple multi-layer feature maps; A neck network is constructed and trained using a learning data set, and all the multi-layer feature maps are input into the neck network. The neck network uses weight branches to upsample a plurality of the multi-layer feature maps and then performs splicing processing to output a plurality of hierarchical information; An initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output.

2. The fish identification method according to claim 1, characterized in that: The backbone network is constructed, the target image is input into the backbone network for feature extraction, the extracted features are iteratively optimized and feature fusion processed, and multiple multi-layer feature maps are output, specifically including: Constructing a backbone network, wherein the backbone network includes a convolution unit, a coupled feedback unit, and a spatial pyramid pooling unit; Acquire a target image input by a user, input the target image into the convolution unit, and the convolution unit preprocesses and extracts features of the target image to obtain multiple multi-scale features of multiple fish targets in the target image; Inputting each of the multi-scale features into the coupling feedback unit, the coupling feedback unit iteratively optimizes each of the multi-scale features to obtain a multi-level detail feature corresponding to each of the multi-scale features; Each of the multi-level detail features is input into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features and then splices them to output a plurality of multi-layer feature maps.

3. The fish identification method according to claim 2, characterized in that: The step of obtaining a target image input by a user, inputting the target image into the convolution unit, and performing preprocessing and feature extraction on the target image to obtain multiple multi-scale features of multiple fish targets in the target image specifically includes: Acquire a target image input by a user, input the target image into the convolution unit, and the convolution unit performs grayscale processing, denoising processing, and intensity enhancement processing on the target image to obtain a preprocessed image; The convolution unit performs edge detection on the preprocessed image by an edge detection algorithm, performs texture analysis on the preprocessed image, obtains edge features of each fish target and texture features of each fish target, and quantizes the edge features of each fish target and the texture features of each fish target; The convolution unit fuses each quantized edge feature and each quantized texture feature to obtain multiple multi-scale features of each fish target.

4. The fish identification method according to claim 2, characterized in that: Inputting each of the multi-scale features into the coupling feedback unit, wherein the coupling feedback unit iteratively optimizes each of the multi-scale features to obtain a multi-level detail feature corresponding to each of the multi-scale features, specifically includes: Input each of the multi-scale features in each fish target into the coupling feedback unit, the coupling feedback unit identifies the exposure of each of the multi-scale features, and fuses the low-exposure multi-scale features and the high-exposure multi-scale features in each fish target to obtain a fusion feature result, wherein the low exposure and the high exposure are both determined according to a preset exposure value; The coupled feedback unit obtains feedback information input by the user, and performs iterative fusion processing on all multi-scale features of each fish target according to the feedback information and the fusion feature result, so as to obtain multiple fused multi-scale features of each fish target; The coupled feedback unit continuously samples multiple fused multi-scale features of each fish target to optimize each fused multi-scale feature and obtain multiple multi-level detail features of each fish target.

5. The fish identification method according to claim 2, characterized in that: The step of inputting each of the multi-level detail features into the spatial pyramid pooling unit, wherein the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features and then splices the features to output a plurality of multi-layer feature maps, specifically includes: Input each of the multi-level detail features into the spatial pyramid pooling unit, and the spatial pyramid pooling unit performs maximum pooling processing on each of the multi-level detail features of each fish target passing through the initial convolutional layer to obtain a pooling result corresponding to each of the multi-level detail features; The spatial pyramid pooling unit splices all multi-level detail features of each fish target according to all the pooling results, and outputs multiple multi-layer feature maps of each fish target through the final convolution layer.

6. The fish identification method according to claim 1, characterized in that: The neck network is constructed and trained using the learning data set, all the multi-layer feature maps are input into the neck network, the neck network uses weight branches to upsample multiple multi-layer feature maps and then performs splicing processing to output multiple hierarchical information, specifically including: Constructing an initial neck network, and inputting a pre-stored learning data set into the initial neck network for training, to obtain a neck network and a weight branch, wherein the weight branch is used to weight different types of fish targets; Inputting all the multi-layer feature maps into the neck network, the neck network weights all the multi-layer feature maps of each fish target according to the weight branch, and extracts a plurality of target key features of each fish target; The neck network integrates all key features of each fish target through an attention mechanism to obtain detailed features and global information of each fish target; The neck network divides all the multi-layer feature maps of each fish target into a plurality of first multi-layer feature maps and a plurality of second multi-layer feature maps, and amplifies the resolution of each of the first multi-layer feature maps to a preset resolution to obtain a corresponding third multi-layer feature map, wherein the resolution of the first multi-layer feature map is lower than a preset value, and the resolutions of the second multi-layer feature map and the third multi-layer feature map are higher than the preset value; The neck network upsamples all the second multi-layer feature maps according to each of the detail features and each of the global information, and concatenates all the second multi-layer feature maps and the third multi-layer feature maps in each of the multi-layer feature maps to obtain and output multiple hierarchical information of each of the fish targets.

7. The fish identification method according to claim 1, characterized in that: The initial detection head is constructed, and a weight coefficient is introduced into the initial detection head for training to obtain a detection head, all the hierarchical information is input into the detection head, and the position information and fish information of all fish targets in the target image are output, specifically including: Constructing an initial detection head, inputting a preset weight coefficient into the initial detection head, adding a weighting term according to the weight coefficient, adjusting the weight coefficient by the weighting term to obtain a loss calculation value, and performing adaptive optimization using the loss calculation value to obtain a detection head; Inputting all the hierarchical information of each fish target into the detection head, the detection head analyzing all the hierarchical information to obtain the boundary overlapping area of ​​multiple fish targets in the target image; The detection head marks the position and type of each fish target according to each hierarchical information, and obtains the position information and fish information of each fish target.

8. A fish identification system, characterized in that: The fish identification system comprises: A feature extraction module is used to construct a backbone network, input the target image into the backbone network for feature extraction, iteratively optimize and fuse the extracted features, and output multiple multi-layer feature maps; A feature splicing module is used to construct and train a neck network using a learning data set, input all the multi-layer feature maps into the neck network, and the neck network uses weight branches to upsample multiple multi-layer feature maps and then splices them to output multiple hierarchical information; The result generation module is used to construct an initial detection head, introduce weight coefficients into the initial detection head for training, obtain a detection head, input all the hierarchical information into the detection head, and output the position information and fish information of all fish targets in the target image.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a fish identification program stored in the memory and executable on the processor. When the fish identification program is executed by the processor, the steps of the fish identification method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a fish identification program, and when the fish identification program is executed by a processor, the steps of the fish identification method according to any one of claims 1 to 7 are implemented.

Images