Sick fish detection method and detection device

Abstract

The invention discloses a diseased fish detection method and device, and belongs to the technical field of computer vision detection. In order to solve the problems that in the prior art, diseased fish disease detection efficiency is low, accuracy is not high, and real-time detection cannot be achieved, a diseased fish detection model is constructed, and the diseased fish detection model comprises a backbone network, a neck network and a detection head network; the method comprises the following steps: inputting a diseased fish image into a backbone network based on a spatial deep convolutional neural network for feature extraction to obtain a visual feature map, and then inputting the visual feature map into a contrast-enhanced attention transfer structure of a neck network for feature fusion processing to obtain a fused feature map; and the fusion feature map is input into a detection head network, and a sick fish target detection result is obtained through detection of a multi-head prediction small target detection algorithm, so that fish school disease detection is performed in real time, the detection efficiency and detection precision of fish diseases are effectively improved, and the method has relatively high practicability and application universality.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision detection, and more specifically to a diseased fish detection method and a detection device. Background Art

[0002] Fish, as a key part of aquaculture, are rich in phosphorus, calcium, beneficial fats, high-quality protein and other nutrients, and are an important source of protein for human daily diet. At present, the fish farming model is gradually transforming into a new factory-based, intelligent and intensive farming model. While greatly improving the farming efficiency, it also faces new difficulties and challenges: under high-density farming conditions, fish diseases are prone to collective infection, causing irreparable losses. Therefore, timely diagnosis and detection of sick fish and avoiding large-scale deaths of fish under factory-based high-density farming conditions are of great practical significance for improving the economic benefits of factories and enterprises.

[0003] Traditional fish disease detection methods often rely on manual sampling and then disease analysis in the laboratory. Although this method has high accuracy, it has problems such as low detection efficiency, long detection time, certain degree of damage to the fish body, and easy stress response. In addition, manual sampling cannot achieve real-time or continuous monitoring of a large area. It can only obtain disease information of individual fish samples, lacking information on the overall disease distribution of fish schools, making it difficult to promptly detect and respond to sudden fish infectious diseases.

[0004] At present, the technology of acquiring visible light spectrum images based on RGB-D cameras provides rich color and depth information to achieve more accurate and stable processing and recognition effects, so it can be used as a key technical means for data collection of diseased fish. Compared with traditional image processing methods, deep learning technology effectively solves the latter's shortcomings in color and texture feature recognition, and reduces the reliance on manual feature extraction. At present, deep learning has been widely used in many important fields such as computer vision and natural language processing (NLP). Among the applications of deep learning in the field of computer vision, target detection is a core task in the field of computer vision. Its main purpose is to find all research-related targets (objects) from images or videos and determine their categories and locations. This task requires not only solving the classification problem, that is, judging which categories of targets are contained in the image, but also solving the positioning problem, that is, determining the specific locations of these targets in the image.

[0005] With the continuous development of deep learning technology, target detection technology has been widely used in the field of fisheries. However, due to the complexity of the water environment in which fish live and the diversity of the actual disease conditions of fish, the work of extracting fish disease characterization information based on target detection still has the following problems: (1) Due to the poor quality and low resolution of water imaging, the target edge is blurred, which makes it difficult to identify and increases the risk of missed detection and false detection; (2) In the process of target detection of fish disease images, the detection of the aggregation of small fish targets is often carried out. The classification of an anchor box is determined by the intersection of the anchor box and the true bounding box and the set threshold condition. If the category is divided based on multiple objects with small overlapping areas, it is easy to cause classification errors, poor anchor box positioning effect, and low accuracy of fish characterization information extraction; (3) The size characteristics of different species of fish and the epigenetic phenomena of the same species of fish will lead to differences in the scale of individual fish. With the increase of the number of feature extraction network layers in the target detection model, frequent downsampling operations make it difficult to effectively detect small-scale fish.

[0006] Therefore, there is an urgent need for an efficient, real-time and highly accurate method for detecting diseases in aquaculture fish. Summary of the invention

[0007] 1. Technical issues to be solved

[0008] In view of the problems in the prior art of low efficiency, low accuracy and inability to detect diseased fish in real time, the present invention provides a diseased fish detection method and detection device, which can realize real-time fish disease detection and effectively improve the detection efficiency and accuracy of fish diseases.

[0009] 2. Technical solution

[0010] The purpose of the present invention is achieved through the following technical solutions.

[0011] A method for detecting diseased fish comprises the following steps:

[0012] Collect fish images, identify diseased fish images in the fish images, and pre-process the diseased fish images;

[0013] Constructing a diseased fish detection model, the diseased fish detection model comprising a backbone network, a neck network and a detection head network; the backbone network comprises a spatial deep convolutional neural network, and the neck network comprises a contrast enhancement transfer attention structure;

[0014] The image of diseased fish is input into the backbone network based on the spatial deep convolutional neural network for feature extraction to obtain a visual feature map; the visual feature map is input into the contrast enhancement transfer attention structure of the neck network for feature fusion processing to obtain a fused feature map; the fused feature map is input into the detection head network and the diseased fish target detection result is obtained through the multi-head prediction small target detection algorithm.

[0015] Furthermore, the steps of preprocessing the diseased fish images include:

[0016] De-noising and normalizing the diseased fish images;

[0017] Enhance the feature information of diseased fish images through automated prominence and contrast enhancement algorithms;

[0018] The automatic prominence and contrast enhancement algorithm includes an image pixel value preprocessing module and an automatic prominence and contrast enhancement calculation module;

[0019] The sick fish image is input into the image pixel value preprocessing module to obtain the image position information of the sick fish image, wherein the image position information includes the reflected light intensity value and the ambient light illumination component value, and the image pixel point processing value is calculated by the reflected light intensity value and the ambient light illumination component value; the image pixel point processing value is input into the automatic protrusion and contrast enhancement calculation module to calculate the pixel information of the sick fish image, and the characteristic information of the sick fish image is enhanced by the pixel information of the sick fish image.

[0020] Furthermore, the image pixel processing value is input into the automatic protrusion and contrast enhancement calculation module to calculate the pixel information of the diseased fish image, and the calculation formula is:

[0021] R' auto-IMES (x, y) = log 10 S(x,y)-log 10 [S(x,y)*K(x,y)]

[0022]

[0023] Among them, R auto-IMES (x,y) represents the pixel value of the diseased fish image contrast pre-enhancement on a single channel, x represents the horizontal coordinate of the pixel, y represents the vertical coordinate of the pixel, S(x,y) represents the image pixel processing value of the diseased fish image at the coordinate (x,y), K(x,y) represents the Cauchy illumination filter at the nth scale at the coordinate (x,y), * Represents the product operation, R auto-IMES(x, y) represents the pixel value of the overall contrast enhancement of the diseased fish image, n represents a natural number, N represents the number of channels, C(x, y) represents the restoration coefficient of the pixel at the coordinate (x, y) on the diseased fish image, G n Represents the gain factor of the nth scale.

[0024] Furthermore, the spatial deep convolutional neural network includes a spatial deep convolutional module and a non-strided convolutional module;

[0025] The deep spatial convolution module crops the diseased fish image into a plurality of sub-feature graphs, and connects the spatial dimensions of the plurality of sub-feature graphs through the channel dimension to obtain a feature graph with a reduced spatial dimension and an increased channel dimension;

[0026] The filter in the non-stride convolution module performs convolution processing on the feature map to obtain a visual feature map.

[0027] Further, the contrast enhancement transfer attention structure includes a regional contrast enhancement module, a channel adaptive weight adjustment module and a channel space contrast enhancement module;

[0028] The regional contrast enhancement module includes a standard convolution layer and a normalization layer;

[0029] The visual feature map is calculated through a standard convolution layer and a normalization layer to obtain a first shallow feature map; the first shallow feature map is expressed as:

[0030]

[0031] Among them, F1 represents the first shallow feature map, F represents the visual feature map, BN represents the normalization layer, and Conv represents the convolutional layer. Represents a 3×3 standard convolutional layer;

[0032] The first shallow feature map is processed by nonlinear activation function to obtain a deep feature map; the deep feature map is expressed as:

[0033]

[0034] Among them, F PCE represents the deep feature map, Sigmoid represents the Sigmoid activation function, LeakyRELU represents the LeakyReLU activation function, Represents a convolution operation.

[0035] Further, the channel adaptive weight adjustment module includes a standard convolution layer, a normalization layer and a depth-separable convolution layer;

[0036] The visual feature map is calculated through a standard convolution layer and a normalization layer to obtain a first shallow feature map; the first shallow feature map is expressed as:

[0037]

[0038] The first shallow feature map is processed by nonlinear activation function and depth-separable convolution layer to obtain a depth-separable convolution feature map; the depth-separable convolution feature map is expressed as:

[0039]

[0040] Among them, F CAWA represents the depth-wise separable convolutional feature map, Represents a 3×3 standard convolutional layer after being processed by a depthwise separable convolutional layer.

[0041] Furthermore, the channel space contrast enhancement module includes a standard convolution layer, a normalization layer and a depth convolution layer;

[0042] The visual feature map is calculated through a standard convolution layer, a normalization layer and a deep convolution layer to obtain a second shallow feature map; the second shallow feature map is expressed as:

[0043]

[0044] Among them, F2 represents the second shallow feature map, represents a 3×3 standard convolutional layer after being processed by a deep convolutional layer;

[0045] The second shallow feature map is processed by nonlinear activation function to obtain a deep convolution feature map; the deep convolution feature map is expressed as:

[0046]

[0047] Among them, F CSCE Represents a deep convolutional feature map.

[0048] Furthermore, the deep feature map, the depth separable convolution feature map and the depth convolution feature map are stacked and added to obtain a fused feature map; the fused feature map is expressed as:

[0049]

[0050] Among them, F' represents the fused feature map, Stack represents the stacking of the deep feature map, the depthwise separable convolutional feature map and the depthwise convolutional feature map according to the color channel dimension, and ⊕ represents the addition of the deep feature map, the depthwise separable convolutional feature map and the depthwise convolutional feature map along the stacked color channel dimension.

[0051] Furthermore, the multi-head prediction small target detection algorithm includes removing the large target detection decoupling head in the detection head network and adding a small target detection decoupling head; the fused feature map is detected by the small target detection decoupling head to obtain the diseased fish target detection result.

[0052] A diseased fish detection device, comprising:

[0053] An acquisition module, which acquires fish images, identifies diseased fish images in the fish images, and pre-processes the diseased fish images;

[0054] A model building module is used to build a diseased fish detection model, wherein the diseased fish detection model includes a backbone network, a neck network, and a detection head network; the backbone network includes a spatial deep convolutional neural network, and the neck network includes a contrast enhancement transfer attention structure;

[0055] The detection module inputs the diseased fish image into the backbone network based on the spatial deep convolutional neural network for feature extraction to obtain a visual feature map; the visual feature map is input into the contrast enhancement transfer attention structure of the neck network for feature fusion processing to obtain a fused feature map; the fused feature map is input into the detection head network and the diseased fish target detection result is obtained through the multi-head prediction small target detection algorithm.

[0056] 3. Beneficial effects

[0057] Compared with the prior art, the advantages of the present invention are:

[0058] (1) A diseased fish detection method and detection device of the present invention obtains pixel information of diseased fish images by constructing an automatic prominence and contrast enhancement algorithm, and uses the pixel information of the diseased fish images to enhance the feature information of the diseased fish images, that is, to enhance the contrast and prominence of the pixel information of the diseased fish images, highlight the feature information of relatively dark areas in the diseased fish images, effectively avoid the defect of color distortion caused by contrast enhancement in local areas of the diseased fish images, improve the local contrast and brightness of the diseased fish images, and effectively improve the detection accuracy of diseased fish.

[0059] (2) The diseased fish detection method and detection device of the present invention improve the processing performance of low-resolution diseased fish images by constructing a spatial deep convolutional neural network, and extract more fine-grained and detailed information of diseased fish images of small target detection objects under low resolution by reducing the step size and reducing the pooling convolution layer, thereby effectively solving the problem of small target object detection information loss during over-upsampling when using the step size and pooling method of traditional convolutional neural network (CNN).

[0060] (3) A diseased fish detection method and detection device of the present invention uses a deep separable convolutional layer in the channel adaptive weight adjustment module to optimize the feature separation between channels, thereby improving the quality of the output feature map and effectively solving the problem of insufficient extraction of feature information of diseased fish images due to low resolution of diseased fish images and uneven light distribution.

[0061] (4) The diseased fish detection method and detection device of the present invention enhance the contrast of the visual feature map through a deep convolution layer in the channel space contrast enhancement module, and enhance the diseased fish detection model's ability to capture and process fine information, thereby effectively improving the performance of the diseased fish detection model in the water surface environment.

[0062] (5) A diseased fish detection method and detection device of the present invention constructs a contrast enhancement transfer attention structure, connects the regional contrast enhancement module, the channel adaptive weight adjustment module and the channel space contrast enhancement module in the contrast enhancement transfer attention structure, and respectively extracts different feature information in the visual feature map through the regional contrast enhancement module, the channel adaptive weight adjustment module and the channel space contrast enhancement module to obtain a fused feature map, effectively improving the contrast of important areas in the fused feature map, while reducing the influence of other areas, and significantly enhancing the local contrast.

[0063] (6) A diseased fish detection method and detection device of the present invention constructs and adds a multi-head prediction small target detection algorithm for scale information extraction in the detection head network, including the removal and optimization of the large target decoupling head in the native network structure and the addition of a small target detection decoupling head, which effectively solves the problem of missed detection of small targets and poor detection effect of the over-large scale detection head. At the same time, the bottom-up part of the feature pyramid is changed to a fusion from the middle layer to the two side layers, that is, the feature information of the P2 layer is fused with the feature information of the P3 layer to retain more small target feature information, thereby significantly improving the detection accuracy of diseased fish. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a method for detecting diseased fish according to an embodiment of the present invention;

[0065] Figure 2 The image of a diseased fish according to an embodiment of the present invention;

[0066] Figure 3 The image of the diseased fish after preprocessing in the embodiment of the present invention;

[0067] Figure 4 This is a structural diagram of a spatial deep convolutional neural network according to an embodiment of the present invention;

[0068] Figure 5 This is a diagram of the contrast enhancement transfer attention structure of an embodiment of the present invention;

[0069] Figure 6 This is a structural diagram of a multi-head prediction small target detection algorithm according to an embodiment of the present invention;

[0070] Figure 7 This is a target detection prediction anchor diagram on the validation set of the original YOLOv8 network diseased fish detection model evaluation index diagram in an embodiment of the present invention;

[0071] Figure 8 It is an F1 value curve diagram of the evaluation index diagram of the original YOLOv8 network diseased fish detection model in the embodiment of the present invention;

[0072] Fig. 9 It is a Recall value curve diagram of the evaluation index diagram of the original YOLOv8 network diseased fish detection model in the embodiment of the present invention;

[0073] Fig.10 The target detection prediction anchor diagram on the validation set of the evaluation index diagram of the diseased fish detection model of the spatial deep convolutional neural network YOLOv8 network is added to the embodiment of the present invention;

[0074] Fig.11 The F1 value curve of the evaluation index diagram of the diseased fish detection model of the spatial deep convolutional neural network YOLOv8 network is added to the embodiment of the present invention;

[0075] Fig.12 The Recall value curve of the evaluation index diagram of the diseased fish detection model of the spatial deep convolutional neural network YOLOv8 network is added to the embodiment of the present invention;

[0076] Fig.13 The target detection prediction anchor diagram on the validation set of the evaluation index diagram of the diseased fish detection model of the YOLOv8 network with contrast enhancement transfer attention structure added to the embodiment of the present invention;

[0077] Fig.14 The F1 value curve of the evaluation index diagram of the diseased fish detection model of the YOLOv8 network with contrast enhancement and transfer attention structure added to the embodiment of the present invention;

[0078] Fig.15 Add the Recall value curve of the YOLOv8 network diseased fish detection model evaluation index diagram for the embodiment of the present invention with contrast enhancement transfer attention structure;

[0079] Fig.16 The target detection prediction anchor diagram on the validation set of the evaluation index diagram of the diseased fish detection model of the YOLOv8 network network, which is a multi-head prediction small target detection algorithm, is added to the embodiment of the present invention;

[0080] Fig.17Add the F1 value curve of the evaluation index diagram of the diseased fish detection model of the multi-head prediction small target detection algorithm YOLOv8 network to the embodiment of the present invention;

[0081] Fig.18 The Recall value curve of the evaluation index diagram of the diseased fish detection model of the YOLOv8 network of the multi-head prediction small target detection algorithm is added to the embodiment of the present invention. DETAILED DESCRIPTION

[0082] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0083] Example

[0084] like Figure 1 As shown, a diseased fish detection method provided by this embodiment includes the following steps: collecting fish images, identifying diseased fish images in the fish images, and preprocessing the diseased fish images; constructing a diseased fish detection model, the diseased fish detection model includes a backbone network, a neck network and a detection head network, wherein the backbone network includes a spatial deep convolutional neural network, and the neck network includes a contrast enhancement transfer attention structure; inputting the diseased fish image into the backbone network based on the spatial deep convolutional neural network to perform feature extraction to obtain a visual feature map, inputting the visual feature map into the contrast enhancement transfer attention structure of the neck network to perform feature fusion processing to obtain a fused feature map, and inputting the fused feature map into the detection head network to obtain a diseased fish target detection result through a multi-head prediction small target detection algorithm.

[0085] Specifically in this embodiment, firstly, a camera is used to capture fish images in a common breeding water environment. In this embodiment, the camera used can be an existing RGB-D camera. In this embodiment, the underwater fish include gregarious breeding fish such as silver carp, grass carp, bighead carp, and perch.

[0086] like Figure 2 As shown, the diseased fish images are identified by manual recognition or machine vision in the collected fish images, and then the diseased fish images are preprocessed. Specifically, the diseased fish images are denoised and standardized, that is, the diseased fish images are denoised by the mean filtering method, and then the denoised diseased fish images are standardized and randomly cropped into diseased fish images with a height of 256 pixels, a width of 256 pixels, and a number of channels of 3, that is, the size of the diseased fish images after cutting becomes [256,256,3]. Therefore, by cutting the diseased fish images, the background can be removed from the diseased fish images and the diseased fish can be obtained, which is convenient for subsequent detection steps; at the same time, segmenting the high-resolution diseased fish images can further reduce the amount of calculation and improve the detection efficiency. It should be noted that in this embodiment, denoising and standardizing the diseased fish images are prior art.

[0087] Furthermore, if Figure 3 As shown, the feature information of the diseased fish image is enhanced by an automated prominence and contrast enhancement algorithm. In this embodiment, the automated prominence and contrast enhancement algorithm includes an image pixel value preprocessing module and an automated prominence and contrast enhancement calculation module. Specifically, after the image of the diseased fish is obtained by shooting with a camera, a horizontal pixel coordinate system is established, and the image position information corresponding to the image of the diseased fish is further obtained by the camera, and the image position information includes the reflected light intensity value and the ambient light illumination component value. In this embodiment, for the reflected light intensity value, the light intensity values ​​of the red channel, green channel, and blue channel are recorded by an RGB-D camera, and the reflected light intensity value is reflected by the light intensity values ​​of the red channel, green channel, and blue channel. In this embodiment, for the ambient light illumination component value, the RGB-D camera and the photometer are used to measure simultaneously in the same environment, and the ambient light illumination component value is obtained by using the photometer. Therefore, in this embodiment, the image pixel processing value is calculated by the reflected light intensity value and the ambient light illumination component value, and the calculation formula is:

[0088] S(x,y)=R(x,y)*L(x,y)

[0089] Where x represents the horizontal coordinate of the pixel, y represents the vertical coordinate of the pixel, S(x,y) represents the image pixel processing value of the diseased fish image at the coordinate (x,y), R(x,y) represents the reflected light intensity value of the diseased fish image at the coordinate (x,y), and R(x,y) represents the ambient light irradiation component value of the diseased fish image at the coordinate (x,y). * Represents the product operation.

[0090] After obtaining the image pixel processing value, the pixel information of the diseased fish is obtained by calculating the image pixel processing value through the automatic prominence and contrast enhancement calculation module, that is, the real pixel value of the diseased fish image at the coordinate (x, y) is obtained. The calculation formula is:

[0091] R' auto-IMES (x,y)=log 10 S(x,y)-log 10 [S(x,y)*K(x,y)]

[0092]

[0093] Among them, R auto-IMES(x, y) represents the pixel value of the diseased fish image contrast pre-enhancement on a single channel, x represents the horizontal coordinate of the pixel, y represents the vertical coordinate of the pixel, S(x, y) represents the image pixel processing value of the diseased fish image at the coordinate (x, y), K(x, y) represents the Cauchy illumination filter at the nth scale at the coordinate (x, y), * represents the product operation, R auto-IMES (x, y) represents the pixel value of the overall contrast enhancement of the diseased fish image, n represents a natural number, N represents the number of channels, C(x, y) represents the restoration coefficient of the pixel at the coordinate (x, y) on the diseased fish image, G n Represents the gain factor of the nth scale.

[0094] In this embodiment, for the Cauchy illumination filter K(x, y) at the nth scale at the coordinate (x, y), the specific function formula is as follows:

[0095]

[0096] Among them, ρ represents the correlation coefficient between the horizontal coordinate x of the pixel point and the vertical coordinate y of the pixel point, and its value range is between [-1,1]. μ x、 μy They represent the position parameters of the horizontal coordinate x and vertical coordinate y of the pixel point in the Cauchy distribution function, σ x , σ y They represent the scale parameters of the horizontal coordinate x and vertical coordinate y of the pixel in the Cauchy distribution function.

[0097] For μ x , μ y , the specific calculation formula is as follows:

[0098]

[0099]

[0100] Where n represents the number of samples, x i ,y i Respectively represent the sample data of the horizontal coordinate x and the vertical coordinate y of the pixel at the i-th position;

[0101] For σ x , σ y , the specific calculation formula is as follows:

[0102]

[0103]

[0104]

[0105]

[0106] In this embodiment, the existing Newton-Raphson method is used to iteratively obtain σ' x ,σ' y , and then select the optimal estimate of the number of iterations to get σ x , σ y ,

[0107]

[0108]

[0109] Among them, n represents the number of samples, k represents the number of iterations, l represents the maximum likelihood simplified function, l' represents the first-order derivative of the maximum likelihood simplified function l, and l'' represents the second-order derivative of the maximum likelihood simplified function l.

[0110] In this embodiment, for the restoration coefficient C(x, y) of the pixel at the coordinate (x, y) on the diseased fish image, its specific function is expressed as:

[0111]

[0112] Where α represents the gain degree and β represents the controlled nonlinear intensity constant.

[0113] Therefore, in this embodiment, the pixel information of the diseased fish image is calculated by constructing an automatic prominence and contrast enhancement algorithm, and the pixel information of the diseased fish image is used to enhance the feature information of the diseased fish image, that is, the contrast and prominence of the pixel information of the diseased fish image are enhanced, and the feature information of the relatively dark areas in the diseased fish image is highlighted, which effectively avoids the defect of color distortion caused by contrast enhancement in local areas of the diseased fish image, improves the local contrast and brightness of the diseased fish image, effectively improves the detection accuracy of diseased fish, realizes accurate classification of the disease condition of the fish school, and obtains the phenotypic information of the diseased fish.

[0114] Furthermore, a diseased fish detection model is constructed. In this embodiment, the diseased fish detection model includes a backbone network, a neck network and a detection head network.

[0115] For the backbone network, in this embodiment, a spatial deep convolutional neural network is introduced into the backbone network. Figure 4As shown in FIG. 1 , the spatial deep convolutional neural network is formed by connecting and stacking dynamic convolution modules based on 3D sparse convolution splitting processing. The dynamic convolution modules based on 3D sparse convolution splitting processing include spatial deep convolution modules and non-stride convolution modules. The spatial deep convolution module is used to reduce the spatial dimension of the input diseased fish image to the channel dimension, thereby reducing the amount of calculation, and then map the feature information of the diseased fish image to a new spatial dimension, namely the depth dimension, to increase the depth of the feature information. The non-stride convolution module performs convolution processing on the diseased fish image to retain the feature information of the diseased fish image to the greatest extent.

[0116] In this embodiment, the deep spatial convolution module crops the diseased fish image into several sub-feature maps, and connects the spatial dimensions of the several sub-feature maps through the channel dimension to obtain a feature map with reduced spatial dimension and increased channel dimension. The filter in the non-stride convolution module convolves the feature map to obtain a visual feature map. Specifically, the size of the input diseased fish image P is set to w×h×Ch1, where w represents the width of the diseased fish image, h represents the height of the diseased fish image, and Chl represents the number of color channels of the diseased fish image. The diseased fish image P is cropped into a series of sub-feature maps f by the spatial deep convolution module. x，y , sub-feature map f x，y The size is unified as Among them, scale represents the number of channels, and the sub-feature map f x，y As the downsampling of the diseased fish image P. When scale = 2, four sub-feature maps are obtained, namely: 0,0 、f 1,0 、f 0,1 、f 1，1 , all sub-feature maps are connected according to the channel dimension to obtain a diseased fish image with the spatial dimension reduced by two times and the channel dimension increased by four times The diseased fish image After mapping the output to the non-strided convolution module containing Ch2 filters, the feature information of the diseased fish image is output as P′ The specific calculation formula is:

[0117] f 0，0 =P[0:w:scale,0:w:scale],...,

[0118] f scale-1，0 =P[scale-1:w:scale,0:w:scale];

[0119] f 0，1 =P[0:w:scale,1:w:scale],...,

[0120] fscale-1，1 =P[scale-1:w:scale,1:w:scale];

[0121] f 0，scale-1 =P[0:w:scale,scale-1:w:scale],...,

[0122] f scale-1，scale-1 =P[scale-1:w:scale, scale-1:w:scale]

[0123] It should be noted that, in this embodiment, f x，y It consists of all i+x and i+y in x(i, j) that are divisible by scale.

[0124] Therefore, in this embodiment, a spatial deep convolutional neural network is introduced into the backbone network, and the processing performance of low-resolution diseased fish images is improved through the spatial deep convolutional neural network. By reducing the step size and reducing the pooling convolution layer, more fine-grained and detailed information of diseased fish images of small target detection objects in low-resolution conditions is extracted, which effectively solves the problem of loss of small target object detection information during excessive upsampling when using step size and pooling methods in traditional convolutional neural networks (CNN), realizes the processing of diseased fish image information into features with learnable parameters, and effectively reduces the amount of calculation by reducing the number of channels.

[0125] For the neck network, a contrast enhancement transfer attention structure is inserted into the neck network. In this embodiment, Figure 5 As shown, the contrast enhancement transfer attention structure includes a regional contrast enhancement module, a channel adaptive weight adjustment module and a channel space contrast enhancement module.

[0126] In this embodiment, the regional contrast enhancement module includes a standard convolution layer and a normalization layer. Specifically, the visual feature map is calculated by the standard convolution layer and the normalization layer to obtain a first shallow feature map, and the first shallow feature map is expressed as:

[0127]

[0128] Among them, F1 represents the first shallow feature map, F represents the visual feature map, BN represents the normalization layer, and Conv represents the convolutional layer. Represents a 3×3 standard convolutional layer;

[0129] The first shallow feature map is processed by the nonlinear activation function to obtain a deep feature map, which is expressed as:

[0130]

[0131] Among them, FPCE represents the deep feature map, Sigmoid represents the Sigmoid activation function, LeakyRELU represents the LeakyReLU activation function, Represents a convolution operation.

[0132] Therefore, in the regional contrast enhancement module, through two convolution operations and nonlinear processing of the activation function, the contrast of important areas in the visual feature map is effectively improved, while the influence of other areas is weakened, significantly enhancing the local contrast.

[0133] In this embodiment, the channel adaptive weight adjustment module includes a standard convolution layer, a normalization layer and a depthwise separable convolution layer (Depthwise Separable Convolution, DSC).

[0134] Specifically, the visual feature map is calculated through a standard convolutional layer and a normalization layer to obtain a first shallow feature map, which is expressed as:

[0135]

[0136] For depthwise separable convolution, depthwise separable convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, and pointwise convolution is used to extract channel features. Therefore, the first shallow feature map is processed by the nonlinear activation function and the depthwise separable convolution layer to obtain the depthwise separable convolution feature map. The depthwise separable convolution feature map is expressed as:

[0137]

[0138] Among them, F CAWA represents the depth-wise separable convolutional feature map, Represents a 3×3 standard convolutional layer after being processed by a depthwise separable convolutional layer.

[0139] Therefore, in the channel adaptive weight adjustment module, the weight of each channel is effectively controlled through the deep separable convolution layer to improve the feature separation between channels.

[0140] It should be noted that, in this embodiment, the depthwise separable convolution layer performs grouped convolutions on the feature dimension, performs independent depthwise convolutions on each channel, and aggregates all channels using a 1×1 convolution before output. The specific implementation method is as follows:

[0141] Set the output depth-separable convolution feature map size to D s ×D s , the convolution kernel size is D k ×D k, M represents the number of input channels, N represents the number of output channels, then the number of parameters required for the depthwise separable convolution operation is:

[0142] D k ×D k ×M+M×N

[0143] The number of parameters required for standard convolution is:

[0144] D k ×D k ×M×N

[0145] The computational cost of depth-separable convolution operation is:

[0146] D k ×D k ×M×N×D s ×D s

[0147] The computational cost of a standard convolution operation is:

[0148] D k ×D k ×M×N×D s ×D s

[0149] Then the ratio of the parameter values ​​required for the depthwise separable convolutional layer (DSC) operation to the standard convolutional layer (Conv) operation is:

[0150]

[0151] The ratio of the amount of computation required for the depthwise separable convolutional layer (DSC) operation to the amount required for the standard convolutional layer (Conv) operation is:

[0152]

[0153] It can be concluded that the depthwise separable convolutional layer uses fewer parameters and reduces the number of input channels, thereby effectively reducing the parameters required for the convolutional layer. At the same time, it has a faster running speed, a streamlined computing model, and can achieve high-precision operations on smaller devices.

[0154] In this embodiment, a deep separable convolutional layer is used in the channel adaptive weight adjustment module to optimize the feature separation between channels, thereby improving the quality of the output feature map and effectively solving the problem of insufficient extraction of feature information of diseased fish images due to low resolution of diseased fish images and uneven light distribution.

[0155] In this embodiment, the channel space contrast enhancement module includes a standard convolution layer, a normalization layer and a depthwise convolution layer (Depthwise Convolution, DC).

[0156] Specifically, the visual feature map is calculated through a standard convolution layer, a normalization layer, and a deep convolution layer to obtain a second shallow feature map, which is expressed as:

[0157]

[0158] Among them, F2 represents the second shallow layer, represents a 3×3 standard convolutional layer after processing by a deep convolutional layer;

[0159] The second shallow feature map is processed by the nonlinear activation function to obtain a deep convolution feature map, which is expressed as:

[0160]

[0161] Among them, F CSCE Represents a deep convolutional feature map.

[0162] In this embodiment, the contrast of the visual feature map is enhanced through a deep convolution layer in the channel space contrast enhancement module, and the ability of the diseased fish detection model to capture and process small information is enhanced, thereby effectively improving the performance of the diseased fish detection model in the water surface environment.

[0163] Therefore, in this embodiment, by constructing a contrast enhancement transfer attention structure, the regional contrast enhancement module, the channel adaptive weight adjustment module and the channel space contrast enhancement module in the contrast enhancement transfer attention structure are connected, and the different feature information in the visual feature map is extracted respectively through the regional contrast enhancement module, the channel adaptive weight adjustment module and the channel space contrast enhancement module to obtain a fused feature map, thereby effectively improving the contrast of important areas in the fused feature map, while weakening the influence of other areas, and significantly enhancing the local contrast.

[0164] Furthermore, in the neck network, the deep feature map F PCE , depth-wise separable convolutional feature map F CAWA And the deep convolutional feature map F CSCE By stacking and adding, we can get the fused feature map of contrast and prominence improvement output by the contrast enhancement transfer attention structure module. The fused feature map is expressed as:

[0165]

[0166] Among them, F' represents the fused feature map, Stack represents the stacking of the deep feature map, the depthwise separable convolutional feature map and the depthwise convolutional feature map according to the color channel dimension, and ⊕ represents the addition of the deep feature map, the depthwise separable convolutional feature map and the depthwise convolutional feature map along the stacked color channel dimension.

[0167] In this embodiment, a multi-head prediction small target detection algorithm for scale information extraction is constructed and added to the detection head network. It should be noted that the multi-head prediction small target detection algorithm for scale information extraction includes the removal optimization of the large target detection decoupling head in the native network structure of the diseased fish detection model and the addition of the small target detection decoupling head. Specifically, in this embodiment, Figure 6 As shown, the diseased fish detection model uses the YOLOv8 network as the native network structure, and 5 feature output layers P1, P2, P3, P4, and P5 of different scales will be output in the backbone network, among which the output corresponding to the P2 layer uses a small target detection decoupling head, which adds two layers of small target information extraction modules in the neck network. In this embodiment, the small target information extraction module includes an upsampling module, a connection module, and a CSPLayer structure. At this time, the detection target map size is 160×160×128, and then the P2 layer and the P3 layer small target information extraction modules are connected and fused again, thereby making the small target detection decoupling head can be used to detect small targets with a size of 4×4 or more, and at the same time delete the original 20×20 large target detection head in the neck network. Specifically, in this embodiment, by adding a feature output layer P2 in the backbone network, the small target information in the fused feature map is prevented from being lost in the sampling and pooling steps of feature extraction in the backbone network. In addition, two feature fusion layers are added to the neck network to fuse the feature maps transmitted by the P2 layer and the P3 layer. Finally, the fused feature maps in the neck network are respectively detected by the small target detection decoupling head and other detection heads of the original detection head layer. The detection steps include convolution layer convolution processing, feature mapping, anchor box prediction of the bounding box, calculation of confidence and category probability detection, etc., thereby obtaining the target detection result. It should be noted that in this embodiment, convolution layer convolution processing, feature mapping, anchor box prediction of the bounding box, calculation of confidence and category probability detection, etc. are existing technologies.

[0168] In this embodiment, in the detection head network, the fusion feature map is detected by a multi-head prediction small target detection algorithm to obtain the diseased fish target detection result, which includes two branches: one branch is used to predict the bounding box, and the other branch is used to predict the category. Each branch reduces the number of channels through the convolution layer to obtain the final output prediction result.

[0169] Therefore, in this embodiment, constructing and adding a multi-head prediction small target detection algorithm in the detection head network can effectively avoid the problem of missed detection and poor detection effect of small targets by over-large-scale detection heads. At the same time, the bottom-up part of the feature pyramid is changed to fusion from the middle layer to the two side layers, that is, the information of the P2 layer is merged with the information of the P3 layer to retain more small target feature information, thereby significantly improving the detection accuracy of diseased fish.

[0170] This embodiment also experimentally verifies the detection accuracy of the above-mentioned diseased fish detection method, and the specific contents are as follows:

[0171] In this embodiment, the RGB_D camera used is a ZED 2i camera. The ZED 2i camera is used to capture fish images in aquaculture water environments. The pixel size of the fish images is 2560 pixels × 1440 pixels. Diseased fish images are manually identified from the fish images. The diseased fish images are used as the Diseased Fish dataset, which are annotated and divided into a dataset, a validation set, and a test set, and then input into a diseased fish model based on the improved YOLOv8 network.

[0172] The experimental parameters are uniformly set to use SGD optimizer, use YOLOv8n.pt training weight parameter file, set epochs to train 150 rounds, and set the initial learning rate to 0.01. The experiment uses precision, recall, average precision (AP) and mean average precision (MAP) as evaluation indicators.

[0173] In this embodiment, the ablation experiment method is used to verify the effects of the spatial deep convolutional neural network, contrast enhancement transfer attention structure and multi-head prediction small target detection algorithm proposed in this embodiment on the detection performance of the diseased fish model.

[0174] In this experiment, spatial deep convolutional neural network, contrast enhancement transfer attention structure and multi-head prediction small target detection algorithm were added to the diseased fish detection model based on YOLOv8 network on the Diseased Fish dataset, and then compared with the original diseased fish detection model based on YOLOv8 network. Figure 7-Figure 18 As shown in the figure, the detection performance comparison results of the original YOLOv8 network diseased fish detection model, the YOLOv8 network diseased fish detection model with added spatial deep convolutional neural network (spdconv_YOLOv8), the YOLOv8 network diseased fish detection model with added contrast enhancement transfer attention structure (CBAM_YOLOv8), and the YOLOv8 network diseased fish detection model with added multi-head prediction small target detection algorithm (detect_head_YOLOv8).

[0175] Depend on Figure 7-Figure 18It can be concluded that the diseased fish detection model of the YOLOv8 network with spatial deep convolutional neural network (spdconv_YOLOv8), the diseased fish detection model of the YOLOv8 network with contrast enhancement transfer attention structure (CBAM_YOLOv8) and the diseased fish detection model of the YOLOv8 network with multi-head prediction small target detection algorithm (detect_head_YOLOv8) respectively improved the MAP accuracy values ​​by 2.6%, 3.7% and 0.8% compared with the original YOLOv8 network diseased fish detection model, and improved the F1 accuracy values ​​by 3.0%, 6.0% and 3.0%. In addition, the diseased fish detection model of the YOLOv8 network with spatial deep convolutional neural network (spdconv_YOLOv8) and the diseased fish detection model of the YOLOv8 network with contrast enhancement transfer attention structure (CBAM_YOLOv8) both improved the recall rate by 2.0% compared with the original YOLOv8 network diseased fish detection model.

[0176] It can be seen that the diseased fish detection model constructed in this embodiment effectively improves the accuracy of target detection and recognition, can efficiently and accurately complete the detection needs of diseased fish under complex lighting and water conditions, and has strong practicality and wide applicability.

[0177] This embodiment also provides a diseased fish detection device, including an acquisition module, a model building module and a detection module. The acquisition module is used to collect fish images, identify diseased fish images in fish images, and pre-process the diseased fish images. The model building module is used to build a diseased fish detection model, which includes a backbone network, a neck network and a detection head network, wherein the backbone network includes a spatial deep convolutional neural network, and the neck network includes a contrast enhancement transfer attention structure. The detection module inputs the diseased fish image into the backbone network based on the spatial deep convolutional neural network to extract features to obtain a visual feature map, inputs the visual feature map into the contrast enhancement transfer attention structure of the neck network to perform feature fusion processing to obtain a fused feature map, and inputs the fused feature map into the detection head network to obtain the diseased fish target detection result through the multi-head prediction small target detection algorithm. The diseased fish detection device provided in this embodiment can implement any method of the diseased fish detection method, and the specific working process of a diseased fish detection device can refer to the corresponding process in the embodiment of the diseased fish detection method. The method and device provided in this embodiment can be implemented in other ways. For example, the device embodiments described above are only illustrative; for example, the division of a module is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual connection or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0178] This embodiment also provides a computer device. A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the diseased fish detection method when executing the computer program.

[0179] This embodiment also provides a computer-readable storage medium. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a diseased fish detection method described in this embodiment is executed. The computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0180] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited thereto, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all belong to the protection scope of the present invention. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate the name, and do not indicate any specific order.

Claims

1. A method for detecting diseased fish, comprising the following steps: Collect fish images, identify diseased fish images in the fish images, and pre-process the diseased fish images; Constructing a diseased fish detection model, the diseased fish detection model comprising a backbone network, a neck network and a detection head network; the backbone network comprises a spatial deep convolutional neural network, and the neck network comprises a contrast enhancement transfer attention structure; The image of diseased fish is input into the backbone network based on the spatial deep convolutional neural network for feature extraction to obtain a visual feature map; the visual feature map is input into the contrast enhancement transfer attention structure of the neck network for feature fusion processing to obtain a fused feature map; the fused feature map is input into the detection head network and the diseased fish target detection result is obtained through the multi-head prediction small target detection algorithm.

2. A diseased fish detection method according to claim 1, characterized in that: The steps of preprocessing the diseased fish images include: De-noising and normalizing the diseased fish images; Enhance the feature information of diseased fish images through automated prominence and contrast enhancement algorithms; The automatic prominence and contrast enhancement algorithm includes an image pixel value preprocessing module and an automatic prominence and contrast enhancement calculation module; The sick fish image is input into the image pixel value preprocessing module to obtain the image position information of the sick fish image, wherein the image position information includes the reflected light intensity value and the ambient light illumination component value, and the image pixel point processing value is calculated by the reflected light intensity value and the ambient light illumination component value; the image pixel point processing value is input into the automatic protrusion and contrast enhancement calculation module to calculate the pixel information of the sick fish image, and the characteristic information of the sick fish image is enhanced by the pixel information of the sick fish image.

3. A diseased fish detection method according to claim 2, characterized in that: The pixel processing value of the image is input into the automatic protrusion and contrast enhancement calculation module to calculate the pixel information of the diseased fish image, and the calculation formula is: R‘ auto-IMES (x,y)=log 10 S(x,y)-log 10 [S(x,y)*K(x,y)] Among them, R auto-IMES (x, y) represents the pixel value of the diseased fish image contrast pre-enhancement on a single channel, x represents the horizontal coordinate of the pixel, y represents the vertical coordinate of the pixel, S(x, y) represents the image pixel processing value of the diseased fish image at the coordinate (x, y), K(x, y) represents the Cauchy illumination filter at the nth scale at the coordinate (x, y), * represents the product operation, R auto-IMES (x, y) represents the pixel value of the overall contrast enhancement of the diseased fish image, n represents a natural number, N represents the number of channels, C(x, y) represents the restoration coefficient of the pixel at the coordinate (x, y) on the diseased fish image, G n Represents the gain factor of the nth scale.

4. A diseased fish detection method according to claim 3, characterized in that: The spatial deep convolutional neural network includes a spatial deep convolutional module and a non-strided convolutional module; The deep spatial convolution module crops the diseased fish image into a plurality of sub-feature graphs, and connects the spatial dimensions of the plurality of sub-feature graphs through the channel dimension to obtain a feature graph with a reduced spatial dimension and an increased channel dimension; The filter in the non-stride convolution module performs convolution processing on the feature map to obtain a visual feature map.

5. A diseased fish detection method according to claim 4, characterized in that: The contrast enhancement transfer attention structure includes a regional contrast enhancement module, a channel adaptive weight adjustment module and a channel space contrast enhancement module; The regional contrast enhancement module includes a standard convolution layer and a normalization layer; The visual feature map is calculated through a standard convolution layer and a normalization layer to obtain a first shallow feature map; the first shallow feature map is expressed as: Among them, F1 represents the first shallow feature map, F represents the visual feature map, BN represents the normalization layer, and Conv represents the convolutional layer. Represents a 3×3 standard convolutional layer; The first shallow feature map is processed by nonlinear activation function to obtain a deep feature map; the deep feature map is expressed as: Among them, F PCE represents the deep feature map, Sigmoid represents the Sigmoid activation function, LeakyRELU represents the LeakyReLU activation function, Represents a convolution operation.

6. A diseased fish detection method according to claim 5, characterized in that: The channel adaptive weight adjustment module includes a standard convolution layer, a normalization layer and a depth-separable convolution layer; The visual feature map is calculated through a standard convolution layer and a normalization layer to obtain a first shallow feature map; the first shallow feature map is expressed as: The first shallow feature map is processed by nonlinear activation function and depth-separable convolution layer to obtain a depth-separable convolution feature map; the depth-separable convolution feature map is expressed as: Among them, F CAWA represents the depth-wise separable convolutional feature map, Represents a 3×3 standard convolutional layer after being processed by a depthwise separable convolutional layer.

7. A diseased fish detection method according to claim 6, characterized in that: The channel space contrast enhancement module includes a standard convolution layer, a normalization layer and a depth convolution layer; The visual feature map is calculated through a standard convolution layer, a normalization layer and a deep convolution layer to obtain a second shallow feature map; the second shallow feature map is expressed as: Among them, F2 represents the second shallow feature map, represents a 3×3 standard convolutional layer after being processed by a deep convolutional layer; The second shallow feature map is processed by nonlinear activation function to obtain a deep convolution feature map; the deep convolution feature map is expressed as: Among them, F CSCE Represents a deep convolutional feature map.

8. A diseased fish detection method according to claim 7, characterized in that: The deep feature map, the depth-separable convolutional feature map and the depth-convolutional feature map are stacked and added to obtain a fused feature map; the fused feature map is expressed as: Among them, F' represents the fused feature map, Stack represents the stacking of the deep feature map, the depthwise separable convolutional feature map and the depthwise convolutional feature map according to the color channel dimension, and ⊕ represents the addition of the deep feature map, the depthwise separable convolutional feature map and the depthwise convolutional feature map along the stacked color channel dimension.

9. A diseased fish detection method according to claim 8, characterized in that: The multi-head prediction small target detection algorithm includes removing the large target detection decoupling head in the detection head network and adding a small target detection decoupling head; the fused feature map is detected by the small target detection decoupling head to obtain the diseased fish target detection result.

10. A diseased fish detection device, characterized in that: include: An acquisition module, which acquires fish images, identifies diseased fish images in the fish images, and pre-processes the diseased fish images; A model building module is used to build a diseased fish detection model, wherein the diseased fish detection model includes a backbone network, a neck network, and a detection head network; the backbone network includes a spatial deep convolutional neural network, and the neck network includes a contrast enhancement transfer attention structure; The detection module inputs the diseased fish image into the backbone network based on the spatial deep convolutional neural network for feature extraction to obtain a visual feature map; the visual feature map is input into the contrast enhancement transfer attention structure of the neck network for feature fusion processing to obtain a fused feature map; the fused feature map is input into the detection head network and the diseased fish target detection result is obtained through the multi-head prediction small target detection algorithm.

Images