A high-precision fish posture measurement method, device and storage medium

Abstract

The present invention provides a high-precision fish posture measurement method, device and storage medium. The measurement method is implemented based on a constructed fish posture measurement model, and the model includes a fish image enhancement module, a fish target detection module and a fish posture measurement module; the fish image enhancement module is designed based on a generative adversarial network architecture, including a generator network and a discriminator network, wherein the generator network introduces an auxiliary guidance module for extracting multi-scale feature details of the target domain image to guide the generation process of the generator network; the fish target detection module is established based on the YOLOV8 model, for obtaining the position information of the fish, and outputting the detection image after the rectangular frame is annotated; the fish posture measurement module performs actual length measurement based on the detection image after the rectangular frame is annotated. The present invention cleverly designs the model structure and the training loss function, realizes the enhancement of fish images, target detection and posture measurement, and greatly improves the efficiency and reliability of fish posture measurement.

Description

Technical Field

[0001] The present invention belongs to the technical field of fish posture measurement based on computer vision, and in particular relates to a high-precision fish posture measurement method, equipment and storage medium. Background Art

[0002] The length and height of fish are not only the key to assessing the health of fish populations and the balance of ecosystems, studying fish growth and environmental adaptability, but also important indicators for species identification and classification, weight estimation, quality selection and grading. At the same time, studies have shown that there is a significant correlation between fish weight and body length traits, but due to the uniqueness of each species' phenotype, there are great differences in body size and weight models. Traditional methods for estimating fish length are mainly manual measurement and sonar measurement. Manual measurement is a contact measurement method with the following limitations: 1) This process is usually time-consuming and labor-intensive and contains many subjective factors, and the reliability and consistency of the results are difficult to guarantee; 2) Fish are very sensitive to changes in the external environment, and may produce stress reactions if they leave the water environment for a short time, causing damage to the fish or even death. Sonar measurement is a contact measurement method that uses imaging sonar to measure fish length, with smaller errors than manual measurement. It has the following limitations: 1) Its resolution and measurement range are limited, and it is difficult to distinguish all types and sizes of fish.

[0003] With the continuous development of artificial intelligence technology, computer vision technology has been widely used in the field of fish body measurement, mainly including traditional image processing methods and machine learning-based methods.

[0004] Traditional image processing methods: For example, using methods such as minimum bounding rectangle and Hough transform to extract the fish body contour. However, these methods can usually only process two-dimensional images and cannot accurately capture the three-dimensional information of the fish body. In addition, image quality is affected by many factors, such as lighting conditions, water quality, and fish posture, which may affect the measurement accuracy.

[0005] Machine learning-based methods: Based on deep learning technology, deep learning models such as convolutional neural networks (CNN) are used to achieve fish body measurement. This type of method can automatically extract features from images and perform high-precision body measurements through training with a large amount of data. However, the existing CNN architecture often suffers from insufficient feature extraction and overfitting problems when processing high-resolution fish images. In addition, the quality of fish images is affected by environmental conditions such as light intensity and water quality during collection, and there is often noise. It may be difficult for the model to retain image details during training, especially to retain important features of the image while removing noise. Summary of the invention

[0006] In view of the above problems, the first aspect of the present invention proposes a method for constructing a high-precision fish body measurement model, comprising the following steps:

[0007] S1, creating a fish image dataset, including the collection and annotation of fish images;

[0008] S2, constructing a fish image paired dataset, including fish image data containing noise and fish image data after denoising and enhancement;

[0009] S3, constructing a fish posture measurement model, which includes a fish image enhancement module, a fish target detection module and a fish posture measurement module;

[0010] The fish image enhancement module is designed based on the generative adversarial network (GAN) architecture, including a generator network and a discriminator network, and is used to enhance the acquired fish images; wherein the generator network introduces an auxiliary guidance module, which is used to extract multi-scale feature details of the target domain image to guide the generation process of the generator network;

[0011] The fish target detection module is established based on the YOLOV8 model, which is used to obtain the position information of each fish and output the detection image marked with a rectangular frame;

[0012] The fish posture measurement module is used to calculate the actual size of the target rectangular frame in the output image based on the detection image after the rectangular frame is marked, the actual width of the target rectangular frame is the body length of the fish, and the actual height is the body height of the fish;

[0013] S4, through the designed fish image enhancement module training loss function and fish target detection module training loss function, and based on the constructed paired image dataset, the model is trained to obtain the final fish body measurement model.

[0014] Preferably, the specific process of preparing the fish image dataset in S1 includes:

[0015] The captured target fish is anesthetized and placed on a flat surface, with a checkerboard calibration board placed next to it to convert pixel measurement values ​​into physical lengths, and horizontal shooting is performed to obtain images;

[0016] The body length and height of each fish were measured and recorded with a vernier caliper, and the body length and height of the fish were manually annotated using Labelimg software. The width of the target frame was from the tip of the fish's snout to the base of the fish's tail fin, and the height of the target frame was from the starting point of the dorsal fin to the ventral surface, representing the body length BL and height BH of the fish, respectively.

[0017] Preferably, the specific structures of the generator network and the discriminator network in the fish image enhancement module are:

[0018] The generator network adopts an encoder stacking structure and a decoder stacking structure. The encoder stacking structure consists of a convolutional layer, a ReLU activation function, and a downsampling pooling layer. i After step-by-step feature extraction and downsampling, the encoder stack structure can be expressed as:

[0019] H i =f enc (S i ; W enc )

[0020] Among them, H i is the output of the encoder stack structure and also the input image S i Representation in latent feature space; f enc (·) indicates the encoder stacking structure; W enc Trainable weight parameters for the encoder stack structure;

[0021] The encoder stack structure contains 3 layers of convolution operations and downsampling operations. The calculation process of the jth layer is:

[0022]

[0023] in, It is the feature map output by the jth layer; is the output of the previous layer; Conv(·) is the convolution operation; W enc j is the convolution kernel weight of the jth layer; ReLU is the linear activation function; DownPool(·) is the downsampling pooling operation;

[0024] After 3 layers of convolution and downsampling, the final encoder stack structure output is a vector H of length 256 i , as the input image S i The encoding feature representation of ;

[0025] The decoder stack structure is used to convert the latent feature vector H output by the encoder stack structure i Reconstruct the original input data; the decoder stacking structure is the reverse symmetric structure of the encoder stacking structure:

[0026]

[0027] in, is the output of the decoder, i.e. the reconstructed image; f dec (·) is the decoder stacking network; W dec Trainable weight parameters for the decoder stack network;

[0028] The decoder stack network contains 3 layers of upsampling operations and deconvolution operations. The calculation process of the jth layer is:

[0029]

[0030] in, is the output feature map of the jth layer; is the output of the previous layer; ConvT(·) is the deconvolution operation; Up(·) is the upsampling pooling operation;

[0031] Through 3 layers of upsampling and deconvolution operations, the decoder stacking structure encodes the feature H i Gradually reconstructed into the same i Images of the same resolution

[0032] The discriminator network is used to judge the quality of the fish image output by the generator network during the training process; the fish image G i The image is divided into 2*2 small blocks, and then the four blocks are input into the discriminator network respectively. Finally, all the results of the discriminator are averaged as the comprehensive discrimination result. The calculation formula is as follows:

[0033]

[0034] in, is the τth region block of the jth image to be discriminated, D is the discriminator’s discriminant result, and Dis(·) is the discriminator’s processing function.

[0035] Preferably, the auxiliary guidance module includes a multi-scale pooling layer, a channel convolution layer, a multi-scale convolution layer, a channel attention layer, a multi-head attention layer and an AdaLN layer; specifically includes:

[0036] The i-th target domain image T in the paired image dataset i Enter the multi-scale pooling layer, the calculation formula is as follows:

[0037]

[0038] in, and They represent the output after average pooling and maximum pooling when the pooling kernel is k, respectively. k is preset to {1, 3, 5}. By using boundary padding to ensure that the output length is consistent after processing with different pooling kernels, the output of the multi-scale pooling layer is finally obtained.

[0039] The output of the multi-scale pooling layer Input channel convolution layer, the specific calculation formula is as follows:

[0040]

[0041] in Represents the output of the channel convolution layer, whose number of channels is consistent with the input dimension of this layer;

[0042] Will Input the multi-scale convolution layer to obtain multi-scale features. The specific calculation formula is as follows:

[0043]

[0044] in represents the output when the convolution kernel size is k, δ represents the Sigmoid activation function, and finally the output of the multi-scale convolution layer is obtained

[0045] Will The three multi-scale features in are input into the channel attention layer respectively to capture the local structural information in the features of different scales, enhance the attention to the fish features, and suppress the unimportant environmental features. The calculation formula is as follows:

[0046]

[0047] Among them, P i，k for The corresponding channel attention weight, MLP (·) is a multi-layer perceptron; P i，k and Perform product operation to obtain channel attention features

[0048] In order to capture the relationship between multi-scale features, and Input the multi-head attention mechanism layer to obtain the weight, and the calculation formula is as follows:

[0049]

[0050] Among them, Q, K, and V are query sequence, key sequence, and value sequence, respectively. m is the dimension of the attention head; the calculation result of the lth attention head is expressed as:

[0051]

[0052] in, are the linear transformation weight matrices of query, key, and value respectively, and connect h attention heads to obtain a multi-head attention mechanism:

[0053] F i =Concat(head1, head2,..., headh )W o

[0054] Among them, W o The linear transformation weight matrix output by the multi-head attention mechanism;

[0055] Using AdaLN layer to normalize F i Normalize the information features and map them to the auxiliary guidance features of the jth layer through the convolution layer

[0056]

[0057] Among them, AdaLN(·) is the layer normalization function, and the auxiliary guide feature The dimension of the decoder layer is the same as the feature restored by have the same dimensions;

[0058] The auxiliary guidance features to be obtained The input of each layer of the decoder stacking structure guides the image enhancement effect in details.

[0059] Preferably, the fish target detection module is established based on the YOLOv8 model, and a dynamic feature fusion module DFF is introduced into the neck network of YOLOv8 to replace the original feature splicing layer. The module can dynamically adjust the feature fusion mode and weight according to different areas and levels of the input fish feature map, and includes a feature alignment layer, an adaptive attention layer and an adaptive fusion layer; the feature alignment layer maps the features to be fused to the same space and then inputs them into the adaptive attention layer, the adaptive attention layer calculates the weight of each feature and then obtains the weighted feature of each feature, and the weighted feature of each feature is input into the adaptive fusion layer for feature splicing;

[0060] An adaptive context enhancement module ACE is introduced into the backbone network. This module aims to improve the detection performance of YOLOV8 by adaptively enhancing the context information of the fish feature map. It includes a dilated convolution layer and a pyramid pooling layer. The dilated convolution layer can effectively expand the receptive field of the convolution layer without increasing the number of parameters and computational complexity, and effectively capture the global context information of the backbone network features. The pyramid pooling layer can extract multi-scale features from the backbone network features by performing pooling operations on sub-regions of different scales in the feature map.

[0061] Preferably, the fish posture measurement module has the following specific processing steps:

[0062] Calculate the actual width of the output image of the fish target detection module; first convert the image into a grayscale image; then use OpenCV to calculate the position of the corner points of the checkerboard calibration plate in the image and the average pixel distance between the corner points; finally, calculate the actual width of the image based on the position of the corner points of the checkerboard calibration plate and the average pixel distance between the corner points;

[0063] Calculate the actual size of the target frame in the output image; calculate the actual size of the target frame using the actual width and image aspect ratio of the image obtained above and the coordinate information of the target frame, the actual width of the target frame is the body length of the fish, and the actual height is the body height of the fish.

[0064] Preferably, the fish image enhancement module training loss function is improved based on the basic loss function of GAN, and high-frequency information loss is introduced on the basic loss function, as follows:

[0065]

[0066] in, and Respectively represent the expectation of the target domain and the source domain, log(·) is the logarithmic function, Haar(·) is the Haar wavelet operator, ||·|| F To calculate the F-norm operator, w1 and w2 are weight factors;

[0067] The training loss function of the fish target detection module is improved based on the basic loss function of YOLOV8. The mean square error loss is replaced by the intersection over union (IoU) loss. The IoU loss directly measures the overlap between the predicted box and the real box. The IoU information is introduced into the confidence loss to obtain the enhanced confidence loss, as follows:

[0068]

[0069] Among them, Intersection is the overlapping area of ​​the predicted box and the real box, and Union is the union area of ​​the predicted box and the real box. i is the true label, whether the target exists or not; is the probability predicted by the model; IoU i is the IoU value between the i-th predicted box and the true box; L cel is the cross entropy classification loss of the basic YOLOV8; γ1, γ2 and γ3 are the weight coefficients of intersection-over-union loss, cross entropy classification loss and enhanced confidence loss, respectively, which are used to balance the importance of different loss items.

[0070] The second aspect of the present invention provides a high-precision fish posture measurement method, which deploys the fish posture measurement model constructed by the construction method described in the first aspect to a back-end server, and includes the following processes:

[0071] S1, receiving the collected fish images in real time;

[0072] S2, the fish image is input into the generator network of the fish image enhancement module to obtain the enhanced fish image;

[0073] S3, the enhanced fish image is input into the fish image detection module to obtain the position information of each fish in the fish image;

[0074] S4, inputting the position information of each fish and the fish image into the fish posture measurement module to obtain the height and length of each fish in the image.

[0075] The third aspect of the present invention provides a high-precision fish posture measurement device, which includes at least one processor and at least one memory, and the processor and the memory are coupled; the memory stores a computer execution program of the fish posture measurement model constructed by the construction method described in the first aspect; when the processor executes the computer execution program stored in the memory, the processor executes a high-precision fish posture measurement method.

[0076] The fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer execution program of the fish body measurement model constructed by the construction method described in the first aspect. When the computer execution program is executed by a processor, the processor executes a high-precision fish body measurement method.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] (1) Significantly improve the quality of fish images: The fish image enhancement module constructed by the present invention has significant effects in removing noise and enhancing contrast. The enhanced fish images are clearer and have richer details.

[0079] (2) Improving the accuracy of fish target detection: The fish image enhancement module constructed by the present invention is based on the well-known target detection model YOVOV8. The dynamic feature fusion module (DFF) and the adaptive context enhancement module (ACE) are introduced into the neck network and the backbone network of YOVOV8 respectively, which significantly improves the prediction accuracy of the model without increasing the complexity of the model too much.

[0080] (3) Enhance the robustness and detail retention capability of the model: The loss function designed for the fish image enhancement module of the present invention optimizes the performance of the model in a complex noise environment and retains the key details of the fish image; the loss function designed for the fish target detection module of the present invention accelerates the convergence speed of the model during training, and the model is more robust;

[0081] (4) Real-time measurement of fish posture: By deploying the trained fish image enhancement module, the trained fish target detection module and the fish posture measurement module to the background server, fish image enhancement, target detection and posture measurement are realized, which greatly improves the efficiency and reliability of fish posture measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, what is described below is only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0083] Figure 1 A flow chart of the high-precision fish posture measurement method and model building method of the present invention;

[0084] Figure 2 A network structure diagram of a fish image enhancement module of the present invention;

[0085] Figure 3 This is a network structure diagram of the fish target detection module of the present invention;

[0086] Figure 4 Flowchart for fish body measurement model deployment and real-time detection;

[0087] Figure 5 This is a schematic diagram of the output result of the fish target detection module in the embodiment;

[0088] Figure 6 This is a structural diagram of the fish posture measurement device of the present invention. DETAILED DESCRIPTION

[0089] The present invention proposes a high-precision fish posture measurement method. First, a fish image dataset is constructed. Secondly, a paired dataset of fish images containing noise and fish images after denoising and enhancement is constructed to provide basic data support for the training of subsequent fish image enhancement models. Then, a fish image enhancement module is designed based on a generative adversarial network (GAN) architecture, and a generator is designed based on convolutional layers, deconvolutional layers and pooling layers to perform fish image enhancement. At the same time, a discriminator network is designed to guide the learning of the generator. On this basis, the present invention further introduces an attention mechanism to extract the deep features of the enhanced image to guide the generator to perform more detailed enhancement processing. Subsequently, a fish image enhancement model is established based on the YOLOV8 model structure. The target detection module is used to improve the neck network and the backbone network of the traditional YOLOV8 model structure to improve the prediction accuracy of the model. Then, the loss functions of the fish image enhancement module and the fish target detection module are designed to improve the convergence speed of the model during training. Then, a fish posture measurement module is designed. The output of the fish target detection module is input into this module to obtain the posture data of individual fish. Finally, the trained image enhancement module, the trained fish target detection module and the fish posture measurement module are deployed to the background server as the overall fish posture measurement model, and the collected fish images are input into the background to realize the posture measurement of all individual fish in the fish images. The overall process is as follows: Figure 1 shown.

[0090] 1. Data Collection and Data Preprocessing

[0091] In this embodiment, the captured fish are anesthetized, placed on a flat surface, classified, and neatly arranged horizontally, and a checkerboard calibration board is placed next to them, and a smartphone is used to take horizontal photos; the calibration board has a specification of 12*9-1cm and is used to convert pixel measurement values ​​into physical lengths;

[0092] The body length and height of each fish were measured and recorded using a vernier caliper; the images captured included images of single and multiple fish. The body length and height of the fish were manually annotated using Labelimg software, with the target frame width starting from the tip of the fish's snout to the base of the fish's caudal fin, and the target frame height starting from the dorsal fin's starting point to the ventral surface, representing the body length (BL) and height (BH) of the fish, respectively;

[0093] The above process is repeated to obtain a fish image dataset containing single-tail fish images and multi-tail fish images.

[0094] Filter the images with high contrast, moderate brightness and as little noise as possible in the fish image dataset, scale their resolution to 800*800 and use them as target domain image data. All target domain image data constitute the target domain image dataset;

[0095] The i-th fish image T in the target domain image dataset is obtained by screening and scaling i, i∈[1,N], add random noise, increase brightness and reduce brightness to obtain the source domain image S i , S i With T i Pairing is performed to obtain paired image data {S i , T i}; Perform the above steps for each image in the target domain image dataset to obtain N groups of paired image data.

[0096] 2. Design of fish image enhancement module

[0097] The present invention designs a fish image enhancement module based on the generative adversarial network (GAN) architecture. Figure 2 As shown in Figure 2, it includes a generator network, an auxiliary guidance module, and a discriminator network. The generator network pairs all source domain data S = {S1, ..., S N The image to be enhanced is taken as input to obtain the enhanced image, and the discriminator is used to guide the training of the generator during the training process, and to assist in guiding the enhancement module to further improve the generation quality of the generator. Finally, the trained generator is used for fish image enhancement.

[0098] Since the input and output of the generator network are the image data of the same size before and after enhancement, the generator network designed by the present invention adopts an encoder stacking structure and a decoder stacking structure. Specifically, the encoder stacking structure consists of a convolutional layer, a ReLU activation function, and a downsampling pooling layer. i After step-by-step feature extraction and downsampling, the encoder stack structure can be expressed as:

[0099] H i =f enc (S i ; W enc )

[0100] Among them, H i is the output of the encoder stack structure and also the input image S i Representation in latent feature space; f enc (·) indicates the encoder stacking structure; W enc The trainable weight parameters of the encoder stack structure.

[0101] Specifically, the encoder stacking structure includes 3 layers of convolution operations and downsampling operations. The calculation process of the jth layer is:

[0102]

[0103] in, It is the feature map output by the jth layer; is the output of the previous layer; Conv(·) is the convolution operation; W enc j is the convolution kernel weight of the jth layer; ReLU is the linear activation function; DownPool(·) is the downsampling pooling operation;

[0104] After 3 layers of convolution and downsampling, the final encoder stack structure output is a vector H of length 256 i , as the input image S i The encoding feature representation of ;

[0105] The decoder stacking structure is used to convert the latent feature vector H output by the encoder stacking structure i Reconstruct back to the original input data. The decoder stack structure is usually the "reverse symmetric" structure of the encoder stack structure:

[0106]

[0107] in, is the output of the decoder, i.e. the reconstructed image; f dec (·) is the decoder stacking network; W dec Trainable weight parameters for the decoder stack network.

[0108] The decoder stack network contains 3 layers of upsampling operations and deconvolution operations. The calculation process of the jth layer is:

[0109]

[0110] in, is the output feature map of the jth layer; is the output of the previous layer; ConvT(·) is the deconvolution operation; Up(·) is the upsampling pooling operation;

[0111] Through 3 layers of upsampling and deconvolution operations, the decoder stacking structure encodes the feature H i Gradually reconstructed into the same i Images of the same resolution

[0112] In order to increase the detailed features of the image generated by the generator network, the present invention designs an auxiliary guidance module, including a multi-scale pooling layer, a channel convolution layer, a multi-scale convolution layer, a channel attention layer, a multi-head attention layer and an AdaLN layer; the auxiliary guidance module extracts the multi-scale feature details of the target domain image to guide the generation process of the generator, specifically including:

[0113] 1) Take the i-th target domain image T in the image dataset i Enter the multi-scale pooling layer, the calculation formula is as follows:

[0114]

[0115] in, and They represent the output after average pooling and maximum pooling when the pooling kernel is k. k is preset to {1, 3, 5}. The output length is consistent after processing with different pooling kernels by using boundary padding. Finally, the output of the multi-scale pooling layer is obtained. Compared with the single-scale pooling layer, the multi-scale pooling layer can better improve the performance of the neural network. The single-scale pooling layer usually uses a fixed pooling window size and stride to reduce the image size, which will lead to information loss and image blur. Especially in fish images, the single-scale pooling layer cannot fully capture the characteristics of fish of different scales in the fish image, resulting in the loss of local details or global structural information of the fish; therefore, the multi-scale pooling layer is introduced to solve these problems;

[0116] 2) The output of the multi-scale pooling layer Input channel convolution layer, the specific calculation formula is as follows:

[0117]

[0118] in represents the output of the channel convolution layer, and its number of channels is consistent with the input dimension of this layer; in the channel convolution, each channel has an independent convolution kernel, and these convolution kernels are convolved with the corresponding channels of the input image respectively; this design enables the network to learn different feature representations on each channel, so as to better capture the diversity and complexity in fish images;

[0119] 3) Input the multi-scale convolution layer to obtain multi-scale features. The specific calculation formula is as follows:

[0120]

[0121] in represents the output when the convolution kernel size is k, δ represents the Sigmoid activation function, and finally the output of the multi-scale convolution layer is obtained

[0122] Multi-scale convolution can overcome the limitations of single-scale convolution and more comprehensively capture various features of fish in fish images. Multi-scale convolution can effectively extract features of fish structures of different scales, thereby enhancing the perception of fish structures.

[0123] 4) The three multi-scale features in are input into the channel attention layer respectively to capture the local structural information in the features of different scales, enhance the attention to the fish features, and suppress the unimportant environmental features. The calculation formula is as follows:

[0124]

[0125] Among them, P i，k for The corresponding channel attention weight, MLP (·) is a multi-layer perceptron; P i，k and Perform product operation to obtain channel attention features

[0126] 5) In order to capture the relationship between multi-scale features, and Input the multi-head attention mechanism layer to obtain the weight, and the calculation formula is as follows:

[0127]

[0128] Among them, Q, K, and V are query sequence, key sequence, and value sequence, respectively. m is the dimension of the attention head. The calculation result of the lth attention head is expressed as:

[0129]

[0130] in, are the linear transformation weight matrices of query, key, and value respectively. Connect h attention heads to get a multi-head attention mechanism:

[0131] F i =Concat(head1, head2,..., head h )W o

[0132] Among them, W o It is the linear transformation weight matrix output by the multi-head attention mechanism.

[0133] 6) Use AdaLN layer to normalize F i Normalize the information features and map them to the auxiliary guidance features of the jth layer through the convolution layer

[0134]

[0135] Among them, AdaLN(·) is the layer normalization function, and the auxiliary guide feature The dimension of the decoder layer is the same as the feature restored by have the same dimensions;

[0136] The auxiliary guidance features to be obtained The input of each layer of the decoder stacking structure guides the image enhancement effect in details.

[0137] Therefore, the decoder stacking structure outputs the feature at layer j as follows:

[0138]

[0139] The decoder stack structure finally passes through a convolutional layer to obtain the output of the overall generator:

[0140] G i =ReLU(Conv(FD3))

[0141] Among them, G i Represents the source domain image S i is the generated image result obtained by input, and FD3 is the output result of the third layer of the encoder stack structure; therefore, the overall network processing of the generator is expressed as:

[0142] G i =Gen(S i )

[0143] Among them, Gen(·) is the generator function;

[0144] The S4-4 discriminator network is used to discriminate the quality of the fish image output by the generator network during the training process. Specifically, the fish image G to be discriminated is i The image is divided into 2*2 small blocks, and then the four blocks are input into the discriminator network respectively. Finally, all the results of the discriminator are averaged as the comprehensive discrimination result. The calculation formula is as follows:

[0145]

[0146] in, is the τth region block of the jth image to be discriminated, D is the discriminator’s discriminant result, and Dis(·) is the discriminator’s processing function.

[0147] 3. Design of fish target detection module

[0148] Fish target module such as Figure 3As shown. A dynamic feature fusion module (DFF) is introduced into the neck network of YOLOv8 to replace the original feature splicing layer. The module can dynamically adjust the feature fusion mode and weight according to the different areas and levels of the input fish feature map, and automatically determine which features should be fused more and which features should be retained or suppressed. In order not to increase the model calculation complexity of YOLOV8 as much as possible, the present invention constructs a lightweight DFF module, which includes a feature alignment layer, an adaptive attention layer and an adaptive fusion layer. The feature alignment layer maps the features to be fused to the same space and then inputs them into the adaptive attention layer. The adaptive attention layer calculates the weight of each feature and then obtains the weighted feature of each feature. The weighted feature of each feature is input into the adaptive fusion layer for feature splicing.

[0149] An adaptive context enhancement module (ACE) is introduced into the backbone network, which aims to improve the detection performance of YOLOV8 by adaptively enhancing the context information of the fish feature map. The ACE module can identify the target boundaries and key feature points in the feature map, and enrich and enhance the context information for these areas; in order to minimize the increase in the model calculation complexity of YOLOV8, the present invention constructs a lightweight ACE module, including a hole convolution layer and a pyramid pooling layer; the hole convolution layer can effectively expand the receptive field of the convolution layer without increasing the number of parameters and the calculation complexity, and effectively capture the global context information of the backbone network features. The pyramid pooling layer can extract multi-scale features in the backbone network features by performing pooling operations on sub-regions of different scales in the feature map, thereby improving the generalization ability of the model.

[0150] 4. Design of loss function

[0151] Design the loss function of the fish image enhancement module; in order to improve the robustness of the fish image enhancement module, the basic loss function based on GAN is improved, and high-frequency information loss is introduced on the basic loss function, as follows:

[0152]

[0153] in, and Respectively represent the expectation of the target domain and the source domain, log(·) is the logarithmic function, Haar(·) is the Haar wavelet operator, ||·|| F To calculate the F-norm operator, w1 and w2 are weight factors;

[0154] The loss function of the fish target detection module is designed; in order to improve the convergence speed of the fish posture measurement module during training, the present invention is improved based on the basic loss function of YOLOV8, and the mean square error loss is replaced by the intersection over union (IoU) loss. The IoU loss directly measures the overlap between the predicted box and the true box, and is more suitable for bounding box regression in target detection tasks; the IoU information is introduced into the confidence loss to obtain the enhanced confidence loss, as follows:

[0155]

[0156] Among them, Intersection is the overlapping area of ​​the predicted box and the real box, and Union is the union area of ​​the predicted box and the real box. i is the true label (whether the target exists or not), is the probability predicted by the model, IoU i is the IoU value between the i-th predicted box and the true box, L cel is the cross entropy classification loss of the basic YOLOV8, γ1, γ2 and γ3 are the weight coefficients of intersection-over-union loss, cross entropy classification loss and enhanced confidence loss, which are used to balance the importance of different loss items.

[0157] 5. Design of fish posture measurement module

[0158] Calculate the actual width of the output image of the fish target detection module; first convert the image into a grayscale image; then use OpenCV to calculate the position of the corner points of the checkerboard calibration plate in the image and the average pixel distance between the corner points; finally, calculate the actual width of the image based on the position of the corner points of the checkerboard calibration plate and the average pixel distance between the corner points;

[0159] Calculate the actual size of the target frame in the output image; use the actual width and image aspect ratio of the image obtained in the above steps and the coordinate information of the target frame to calculate the actual size of the target frame, the actual width of the target frame is the body length of the fish, and the actual height is the body height of the fish.

[0160] 6. Model deployment and real-time detection

[0161] Fish real-time detection process Figure 4 As shown in the figure, the trained fish body measurement model is deployed to the backend server, and the following processes are included:

[0162] Receive the collected fish images in real time;

[0163] The fish image is input into the generator network to obtain the enhanced fish image;

[0164] The enhanced fish image is input into the fish image detection module to obtain the position information of each fish in the fish image;

[0165] The position information and fish image of each fish are input into the fish posture measurement module to obtain the body height and length of each fish in the image.

[0166] VII. Related Experiments

[0167] The model proposed in the present invention was used to predict the body length and height of 98 fish. According to the above verification method, the detection error results are shown in Table 1. The absolute error range of body length is 0.14-0.21cm, and the average absolute error is 0.17cm; the absolute error range of body height is 0.08-0.17cm, and the average absolute error is 0.12cm. The relative error range of body length is 0.08-2.76%, and the average relative error is 0.096%; the relative error range of body height is 0.10-3.24%, and the average relative error is 1.21%, indicating that the detection results of body length and height are highly accurate, and the detection accuracy can meet the design requirements. The output results of the fish image detection module are shown in Figure 1. Figure 5 Example shown.

[0168] Table 1 Body length and height detection errors

[0169]

[0170] The present invention also provides a high-precision fish posture measurement device, which includes at least one processor and at least one memory, and also includes a communication interface and an internal bus; a computer execution program is stored in the memory; a computer execution program of the fish posture measurement model constructed by the above-mentioned construction method is stored in the memory; when the processor executes the computer execution program stored in the memory, the processor can execute a high-precision fish posture measurement method. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0171] like Figure 6As shown, the device can be provided as a terminal, a server or other form of device. The device may include one or more of the following components: a processing component, a memory, a power component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component. The processing component generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component may include one or more processors to execute instructions to complete all or part of the steps of the above method. In addition, the processing component may include one or more modules to facilitate interaction between the processing component and other components. For example, the processing component may include a multimedia module to facilitate interaction between the multimedia component and the processing component.

[0172] The memory is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0173] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0174] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0175] The present invention also provides a computer-readable storage medium, in which a computer execution program of the fish body measurement model constructed by the above-mentioned construction method is stored. When the computer execution program is executed by a processor, the processor can execute a high-precision fish body measurement method.

[0176] Specifically, a system, device or equipment equipped with a readable storage medium may be provided, on which a software program code for implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system, device or equipment reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the above-mentioned embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0177] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk (such as CD-ROM, CD-R, CD-RW, DVD-20ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tape, etc. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0178] It should be understood that the storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in a terminal or server.

[0179] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0180] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, etc., and conventional procedural programming languages-such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network-including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0181] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0182] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing a high-precision fish body measurement model, characterized in that: The process includes: S1, creating a fish image dataset, including the collection and annotation of fish images; S2, constructing a fish image paired dataset, including fish image data containing noise and fish image data after denoising and enhancement; S3, constructing a fish posture measurement model, which includes a fish image enhancement module, a fish target detection module and a fish posture measurement module; The fish image enhancement module is designed based on the generative adversarial network (GAN) architecture, including a generator network and a discriminator network, which are used to enhance the acquired fish images; the generator network introduces an auxiliary guidance module to extract multi-scale feature details of the target domain image to guide the generation process of the generator network; the specific structures of the generator network and the discriminator network in the fish image enhancement module are as follows: The generator network adopts an encoder stacking structure and a decoder stacking structure. The encoder stacking structure consists of a convolutional layer, a ReLU activation function, and a downsampling pooling layer. i Perform step-by-step feature extraction and downsampling, and the encoder stack structure is expressed as: H i =f enc (S i ;W enc ) Among them, H i is the output of the encoder stack structure and also the input image S i Representation in latent feature space; f enc (·) indicates the encoder stacking structure; W enc Trainable weight parameters for the encoder stack structure; The encoder stack structure contains 3 layers of convolution operations and downsampling operations. The calculation process of the jth layer is: in, It is the feature map output by the jth layer; is the output of the previous layer; Conv(·) is the convolution operation; W enc j is the convolution kernel weight of the jth layer; ReLU is the linear activation function; DownPool(·) is the downsampling pooling operation; After 3 layers of convolution and downsampling, the final encoder stack structure output is a vector H of length 256 i , as the input image S i The encoding feature representation of ; The decoder stack structure is used to convert the latent feature vector H output by the encoder stack structure i Reconstruct the original input data; the decoder stacking structure is the reverse symmetric structure of the encoder stacking structure: in, is the output of the decoder, i.e. the reconstructed image; f dec (·) is the decoder stacking network; W dec Trainable weight parameters for the decoder stack network; The decoder stack network contains 3 layers of upsampling operations and deconvolution operations. The calculation process of the jth layer is: in, is the output feature map of the jth layer; is the output of the previous layer; ConvT(·) is the deconvolution operation; Up(·) is the upsampling pooling operation; Through 3 layers of upsampling and deconvolution operations, the decoder stacking structure encodes the feature H i Gradually reconstructed into the same i Images of the same resolution The discriminator network is used to judge the quality of the fish image output by the generator network during the training process; the fish image G i The image is divided into 2*2 small blocks, and then the four blocks are input into the discriminator network respectively. Finally, all the results of the discriminator are averaged as the comprehensive discrimination result. The calculation formula is as follows: in, is the τth region block of the jth image to be discriminated, D is the discriminator’s discriminant result, Dis(·) is the discriminator’s processing function; The fish target detection module is established based on the YOLOV8 model, which is used to obtain the position information of each fish and output the detection image marked with a rectangular frame; The fish posture measurement module is used to calculate the actual size of the target rectangular frame in the output image based on the detection image after the rectangular frame is marked, the actual width of the target rectangular frame is the body length of the fish, and the actual height is the body height of the fish; S4, through the designed fish image enhancement module training loss function and fish target detection module training loss function, and based on the constructed paired image dataset, the model is trained to obtain the final fish body measurement model.

2. A method for constructing a high-precision fish body measurement model as claimed in claim 1, characterized in that: The specific process of making the fish image dataset in S1 includes: The captured target fish is anesthetized and placed on a flat surface, with a checkerboard calibration board placed next to it to convert pixel measurement values ​​into physical lengths, and horizontal shooting is performed to obtain images; The body length and height of each fish were measured and recorded with a vernier caliper, and the body length and height of the fish were manually annotated using Labelimg software. The width of the target frame was from the tip of the fish's snout to the base of the fish's tail fin, and the height of the target frame was from the starting point of the dorsal fin to the ventral surface, representing the body length BL and height BH of the fish, respectively.

3. A method for constructing a high-precision fish body measurement model as claimed in claim 1, characterized in that: The auxiliary guidance module includes a multi-scale pooling layer, a channel convolution layer, a multi-scale convolution layer, a channel attention layer, a multi-head attention layer and an AdaLN layer; specifically includes: The i-th target domain image T in the paired image dataset i Enter the multi-scale pooling layer, the calculation formula is as follows: in, and They represent the output after average pooling and maximum pooling when the pooling kernel is k, respectively. k is preset to {1, 3, 5}. By using boundary padding to ensure that the output length is consistent after processing with different pooling kernels, the output of the multi-scale pooling layer is finally obtained. The output of the multi-scale pooling layer Input channel convolution layer, the specific calculation formula is as follows: in Represents the output of the channel convolution layer, whose number of channels is consistent with the input dimension of this layer; Will Input the multi-scale convolution layer to obtain multi-scale features. The specific calculation formula is as follows: in represents the output when the convolution kernel size is k, δ represents the Sigmoid activation function, and finally the output of the multi-scale convolution layer is obtained Will The three multi-scale features in are input into the channel attention layer respectively to capture the local structural information in the features of different scales, enhance the attention to the fish features, and suppress the unimportant environmental features. The calculation formula is as follows: Among them, P i,k for The corresponding channel attention weight, MLP (·) is a multi-layer perceptron; P i,k and Perform product operation to obtain channel attention features In order to capture the relationship between multi-scale features, and Input the multi-head attention mechanism layer to obtain the weight, and the calculation formula is as follows: Among them, Q, K, and V are query sequence, key sequence, and value sequence, respectively. m is the dimension of the attention head; the calculation result of the lth attention head is expressed as: in, are the linear transformation weight matrices of query, key, and value respectively, and connect h attention heads to obtain a multi-head attention mechanism: F i =Concat(head1,head2,…,head h )W o Among them, W o The linear transformation weight matrix output by the multi-head attention mechanism; Using AdaLN layer to normalize F i Normalize the information features and map them to the auxiliary guidance features of the jth layer through the convolution layer Among them, AdaLN(·) is the layer normalization function, and the auxiliary guide feature The dimension of the decoder layer is the same as the feature restored by have the same dimensions; The auxiliary guidance features to be obtained The input of each layer of the decoder stacking structure guides the image enhancement effect in details.

4. A method for constructing a high-precision fish body measurement model as claimed in claim 1, characterized in that: The fish target detection module is established based on the YOLOV8 model. A dynamic feature fusion module DFF is introduced into the neck network of YOLOv8 to replace the original feature splicing layer. The module can dynamically adjust the feature fusion mode and weight according to different areas and levels of the input fish feature map, including a feature alignment layer, an adaptive attention layer and an adaptive fusion layer. The feature alignment layer maps the features to be fused to the same space and then inputs them into the adaptive attention layer. The adaptive attention layer calculates the weight of each feature to obtain the weighted feature of each feature, and then inputs the weighted feature of each feature into the adaptive fusion layer for feature splicing. An adaptive context enhancement module ACE is introduced into the backbone network. This module aims to improve the detection performance of YOLOV8 by adaptively enhancing the context information of the fish feature map. It includes a dilated convolution layer and a pyramid pooling layer. The dilated convolution layer can effectively expand the receptive field of the convolution layer without increasing the number of parameters and computational complexity, and effectively capture the global context information of the backbone network features. The pyramid pooling layer can extract multi-scale features from the backbone network features by performing pooling operations on sub-regions of different scales in the feature map.

5. A method for constructing a high-precision fish body measurement model as claimed in claim 1, characterized in that: The fish posture measurement module specifically processes as follows: Calculate the actual width of the output image of the fish target detection module; first convert the image into a grayscale image; then use OpenCV to calculate the position of the corner points of the checkerboard calibration plate in the image and the average pixel distance between the corner points; finally, calculate the actual width of the image based on the position of the corner points of the checkerboard calibration plate and the average pixel distance between the corner points; Calculate the actual size of the target frame in the output image; calculate the actual size of the target frame using the actual width and image aspect ratio of the image obtained above and the coordinate information of the target frame, the actual width of the target frame is the body length of the fish, and the actual height is the body height of the fish.

6. A method for constructing a high-precision fish body measurement model as claimed in claim 1, characterized in that: The fish image enhancement module training loss function is improved based on the basic loss function of GAN, and high-frequency information loss is introduced on the basic loss function, as follows: in, and They represent the expectations of the target domain and the source domain, respectively. log(·) is the logarithmic function, Haar(·) is the Haar wavelet operator, ‖·‖ F To calculate the F-norm operator, w1 and w2 are weight factors; The training loss function of the fish target detection module is improved based on the basic loss function of YOLOV8. The mean square error loss is replaced by the intersection over union (IoU) loss. The IoU loss directly measures the overlap between the predicted box and the real box. The IoU information is introduced into the confidence loss to obtain the enhanced confidence loss, as follows: Among them, Intersection is the overlapping area of ​​the predicted box and the real box, and Union is the union area of ​​the predicted box and the real box. i is the true label, whether the target exists or not; is the probability predicted by the model; IoU i is the IoU value between the i-th predicted box and the true box; L cel is the cross entropy classification loss of the basic YOLOV8; γ1, γ2 and γ3 are the weight coefficients of intersection-over-union loss, cross entropy classification loss and enhanced confidence loss, respectively, which are used to balance the importance of different loss items.

7. A high-precision fish posture measurement method, characterized in that: The fish body measurement model constructed by the construction method according to any one of claims 1 to 6 is deployed on a back-end server, and includes the following processes: S1, receiving the collected fish images in real time; S2, the fish image is input into the generator network of the fish image enhancement module to obtain the enhanced fish image; S3, the enhanced fish image is input into the fish image detection module to obtain the position information of each fish in the fish image; S4, inputting the position information of each fish and the fish image into the fish posture measurement module to obtain the height and length of each fish in the image.

8. A high-precision fish posture measurement device, characterized by: The device includes at least one processor and at least one memory, the processor and the memory are coupled; the memory stores a computer execution program of a fish body measurement model constructed by the construction method as described in any one of claims 1 to 6; when the processor executes the computer execution program stored in the memory, the processor executes a high-precision fish body measurement method.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer execution program of the fish posture measurement model constructed by the construction method according to any one of claims 1 to 6. When the computer execution program is executed by the processor, the processor executes a high-precision fish posture measurement method.

Images