Fish body color recognition method based on fish body color recognition model and electronic device

Abstract

The application provides a fish body color recognition method based on a fish body color recognition model and an electronic device. The method comprises the following steps: constructing a fish body color dataset, wherein the fish body color dataset comprises fish body images and corresponding true value labels; establishing an initial fish body color recognition model, wherein the initial fish body color recognition model is established based on a YOLOv5s network model; training the initial fish body color recognition model by using a training set, wherein the fish body images and the true value labels in the training set are used as inputs during the training, and the initial fish body color recognition model outputs fish body color prediction results; obtaining errors between the fish body color prediction results and the true value labels according to a loss function, updating and optimizing parameters of the initial fish body color recognition model through multiple iterations, and obtaining a fish body color recognition model; and recognizing the body color of a target fish in a target region based on the fish body color recognition model. The method has high recognition accuracy for the body color of the fish.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fish body color recognition, and in particular to a fish body color recognition method based on a fish body color recognition model and an electronic device. BACKGROUND

[0002] Fish body color not only reflects the physiological function of the fish body, but also is one of the important attributes that determine the market value and economic value. The East Star is a representative fish species with high economic value and ornamental value due to its bright body color in recent years. The body color directly affects its value and sales. The detection of the body color of the East Star is not only beneficial to the adjustment of the proportion of colored feed, which can save the cost of aquaculture, but also beneficial to the realization of quality grading and the estimation of the overall economic value of the East Star in the aquaculture area. Therefore, the detection of the body color of the East Star is of great significance to the regulation of aquaculture and the evaluation of economic benefits.

[0003] In a factory-like breeding workshop, the light source needs to be adjusted according to the growth characteristics of fish and the production and breeding needs, and is not stable and consistent. In addition, the refraction and absorption of seawater to light can also cause serious image color deviation problems. Therefore, fish body color recognition needs to be realized on the basis of image calibration.

[0004] At present, there are mainly two types of image color calibration methods: image restoration-based method and data-driven method. The image restoration-based method restores the image to the effect before degradation by using prior knowledge. The data-driven method selects the data set after acquisition or gain to train the color calibration network model, which is used for subsequent color calibration. The data-driven method can better guarantee the color constancy, but the parameter quantity is large and not easy to carry the fish body color recognition network. The image restoration-based method can be easily carried by the body color recognition network, but it focuses more on global color calibration, and the local detail calibration effect is poor. SUMMARY

[0005] The present application provides a fish body color recognition method based on a fish body color recognition model and an electronic device, which solves the problem of low accuracy of fish body color recognition in the prior art, and realizes a method for establishing a model with high recognition rate for fish body color to recognize the fish body color.

[0006] The present application provides a fish body color recognition method based on a fish body color recognition model, comprising:

[0007] Constructing a fish body color data set, the fish body color data set comprising fish body images and true value labels corresponding to the fish body images;

[0008] An initial fish body color recognition model is established, and the initial fish body color model is established based on a YOLOv5s network model and has an image calibration function based on a variance gray world algorithm;

[0009] The initial fish body color recognition model is trained by using a training set, the training set includes fish body images and corresponding true value labels extracted from the fish body color dataset, and the fish body images and the true value labels in the training set are used as inputs during training, and the initial fish body color recognition model outputs fish body color prediction results;

[0010] According to the loss function, the error between the fish body color prediction result and the true value label is obtained, and through multiple iterations, the parameters of the initial fish body color recognition model are updated and optimized to obtain a fish body color recognition model;

[0011] Based on the fish body color recognition model, the body color of the target fish in the target region is recognized.

[0012] According to the fish body color recognition model provided by the application, the initial fish body color recognition model is established, and the initial fish body color model is established based on a YOLOv5s network model, which comprises:

[0013] Based on the YOLOv5s network model, the data processing module of the YOLOv5s network model is improved, so that the improved data processing module includes an image calibration algorithm based on a variance gray world algorithm and a data enhancement algorithm;

[0014] Based on the YOLOv5s network model, the backbone network part of the YOLOv5s network model is improved, and sequentially connected CBS modules, CBS modules, BottleneckCSP modules, CBS modules, BottleneckCSP modules, CBS modules, BottleneckCSP modules, CBS modules, C3TR modules and HSPP modules constitute the improved backbone network;

[0015] Based on the YOLOv5s network model, the neck network part of the YOLOv5s network model is improved, and the improved neck network is sequentially connected with a CBS module, an Upsampling2d module, a concat module, a BottleneckCSP module, a CBS module, an Upsampling2d module, a concat module, a BottleneckCSP module, a CBS module, a concat module, a BottleneckCSP module, a CBS module, a concat module and a C3TR module, and the second BottleneckCSP module is connected with the Detect1 module of the head network of the YOLOv5s network model, and the third BottleneckCSP module is connected with the Detect2 module of the head network, and the C3TR module is connected with the Detect3 module of the head network.

[0016] The data processing module, the backbone network, the neck network and the head network are sequentially connected to obtain the initial fish body color recognition model.

[0017] According to the fish body color recognition method based on the fish body color recognition model provided by the application, the HSPP module is a pooling module improved based on a spatial pyramid pooling model (SPP) and a faster spatial pyramid pooling model (SPPF) and is composed of CBMe, a double-size MaxPool, a concat and CBMe sequentially connected.

[0018] According to the fish body color recognition method based on the fish body color recognition model provided by the application, the initial fish body color recognition model is trained by using a training set, the training set includes fish body images and corresponding true value labels extracted from the fish body color data set, and the fish body images and the true value labels in the training set are used as inputs during training, and the initial fish body color recognition model outputs fish body color prediction results, including:

[0019] The fish body images and the corresponding true value labels are extracted from the fish body color data set as a training set.

[0020] The fish body images and the true value labels are input into the initial fish body color recognition model, and the fish body images are calibrated based on a variance gray world algorithm by using the image calibration algorithm.

[0021] The fish body image after calibration is enhanced by the data enhancement module to obtain a gain fish body image set corresponding to a true value label after the fish body image after calibration is amplified.

[0022] The fish body image after gain is extracted by the improved backbone network and the improved neck network to obtain feature information.

[0023] Based on the feature information, the fish body color prediction result of the fish body image after gain is output by the head network.

[0024] According to the fish body color recognition method based on the fish body color recognition model provided by the application, the fish body color data set is constructed, the fish body color data set includes fish body images and true value labels corresponding to the fish body images, and the fish body color data set includes:

[0025] Video data of fish bodies in a water area is shot.

[0026] The video data collected is intercepted to obtain fish body images.

[0027] The fish body images are manually labeled to obtain the true value labels corresponding to the fish body images.

[0028] The fish body images and the true value labels corresponding to the fish body images constitute the fish body color data set.

[0029] According to the fish body color recognition method based on the fish body color recognition model provided by the application, the fish body color recognition method further includes:

[0030] The fish body color recognition model is tested by using a test set, and the test set includes fish body images and corresponding true value labels remaining in the fish body color data set after a training set is extracted.

[0031] During the testing, the fish body images and the true value labels in the test set are taken as inputs, and fish body color test results are output to calculate the intersection over union of the fish body color test results and the true value labels corresponding to the test set.

[0032] Based on the intersection over union, the precision, the recall, the first average precision and the second average precision are taken as measurement indexes to determine whether the performance of the fish body color recognition model is better than the performance of the YOLOv5s network model.

[0033] The application further provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the fish body color recognition method based on the fish body color recognition model according to any one of the above when executing the program.

[0034] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the fish body color recognition method based on the fish body color recognition model.

[0035] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the fish body color recognition method based on the fish body color recognition model.

[0036] The fish body color recognition method based on the fish body color recognition model and the electronic device provided by the application establish an initial fish body color recognition model based on a YOLOv5s network model, so that the initial fish body color recognition model has an image calibration function based on a variance gray world algorithm, greatly improving the calibration effect of the image, thereby further improving the recognition accuracy of the fish body color recognition model trained based on the initial fish body color recognition model, and efficiently and accurately recognizing the body color of the multi-target object in the breeding environment without affecting the activity of the fish, thereby providing support for adjusting the breeding link and evaluating the economic benefits in fish breeding. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 is one of the flowcharts of the fish body color recognition method based on the fish body color recognition model provided by the application;

[0039] Figure 2 is the flowchart of the method for establishing the initial fish body color recognition model provided by the application;

[0040] Figure 3 is the structural diagram of the initial fish body color recognition model provided by the application;

[0041] Figure 4 is the structural diagram of the HSPP module provided by the application;

[0042] Figure 5 is the flowchart of the method for training the initial fish body color recognition model by using the training set provided by the application;

[0043] Figure 6 is the flowchart of the method for constructing the fish body color data set provided by the application;

[0044] Figure 7 Figure 2 is a flowchart of a fish body color recognition method based on a fish body color recognition model according to the present application;

[0045] Figure 8 Figure 3 is a structural schematic diagram of an electronic device according to the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] The fish body color recognition method based on a fish body color recognition model according to the present application will be described below. Figures 1-7 The fish body color recognition method based on a fish body color recognition model according to the present application includes the following steps.

[0048] S101: Construct a fish body color dataset, which includes fish body images and true value labels corresponding to the fish body images.

[0049] For example, the fish body color dataset is constructed, which includes multiple fish body images and true value labels corresponding to each fish body image. The true value labels corresponding to the fish body images are manually labeled, and specifically, the makesense.ai online labeling software can be used to label the fish body color in the fish body images.

[0050] S102: Establish an initial fish body color recognition model, which is established based on a YOLOv5s network model and has an image calibration function based on a variance gray world algorithm.

[0051] Specifically, the initial fish body color recognition model is established based on the YOLOv5s network model. The YOLOv5 network model is improved based on the YOLOv3 network model, and there are four models, YOLOv5s, YOLOv5m, YOLOv5l and YOLOv5x. The YOLOv5s network model is composed of a data processing module, a backbone network, a neck network and a head network. The YOLOv5s network model provides an end-to-end target detection method, which converts the target detection problem into a regression problem, and the target and its position can be directly recognized by processing the image through a single convolutional neural network.

[0052] The initial fish body color recognition model has an image calibration function based on a variance gray world algorithm, which is integrated in the input network part.

[0053] S103: training the initial fish body color recognition model using a training set, the training set including fish images extracted from the fish body color dataset and corresponding true value labels, the fish images and the true value labels in the training set being used as input during the training, and the initial fish body color recognition model outputting fish body color prediction results.

[0054] Specifically, the fish images extracted from the fish body color dataset and the corresponding true value labels are used as the training set. For example, the fish images and the corresponding true value labels in the fish body color dataset can be divided into two groups in a ratio of 8:2. After the division, 80% of the fish images and the corresponding labels are used as the training set, and the remaining fish images and the corresponding true value labels are used as the test set.

[0055] For example, before the fish images in the training set are input into the initial fish body color recognition model, the fish images can be scaled to 640x640 and then input.

[0056] S104: obtaining the error between the fish body color prediction results and the true value labels according to a loss function, and updating and optimizing the parameters of the initial fish body color recognition model through multiple iterations to obtain a fish body color recognition model.

[0057] Specifically, the true value labels corresponding to the fish images and the fish body color prediction results corresponding to the fish images may have certain errors. The purpose of training the initial fish body color recognition model is to make the fish body color prediction results corresponding to the fish images as close to the true value labels as possible. The error between the fish body color prediction results and the true value labels can be obtained through a loss function. The parameters of the initial fish body color recognition model are updated and optimized through multiple iterations to make the model converge, and a fish body color recognition model is obtained, in which the fish body color prediction results are as close to the true value labels as possible.

[0058] The loss function Loss used in the training of the initial fish body color recognition model includes a bounding box positioning loss (L CIoU ), a confidence loss (L confidence ), and a category loss (L class ):

[0059] Loss = L CIoU + L confidence + L class

[0060] S105: performing body color recognition on a target fish in a target region based on the fish body color recognition model.

[0061] The fish body color recognition model obtained through the above steps can be directly used for efficient and accurate recognition of the fish body color, and the fish body color recognition model can be used to recognize the fish body color of any target fish in any target region.

[0062] The fish body color recognition method based on the fish body color recognition model provided by the application can greatly improve the calibration effect of the image, thereby further improving the recognition accuracy of the fish body color recognition model obtained through training based on the initial fish body color recognition model, and can efficiently and accurately recognize the body color of the multi-target object in the breeding environment without affecting the activity of the fish, thereby providing support for adjusting the breeding link and evaluating the economic benefits in fish breeding.

[0063] In one embodiment, as shown in Figure 2 , the initial fish body color recognition model is established based on the YOLOv5s network model, and includes:

[0064] S201: Based on the YOLOv5s network model, the data processing module of the YOLOv5s network model is improved, so that the improved data processing module includes an image calibration algorithm based on the variance gray world algorithm and a data enhancement algorithm.

[0065] Specifically, the YOLOv5s network model is composed of a data processing module, a backbone network, a neck network and a head network. As shown in Figure 3 , based on the YOLOv5s network model, the image calibration algorithm based on the variance gray world algorithm is added to the data processing module of the YOLOv5s network model, and the data enhancement algorithm inherent in the data processing module of the YOLOv5s network model is combined, so that the improved data processing module includes the image calibration algorithm and the data enhancement algorithm. As shown in Figure 3 , the dashed box is a structure diagram of the improved data processing module (Input).

[0066] Since the imbalance of fish body color category samples will affect the recognition accuracy of the fish body color recognition model obtained after training, the fish body image is data enhanced through a data enhancement algorithm, such as a mosaic enhancement algorithm. The mosaic enhancement algorithm realizes the expansion of limited fish body images by splicing, scaling and randomly cropping multiple images to synthesize new images, which can alleviate the problem of sample category imbalance to a certain extent, and the robustness of the fish body color recognition model obtained after training can be improved by increasing the number of multi-category fish body images through the mosaic enhancement algorithm.

[0067] Further, in a relatively closed aquaculture environment, the workshop light is mainly provided by external light sources such as lamp posts. However, the external light source is not continuously and stably provided, and will be changed according to the demand of fish body to the light source and the demand of production and cultivation. At the same time, seawater also has the effects of absorption and refraction on light, so that the collected image has the problems of color distortion and color cast. In order to solve the above problems, an image calibration algorithm based on variance gray world algorithm is added to the data processing module part of the YOLOv5s network model. The variance gray world algorithm is based on the gray world algorithm, and considers the channel variance of the image color space to optimize the mean weight of each channel.

[0068] In the gray world algorithm, it is first assumed that the pixel mean values of R, G and B color channels in the fish body image tend to the same gray value K', and then the channel adjustment coefficient of three channels is obtained by using the gray value K' and the channel pixel mean value. Finally, the pixel value in the image is calibrated according to the channel adjustment coefficient, so as to realize the color calibration of the whole image.

[0069] The gray value K' of the gray world algorithm is:

[0070] K' = (R + G + B) / 3 avg + G avg +B avg ) / 3

[0071] Wherein, K' represents the gray value of the gray world algorithm, R avg , G avg , B avg are the pixel mean values of R, G and B three channels of the image respectively.

[0072] However, in the image with color cast and color distortion, there is a large difference among R, G and B three channels. Using the channel mean value as the only calibration factor will cause the problems of insufficient channel compensation and incomplete calibration. The variance gray world algorithm uses variance and mean as calibration parameters to analyze the value of gray value K according to the color cast and color distortion of the image.

[0073] The gray value K of the variance gray world algorithm is shown in the following formula:

[0074] K = (C r *R avg +C g *G avg +C b *B avg ) / 3

[0075] Wherein, K represents the gray value of the variance gray world algorithm, C r , C g , C b are Rαvg , C gvg , B avg The weight coefficient of the mean value of the three channels of pixels.

[0076] Due to the absorption of water to light, the collected image will have the problem of color imbalance of R, G and B channels, for example, the image can be blue or green.

[0077] Therefore, C r , C g , C b The weight is set as follows.

[0078] In the first case, when the image is blue, that is, B avg > max(G avg , R avg ), the weight coefficients of the mean values of R avg , G avg , B avg channels are:

[0079]

[0080] Where, R var , G var , B var are the variances of the pixel values of the R, G and B channels of the image respectively.

[0081] In the second case, when the image is green, that is, G avg > max(R avg , B avg ), the weight coefficients of the mean values of R avg , G avg , B avg channels are:

[0082]

[0083] Where, R var , G var , B var are the variances of the pixel values of the R, G and B channels of the image respectively.

[0084] In the third case, when there is no G and B channel color highlight, the variance gray world algorithm is the gray world algorithm, and the weight coefficients of the mean values of R avg , G avg , B avg channels are:

[0085] C r = C g = C b = 1

[0086] In summary, the channel adjustment coefficient calculation formula of the variance gray world algorithm is as follows:

[0087]

[0088] K R, K G and K B are the channel adjustment coefficients of the R, G and B channels of the image respectively. r g b K R, K G and K B are the channel adjustment coefficients of the R, G and B channels of the image respectively.

[0089] According to the channel adjustment coefficients of the three channels, the pixel values of the three channels of the fish body image are color calibrated:

[0090]

[0091] R, G and B are the three-channel pixel values of the original image, and R', G' and B' are the three-channel pixel values of the color calibrated image.

[0092] S202: Based on the YOLOv5s network model, the backbone network part of the YOLOv5s network model is improved, and sequentially connected CBS module, CBS module, Bottlen eckCSP module, CBS module, Bottlen eckCSP module, CBS module, Bottlen eckCSP module, CBS module, C3TR module and HSPP module constitute the improved backbone network.

[0093] Specifically, as shown in Figure 3 , the box part is the improved part, and the module in the box is the improved module. As can be seen from Figure 3 , the sequentially connected CBS module, CBS module, Bottlen eckCSP module, CBS module, Bottlen eckCSP module, CBS module, Bottlen eckCSP module, CBS module, C3TR module and HSPP module constitute the improved backbone network (Backbone).

[0094] ​​The C3TR module is implemented by improving the Transformer-Encoder. The Transformer is an Encoder-Decoder model based on the self-attention mechanism to improve the training speed of the model. It not only can expand the focus of the fish body color recognition model on multiple regions of interest, but also can fuse the feature information of multiple representation subspaces, which is conducive to the realization of target positioning and feature extraction of the region of interest. The Encoder module is composed of multiple identical sublayers. Each sublayer includes a multi-head self-attention mechanism (MSA) and a fully connected feed-forward neural network (FFN).

[0095] The multi-head self-attention mechanism MSA is based on the self-attention mechanism. It first maps the Query, Key, and Value linearly to multiple subspaces, i.e., scaled dot-Product attention, and finally concatenates the feature information of multiple subspaces to obtain the final attention feature information.

[0096] In addition to the multi-head self-attention mechanism, each sublayer also includes a fully connected feed-forward neural network FFN. The fully connected feed-forward neural network is composed of two linear transformation layers and an activation function ReLU.

[0097] The BottleneckCSP module uses a residual block applied to a deep network, i.e., Bottleneck, which reduces the possibility of gradient dispersion and to some extent preserves more original information. Then, a convolution layer, batch normalization, and an activation function are added after the Concat layer to increase the depth of the network to learn the fused feature information.

[0098] S203: Based on the YOLOv5s network model, the neck network part of the YOLOv5s network model is improved, and the improved neck network is sequentially connected CBS module, Upsampling2d module, concat module, BottleneckCSP module, CBS module, Upsampling2d module, concat module, BottleneckCSP module, CBS module, concat module, BottleneckCSP module, CBS module, concat module and C3TR module, and the second BottleneckCSP module is connected with the CBS module, and at the same time, it is connected with the Detect1 module of the head network of the YOLOv5s network model, the third BottleneckCSP module is connected with the CBS module, and at the same time, it is connected with the Detect2 module of the head network, and the C3TR module is connected with the Detect3 module of the head network.

[0099] Specifically, referring to Figure 3 , the neck network part of the YOLOv5s network model is improved, and the box part is the improved part, and the module in the box is the improved module. As can be seen from Figure 3 , the sequentially connected CBS module, Upsampling2d module, concat module, BottleneckCSP module, CBS module, Upsampling2d module, concat module, BottleneckCSP module, CBS module, concat module, BottleneckCSP module, CBS module, concat module and C3TR module are the improved neck network.

[0100] S204: The improved data processing module, the improved backbone network, the improved neck network and the head network are sequentially connected to obtain an initial fish body color recognition model.

[0101] Specifically, the improved input network, the improved backbone network, the improved neck network and the head network are sequentially connected to obtain an initial fish body color recognition model, and the head network is the head network of the YOLOv5s network model.

[0102] In one embodiment, the HSPP module is a pooling module improved based on a spatial pyramid pooling model (SPP) and a faster spatial pyramid pooling model (SPPF), mainly composed of CBMe, double-size MaxPool, Concat and CBMe sequentially connected.

[0103] For details, please refer to Figure 4 As shown, HSPP is an improved pooling module based on SPP (Spatial Pyramid Pooling) and SPPF (Faster Spatial Pyramid Pooling). It mainly consists of four interconnected parts: CBMe, a dual-size MaxPool, Concat, and CBMe. The HSPP module reduces the pooling size while incorporating parallel and cascaded structures, changing from concatenating four features to concatenating eight. Furthermore, the HSPP module replaces the SiLU activation function with the MetaAconC activation function, adaptively adjusting the activation function's performance. This reduces the number of parameters while increasing the robustness and applicability of the HSPP module.

[0104] Unlike the Acon series, which switches between linear and non-linear activation functions by setting a beta value, the MetaAconC activation function dynamically learns its beta based on the input x. c The activation function is set to an adaptive value to achieve linear or nonlinear control, which is beneficial for improving the generalization ability and transfer performance of the HSPP module. The activation function is as follows:

[0105] MetaAconC=(P1-P2)x·σ(β c (P1-P2)x)+P2

[0106] Where x is the input feature map of the activation function, P1 is the first derivative of x as x approaches positive infinity, P2 is the first derivative of x as x approaches negative infinity, and β is the first derivative of x as x approaches negative infinity. c σ represents the performance coefficient of the activation function, where σ denotes the sigmoid activation function.

[0107] This embodiment uses channel space to learn β. c value:

[0108]

[0109] Where W1 and W2 represent the input and output channel weights, W represents the width of the input feature map of the activation function, H represents the height of the input feature map of the activation function, c represents the number of channels, and σ represents the sigmoid activation function.

[0110] In one embodiment, the initial fish body color recognition model is trained using a training set, wherein the training set includes fish images extracted from the fish body color dataset and corresponding ground truth labels. During training, the fish images and ground truth labels in the training set are used as input, and the initial fish body color recognition model outputs a fish body color prediction result, including:

[0111] S501: Extract fish body image and corresponding true value label from the fish body color data set as a training set.

[0112] Specifically, the fish body image and the corresponding true value label in the fish body color data set are divided into groups in a ratio of 8:2, and the group with more fish body images and corresponding true value labels is used as the training set.

[0113] S502: Input the fish body image and the true value label into the initial fish body color recognition model, and perform image calibration on the fish body image based on the variance gray world algorithm through the image calibration algorithm.

[0114] Specifically, the image calibration algorithm calibrates the fish body image based on the variance gray world algorithm. The variance gray world algorithm considers different calibration algorithms under different color cast conditions of the fish body image, and finally obtains calibrated image data, which can improve the recognition accuracy of the fish body color recognition model.

[0115] S503: Perform mosaic enhancement on the calibrated fish body image through the data enhancement module to expand the calibrated fish body image to obtain the gain fish body image and the corresponding true value label.

[0116] Specifically, the fish body image can be scaled to 640x640 and input into the initial fish body color recognition model. The data enhancement module performs mosaic enhancement on the fish body image to obtain the gain fish body image and the corresponding true value label.

[0117] S504: Extract feature information from the gain fish body image through the improved backbone network and the improved neck network.

[0118] S505: Based on the feature information, output the fish body color prediction result of the gain fish body image through the head network.

[0119] Specifically, the head network outputs the fish body color prediction result, which is the target recognized on the fish body image and the position of the target.

[0120] In one embodiment, the fish body color data set is constructed, and the fish body color data set includes fish body images and corresponding true value labels of the fish body images, which includes:

[0121] S601: Capture video data of fish bodies in a water area.

[0122] Exemplarily, the fish body image acquisition mode can be: first, a fish body image acquisition platform is built in an aquaculture base, a fish group overhead video is collected, and fish group video data is obtained. The platform includes devices including a camera, a camera support, a camera memory card, a computer, etc. The camera can be, for example, a Hikvision 3T86FWDV2-I3S, 8 million pixel, 4 mm focal length camera. The camera memory card is connected with the computer through a network cable, and is used to import the collected fish group video data into the computer for storage. The camera support is used to fix the camera directly above the culture pond, at a distance of about 1.5 m from the water surface, so that the camera can shoot overhead.

[0123] S602: The collected video data is intercepted to obtain a fish body image.

[0124] Specifically, after the fish group video is collected, a video frame is intercepted and cropped in the collected fish group video data by using a video frame division technology, and a fish group overhead view is obtained. The collected fish group overhead view is an RGB (Red-Green-Blue) color image, the image format is jpg format, and the image resolution is 1200*800. Finally, the fish group overhead view is randomly cropped to obtain a local image, which is the fish body image.

[0125] S603: The fish body image is manually labeled to obtain the true value label corresponding to the fish body image.

[0126] Specifically, the fish body image obtained is manually labeled to obtain the true value label. Exemplarily, the makesense.ai online labeling software can be used to label the fish body color in the fish body image.

[0127] S604: The fish body image and the true value label corresponding to the fish body image constitute the fish body color data set.

[0128] In one embodiment, the fish body color recognition method based on the fish body color recognition model further includes:

[0129] S701: The fish body color recognition model is tested by using a test set, and the test set includes the fish body images and the corresponding true value labels remaining in the fish body color data set after the training set is extracted.

[0130] Specifically, the fish body images and the corresponding true value labels in the fish body color data set are extracted as a test set. Exemplarily, the fish body images and the corresponding true value labels in the fish body color data set can be divided into groups in a ratio of 8:2, and the group with fewer fish body images and corresponding true value labels is used as a verification set after the division.

[0131] S702: input the fish body image and the true value label in the test set as input at the time of test, output the fish body color test result, and calculate the intersection over union of the fish body color test result and the corresponding true value label in the test set.

[0132] Specifically, the intersection over union is the coincidence of the position area of the fish to which the fish body color belongs in the fish body color test information and the position area of the fish labeled in the true value label. When there is no coincidence or the coincidence degree is low, it indicates that the recognition performance of the fish body color recognition model is poor.

[0133] S703: based on the intersection over union, the precision, recall, first average precision and second average precision are used as measurement indexes to determine whether the performance of the fish body color recognition model is better than that of the YOLOv5s network model.

[0134] Specifically, the intersection over union (threshold value greater than 0.5) of the output obtained by inputting the test set into the fish body color recognition model and the labeled true value label is used as the test result of the fish body color recognition model. The intersection over union threshold value greater than 0.5 means that the coincidence area of the position area of the fish to which the fish body color belongs in the fish body color test information and the position area of the fish labeled in the true value label is greater than 0.5 of the position area of the fish.

[0135] The precision (Precision), recall (Recall), first average precision (AP50) and second average precision (mAP@0.5:0.95) are selected as measurement indexes to determine the recognition effect of the fish body color recognition model. mAP@0.5 The first average precision refers to the threshold value of the intersection over union being 0.5, and the second average precision refers to the average value obtained by averaging the different average precision values when the intersection over union threshold value is respectively taken as 0.5 / 0.55 / 0.6 / 0.65 / 0.7 / … / 0.9 / 0.95.

[0136] The specific formula is as follows:

[0137]

[0138]

[0139] wherein, T P represents the number of samples that are actually positive and the test result is also positive, F P represents the number of samples that are actually negative but the test result is positive, F N represents the number of samples that are actually positive but the test result is negative. Exemplarily, T P represents the number of samples that are actually positive and the prediction is also positive, for example, for the red category, T P represents the number of samples that are true value label red and the test result is also red, F PPrecision, Recall, F1, and IoU are defined as follows: Precision = TP / (TP + FP) Recall = TP / (TP + FN) F1 = 2 * Precision * Recall / (Precision + Recall) IoU = TP / (TP + FP + FN) N Precision, Recall, F1, and IoU are defined as follows: Precision = TP / (TP + FP) Recall = TP / (TP + FN) F1 = 2 * Precision * Recall / (Precision + Recall) IoU = TP / (TP + FP + FN)

[0140] The above indicators are used to determine the recognition effect of the fish body color recognition model for each color category.

[0141] The above precision (Precision) and recall (Recall) are calculated based on mAP@0.5, and the calculation formulas of mAP@0.5 and mAP@0.5:0.95 are as follows:

[0142]

[0143]

[0144] wherein mAP@0.5 refers to the average AP of all categories when the intersection over union threshold (IoU) is set to 0.5, and mAP@0.5:0.95 refers to the average mAP under different intersection over union thresholds (IoU), the intersection over union threshold (IoU) ranges from 0.5 to 0.95 with a step of 0.05; P is the precision (Precision), and R is the recall (Recall).

[0145] The performance of the fish body color recognition model obtained in the above embodiment is determined, and it is found that the fish body color recognition model has higher recognition accuracy than the YOLOv5s network model, and has better effect for fish body color recognition.

[0146] The fish body color recognition method based on the fish body color recognition model provided by the application establishes an initial fish body color recognition model based on the YOLOv5s network model, and the initial fish body color recognition model has an image calibration function based on the variance gray world algorithm, which greatly improves the calibration effect of the image, thereby further improving the recognition accuracy of the fish body color recognition model trained based on the initial fish body color recognition model, and efficiently and accurately recognizing the body color of multiple target objects in the breeding environment without affecting the activity of the fish, thereby providing support for adjusting the breeding link and evaluating the economic benefits in fish breeding.

[0147] Figure 8 An example of an electronic device is shown in the physical structure diagram as follows: Figure 8As shown, the electronic device can include a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 complete mutual communication through the communications bus 840. The processor 810 can invoke a logic instruction in the memory 830 to execute a fish body color recognition method based on a fish body color recognition model, the method comprising: constructing a fish body color dataset, the fish body color dataset comprising fish images and true value labels corresponding to the fish images; establishing an initial fish body color recognition model, the initial fish body color recognition model being established based on a YOLOv5s network model and having an image calibration function based on a variance gray world algorithm; training the initial fish body color recognition model using a training set, the training set comprising fish images and corresponding true value labels extracted from the fish body color dataset, the fish images and the true value labels in the training set being used as input during training, and the initial fish body color recognition model outputting fish body color prediction results; obtaining an error between the fish body color prediction results and the true value labels according to a loss function, updating and optimizing parameters of the initial fish body color recognition model through multiple iterations to obtain a fish body color recognition model; and identifying the body color of a target fish in a target region based on the fish body color recognition model.

[0148] In addition, the logic instruction in the memory 830 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0149] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer-readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the fish body color recognition method based on the fish body color recognition model provided by the above-mentioned methods, which comprises: constructing a fish body color dataset, the fish body color dataset comprising fish images and true value labels corresponding to the fish images; establishing an initial fish body color recognition model, the initial fish body color recognition model being established based on a YOLOv5s network model and having an image calibration function based on a variance gray world algorithm; training the initial fish body color recognition model using a training set, the training set comprising fish images and corresponding true value labels extracted from the fish body color dataset, the fish images and the true value labels in the training set being used as inputs during training, and the initial fish body color recognition model outputting fish body color prediction results; obtaining errors between the fish body color prediction results and the true value labels according to a loss function, updating and optimizing parameters of the initial fish body color recognition model through multiple iterations to obtain a fish body color recognition model; and performing fish body color recognition on a target fish in a target area based on the fish body color recognition model.

[0150] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the fish body color recognition method based on the fish body color recognition model provided by the above-mentioned methods, which comprises: constructing a fish body color dataset, the fish body color dataset comprising fish images and true value labels corresponding to the fish images; establishing an initial fish body color recognition model, the initial fish body color recognition model being established based on a YOLOv5s network model and having an image calibration function based on a variance gray world algorithm; training the initial fish body color recognition model using a training set, the training set comprising fish images and corresponding true value labels extracted from the fish body color dataset, the fish images and the true value labels in the training set being used as inputs during training, and the initial fish body color recognition model outputting fish body color prediction results; obtaining errors between the fish body color prediction results and the true value labels according to a loss function, updating and optimizing parameters of the initial fish body color recognition model through multiple iterations to obtain a fish body color recognition model; and performing fish body color recognition on a target fish in a target area based on the fish body color recognition model.

[0151] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0153] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A fish body color recognition method based on a fish body color recognition model, characterized in that, The method comprises the following steps: constructing a fish body color dataset, the fish body color dataset comprising fish body images and true value labels corresponding to the fish body images; establishing an initial fish body color recognition model, the initial fish body color recognition model being established based on a YOLOv5s network model and having an image calibration function based on a variance gray world algorithm; The establishment of the initial fish body color recognition model based on the YOLOv5s network model comprises: Based on the YOLOv5s network model, the data processing module of the YOLOv5s network model is improved, so that the improved data processing module comprises an image calibration algorithm based on a variance gray world algorithm and a data enhancement algorithm, which is used to dynamically adjust the RGB channel weight to eliminate underwater color cast; Based on the YOLOv5s network model, the backbone network part of the YOLOv5s network model is improved, and the sequentially connected CBS module, CBS module, BottleneckCSP module, CBS module, BottleneckCSP module, CBS module, BottleneckCSP module, CBS module, C3TR module and HSPP module constitute the improved backbone network; Based on the YOLOv5s network model, the neck network part of the YOLOv5s network model is improved, and the improved neck network is sequentially connected CBS module, Upsampling2d module, concat module, BottleneckCSP module, CBS module, Upsampling2d module, concat module, BottleneckCSP module, CBS module, concat module, BottleneckCSP module, CBS module, concat module and C3TR module, and the second BottleneckCSP module is connected with the CBS module while also connected with the Detect1 module of the head network of the YOLOv5s network model, the third BottleneckCSP module is connected with the CBS module while also connected with the Detect2 module of the head network, and the C3TR module is connected with the Detect3 module of the head network; The improved data processing module, the improved backbone network, the improved neck network and the head network are sequentially connected in order to obtain the initial fish body color recognition model; extracting fish body images and corresponding true value labels from the fish body color dataset as a training set; inputting the fish body images and the true value labels into the initial fish body color recognition model, and performing image calibration on the fish body images based on the variance gray world algorithm through the image calibration algorithm; performing mosaic enhancement on the calibrated fish body images through the data enhancement module to expand the calibrated fish body images to obtain gain fish body images and corresponding true value labels; The improved backbone network and the improved neck network are used for feature extraction on the gain fish body image to obtain feature information; Based on the feature information, the head network is used to output a fish body color prediction result of the gain fish body image; According to a loss function, an error between the fish body color prediction result and the true value label is obtained, and through multiple iterations, parameters of the initial fish body color recognition model are updated and optimized to obtain a fish body color recognition model; Based on the fish body color recognition model, a target fish body in a target region is recognized in terms of body color. 2.The fish body color recognition method based on the fish body color recognition model according to claim 1, characterized in that, The HSPP module is a pooling module improved based on a spatial golden pyramid pooling model and a faster golden pyramid pooling model, and is composed of CBMe, a double-size MaxPool, a Concat, and a CBMe sequentially connected. 3.The fish body color recognition method based on the fish body color recognition model according to claim 1 or 2, characterized in that, The fish body color dataset is constructed, and the fish body color dataset includes fish body images and true value labels corresponding to the fish body images, and includes: Video data of fish bodies in a water area is captured; The captured video data is intercepted to obtain fish body images; The fish body images are manually labeled to obtain the true value labels corresponding to the fish body images; The fish body images and the true value labels corresponding to the fish body images constitute the fish body color dataset. 4.The fish body color recognition method based on the fish body color recognition model according to claim 3, characterized in that, Further comprising: The fish body color recognition model is tested by using a test set, and the test set includes fish body images and corresponding true value labels remaining in the fish body color dataset after the training set is extracted; During the testing, the fish body images and the true value labels in the test set are taken as inputs, and a fish body color test result is outputted, and an intersection over union of the fish body color test result and the true value labels corresponding to the test set is calculated; Based on the intersection over union, whether the performance of the fish body color recognition model is better than the performance of the YOLOv5s network model is determined by taking precision, recall, first average precision, and second average precision as measurement indexes.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the fish body color recognition method based on the fish body color recognition model according to any one of claims 1 to 4. 6.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the fish body color recognition method based on the fish body color recognition model according to any one of claims 1 to 4.

7. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the fish body color recognition method based on the fish body color recognition model according to any one of claims 1 to 4.

Images