A method, device and storage medium for detecting fish descaling effect

The process of fish scale removal rate detection images through neural network models solves the problems of low detection accuracy and high cost in the prior art, and realizes an efficient and economical solution for fish scale removal rate detection.

CN113763347BActive Publication Date: 2024-11-19FISHERY MACHINERY & INSTR RES INST CHINESE ACADEMY OF FISHERY SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111018234.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-01
Publication Date
2024-11-19
Estimated Expiration
2041-09-01

AI Technical Summary

Technical Problem

In the prior art, the detection method of fish scale removal rate is low in detection accuracy and high cost, and lacks efficient and economical detection methods.

Method used

Using neural network model combined with image processing technology, by taking images of descaling fish bodies, using skeleton network, regional suggestion network and post-processing network to calculate descaling rate and damage rate, optimize model parameters through training data sets and verification data sets, and improve the Detection accuracy.

Benefits of technology

It realizes high accuracy and low cost for fish scale removal rate detection, providing an efficient and economical detection method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113763347B_ABST
    Figure CN113763347B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device and storage medium for detecting the descaling effect of fish, which are used to quickly and accurately detect the descaling effect of fish. The method for detecting the descaling effect of fish disclosed in the present application comprises: taking a first image of a descaled fish body; importing the first image of the descaled fish body into a predetermined model for detection; obtaining the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part according to the detection result; calculating the descaling rate and the damage rate according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part. The present application also provides a device and storage medium for detecting the descaling effect of fish.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device and storage medium for detecting fish descaling effects. Background Art

[0002] Descaling is a very important step in the pre-processing of freshwater fish. In the prior art, descaling equipment detects the descaling effect based on sensory evaluation or marks the area of ​​remaining scales after processing on coordinate paper, and then calculates the descaling rate. Alternatively, the descaling rate of fish can be detected by a three-dimensional laser scanner, but it is necessary to stick target stickers on the surface of the fish body and import the scanned model into the software ProE for calculation. The detection steps are numerous and the equipment used is expensive. At present, there is still a lack of a descaling rate detection method with high detection accuracy and low cost. Summary of the invention

[0003] In response to the above technical problems, the embodiments of the present application provide a method, device and storage medium for detecting the descaling effect of fish, so as to improve the accuracy of descaling rate detection and reduce the detection cost.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting the descaling effect of fish, comprising:

[0005] capturing a first image of the descaled fish body;

[0006] Importing the first image of the descaled fish body into a predetermined model for detection;

[0007] According to the detection results, the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body are obtained;

[0008] The descaling rate and the damage rate are calculated according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part.

[0009] Preferably, the capturing of the first image of the descaled fish body comprises:

[0010] Take an image every time T1, or

[0011] Capture images when a fish is detected passing by;

[0012] Wherein T1 is the preset shooting time interval.

[0013] The descaling rate is equal to the number of pixels of the scale-free part divided by the number of pixels of the fish body part, and the damage rate is equal to the number of pixels of the damaged part divided by the number of pixels of the fish body part.

[0014] Preferably, the predetermined model is determined according to the following steps:

[0015] Step A: Determine the training data set, validation data set and test data set;

[0016] Step B: constructing the first neural network model;

[0017] Step C: importing the training data set and the verification data set into the first neural network model for learning;

[0018] Step D: When the number of training times exceeds the preset number threshold, and the average precision obtained by training is greater than the preset precision threshold, then execute step E, otherwise adjust the parameters and continue to execute step C;

[0019] Step E: Use the test data set to test the model. When the average accuracy obtained by the test is less than the preset accuracy threshold, adjust the parameters to continue to execute step C; when the average accuracy obtained by the test is greater than or equal to the preset accuracy threshold, stop training to obtain the predetermined model.

[0020] Preferably, the determining of the training data set, the validation data set and the test data set comprises:

[0021] capturing a second image of the descaled fish;

[0022] Marking the second image into a fish scale-free part, a damaged part, and a fish body part to obtain a first data set;

[0023] The first data set is divided into a training data set, a validation data set, and a test data set.

[0024] As a preferred example, the preset number threshold is 200 times, and the accuracy threshold is 90%.

[0025] Preferably, the first neural network model includes:

[0026] The first neural network model includes a skeleton network, a region proposal network and a post-processing network;

[0027] The skeleton network is connected to the region proposal network, and the region proposal network is connected to the post-processing network;

[0028] The skeleton network is used to extract feature images, generate anchor frames according to the feature images, and input the anchor frames into the region proposal network;

[0029] The region proposal network is used to perform binary classification and bounding box regression according to the input anchor boxes, delete useless anchor boxes, and input the remaining anchor boxes into the post-processing network;

[0030] The post-processing network performs classification, boundary box regression and mask generation according to the input anchor box.

[0031] The skeleton network includes:

[0032] The skeleton network is divided into C layer, M layer and P layer from left to right;

[0033] The C layer is a residual network, including 5 layers, which are divided into C1 layer, C2 layer, C3 layer, C4 layer and C5 layer from bottom to top, and C1 layer is the input layer of the first image;

[0034] The M layer is an FPN network, which includes 5 layers, which are divided into M1 layer, M2 layer, M3 layer, M4 layer and M5 layer from bottom to top;

[0035] The P layer is a horizontal connection network, including 5 layers, which are divided into P2 layer, P3 layer, P4 layer, P5 layer and P6 layer from bottom to top;

[0036] The input of the C2 layer is the output of the C1 layer, the input of the C3 layer is the output of the C2 layer, the input of the C4 layer is the output of the C3 layer, and the input of the C5 layer is the output of the C4 layer;

[0037] The input of the M5 layer is the output of the C5 layer, the input of the M4 layer is the output of the M5 layer, the input of the M3 layer is the output of the M4 layer, the input of the M2 layer is the output of the M3 layer, and the input of the M1 layer is the output of the M2 layer;

[0038] The input of the P6 layer is the output of the M5 layer, the input of the P5 layer is the output of the M5 layer, the input of the P4 layer is the sum of the output of the C4 layer and the output of the M4 layer, the input of the P3 layer is the sum of the output of the C3 layer and the output of the M3 layer, and the input of the P2 layer is the sum of the output of the C2 layer and the output of the M2 layer.

[0039] Specifically, in the C layer, the resolution of the feature map output by each layer is one quarter of the input. In the M layer, the output of the M5 layer is the convolution calculation of the input, and the output of the other M layers is the upsampling of the input. In the P layer, the output of the P6 layer is the maximum pooling of the input, and the output of the other P layers is the convolution calculation of the input; wherein the output of the P6 is used as the input of the region proposal network, and the output of the other P layers is output as a feature map and input into the post-processing network.

[0040] Preferably, the C1 layer, C2 layer, C3 layer and C4 layer further include an attention module;

[0041] The loss functions of the C1 layer, C2 layer, C3 layer and C4 layer are:

[0042] L2=L cls +L box +Lmask +L edge ,

[0043] Among them, L edge is the marginal loss, and:

[0044]

[0045] L cls is the classification loss

[0046] L box is the bounding box loss

[0047] L mask is the mask loss

[0048] y is the marginal true value

[0049] y^ is the predicted margin

[0050] Preferably, the attention module is used to:

[0051] Each feature map channel is multiplied by the corresponding weight coefficient;

[0052] Among them, the weight coefficient of the qth channel is:

[0053] z q =σ(W 1q P1+W 2q P2+b q ),

[0054] Among them, z q is the weight coefficient of the qth channel;

[0055] W 1q is the qth element of the sequence W1 obtained through learning;

[0056] W 2q is the qth element of the sequence W2 obtained through learning;

[0057] b q is the qth element of the sequence b obtained through learning;

[0058] x is an arbitrary value.

[0059] Using the method provided by the present invention, before detection, a detection model is first constructed, an attention module is inserted into the residual network of the detection model, and a weight coefficient is added to each feature map channel. More important channels have larger weight coefficients, so that the network pays more attention to more important channels. After the model is constructed, the model is trained with a training data set and a verification data set to obtain a monitoring model with a higher detection accuracy, and then the descaled picture of the fish to be detected is input into the model for detection.

[0060] In a second aspect, the present application also provides a device for detecting fish descaling effect, comprising:

[0061] A model building module is configured to build a neural network model for monitoring descaling effect;

[0062] An image capturing module is configured to capture a first image of the descaled fish body, and import the first image of the descaled fish body into a predetermined neural network model for detection;

[0063] The result calculation module is configured to obtain the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part according to the detection results; and calculate the descaling rate and the damage rate according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part.

[0064] In a third aspect, an embodiment of the present application further provides a fish descaling effect detection device, comprising: a memory, a processor, and a user interface;

[0065] The memory is used to store computer programs;

[0066] The user interface is used to interact with the user;

[0067] The processor is used to read the computer program in the memory. When the processor executes the computer program, the fish descaling effect detection method provided by the present invention is implemented.

[0068] In a fourth aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and when the processor executes the computer program, the fish descaling effect detection method provided by the present invention is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 A schematic diagram of a method for detecting fish descaling rate provided in an embodiment of the present application;

[0071] Figure 2 A schematic diagram of the detection model training process provided in an embodiment of the present application;

[0072] Figure 3 A schematic diagram of the detection model structure provided in the embodiment of the present application;

[0073] Figure 4 A schematic diagram of a region of interest ROI provided in an embodiment of the present application;

[0074] Figure 5 A schematic diagram of the attention module structure provided in an embodiment of the present application;

[0075] Figure 6 A schematic diagram for comparing residual networks provided in an embodiment of the present application;

[0076] Figure 7 A schematic diagram of the structure of a descaling rate detection device provided in an embodiment of the present application;

[0077] Figure 8 A schematic diagram of the structure of another descaling rate detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] The following are explanations of some of the terms that appear in the text:

[0080] 1. In the embodiments of the present invention, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0081] 2. In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0082] 3. CNN,

[0083] 4. ROI (Region of Interest) - Region of Interest; FPN (Feature Pyramid Networks) - Feature Pyramid Network; RPN (Region Proposal Networks) - Region Proposal Network

[0084] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0085] It should be noted that the order in which the embodiments of the present application are presented only represents the order of precedence of the embodiments, and does not represent the advantages or disadvantages of the technical solutions provided by the embodiments.

[0086] Embodiment 1

[0087] See also Figure 1 , a schematic diagram of a method for detecting fish descaling effect provided in an embodiment of the present application, such as Figure 1 As shown, the method includes steps S101 to S104:

[0088] S101, capturing a first image of a descaled fish body;

[0089] S102, importing the first image of the descaled fish body into a predetermined model for detection;

[0090] S103, obtaining the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part according to the detection result;

[0091] S104, calculating the descaling rate and the damage rate according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part.

[0092] In the embodiment of the present invention, the scaled fish body refers to a fish that has been descaled, and the descaling process is to remove the scales on the fish body.

[0093] In the embodiment of the present invention, the method for capturing the first image of the descaled fish body may include the following method 1 or method 2:

[0094] Method 1: taking a first image of a descaled fish body at intervals of time T1;

[0095] T1 is a preset shooting time interval. For example, an image of a descaled fish body is taken every 2 seconds, and the captured image is imported into the trained model for detection.

[0096] Method 2: When a fish body is detected passing by, a first image of the descaled fish body is captured. For example, a photoelectric sensor is set up to capture an image when a fish body is detected passing by, and the captured image is imported into the trained model for detection.

[0097] In the embodiment of the present invention, the descaling rate is equal to the number of pixels of the scale-free part divided by the number of pixels of the fish body part, and the damage rate is equal to the number of pixels of the damaged part divided by the number of pixels of the fish body part, that is:

[0098]

[0099]

[0100] The descaling rate may be expressed as a percentage or a decimal, which is not limited in this embodiment.

[0101] In the embodiment of the present invention, before the first image of the descaled fish body is imported into a predetermined model for detection, a detection model needs to be constructed. As a preferred example, Figure 2 As shown, the predetermined model is determined according to the following steps:

[0102] Step A: Determine the training data set, validation data set and test data set;

[0103] Step B: constructing the first neural network model;

[0104] Step C: importing the training data set and the verification data set into the first neural network model for learning;

[0105] Step D: When the number of training times exceeds the preset number threshold, and the average precision obtained by training is greater than the preset precision threshold, then execute step E, otherwise adjust the parameters and continue to execute step C;

[0106] Step E: Use the test data set to test the model. When the average accuracy obtained by the test is less than the preset accuracy threshold, adjust the parameters to continue to execute step C; when the average accuracy obtained by the test is greater than or equal to the preset accuracy threshold, stop training to obtain the predetermined model.

[0107] Preferably, the above step A may include:

[0108] capturing a second image of the descaled fish;

[0109] Marking the second image into a fish scale-free part, a damaged part, and a fish body part to obtain a first data set;

[0110] The first data set is divided into a training data set, a validation data set, and a test data set.

[0111] As a preferred example, further, in order to unify the format, all the second image sizes are set to a uniform size. For example, all the second image sizes are unified to a size of 640 pixels × 480 pixels, and the scale-free part, the damaged part, and the fish body part are marked. The data set is divided into a training set, a validation set, and a test set in a ratio of 3:1:1. For example, a total of 1,000 second images of scaled fish are collected, the training set is 600 auxiliary images, the validation set is 200 images, and the test set is 200 images.

[0112] It should be noted that the scale-free part can be marked using existing marking software, which is not limited in this embodiment.

[0113] As a preferred example, in the above step D, the preset number threshold is 200 times, and the accuracy threshold is 90%. For example: put the training set and the validation set into the network for learning, and stop training when the mean average precision mAP (mean average precision) is greater than 90%; and set the maximum number of training epochs (when all samples are trained once, it is called an epoch) to 200, and force the training to stop after 200 epochs. If the mAP is less than 90% after the training, return to step C, adjust the parameters, and train again.

[0114] As a preferred example, in the above step E, based on the completion of step D, the model is verified using the test set data. For example, when the mAP on the test set is less than 90%, return to step C, adjust the parameters, and train again. If the mAP of the model on the test set is greater than 90%, it is considered the final model. That is, stop training to obtain the predetermined model.

[0115] In the embodiment of the present invention, the first neural network is as follows: Figure 3 As shown, the first neural network includes:

[0116] The first neural network model includes a skeleton network, a region proposal network and a post-processing network;

[0117] The skeleton network is connected to the region proposal network, and the region proposal network is connected to the post-processing network;

[0118] The skeleton network is used to extract feature images, generate anchor frames according to the feature images, and input the anchor frames into the region proposal network;

[0119] The region proposal network is used to perform binary classification and bounding box regression according to the input anchor boxes, delete useless anchor boxes, and input the remaining anchor boxes into the post-processing network;

[0120] The post-processing network performs classification, boundary box regression and mask generation according to the input anchor box.

[0121] It should be noted that, in the embodiment of the present invention, the skeleton network is Figure 3 The backbone network in the region proposal network is Figure 3 The RPN network (Region Proposal Network, referred to as RPN) in Figure 3 The networks other than the skeleton network and the region proposal network are post-processing networks.

[0122] It should be noted that in the embodiment of the present invention, the first neural network is also called Mask R-CNN. Specifically, its composition and working process are as follows:

[0123] The structure of Mask R-CNN is as follows Figure 3 As shown in the figure, the input image first passes through the backbone network to extract the feature map. Each point on the feature map will generate multiple anchor boxes and enter the region proposal network (RPN) for binary classification and bounding box regression. The useless anchor boxes are removed, and the region of interest alignment (ROI Align) unifies the size of the remaining region of interest (ROI) and corresponds to the original image. Finally, classification, bounding box regression and mask generation are performed on the obtained region of interest (ROI). Figure 4 As shown in the figure, the dotted part represents the feature map, and the solid part represents the ROI (region of interest). Here, the ROI is divided into 2×2 cells. Assuming that the number of sampling points is 4, each cell is divided into 2×2 small cells, and the center point is the sampling point. However, the coordinates of the sampling points are generally floating point numbers, so bilinear interpolation is used to obtain the value of the center point, and then the maximum pooling is performed on the four sampling points in each cell to obtain the result of ROI Align (region of interest align).

[0124] The backbone network used by Mask R-CNN consists of three parts: bottom-up, top-down, and lateral connections. Bottom-up means using the residual network (ResNet) to extract feature maps of different scales from the original image. The C1 to C5 layers on the left are the structural layers of ResNet. The stride (Stride) of each layer is 2, that is, the resolution of the feature map output at each stage is one-fourth of the previous layer. The middle M5 layer is obtained by calculating the C5 layer through a 1x1 convolution. The subsequent bottom-up is to use the Feature Pyramid Networks (FPN) network to gradually upsample the M5 layer to obtain the M4 to M1 layers. The final lateral connection is that each layer of C2, C3, C4, and C5 undergoes a 1x1 convolution calculation, and then adds them to M2, M3, M4, and M5 respectively to obtain P2, P3, P4, and P5, and a 3x3 convolution is used to eliminate the aliasing effect. P6 is obtained by the maximum pooling of M5. P6 is only used in the subsequent RPN network and is not output as a feature map. The feature extraction network combining ResNet and FPN improves the detection ability of small-scale objects while keeping the computational complexity unchanged.

[0125] Anchors are automatically generated on the feature maps extracted by Backbone. A branch in the RPN classifies the anchors into two categories: positive examples that include the target object and negative examples that do not. Another branch is used to calculate the bounding box regression offset of the anchor to obtain an accurate proposal. After eliminating anchors that are beyond the boundary or too small, non-maximum suppression is used to filter the anchors that contain the target object to obtain the ROI.

[0126] The ROI is mapped to a fixed size through ROI Align, and the obtained ROI is used for bounding box regression, classification, and mask generation through three branches, and finally the results of target recognition and instance segmentation are obtained at the same time.

[0127] The skeleton network of the embodiment of the present application is further described as follows:

[0128] The skeleton network is divided into C layer, M layer and P layer from left to right;

[0129] The C layer is a residual network, including 5 layers, which are divided into C1 layer, C2 layer, C3 layer, C4 layer and C5 layer from bottom to top, and C1 layer is the input layer of the first image;

[0130] The M layer is an FPN network, which includes 5 layers, which are divided into M1 layer, M2 layer, M3 layer, M4 layer and M5 layer from bottom to top;

[0131] The P layer is a horizontal connection network, including 5 layers, which are divided into P2 layer, P3 layer, P4 layer, P5 layer and P6 layer from bottom to top;

[0132] The input of the C2 layer is the output of the C1 layer, the input of the C3 layer is the output of the C2 layer, the input of the C4 layer is the output of the C3 layer, and the input of the C5 layer is the output of the C4 layer;

[0133] The input of the M5 layer is the output of the C5 layer, the input of the M4 layer is the output of the M5 layer, the input of the M3 layer is the output of the M4 layer, the input of the M2 layer is the output of the M3 layer, and the input of the M1 layer is the output of the M2 layer;

[0134] The input of the P6 layer is the output of the M5 layer, the input of the P5 layer is the output of the M5 layer, the input of the P4 layer is the sum of the output of the C4 layer and the output of the M4 layer, the input of the P3 layer is the sum of the output of the C3 layer and the output of the M3 layer, and the input of the P2 layer is the sum of the output of the C2 layer and the output of the M2 layer.

[0135] As a preferred example, in the C layer, the resolution of the feature map output by each layer is one quarter of the input.

[0136] As a preferred example, among the M layers, the output of the M5 layer is the convolution calculation of the input, and the outputs of the other M layers are upsampling of the input.

[0137] As a preferred example, in the P layers, the output of the P6 layer is the maximum pooling of the input, and the outputs of other P layers are the convolution calculations of the input;

[0138] Among them, the output of P6 is used as the input of the region proposal network, and the outputs of other P layers are output as feature maps and input into the post-processing network.

[0139] As a preferred example, in the skeleton network of the embodiment of the present invention, the C2 layer, the C3 layer and the C4 layer also include an attention module, such as Figure 5 In the embodiment of the present invention, an attention module is added to the feature extraction network, and a weight is added to each feature map channel. Important channels have a larger weight coefficient, so that the network pays more attention to important channels.

[0140] like Figure 5As shown in the figure, the feature map U extracted by the previous network has a size of H×W×Q, where H is the height, W is the width, and Q is the number of channels. Through average pooling calculation and maximum pooling calculation, two series of sizes of 1×1×Q are obtained respectively. The average pooling calculation formula is as follows:

[0141]

[0142] U=[u1,u2,...,u C ]......Formula 2

[0143] U Q represents the Q-th matrix constituting U, and P1 represents the calculation result of average pooling.

[0144] The maximum pooling calculation formula is as follows:

[0145] P2=F max pooling (u C )=max(u C )......Formula 3

[0146] P2 represents the maximum pooling calculation result.

[0147] The obtained P1 and P2 are then passed through a linear regression equation to obtain the final weight series:

[0148] Among them, the weight coefficient of the qth channel is:

[0149] z q =σ(W 1q P1+W 2q P2+b q ),

[0150] Among them, z q is the weight coefficient of the qth channel;

[0151] W 1q is the qth element of the sequence W1 obtained through learning;

[0152] W 2q is the qth element of the sequence W2 obtained through learning;

[0153] b q is the qth element of the sequence b obtained through learning;

[0154] x is an arbitrary value.

[0155] The calculated weights are weighted to the previous feature map channel by channel through multiplication.

[0156] As a preferred example, in the embodiment of the present invention, ResNet50 and FPN are used as the backbone, the above-mentioned attention module is inserted into the residual block (ResidualBlock) in ResNet50, and a layer reduction process is performed. Figure 6 As shown in the figure, the attention module is added to C1 to C4 of the above skeleton network, and C5 remains unchanged. Before the attention module is inserted, the loss function is:

[0157] L1=L cls +L box +L mask ...Formula 5

[0158] After inserting the attention module, the marginal loss is added:

[0159]

[0160] Therefore, after inserting the attention module, the loss function is:

[0161] L2=L cls +L box +L mask +L edge ...Formula 7

[0162] Among them, L cls is the classification loss

[0163] L box is the bounding box loss

[0164] L mask is the mask loss

[0165] y is the marginal true value

[0166] y^ is the predicted margin

[0167] Using the method provided by the present invention, before detection, a detection model is first constructed, an attention module is inserted into the residual network of the detection model, and a weight coefficient is added to each feature map channel. More important channels have larger weight coefficients, so that the network pays more attention to more important channels. After the model is constructed, the model is trained with a training data set and a verification data set to obtain a monitoring model with a higher detection accuracy, and then the descaled picture of the fish to be detected is input into the model for detection.

[0168] Embodiment 2

[0169] Based on the same inventive concept, the embodiment of the present invention also provides a fish descaling effect detection device, such as Figure 7 As shown, the device comprises:

[0170] A model building module 701 is configured to build a neural network model for monitoring descaling effect;

[0171] An image capturing module 702 is configured to capture a first image of a descaled fish body, and import the first image of the descaled fish body into a predetermined neural network model for detection;

[0172] The result calculation module 703 is configured to obtain the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part according to the detection results; and calculate the descaling rate and the damage rate according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part.

[0173] Preferably, the model building module 701 is further used to build a neural network model for monitoring the descaling effect according to the following steps:

[0174] Step A: Determine the training data set, validation data set and test data set;

[0175] Step B: constructing the first neural network model;

[0176] Step C: importing the training data set and the verification data set into the first neural network model for learning;

[0177] Step D: When the number of training times exceeds the preset number threshold, and the average precision obtained by training is greater than the preset precision threshold, then execute step E, otherwise adjust the parameters and continue to execute step C;

[0178] Step E: Use the test data set to test the model. When the average accuracy obtained by the test is less than the preset accuracy threshold, adjust the parameters to continue to execute step C; when the average accuracy obtained by the test is greater than or equal to the preset accuracy threshold, stop training to obtain the predetermined model.

[0179] Preferably, the model building module 701 is further used to determine a training data set, a validation data set and a test data set according to the following steps:

[0180] capturing a second image of the descaled fish;

[0181] Marking the second image into a fish scale-free part, a damaged part, and a fish body part to obtain a first data set;

[0182] The first data set is divided into a training data set, a validation data set, and a test data set.

[0183] Preferably, the first neural network model constructed by the model construction module 701 includes:

[0184] The first neural network model includes a skeleton network, a region proposal network and a post-processing network;

[0185] The skeleton network is connected to the region proposal network, and the region proposal network is connected to the post-processing network;

[0186] The skeleton network is used to extract feature images, generate anchor frames according to the feature images, and input the anchor frames into the region proposal network;

[0187] The region proposal network is used to perform binary classification and bounding box regression according to the input anchor boxes, delete useless anchor boxes, and input the remaining anchor boxes into the post-processing network;

[0188] The post-processing network performs classification, boundary box regression and mask generation according to the input anchor box.

[0189] Preferably, the skeleton network comprises:

[0190] The skeleton network is divided into C layer, M layer and P layer from left to right;

[0191] The C layer is a residual network, including 5 layers, which are divided into C1 layer, C2 layer, C3 layer, C4 layer and C5 layer from bottom to top, and C1 layer is the input layer of the first image;

[0192] The M layer is an FPN network, which includes 5 layers, which are divided into M1 layer, M2 layer, M3 layer, M4 layer and M5 layer from bottom to top;

[0193] The P layer is a horizontal connection network, including 5 layers, which are divided into P2 layer, P3 layer, P4 layer, P5 layer and P6 layer from bottom to top;

[0194] The input of the C2 layer is the output of the C1 layer, the input of the C3 layer is the output of the C2 layer, the input of the C4 layer is the output of the C3 layer, and the input of the C5 layer is the output of the C4 layer;

[0195] The input of the M5 layer is the output of the C5 layer, the input of the M4 layer is the output of the M5 layer, the input of the M3 layer is the output of the M4 layer, the input of the M2 layer is the output of the M3 layer, and the input of the M1 layer is the output of the M2 layer;

[0196] The input of the P6 layer is the output of the M5 layer, the input of the P5 layer is the output of the M5 layer, the input of the P4 layer is the sum of the output of the C4 layer and the output of the M4 layer, the input of the P3 layer is the sum of the output of the C3 layer and the output of the M3 layer, and the input of the P2 layer is the sum of the output of the C2 layer and the output of the M2 layer.

[0197] Preferably, the C1 layer, C2 layer, C3 layer and C4 layer further include an attention module;

[0198] The loss functions of the C1 layer, C2 layer, C3 layer and C4 layer are:

[0199] L=L cls +L box +L mask +L edge ,

[0200] Among them, L edge is the marginal loss, and:

[0201]

[0202] L cls is the classification loss

[0203] L box is the bounding box loss

[0204] L mask is the mask loss

[0205] y is the marginal true value

[0206] y^ is the predicted margin

[0207] It should be noted that the model building module 701 provided in this embodiment can realize all the functions included in building the first neural network model in the first embodiment, solve the same technical problems, and achieve the same technical effects, and will not be repeated here;

[0208] It should be noted that the image capture module 702 provided in this embodiment can realize all the functions of S101 in the first embodiment, solve the same technical problems, and achieve the same technical effects, and will not be described in detail here.

[0209] It should be noted that the result calculation module 703 provided in this embodiment can realize all the functions of S103 and S104 in the first embodiment, solve the same technical problems, and achieve the same technical effects, and will not be described in detail here;

[0210] It should be noted that the device provided in Example 2 and the method provided in Example 1 belong to the same inventive concept, solve the same technical problems, and achieve the same technical effects. The device provided in Example 2 can implement all the methods of Example 1, and the similarities will not be repeated here.

[0211] Embodiment 3

[0212] Based on the same inventive concept, the embodiment of the present invention also provides a fish descaling effect detection device, such as Figure 8 As shown, the device comprises:

[0213] It includes a memory 802, a processor 801 and a user interface 803;

[0214] The memory 802 is used to store computer programs;

[0215] The user interface 803 is used to interact with the user;

[0216] The processor 801 is configured to read the computer program in the memory 802. When the processor 801 executes the computer program, the following is achieved:

[0217] capturing a first image of the descaled fish body;

[0218] Importing the first image of the descaled fish body into a predetermined model for detection;

[0219] According to the detection results, the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body are obtained;

[0220] The descaling rate and the damage rate are calculated according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part.

[0221] Among them, Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 801 and memory represented by memory 802. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The processor 801 is responsible for managing the bus architecture and general processing, and the memory 802 may store data used by the processor 801 when performing operations.

[0222] The processor 801 may be a CPU, an ASIC, an FPGA or a CPLD, and the processor 801 may also adopt a multi-core architecture.

[0223] When the processor 801 executes the computer program stored in the memory 802, the method for detecting the descaling effect of any fish in the first embodiment is implemented.

[0224] It should be noted that the device provided in Example 3 and the method provided in Example 1 belong to the same inventive concept, solve the same technical problems, and achieve the same technical effects. The device provided in Example 3 can implement all the methods of Example 1, and the similarities will not be repeated here.

[0225] The present application also proposes a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and when the processor executes the computer program, the method for detecting the descaling effect of any fish in the first embodiment is implemented.

[0226] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0227] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0228] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0229] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0230] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting the descaling effect of fish, characterized in that: include: capturing a first image of the descaled fish body; Importing the first image of the descaled fish body into a predetermined model for detection; According to the detection results, the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body are obtained; Calculating the descaling rate and the damage rate according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part; The predetermined model is determined according to the following steps: Step A: Determine the training data set, validation data set and test data set; Step B: constructing the first neural network model; Step C: importing the training data set and the verification data set into the first neural network model for learning; Step D: When the number of training times exceeds the preset number threshold, and the average precision obtained by training is greater than the preset precision threshold, then execute step E, otherwise adjust the parameters and continue to execute step C; Step E: Use the test data set to test the model. When the average accuracy obtained by the test is less than the preset accuracy threshold, adjust the parameters and continue to execute step C; When the average accuracy value obtained by the test is greater than or equal to the preset accuracy threshold, stop training to obtain the predetermined model; The first neural network model includes: The first neural network model includes a skeleton network, a region proposal network and a post-processing network; The skeleton network is connected to the region proposal network, and the region proposal network is connected to the post-processing network; The skeleton network is used to extract feature images, generate anchor frames according to the feature images, and input the anchor frames into the region proposal network; The region proposal network is used to perform binary classification and bounding box regression according to the input anchor boxes, delete useless anchor boxes, and input the remaining anchor boxes into the post-processing network; The post-processing network performs classification, boundary box regression and mask generation according to the input anchor box.

2. The method according to claim 1, characterized in that The capturing of the first image of the descaled fish body comprises: Take an image every time T1, or Capture images when a fish is detected passing by; Wherein T1 is the preset shooting time interval.

3. The method according to claim 1, characterized in that: The descaling rate is equal to the number of pixels of the scale-free part divided by the number of pixels of the fish body part, and the damage rate is equal to the number of pixels of the damaged part divided by the number of pixels of the fish body part.

4. The method according to claim 1, characterized in that Determining the training data set, the validation data set, and the test data set includes: capturing a second image of the descaled fish; Marking the second image into a fish scale-free part, a damaged part, and a fish body part to obtain a first data set; The first data set is divided into a training data set, a validation data set, and a test data set.

5. The method according to claim 1, characterized in that The preset number threshold is 200 times, and the accuracy threshold is 90%.

6. The method according to claim 1, characterized in that The skeleton network includes: The skeleton network is divided into C layer, M layer and P layer from left to right; The C layer is a residual network, including 5 layers, which are divided into C1 layer, C2 layer, C3 layer, C4 layer and C5 layer from bottom to top, and C1 layer is the input layer of the first image; The M layer is an FPN network, which includes 5 layers, which are divided into M1 layer, M2 layer, M3 layer, M4 layer and M5 layer from bottom to top; The P layer is a horizontal connection network, including 5 layers, which are divided into P2 layer, P3 layer, P4 layer, P5 layer and P6 layer from bottom to top; The input of the C2 layer is the output of the C1 layer, the input of the C3 layer is the output of the C2 layer, the input of the C4 layer is the output of the C3 layer, and the input of the C5 layer is the output of the C4 layer; The input of the M5 layer is the output of the C5 layer, the input of the M4 layer is the output of the M5 layer, the input of the M3 layer is the output of the M4 layer, the input of the M2 layer is the output of the M3 layer, and the input of the M1 layer is the output of the M2 layer; The input of the P6 layer is the output of the M5 layer, the input of the P5 layer is the output of the M5 layer, the input of the P4 layer is the sum of the output of the C4 layer and the output of the M4 layer, the input of the P3 layer is the sum of the output of the C3 layer and the output of the M3 layer, and the input of the P2 layer is the sum of the output of the C2 layer and the output of the M2 layer.

7. The method according to claim 6, characterized in that In the C layer, the resolution of the feature map output by each layer is one quarter of the input.

8. The method according to claim 6, characterized in that Among the M layers, the output of the M5 layer is the convolution calculation of the input, and the output of the other M layers is the upsampling of the input.

9. The method according to claim 6, characterized in that In the P layer, the output of the P6 layer is the maximum pooling of the input, and the output of other P layers is the convolution calculation of the input; Among them, the output of P6 is used as the input of the region proposal network, and the outputs of other P layers are output as feature maps and input into the post-processing network.

10. The method according to claim 6, characterized in that The C1 layer, C2 layer, C3 layer and C4 layer also include an attention module; The loss functions of the C1 layer, C2 layer, C3 layer and C4 layer are: L2=L cls +L box +L mask +L edge , Among them, L edge is the marginal loss, and: L cls is the classification loss L box is the bounding box loss L mask is the mask loss y is the marginal true value y^ is the predicted margin.

11. The method according to claim 10, characterized in that The attention module is used to: Each feature map channel is multiplied by the corresponding weight coefficient; Among them, the weight coefficient of the qth channel is: WITH Q =σ(W 1q P1+W 2q P2+b q ), Among them, Z q is the weight coefficient of the qth channel; W 1q is the qth element of the sequence W1 obtained through learning; W 2q is the qth element of the sequence W2 obtained through learning; b q is the qth element of the sequence b obtained through learning; P1 represents the calculation result of average pooling; P2 represents the maximum pooling calculation result.

12. A fish descaling effect detection device, characterized in that: include: A model building module is configured to build a neural network model for monitoring descaling effect; An image capturing module is configured to capture a first image of the descaled fish body, and import the first image of the descaled fish body into a predetermined neural network model for detection; A result calculation module is configured to obtain the number of pixels of the fish scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part according to the detection result; Calculating the descaling rate and the damage rate according to the number of pixels of the scale-free part, the number of pixels of the damaged part and the number of pixels of the fish body part; The predetermined neural network model is determined according to the following steps: Step A: Determine the training data set, validation data set and test data set; Step B: constructing the first neural network model; Step C: importing the training data set and the verification data set into the first neural network model for learning; Step D: When the number of training times exceeds the preset number threshold, and the average precision obtained by training is greater than the preset precision threshold, then execute step E, otherwise adjust the parameters and continue to execute step C; Step E: Use the test data set to test the model. When the average accuracy obtained by the test is less than the preset accuracy threshold, adjust the parameters and continue to execute step C; When the average accuracy value obtained by the test is greater than or equal to the preset accuracy threshold, stop training to obtain the predetermined model; The first neural network model includes: The first neural network model includes a skeleton network, a region proposal network and a post-processing network; The skeleton network is connected to the region proposal network, and the region proposal network is connected to the post-processing network; The skeleton network is used to extract feature images, generate anchor frames according to the feature images, and input the anchor frames into the region proposal network; The region proposal network is used to perform binary classification and bounding box regression according to the input anchor boxes, delete useless anchor boxes, and input the remaining anchor boxes into the post-processing network; The post-processing network performs classification, boundary box regression and mask generation according to the input anchor box.

13. A fish descaling effect detection device, characterized in that: including a memory, a processor and a user interface; The memory is used to store computer programs; The user interface is used to interact with the user; The processor is used to read the computer program in the memory, and when the processor executes the computer program, the fish descaling effect detection method according to any one of claims 1 to 11 is implemented.

14. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and when the processor executes the computer program, the fish descaling effect detection method according to any one of claims 1 to 11 is implemented.