Automatic shrimp fry counting method and system based on visual detection and fusion of multiple dynamic thresholds

Abstract

The present invention discloses a method and system for automatic counting of shrimp seedlings based on visual detection fusion of multiple dynamic thresholds, the method comprising: obtaining a shrimp seedling image data set; constructing a target detection model; training the target detection model using the shrimp seedling image data set to obtain a shrimp seedling target detection model; obtaining shrimp seedling images to be counted, and preprocessing the shrimp seedling images to be counted; inputting the preprocessed shrimp seedling images to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map; processing the first shrimp seedling area map to obtain a second shrimp seedling area map; based on the detection results of the shrimp seedlings, selecting shrimp seedling samples, and performing multi-dynamic threshold processing on the second shrimp seedling area map; using an optimized matching strategy to process the second shrimp seedling area map and the shrimp seedling detection results after multi-dynamic threshold processing, to realize automatic counting of shrimp seedlings. The present invention can realize automatic, rapid, accurate, and non-dehydrated counting of shrimp seedlings of different ages under complex backgrounds, thereby improving counting efficiency.

Description

Technical Field

[0001] The invention relates to a method and system for automatically counting shrimp fry based on visual detection and fusion of multiple dynamic thresholds, belonging to the technical field of aquatic product image processing. Background Art

[0002] Shrimp fry counting is an important part of shrimp fry breeding and trading, and is the basis and basis for shrimp fry trading parties to reach a deal. At present, shrimp fry counting mainly relies on the method of manual sampling and counting. After the shrimp fry are evenly packed and labeled, the two parties to the transaction each give a number, and the sum of the two numbers or the sum of the two numbers minus the total number of bags is used as the sampling bag number. The bag of shrimp fry is then evenly divided into 10 parts, and one part is randomly selected for counting, and finally the total number of shrimp fry is estimated [1]. For sampling counting methods, there is a frame-format fish (shrimp) counting method [2]. The fish (shrimp) fry is placed in a square composed of several horizontal and vertical lines, and two or three squares are randomly selected for counting, and then the average value is taken, and then the number of fish (shrimp) fry is calculated by multiplying it by the number of squares. There is also a counting method based on the age, body length, and unit weight of the shrimp fry to obtain the number of tails contained in 500g of shrimp fry [3]. Although such methods can guarantee a certain degree of credibility, they require a lot of manpower and material resources, and increase the time cost during the transaction and transportation process. Moreover, during the counting process, people are easily affected by the environment, feel tired, and make mistakes in memory and counting, which leads to deviations between the counting results and the actual number, and ultimately makes the counting results unconvincing to both parties of the transaction. At the same time, the shrimp fry often need to be dehydrated during manual counting, which often causes the death of some shrimp fry with poor vitality, which is not conducive to the subsequent breeding and sales of shrimp fry.

[0003] In view of the difficulty of traditional shrimp fry counting methods, some researchers have begun to design shrimp fry counting instruments based on photoelectric signals. For example, the channel-type shrimp fry counter [4] uses a multi-channel television scanning method to transmit the information to the microprocessor, and the counting signal is obtained by judging the image signal of each shrimp fry in each channel. There is also a shrimp fry counter combined with a weighing type [5]. It uses a capacitive weight sensor to convert the weight into an electrical frequency, and then uses a microprocessor to calculate the number of shrimp fry. [6] The shrimp fry weight information is obtained by using a scale plate and a pressure sensor, and then the age of the shrimp fry is input to obtain the weight of a single shrimp fry stored in the chip to estimate the number of shrimp fry in the scale plate. [7] The weight of the mixture of fish (shrimp) fry and water is continuously detected by combining a microcomputer with dynamic gravity measurement technology, and the weight and counting of the fish (shrimp) fry are carried out by controlling the water content when weighing the fish (shrimp) fry. [8] A fish (shrimp) counting machine is designed, which uses an optical fiber sensor to sense the number of fish (shrimp) fry in the box to achieve automatic counting.

[0004] With the rapid development of computer vision technology, some researchers have estimated the number of shrimp fry by using photos of shrimp fry. At present, the research on computer vision technology in shrimp fry counting mainly focuses on the following four aspects:

[0005] One is based on traditional image processing methods, such as [9] using the image restoration model TV-L1 to preprocess the image, then using the maximum inter-class variance method to obtain the optimal threshold for shrimp fry segmentation, and then estimating the number of shrimp fry based on the area of ​​the connected domain. [10-12] designed a device including a darkroom, an industrial camera and an auxiliary light source to capture shrimp fry images, which can obtain images of good quality, and use image processing and recognition algorithms to realize automatic counting of shrimp fry.

[13] A method combining Canny edge detection and Blob processing technology is used to search and count shrimp fry in the image. It is worth noting that the image processing-based method uses a threshold to process the entire image, and it is impossible to determine the specific processing area. At the same time, it has high requirements on the pixels and lighting of the image, and requires uniform backlight. Therefore, when processing the image, only plates and shrimp fry can appear, and there should be no interference from other background information, no light reflection, reflection, etc., and additional darkrooms and lights are required.

[0006] The second method is to use the method of generating density maps to estimate the number of shrimp fry. For example,

[14] designed a shrimp fry counter to take photos of shrimp fry, then used the improved VGG network to learn image features to generate an estimated density map, and finally used the density map to estimate the total number of shrimp fry in the entire field of view. It is worth noting that the method based on generating density maps requires a large amount of labeled data to train the model. The quality of the model directly affects the generation of the density map, and the results of shrimp fry counting are completely dependent on the quality of the generated density map, which can easily lead to inaccurate shrimp fry statistics. Moreover, only providing the density map and the total number of shrimp fry cannot provide a more intuitive result.

[0007] The third is the method based on target detection. This method identifies and detects the shrimp fry in the image, and then counts the number of target boxes obtained, which is the total number of shrimp fry.

[15] The YOLOv3 model is directly used to count shrimp fry, with an accuracy rate of 76.48%.

[16] Based on the YOLOv4 detection model, the size of shrimp fry is first counted by clustering method, and then the distribution characteristics of shrimp fry size are obtained, and the preset 8 anchor boxes are reduced to 4. At the same time, the attention module is added to the output stage of the YOLOv4 network to suppress unnecessary features, and a higher counting accuracy rate is achieved in its shrimp fry dataset. It is worth noting that the method based on target detection also requires a large amount of labeled data to train the model, and the detector has poor detection effect on shrimp fry dense areas, occlusion and other situations. Therefore, it is easy to miss detection and misdetection, resulting in inaccurate shrimp fry counting.

[0008] Fourth, based on the image segmentation method,

[17] designed a set of devices including containers, boxes, sensors and lights, and used the improved Mask R-CNN model to segment the area of ​​each shrimp fry in the image, and then calculated the number of each area to estimate the total number of shrimp fry, with an accuracy of 97.48%.

[18] used a two-stage Mask R-CNN to segment and count shrimp fry, and the counting accuracy of shrimp fry images with medium overlap was improved. It is worth noting that the image segmentation-based method also requires a large amount of labeled data to train the model. When the number of shrimp fry in the image is small or large, the segmentation effect is poor and the processing speed is slow.

[0009] References are as follows:

[0010] [1] Li Sheng, Things to note when buying white shrimp fry. Scientific Fish Farming, 2006(02): pp. 45-47.

[0011] [2] Zhang Kangde, Frame format fish (shrimp) fry counting method. Rural Aquaculture Technology, 2000 (No. 3): p. 13.

[0012] [3]Xue Zhining and Duan Jiwang, Study on the measurement of Japanese shrimp fry out of the pond by weight method. Hebei Fisheries, 2000 (No. 5): pp. 11-16.

[0013] [4] Li Yuzhen, Channel type fish and shrimp counter. Marine Technology, 1993(01): pp. 75-76.

[0014] [5] Huang Yingsheng and Xu Feili, YJ-2 fry counter. Fishery Machinery and Instruments, 1989 (No. 3): pp. 29-30, 20.

[0015] [6] He Puqing, Intelligent shrimp fry counter and counting method, 2020.

[0016] [7] Fu Xijin et al., Method and equipment for counting fish (shrimp) fry by weight, 1988.

[0017] [8] Qu Jiao, Fish (shrimp) seedling counting machine, 2014.

[0018] [9] Ji Yuyao et al., Shrimp fry counting method based on improved TV-L1 model. Journal of Qingdao University (Natural Science Edition), 2018 (Issue 4): pp. 62-68, 82.

[0019]

[10] Hu Lihua et al., A shrimp fry counting device and method, 2020.

[0020]

[11] Solahudin, M., W. Slamet and ASDwi, Vaname (Litopenaeus vannamei) Shrimp Fry Counting Based on Image Processing Method. IOP Conference Series: Earth and Environmental Science, 2018 (No.0): p.012014.

[0021]

[12] Kesvarakul, R., C. Chianrabutra and S. Chianrabutra. Baby ShrimpCounting via Automated Image Processing. in ICMLC 2017: Proceedings of the 9thInternational Conference on Machine Learning and Computing. 2017. Singapore, Singapore.

[0022]

[13] Awalludin,EA,et al.Combination of Canny Edge Detection and BlobProcessing Techniques for Shrimp Larvae Counting.in 2019IEEE InternationalConference on Signal and Image Processing Applications(ICSIPA).2019.KualaLumpur,Malaysia,Malaysia.

[0023]

[14] Fan Songwei, Lin Xiangyu and Zhou Ping. Research on automatic counting of shrimp fry based on improved convolutional neural network. Fisheries Modernization, 2020(Issue 6): pp. 35-41.

[0024]

[15] Armalivia, S., et al. Automatic Counting Shrimp Larvae Based YouOnly Look Once (YOLO). in 2021International Conference on Artificial Intelligence and Mechatronics Systems (AIMS). 2021. Bandung, Indonesia.

[0025]

[16] Yu Qiuyu, Research on intelligent identification algorithm of shrimp fry based on improved YOLOv4. Henan Science and Technology, 2021.40(06): pp. 25-28.

[0026]

[17] Khai, TH, et al., Underwater Fish Detection and Counting UsingMask Regional Convolutional Neural Network. Water, 2022 (No. 222): p. 222.

[0027]

[18] Nguyen,K.,et al.Two-Phase Instance Segmentation for WhitelegShrimp Larvae Counting.in 2020IEEE International Conference on Consumer Electronics(ICCE).2020.Las Vegas,NV,USA,USA. Summary of the invention

[0028] In view of this, the present invention provides a method and system for automatic counting of shrimp fry based on visual detection and fusion of multiple dynamic thresholds. By constructing a shrimp fry target detection model and utilizing multiple dynamic thresholds and optimizing matching strategy processing, it can solve the problems of missed detection of shrimp fry and slow counting of shrimp fry, thereby improving the accuracy and efficiency of shrimp fry counting.

[0029] The first object of the present invention is to provide a method for automatically counting shrimp fry based on visual detection and fusion of multiple dynamic thresholds.

[0030] The second object of the present invention is to provide an automatic shrimp fry counting system based on visual detection and fusion of multiple dynamic thresholds.

[0031] A third object of the present invention is to provide a computer device.

[0032] A fourth object of the present invention is to provide a storage medium.

[0033] The first object of the present invention can be achieved by adopting the following technical solutions:

[0034] A method for automatically counting shrimp fry based on visual detection and fusion of multiple dynamic thresholds, the method comprising:

[0035] Get the shrimp fry image dataset;

[0036] Build an object detection model;

[0037] The shrimp seedling image dataset is used to train and verify the target detection model, and the shrimp seedling target detection model is obtained;

[0038] Acquire the shrimp fry image to be counted, and pre-process the shrimp fry image to be counted;

[0039] Inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes the shrimp seedling detection result;

[0040] Performing background removal and grayscale processing on the first shrimp fry region map to obtain a second shrimp fry region map;

[0041] Based on the shrimp fry detection results, a complete shrimp fry sample is selected, and a multi-dynamic threshold processing is performed on the second shrimp fry area map;

[0042] The optimized matching strategy is used to process the second shrimp seedling area map and the shrimp seedling detection results after multi-dynamic threshold processing to realize automatic counting of shrimp seedlings.

[0043] Furthermore, the target detection model includes a backbone network, a neck structure and an output layer; the backbone network adopts a CSPDarknet architecture, the neck structure includes FPN and PAN layers, and the output layer corresponds to predicting targets of three different scales;

[0044] The FPN and PAN layers are added with a CBAM attention module, and the CBAM attention module is used to avoid interference from background features.

[0045] Furthermore, the CBAM attention module is used to avoid interference from background features, specifically including:

[0046] The CBAM attention module injects the attention map along the channel dimension and spatial dimension of the input feature map, and then multiplies the injected attention map by the input feature map to achieve adaptive feature refinement of the input feature map, thereby avoiding interference from background features.

[0047] Furthermore, in the process of training the target detection model using the shrimp seedling image dataset, a first loss function is used for regression prediction, and a second loss function is used for classification prediction;

[0048] The first loss function is as follows:

[0049] L α-IoU =1-IoU α

[0050] Among them, IoU represents the intersection-over-union ratio of the predicted box and the real box;

[0051] The second loss function is as follows:

[0052]

[0053] Among them, t[i] represents the true label value; o[i] represents the predicted value, which is 0 or 1.

[0054] Furthermore, the detection result based on the shrimp fry is used to select a complete shrimp fry sample and perform multi-dynamic threshold processing on the second shrimp fry area map, specifically including:

[0055] Based on the second shrimp seedling area map, the maximum inter-class variance method was used to obtain the dynamic binarization threshold;

[0056] The second shrimp seedling area map is threshold processed by using a dynamic binary threshold to obtain a binary map;

[0057] Obtain a complete single shrimp fry sample from the shrimp fry detection results and calculate the dynamic area threshold;

[0058] The pixel values ​​of the connected domains of the binary image that are smaller than the dynamic area threshold are set as preset values.

[0059] Furthermore, the complete single shrimp fry sample in the shrimp fry detection result is obtained, and the dynamic area threshold is calculated, specifically including:

[0060] Obtain a complete single shrimp fry sample in the shrimp fry detection result, and obtain two eyes and a head of the processed single shrimp fry sample according to threshold processing and morphological operations;

[0061] Dynamically calculating a maximum eye area threshold according to the maximum value of the eye area;

[0062] Dynamically calculate the minimum shrimp fry area threshold according to the minimum value of the head area;

[0063] The maximum eye area threshold and the minimum shrimp fry area threshold are dynamic area thresholds.

[0064] Further, the method uses an optimized matching strategy to process the second shrimp seedling area map and the shrimp seedling detection results after the multi-dynamic threshold processing to realize automatic counting of shrimp seedlings, specifically including:

[0065] According to the second shrimp fry area map after multi-dynamic threshold processing, the center coordinate positions of all shrimp fry connected domains are obtained and saved, thereby obtaining a point set;

[0066] According to the detection results of shrimp fry, the prediction frames of all shrimp fry are saved to obtain a frame set;

[0067] The point set and the frame set are optimally matched by using an optimization matching strategy to obtain a calculation result, thereby realizing automatic counting of shrimp fry.

[0068] The second object of the present invention can be achieved by adopting the following technical solutions:

[0069] A shrimp seed automatic counting system based on visual detection and fusion of multiple dynamic thresholds, the system comprising:

[0070] A first acquisition unit is used to acquire a shrimp seedling image dataset;

[0071] A model building unit, used to build a target detection model;

[0072] A model training unit is used to train and verify the target detection model using the shrimp seedling image data set to obtain the shrimp seedling target detection model;

[0073] The second acquisition unit is used to acquire the shrimp fry image to be counted and pre-process the shrimp fry image to be counted;

[0074] A model detection unit, used for inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes the detection result of the shrimp seedling;

[0075] A first processing unit is used to perform background removal and grayscale processing on the first shrimp fry area map to obtain a second shrimp fry area map;

[0076] The second processing unit is used to select a complete shrimp fry sample based on the shrimp fry detection result, and perform multi-dynamic threshold processing on the second shrimp fry area map;

[0077] The shrimp seed counting unit is used to process the second shrimp seed area map and the shrimp seed detection results after multi-dynamic threshold processing by using an optimized matching strategy to realize automatic shrimp seed counting.

[0078] The third object of the present invention can be achieved by adopting the following technical solutions:

[0079] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned method for automatically counting shrimp fry is implemented.

[0080] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0081] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned shrimp fry automatic counting method is implemented.

[0082] The present invention has the following beneficial effects compared with the prior art:

[0083] The present invention only utilizes tools such as scoops and plates commonly used in the shrimp fry breeding process to take samples, and then uses a mobile phone to take photos of the shrimp fry without the need for other auxiliary equipment. It can realize automatic, rapid, relatively accurate, and non-dehydrated counting of shrimp fry of different ages under a complex background, and can draw a mark at the head position of each shrimp fry in the picture for reference, thereby improving the efficiency of shrimp fry counting and trading. In addition, the method can also be deployed in a WeChat applet, providing a simple, convenient, efficient, and mutually trusted shrimp fry counting platform for both parties in the shrimp fry transaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0085] Figure 1 This is a specific flow chart of the method for automatic counting of shrimp fry based on visual detection and fusion of multiple dynamic thresholds according to Example 1 of the present invention.

[0086] Figure 2 The present invention is a simplified flowchart of the method for automatic counting of shrimp fry based on visual detection and fusion of multiple dynamic thresholds according to Example 1 of the present invention.

[0087] Figure 3 (a) to (d) are schematic diagrams of pictures of shrimp fry collected from different backgrounds and containers according to Example 1 of the present invention.

[0088] Figure 4 The present invention is a flowchart of constructing a shrimp seedling data set according to embodiment 1.

[0089] Figure 5 This is an architecture diagram of the target detection model of Example 1 of the present invention.

[0090] Figure 6 This is a flowchart of multi-dynamic threshold processing according to Embodiment 1 of the present invention.

[0091] Figure 7 This is a schematic diagram of the optimized matching strategy of Example 1 of the present invention.

[0092] Figure 8 This is a structural block diagram of an automatic shrimp fry counting system based on visual detection and fusion of multiple dynamic thresholds according to Example 2 of the present invention.

[0093] Fig. 9 This is a structural block diagram of a computer device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0095] Embodiment 1:

[0096] like Figure 1 and Figure 2 As shown, this embodiment provides a method for automatically counting shrimp fry based on visual detection and fusion of multiple dynamic thresholds, and the method includes the following steps:

[0097] S101, obtaining a shrimp fry image dataset.

[0098] Before this step, the shrimp fry are sampled using tools such as scoops and plates that can be found everywhere in the shrimp fry breeding process. Specifically, the shrimp fry are poured into a scoop or plate, and then photos of the shrimp fry are taken vertically from top to bottom using a mobile phone at a moderate height. At the same time, the water surface should be prevented from being reflected by light as much as possible; after taking photos of a scoop or a plate of shrimp fry, pour it out, continue to pour in the remaining shrimp fry, repeat the above method to shoot, and finally obtain the shrimp fry image data.

[0099] In the above shooting process, no other lighting and equipment are needed, and the user is allowed to capture other background information. Some of the collected shrimp fry pictures are shown in Figure 3 (a) to (d).

[0100] like Figure 4As shown, in this step, the shrimp fry image data is preprocessed, and then the preprocessed shrimp fry image data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1, and then the training set, the validation set, and the test set are annotated with horizontal rectangular frames using an annotation tool, and normalized, so as to construct a shrimp fry image data set; wherein: the data preprocessing is: removing abnormal images, such as removing images with severe reflections in the field of view; the normalization is: scaling the images of the training set, the validation set, and the test set to 1280×1280, and then processing the scaled images with a mean of 0.5 and a standard deviation of 0.5.

[0101] S102: Build a target detection model.

[0102] like Figure 5 As shown, the target detection model used in this step is an improved YOLOX network. The target detection model does not need to cluster the prior frames through the K-means algorithm, but is based on the Anchor-Free detection algorithm, which eliminates the step of setting the size and number of prior frames.

[0103] Specifically, the target detection model includes a backbone network, a neck structure and an output layer; among them: the backbone network adopts the CSPDarknet architecture to extract fine-grained features of the image, the neck structure includes FPN and PAN layers to strengthen the feature fusion between different layers, and the output layer corresponds to predicting targets of three different scales (large, medium and small).

[0104] In this step, a CBAM attention module is added to the FPN and PAN layers. The CBAM attention module is used to avoid interference from other background features.

[0105] Furthermore, the CBAM attention module is an end-to-end, simple and effective convolutional neural network attention module, which can be seamlessly integrated into the network model of this embodiment to participate in end-to-end training; it injects the attention map along the two independent dimensions of the channel and space of the input feature map, and then multiplies the injected attention map by the input feature map to achieve adaptive feature refinement of the input feature map, thereby avoiding interference from background features.

[0106] S103, using the shrimp seedling image data set to train and verify the target detection model to obtain a shrimp seedling target detection model.

[0107] In this step, the training set and verification set of the shrimp seed image data set are input into the target detection model for training and verification. In the process of training the target detection model using the training set, the regression prediction uses the first loss function to replace the original IoUloss function for prediction, and the classification prediction uses the second loss function for prediction, and the target detection model is trained using mosaic data enhancement to generate a weight file; after obtaining the trained target detection model, the trained target detection model is verified using the verification set, and the target detection model with the highest shrimp seed detection rate is selected as the shrimp seed target detection model. Of course, other target detection algorithms are also applicable in this embodiment.

[0108] The first loss function of this embodiment is as follows:

[0109] L α-IoU =1-IoU α

[0110] Among them, IoU represents the intersection over union ratio of the predicted box and the true box, α=3.

[0111] The second loss function of this embodiment is as follows:

[0112]

[0113] Among them, t[i] represents the true label value; o[i] represents the predicted value, which is 0 or 1.

[0114] S104, obtaining the shrimp seedlings image to be counted, and preprocessing the shrimp seedlings image to be counted.

[0115] This step preprocesses the shrimp fry image to be counted, specifically: performing data preprocessing and normalization processing of step S801 on the shrimp fry image to be counted, so as to obtain the preprocessed shrimp fry image to be counted.

[0116] S105, inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map.

[0117] In this step, the preprocessed shrimp seedling image to be counted is input into the shrimp seedling target detection model to obtain the first shrimp seedling area map; wherein: the first shrimp seedling area map is an image containing the detection results of other background areas and shrimp seedlings, and the detection results of shrimp seedlings are the prediction boxes of all shrimp seedlings.

[0118] S106, performing background removal and grayscale processing on the first shrimp fry area map to obtain a second shrimp fry area map.

[0119] According to the coordinate values ​​of the leftmost, rightmost, topmost and bottommost prediction boxes, the background of the first shrimp fry area map is removed and grayscale processed to obtain the second shrimp fry area map.

[0120] The above prediction box is also obtained based on the shrimp fry target detection model, but only the most boundary prediction box is found out in order to further obtain the shrimp fry counting area.

[0121] S107. Based on the detection results of the shrimp fry, a complete shrimp fry sample is selected, and a multi-dynamic threshold processing is performed on the second shrimp fry area map.

[0122] like Figure 6 As shown in , this step is specifically as follows: based on the second shrimp seedling area map, the maximum inter-class variance method proposed by OTSU is used to obtain the dynamic binary threshold, and then the second shrimp seedling area map is thresholded using the dynamic binary threshold to obtain a binary map; however, there are still some shrimp eyes in the binary map, so some detection results of all shrimp seedlings are used as shrimp seedling analysis sample data, and the two eyes and head of the shrimp seedlings can be obtained by selecting a complete single shrimp seedling sample, and after the above threshold processing and morphological operations, the minimum shrimp seedling area threshold is dynamically calculated according to the minimum value of the shrimp seedling head area, and at the same time, the maximum eye area threshold is dynamically calculated according to the maximum value of the shrimp seedling eye area, wherein the maximum eye area threshold and the minimum shrimp seedling area threshold are both dynamic area thresholds; the pixel value of the connected domain of the binary map that is less than the dynamic area threshold is set to (255,255,255), and the remaining connected domain is the shrimp seedling after multiple dynamic threshold processing, thereby obtaining the second shrimp seedling area map after multiple dynamic threshold processing.

[0123] S108, using an optimized matching strategy to process the second shrimp seedling area map and the shrimp seedling detection results after multi-dynamic threshold processing to achieve automatic counting of shrimp seedlings.

[0124] like Figure 7 As shown, this step is specifically as follows: according to the second shrimp seedling area map after multi-dynamic threshold processing, the center coordinate position of the connected domain of the shrimp seedlings is obtained, and the center coordinate position of the connected domain of all shrimp seedlings is saved to obtain a point set; then according to the first shrimp seedling area map, the prediction frames of all shrimp seedlings are saved to obtain a frame set; it is not difficult to find that due to occlusion or the shrimp seedlings are too small, the number of predicted frames of shrimp seedlings obtained using the shrimp seedling target detection model is generally less than the actual number of shrimp seedlings; and the number of connected domains is often closer to the actual number of shrimp seedlings, but the number of connected domains may be missed due to the setting of the threshold, so it is necessary to optimize and match the two results to obtain a more accurate counting result.

[0125] This embodiment proposes a new solution for shrimp fry counting, that is, the detection results obtained by using a neural network are fused with the results of multi-dynamic threshold processing for shrimp fry counting; the counting accuracy of this example for medium-density shrimp fry images (200 to 500 shrimp fry) reaches 98.45%, with an average error of 5; the counting accuracy for high-density shrimp fry images (600 to 3000 shrimp fry) reaches 97.12%, with an average error of 40; the above counting time does not exceed 5 seconds.

[0126] In this embodiment, the method of steps S801 to S807 can be applied to a WeChat applet or APP to provide a simple, convenient, efficient, and mutually trusted shrimp seedling counting platform for both parties in the shrimp seedling transaction.

[0127] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium.

[0128] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0129] Embodiment 2:

[0130] like Figure 8 As shown, this embodiment provides a shrimp seed automatic counting system based on visual detection fusion multiple dynamic thresholds, the system includes a first acquisition unit 801, a model building unit 802, a model training unit 803, a second acquisition unit 804, a model detection unit 805, a first processing unit 806, a second processing unit 807 and a shrimp seed counting unit 808, and the specific functions of each unit are as follows:

[0131] The first acquisition unit 801 is used to acquire a shrimp seedling image dataset;

[0132] A model building unit 802, used to build a target detection model;

[0133] The model training unit 803 is used to train and verify the target detection model using the shrimp seedling image data set to obtain the shrimp seedling target detection model;

[0134] The second acquisition unit 804 is used to acquire the shrimp seedling image to be counted and pre-process the shrimp seedling image to be counted;

[0135] A model detection unit 805 is used to input the preprocessed shrimp seedling image to be counted into a shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes a detection result of the shrimp seedlings;

[0136] The first processing unit 806 is used to perform background removal and grayscale processing on the first shrimp fry area map to obtain a second shrimp fry area map;

[0137] The second processing unit 807 is used to select a complete shrimp fry sample based on the shrimp fry detection result, and perform multi-dynamic threshold processing on the second shrimp fry area map;

[0138] The shrimp seed counting unit 808 is used to process the second shrimp seed area map and the shrimp seed detection results after multi-dynamic threshold processing using an optimized matching strategy to achieve automatic shrimp seed counting.

[0139] Embodiment 3:

[0140] like Fig. 9 As shown, this embodiment provides a computer device, which includes a processor 902, a memory, an input device 903, a display device 904 and a network interface 905 connected through a system bus 901. Among them, the processor 902 is used to provide computing and control capabilities, the memory includes a non-volatile storage medium 906 and an internal memory 907, the non-volatile storage medium 906 stores an operating system, a computer program and a database, and the internal memory 907 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium 906. When the computer program is executed by the processor 902, the shrimp seed automatic counting method of the above-mentioned embodiment 1 is implemented as follows:

[0141] Get the shrimp fry image dataset;

[0142] Build an object detection model;

[0143] The shrimp seedling image dataset is used to train and verify the target detection model, and the shrimp seedling target detection model is obtained;

[0144] Acquire the shrimp fry image to be counted, and pre-process the shrimp fry image to be counted;

[0145] Inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes the shrimp seedling detection result;

[0146] Performing background removal and grayscale processing on the first shrimp fry region map to obtain a second shrimp fry region map;

[0147] Based on the shrimp fry detection results, a complete shrimp fry sample is selected, and a multi-dynamic threshold processing is performed on the second shrimp fry area map;

[0148] The optimized matching strategy is used to process the second shrimp seedling area map and the shrimp seedling detection results after multi-dynamic threshold processing to realize automatic counting of shrimp seedlings.

[0149] Embodiment 4:

[0150] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for automatically counting shrimp fry in the above embodiment 1 is implemented as follows:

[0151] Get the shrimp fry image dataset;

[0152] Build an object detection model;

[0153] The shrimp seedling image dataset is used to train and verify the target detection model, and the shrimp seedling target detection model is obtained;

[0154] Acquire the shrimp fry image to be counted, and pre-process the shrimp fry image to be counted;

[0155] Inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes the shrimp seedling detection result;

[0156] Performing background removal and grayscale processing on the first shrimp fry region map to obtain a second shrimp fry region map;

[0157] Based on the shrimp fry detection results, a complete shrimp fry sample is selected, and a multi-dynamic threshold processing is performed on the second shrimp fry area map;

[0158] The optimized matching strategy is used to process the second shrimp seedling area map and the shrimp seedling detection results after multi-dynamic threshold processing to realize automatic counting of shrimp seedlings.

[0159] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0160] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0161] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Java, Python, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0162] In summary, the present invention only uses tools such as scoops and plates commonly used in the shrimp fry breeding process for sampling, and then takes photos of the shrimp fry by mobile phones without the need for auxiliary other equipment. It can realize the automatic, fast, relatively accurate, and non-dehydrated counting of shrimp fry of different ages under complex backgrounds, and can draw marks at the head position of each shrimp fry in the picture for reference, thereby improving the efficiency of shrimp fry counting and trading; in addition, the method can also be deployed in the WeChat applet to provide a simple, convenient, efficient, and mutually trusted shrimp fry counting platform for both parties of the shrimp fry transaction.

[0163] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which shall fall within the protection scope of the present invention.

Claims

1. A method for automatic counting of shrimp fry based on visual detection and fusion of multiple dynamic thresholds, characterized in that: The method comprises: Get the shrimp fry image dataset; Build an object detection model; The shrimp seedling image dataset is used to train and verify the target detection model, and the shrimp seedling target detection model is obtained; Acquire the shrimp fry image to be counted, and pre-process the shrimp fry image to be counted; Inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes the shrimp seedling detection result; Performing background removal and grayscale processing on the first shrimp fry region map to obtain a second shrimp fry region map; Based on the shrimp fry detection results, a complete shrimp fry sample is selected, and a multi-dynamic threshold processing is performed on the second shrimp fry area map; The second shrimp seedling area map and shrimp seedling detection results after multi-dynamic threshold processing are processed using an optimized matching strategy to achieve automatic shrimp seedling counting; The method of selecting a complete shrimp fry sample based on the shrimp fry detection result and performing multi-dynamic threshold processing on the second shrimp fry area map specifically includes: Based on the second shrimp seedling area map, the maximum inter-class variance method was used to obtain the dynamic binarization threshold; The second shrimp seedling area map is threshold processed by using a dynamic binary threshold to obtain a binary map; Obtain a complete single shrimp fry sample from the shrimp fry detection results and calculate the dynamic area threshold; Setting the pixel value of the connected domain of the binary image that is smaller than the dynamic area threshold to a preset value; The step of obtaining a complete single shrimp fry sample from the shrimp fry detection result and calculating the dynamic area threshold specifically includes: Obtain a complete single shrimp fry sample in the shrimp fry detection result, and obtain two eyes and a head of the processed single shrimp fry sample according to threshold processing and morphological operations; Dynamically calculating a maximum eye area threshold according to the maximum value of the eye area; Dynamically calculate the minimum shrimp fry area threshold according to the minimum value of the head area; The maximum eye area threshold and the minimum shrimp fry area threshold are dynamic area thresholds.

2. The method for automatically counting shrimp fry according to claim 1, characterized in that: The target detection model includes a backbone network, a neck structure and an output layer; the backbone network adopts the CSPDarknet architecture, the neck structure includes FPN and PAN layers, and the output layer corresponds to predicting targets of three different scales; The FPN and PAN layers are added with a CBAM attention module, and the CBAM attention module is used to avoid interference from background features.

3. The method for automatically counting shrimp fry according to claim 2, characterized in that: The CBAM attention module is used to avoid interference from background features, specifically including: The CBAM attention module injects the attention map along the channel dimension and spatial dimension of the input feature map, and then multiplies the injected attention map by the input feature map to achieve adaptive feature refinement of the input feature map, thereby avoiding interference from background features.

4. The method for automatically counting shrimp fry according to claim 1, characterized in that: In the process of training the target detection model using the shrimp seedling image data set, a first loss function is used for regression prediction, and a second loss function is used for classification prediction; The first loss function is as follows: L α-IoU =1-IoU α Among them, IoU represents the intersection-over-union ratio of the predicted box and the real box; The second loss function is as follows: Among them, t[i] represents the true label value; o[i] represents the predicted value, which is 0 or 1.

5. The method for automatically counting shrimp fry according to claim 1, characterized in that: The method of using an optimized matching strategy to process the second shrimp seedling area map and the shrimp seedling detection results after the multi-dynamic threshold processing to realize automatic shrimp seedling counting specifically includes: According to the second shrimp fry area map after multi-dynamic threshold processing, the center coordinate positions of all shrimp fry connected domains are obtained and saved, thereby obtaining a point set; According to the detection results of shrimp fry, the prediction frames of all shrimp fry are saved to obtain a frame set; The point set and the frame set are optimally matched by using an optimization matching strategy to obtain a calculation result, thereby realizing automatic counting of shrimp fry.

6. An automatic shrimp fry counting system based on visual detection and fusion of multiple dynamic thresholds, characterized in that: The system comprises: A first acquisition unit is used to acquire a shrimp seedling image dataset; A model building unit, used to build a target detection model; A model training unit is used to train and verify the target detection model using the shrimp seedling image data set to obtain the shrimp seedling target detection model; The second acquisition unit is used to acquire the shrimp fry image to be counted and pre-process the shrimp fry image to be counted; A model detection unit, used for inputting the preprocessed shrimp seedling image to be counted into the shrimp seedling target detection model to obtain a first shrimp seedling area map, wherein the first shrimp seedling area map includes the detection result of the shrimp seedling; A first processing unit is used to perform background removal and grayscale processing on the first shrimp fry area map to obtain a second shrimp fry area map; The second processing unit is used to select a complete shrimp fry sample based on the shrimp fry detection result, and perform multi-dynamic threshold processing on the second shrimp fry area map; A shrimp seed counting unit is used to process the second shrimp seed area map and the shrimp seed detection results after multi-dynamic threshold processing by using an optimized matching strategy to realize automatic shrimp seed counting; The method of selecting a complete shrimp fry sample based on the shrimp fry detection result and performing multi-dynamic threshold processing on the second shrimp fry area map specifically includes: Based on the second shrimp seedling area map, the maximum inter-class variance method was used to obtain the dynamic binarization threshold; The second shrimp seedling area map is threshold processed by using a dynamic binary threshold to obtain a binary map; Obtain a complete single shrimp fry sample from the shrimp fry detection results and calculate the dynamic area threshold; Setting the pixel value of the connected domain of the binary image that is smaller than the dynamic area threshold to a preset value; The step of obtaining a complete single shrimp fry sample from the shrimp fry detection result and calculating the dynamic area threshold specifically includes: Obtain a complete single shrimp fry sample in the shrimp fry detection result, and obtain two eyes and a head of the processed single shrimp fry sample according to threshold processing and morphological operations; Dynamically calculating a maximum eye area threshold according to the maximum value of the eye area; Dynamically calculate the minimum shrimp fry area threshold according to the minimum value of the head area; The maximum eye area threshold and the minimum shrimp fry area threshold are dynamic area thresholds.

7. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the method for automatically counting shrimp fry described in any one of claims 1 to 5 is implemented.

8. A storage medium storing a program, characterized in that: When the program is executed by a processor, the method for automatically counting shrimp fry described in any one of claims 1 to 5 is implemented.

Images