An intelligent detection method for bivalve planktonic larvae based on deep learning

Abstract

The present invention discloses a method for intelligent detection of bivalve planktonic larvae based on deep learning, comprising: collecting microscopic images of bivalve planktonic larvae with ink dots, image preprocessing, preparation of a planktonic larvae data set, training and acquisition of an application model, and planktonic larvae detection. The method uses a deep learning method to achieve intelligent detection of the number and shell length of bivalve planktonic larvae under a microscopic field of view, thereby reducing the workload and working time of manual microscopic observation of larvae, and greatly improving the work efficiency of larvae observation in the aquatic field.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and marine shellfish seed breeding, and in particular to an intelligent detection method for bivalve planktonic larvae based on deep learning. Background Art

[0002] Shellfish are the most diverse and dominant group of marine benthic animals. Among them, large bivalve species such as oysters, clams, clams, scallops, etc. have a large population and are the main food, breeding and fishing targets, with important economic value. Since the 1990s, the wave of marine bivalve aquaculture represented by scallop aquaculture has promoted the rise of the shellfish seed breeding industry, and at the same time put forward higher requirements for bivalve seed breeding technology and management level. Planktonic larvae are the larval stage in the life history of shellfish that rely on planktonic life, and are also the most important management stage in the process of shellfish seed breeding. Changes in the shell length of planktonic larvae are a direct reflection of the growth rate and health status of the larvae; changes in the number and density of planktonic larvae are closely related to the survival rate of the larvae, and are the result of the larvae's adaptation to environmental factors such as water quality, bait, density, and management. Therefore, daily inspection of bivalve planktonic larvae is an essential task for seedling breeding practitioners and an important guarantee for the smooth development of seedling breeding and the production of healthy seedlings.

[0003] At present, more than 90% of the seedlings needed for domestic bivalve aquaculture are still obtained through artificial seedling cultivation. The bivalve planktonic larvae cultivated in factories are characterized by large numbers (billions to tens of billions), small individuals (shell length is only tens to hundreds of microns), and rapid changes (shell length increases by several microns to more than 20 microns per day). Traditional inspection methods require the use of microscopes and other equipment for manual counting and measurement, which is labor-intensive and redundant. Long-term observation can easily cause eye fatigue or damage to the observer, greatly reducing work efficiency. Therefore, it is urgent to develop intelligent detection technologies and methods for bivalve planktonic larvae based on deep learning to improve the quality of bivalve planktonic larvae detection.

[0004] At present, the detection of marine organisms based on deep learning technology mainly focuses on the identification of species, and has not yet achieved the application-valuable biological counting and biological size prediction functions. An automatic detection method for bivalve planktonic larvae based on deep learning with great application value is urgently needed. Summary of the invention

[0005] According to industry needs and problems existing in the prior art, the present invention discloses an intelligent detection method for bivalve planktonic larvae based on deep learning, which solves the functions of automatic counting and physical shell length measurement of planktonic larvae under microscope observation conditions in the current bivalve seed breeding industry.

[0006] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0007] A method for intelligent detection of bivalve planktonic larvae based on deep learning, comprising the following steps:

[0008] S1. Original image acquisition: Original microscopic images of bivalve planktonic larvae placed in embryo dishes were taken using an optical microscope;

[0009] S2, image preprocessing: adjust image resolution, perform image noise reduction and image enhancement;

[0010] S3. Create a planktonic larvae dataset: Use the LabelImg tool to annotate the microscopic images, create a planktonic larvae dataset, and divide it into a training set and a test set in a ratio of 8:2;

[0011] S4, training and obtaining application models: training the obtained training set data under the YOLOv4 network structure, adjusting the network model hyperparameters according to the detection box results of the test set, and obtaining the optimized application model;

[0012] S5. Automatic detection of planktonic larvae: The microscopic image of the planktonic larvae to be detected is input into the optimized YOLOv4 application model, which automatically calculates and outputs the number of prediction boxes, the average of the prediction box length and width, and realizes the counting of bivalve planktonic larvae and the measurement of their growth data.

[0013] The method is to anesthetize the bivalve planktonic larvae with iodine tincture and collect microscopic images after the larvae sink to the bottom.

[0014] The image resolution is set to 3000×4000 pixels; the image noise reduction is Gaussian filtering and median filtering; the image enhancement is random flipping, rotation, and scaling.

[0015] The optical microscope is equipped with both a micrometer scale and ink dots. The ink dots are circles with a diameter of 100 microns pre-engraved by laser on the micrometer and are used to calibrate the physical size of a unit pixel. The physical size of a unit pixel is the ratio of the actual size of the ink dot to the number of pixels occupied by the ink dot in the microscopic image.

[0016] When labeling with the labelImg tool, the marking box is tangent to the larvae. The labeled planktonic larvae dataset xml file contains the length and width of the detection box, and stores the path, category name, quantity, and position of the marking box in the image in the Pascal VOC data format.

[0017] The optimized YOLOv4 application model uses the network model hyperparameters obtained from the training set and the features extracted from the training set images to make predictions on the test set images.

[0018] The prediction results include the number of bivalve planktonic larvae and the average shell length of the bivalve planktonic larvae, and a text file recording the shell length of each larva is automatically exported.

[0019] The number of the prediction boxes is the number of bivalve planktonic larvae in the image; the average value of the length and width of the prediction box is approximately the shell length of the bivalve planktonic larvae, and the specific formula calculation process is as follows:

[0020] L=(a+b) / 2

[0021] Where L is the shell length of bivalve planktonic larvae, a is the physical length of the prediction box, and b is the physical width of the prediction box.

[0022] The physical length of the prediction box is the number of pixels occupied by the long side multiplied by the physical size of the unit pixel, and the physical width of the prediction box is the number of pixels occupied by the short side multiplied by the physical size of the unit pixel.

[0023] A deep learning-based intelligent detection system for bivalve planktonic larvae, comprising a storage unit and a processing unit, wherein the storage unit stores a program, and the processing unit loads the program to execute the method steps according to any one of claims 1 to 9, thereby realizing the counting of bivalve planktonic larvae and the detection of growth data of bivalve planktonic larvae.

[0024] The present invention has the following beneficial effects and advantages:

[0025] The present invention can intuitively obtain the number of bivalve larvae under a microscope through an intelligent detection method, and can obtain the shell length data of the bivalve larvae, thereby increasing the efficiency of larvae observation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of an embodiment of the present invention.

[0027] Figure 2 This is a data augmentation graph.

[0028] Figure 3 The label image of labelImg

[0029] Figure 4 Flowchart for YOLOv4

[0030] Figure 5 This is the test result picture. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0032] This embodiment provides a method for intelligently detecting bivalve larvae based on deep learning. Figure 1As shown, the following steps are included:

[0033] (1) Collection of microscopic images of bivalve planktonic larvae: artificially bred bivalve planktonic larvae of different periods were placed in embryo dishes. After the larvae species were identified, iodine tincture was added for anesthesia. After the larvae sank to the bottom, the embryo dishes were placed under an optical microscope with a micrometer with a laser-engraved dot of 100 microns in diameter. Microscopic images of bivalve planktonic larvae were obtained by manual photography. The number of images should be as large as possible to better train the model and improve recognition accuracy.

[0034] The bivalve planktonic larvae in step (1) are screened and classified by professionals in shellfish biology, and larvae of different sizes, developmental stages, and types are selected.

[0035] The optical microscope is equipped with a micrometer, on which a circle with a diameter of 100 microns is laser engraved as an ink dot, so that the microscopic image of the bivalve planktonic larvae has both scale and ink dots.

[0036] (2) Image preprocessing: The image resolution is set to a fixed 3000 × 4000 pixels, and the image is subjected to Gaussian filtering and median filtering using an algorithm, and the image is randomly flipped, rotated, and scaled ( Figure 2 ).

[0037] (3) Preparation of planktonic larvae dataset: Figure 3 As shown in the figure, the LabelImg tool is used to manually annotate the microscopic images of bivalve planktonic larvae. When annotating, care should be taken not to include Chinese characters in the path, and the marking box should be as tangent to the larvae as possible. The xml file generated after annotation contains the length and width of the detection box, so as to create a planktonic larvae dataset, and the obtained planktonic larvae dataset is divided into a training set and a test set in a ratio of 8:2.

[0038] The XML file generated after the annotation follows the Pascal VOC format, and includes the path, category name, quantity, and position of the annotation box in the image of the target. The category names are: larval size, development stage, and species.

[0039] (4) Training and acquisition of application model: The acquired training set is input into the YOLOv4 detection model for training. The experimental parameters of the training set are trained under given hyperparameters. The specific settings of the experimental parameters are shown in Table 1. The error of the model is controlled within an acceptable range, the generalization ability of the network is verified, and the hyperparameters are adjusted according to the results to obtain an application model with excellent generalization ability ( Figure 4). Specifically, the training set images will be input into the YOLOv4 backbone network CSPDarkNet53 in the form of length × width × depth for deep feature extraction. The neck network (Neck) structure of YOLOv4 is mainly composed of SPP (Spatial Pyramid Pooling), FPN (Feature Pyramid Networks) + PAN (Path Aggregation Network), where the SPP layer is used to increase feature extraction and separate important contextual features. The FPN structure is mainly used to strengthen the semantic information in the high-level from top to bottom, and the PAN structure strengthens the positioning information in the high-level from bottom to top. The head network (Head) of the YOLOv4 model still uses the head structure of YOLOv3. The acquired data set is trained under the YOLOv4 network structure, and the hyperparameters are adjusted according to the detection results of the test set to obtain an application model that can realize intelligent detection of bivalve planktonic larvae and has excellent generalization ability.

[0040] Table 1 Experimental parameters

[0041]

[0042] The network input is the XML file generated by the picture and the markup. The XML file records the category and location information of the target. The network output is the detection box and the center position of the detection box.

[0043] (5) Automatic detection of planktonic larvae: Based on the features extracted from the training set images, the trained YOLOv4 application model is used to detect the planktonic larvae images. The labeled images of bivalve planktonic larvae are input into the counting algorithm. The bivalve planktonic larvae counting function is realized by counting the number of prediction boxes. The shell length of the bivalve planktonic larvae is approximately calculated by the average value of the prediction box length and width, and a text file recording the shell length of each larva is automatically exported.

[0044] The YOLOv4 application model in step (5) uses the network output content obtained from the training set and the features extracted from the training set images to predict the content of the test set images.

[0045] The shell length of the bivalve planktonic larvae is calculated by the following method:

[0046] The length of the prediction box is the number of pixels occupied by the long side multiplied by the physical size of the unit pixel, and the width of the prediction box is the number of pixels occupied by the short side multiplied by the physical size of the unit pixel. The average value of the length and width of the prediction box is approximated as the shell length of the bivalve planktonic larvae. The specific calculation process is as follows:

[0047] L=(a+b) / 2

[0048] Where L is the shell length of bivalve planktonic larvae, a is the physical length of the prediction box, and b is the physical width of the prediction box.

[0049] The present invention also includes a deep learning-based intelligent detection system for bivalve planktonic larvae, including a storage unit and a processing unit. The storage unit stores a program, and the processing unit loads the program to execute the method steps described above to achieve the counting of bivalve planktonic larvae and the detection of growth data of bivalve planktonic larvae.

[0050] like Figure 5 As shown, the test results showed that the number of bivalve planktonic larvae was 24, and the average larval shell length was 149.78 μm. The results of manual measurement were that the number of bivalve planktonic larvae was 24, and the average larval shell length was 146.78 μm, with extremely high accuracy.

[0051] The above embodiments are only for illustrating the technical concept and features of the present invention, and cannot be used to limit the protection scope of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A method for intelligent detection of bivalve planktonic larvae based on deep learning, characterized in that: The following steps are performed to detect and measure the growth data of marine shellfish planktonic seedlings during the breeding period. The method includes the following steps: S1. Original image acquisition: an original microscopic image of bivalve planktonic larvae placed in an embryo dish is taken by an optical microscope; the optical microscope is equipped with a micrometer scale and ink dots, the ink dots are circles with a diameter of 100 microns pre-engraved by laser on the micrometer, and are used to calibrate the physical size of a unit pixel; the physical size of a unit pixel is the ratio of the actual size of the ink dots to the number of pixels occupied by the ink dots in the microscopic image; S2, image preprocessing: adjust image resolution, perform image noise reduction and image enhancement; S3. Create a planktonic larvae dataset: Use the LabelImg tool to annotate the microscopic images, create a planktonic larvae dataset, and divide it into a training set and a test set in a ratio of 8:2; S4, training and obtaining application models: training the obtained training set data under the YOLOv4 network structure, adjusting the network model hyperparameters according to the detection box results of the test set, and obtaining the optimized application model; S5. Automatic detection of planktonic larvae: Input the microscopic image of the planktonic larvae to be detected into the optimized YOLOv4 application model, automatically calculate and output the number of prediction boxes, the average of the length and width of the prediction boxes, and realize the counting of bivalve planktonic larvae and the measurement of the growth data of bivalve planktonic larvae; The number of the prediction boxes is the number of bivalve planktonic larvae in the image; the average of the length and width of the prediction boxes is approximately the shell length of the bivalve planktonic larvae, and the specific formula calculation process is as follows: L=(a+b) / 2 Where L is the shell length of bivalve planktonic larvae, a is the physical length of the prediction box, and b is the physical width of the prediction box.

2. According to claim 1, a method for intelligent detection of bivalve planktonic larvae based on deep learning, characterized in that: The method is to anesthetize the bivalve planktonic larvae with iodine tincture and collect microscopic images after the larvae sink to the bottom.

3. According to claim 1, a method for intelligent detection of bivalve planktonic larvae based on deep learning, characterized in that: The image resolution is set to 3000×4000 pixels; the image noise reduction is Gaussian filtering and median filtering; the image enhancement is random flipping, rotation, and scaling.

4. The method for intelligent detection of bivalve planktonic larvae based on deep learning according to claim 1, characterized in that: When labeling with the labelImg tool, the marking box is tangent to the larvae. The labeled planktonic larvae dataset xml file contains the length and width of the detection box, and stores the path, category name, quantity, and position of the marking box in the image in the Pascal VOC data format.

5. The method for intelligent detection of bivalve planktonic larvae based on deep learning according to claim 1, characterized in that: The optimized YOLOv4 application model uses the network model hyperparameters obtained from the training set and the features extracted from the training set images to make predictions on the test set images.

6. The method for intelligent detection of bivalve planktonic larvae based on deep learning according to claim 5, characterized in that: The prediction results include the number of bivalve planktonic larvae and the average shell length of the bivalve planktonic larvae, and a text file recording the shell length of each larva is automatically exported.

7. The method for intelligent detection of bivalve planktonic larvae based on deep learning according to claim 1, characterized in that: The physical length of the prediction box is the number of pixels occupied by the long side multiplied by the physical size of the unit pixel, and the physical width of the prediction box is the number of pixels occupied by the short side multiplied by the physical size of the unit pixel.

8. An intelligent detection system for bivalve planktonic larvae based on deep learning, characterized in that: The method comprises a storage unit and a processing unit. The storage unit stores a program. The processing unit loads the program to execute the method steps according to any one of claims 1 to 7, so as to realize the counting of bivalve planktonic larvae and the detection of the growth data of bivalve planktonic larvae.

Images