Automatic fry measuring system for enhancement and release

Abstract

The invention, which relates to the technical field of fishery proliferation, discloses an automatic fry measurement system for proliferation and releasing, comprising a weighing module for collecting net weight data of fry samples, an image acquisition module for acquiring RGB-D images of the fry samples at different angles, and a basic data module for acquiring the sampling time and place of the fry samples. The data collection module sends the net weight data, the sampling time and the sampling place to the record report generation module, and sends the image data to the data processing module; the data processing module obtains type data and body length data of the fry samples through the RGB-D images and sends the data to the record report generation module; and the record report generation module generates an enhancement release record report according to the collected data. By means of the system, sampling detection of the weight, the body length, the type and the sample number of the enhancement and release offspring seed samples and automatic generation of an enhancement and release record report are achieved, the current enhancement and release situation is accurately recorded, and a basis is provided for enhancement and release management and control.

Description

Technical Field

[0001] The invention relates to the technical field of fishery proliferation, and more particularly to an automatic measuring system for seedlings used for proliferation and release. Background Art

[0002] At present, in order to protect aquatic biological resources, increase the number of fishery resources, protect the marine ecological environment, and improve the economic benefits of the ocean, artificial reproduction and release activities have been widely carried out around the world to restore the increasingly depleted fishery resources. The effect of artificial reproduction and release should be said to be very good, which can be reflected in the following aspects: 1) Through artificial reproduction and release, aquatic biological resources can be supplemented and restored; 2) For some endangered species, artificial reproduction and release can increase the number of wild populations, promote genetic exchanges between populations, and play a certain protective role for these endangered species. In summary, artificial reproduction and release can not only increase wild resources and improve the population structure of organisms, but also maintain biological diversity. It is one of the effective ways to deal with the decline or even disappearance of aquatic species and resources caused by human activities.

[0003] However, with the continuous advancement of the enhancement and release project, accurate, effective and intelligent supervision of enhancement and release has become an area of ​​focus for the competent authorities and release organizers. At present, enhancement and release adopts the method of manual random sampling and testing at the release site, manually distinguishing the species composition of the samples, and manually measuring the length and weight of the sampled samples to estimate the input of this enhancement and release. The efficiency and accuracy are low, and each link is difficult to effectively supervise and control. The relevant sampling records are also not standardized. In some cases, there is a possibility that some releasers purchase alien invasive species and release them. Some seedling suppliers also have the situation of not meeting specifications, poor vitality, and selling inferior products as good ones. Therefore, it is necessary to use automated and intelligent equipment to accurately identify the types of seedlings for enhancement and release, eliminate the existence of alien invasive species, accurately measure the sampled samples, and accurately calculate and standardize the number of enhancement and release.

[0004] Therefore, how to efficiently and accurately measure and standardize the records of the released seedlings is an urgent problem that needs to be solved by technical personnel in this field. Summary of the invention

[0005] In view of this, the present invention provides an automatic measurement system for seedlings for proliferation and release, which can accurately record the proliferation and release situation and provide a basis for proliferation and release control by randomly sampling the seedlings for this proliferation and release, detecting the weight, body length, type and sample quantity of the seedling samples and automatically generating a proliferation and release record report.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A seedling automatic measurement system for proliferation and release comprises: a data collection module, a data processing module, a PC main control module, and a record report generation module, wherein the data collection module comprises a weighing module, an image acquisition module, and a basic data module, wherein the weighing module collects net weight data of seedling samples, the image acquisition module collects RGB-D images of seedling samples at different angles, the basic data module obtains sampling time and location of seedling samples, the data collection module sends the net weight data, sampling time and location to the record report generation module through the PC main control module, and sends the image data to the data processing module through the PC main control module; the data processing module obtains the species data and body length data of the seedling samples through the RGB-D images, and sends them to the record report generation module through the PC main control module; the record report generation module generates a proliferation and release record report according to the net weight data, species data, body length data, sampling time and location.

[0008] Preferably, the data processing module includes a species identification module and a body length detection module, the species identification module acquires species data based on the head, trunk and tail images of the seedling sample; the body length detection module calculates the body length pixel value of the seedling sample and further calculates the body length data.

[0009] Preferably, the type identification module specifically includes:

[0010] A sample segmentation module, which obtains a complete sample image by splicing the RGB-D images at different angles, and segments the head, trunk and tail images of the seedling sample from the complete sample image;

[0011] A model building module collects RGB-D images of different types of seedling samples from different angles, splices them to obtain complete images of the seedling samples, separates the head, trunk and tail images of the different types of seedling samples from the complete images; builds training samples based on the head, trunk and tail images of the different types of seedling samples, trains the deep convolution classification model using the training samples, and obtains a seedling classification network model when the recognition accuracy of the deep convolution classification model reaches a preset threshold;

[0012] The seedling classification module inputs the RGB-D image of the seedling sample to be classified into the seedling classification network model to obtain the type of the seedling sample to be classified.

[0013] Preferably, the complete sample image data obtained in the sample segmentation module specifically includes: extracting multiple feature points of the RGB-D image at different angles, and calculating the histograms of the neighborhood geometric information of the multiple feature points with the multiple feature points as the center points; using the histogram similarity to perform a two-way check on the feature points, and deleting the feature points with a similarity less than a certain value; using a random sampling consensus algorithm to align the feature points between the RGB-D images at different angles; performing spatial transformation and splicing on the RGB-D images at different angles to form the complete sample image data; using depth and color information to find the edges of the seedling samples to be classified, separating the seedling samples to be classified from the background, and using depth, color and shape information to separate the head, trunk and tail images from the seedling samples to be classified.

[0014] Preferably, the seedling classification module specifically includes:

[0015] A first confidence module, using the seedling classification network model to identify the type of the head of the seedling sample, and identify the type to which the seedling sample to be classified belongs and a first confidence corresponding to the type;

[0016] A second confidence module, using the seedling classification network model to identify the type of the trunk of the seedling sample, to identify the type to which the seedling sample to be classified belongs and a second confidence corresponding to the type;

[0017] A third confidence module, using the seedling classification network model to identify the type of the tail of the seedling sample, to identify the type to which the seedling sample to be classified belongs and a third confidence corresponding to the type;

[0018] A weighted fusion module calculates the confidence of identifying the seedling sample to be classified as the category based on the first confidence, the second confidence and the third confidence corresponding to the same category through weighted fusion; and selects the category with the highest confidence as the category of the seedling sample to be classified.

[0019] Preferably, the body length detection module specifically includes: taking the tail of the seedling sample in the RGB-D image as the image boundary, calculating the pixel difference between the mouth and the image end position of the mouth corresponding to the seedling sample in the RGB-D image, and judging whether the pixel difference is less than or equal to a determined pixel difference threshold; when it is judged that the pixel difference is less than or equal to the pixel difference threshold, calculating the length pixel value of the seedling sample in the RGB-D image; and calculating the body length data of the seedling sample according to the length pixel value.

[0020] Preferably, before the body length detection module calculates the pixel difference, it also includes a completeness detection module: judging whether the seedling sample completely appears in the RGB-D image according to the RGB-D image of the seedling sample; when it is judged that it completely appears in the RGB-D image, calculating the body length data of the seedling sample according to the length pixel value; when it is judged that it does not completely appear in the RGB-D image, obtaining the blind area length of the seedling sample, and the blind area length of the seedling sample is used to indicate the actual length corresponding to the partial image of the seedling sample that does not appear in the RGB-D image;

[0021] Among them, calculating the body length data of the seedling sample according to the length pixel value includes: determining the length matching the length pixel value, and calculating the sum of the length matching the length pixel value and the blind area length of the seedling sample as the body length data of the seedling sample.

[0022] Preferably, obtaining the blind area length of the seedling sample in the integrity detection module specifically includes: obtaining the RGB-D image acquisition position where the seedling sample is currently located, and determining the horizontal distance between the image acquisition position and the position of the image acquisition module based on the RGB-D image acquisition position, and determining the blind area length of the seedling sample based on the horizontal distance.

[0023] Through the above technical scheme, it can be known that compared with the prior art, the present invention discloses an automatic measurement system for seedlings for proliferation and release, which collects the net weight data of seedling samples, RGB-D images at different angles, and sampling time and place through the weighing module, image acquisition module and basic data module in the data collection module respectively. The data source is rich and comprehensive. The data processing module obtains the species data and body length data of the seedling samples using RGB-D images. The species recognition module analyzes the head, trunk and tail of the seedling samples and combines the seedling classification network model obtained by training the deep convolution classification model to accurately identify the seedling species; the body length detection module determines the body length of the seedlings more accurately by calculating the body length pixel value of the seedling samples; the sample segmentation module performs splicing, feature point analysis, Operations such as image registration can obtain a complete sample image and separate the head, trunk and tail images from the background and sample, providing an accurate image basis for subsequent species identification and body length detection; the seedling classification module in the species identification module identifies the species of the head, trunk and tail of the seedling sample respectively, and obtains the final species confidence through weighted fusion calculation, which improves the accuracy and reliability of species identification; before calculating the pixel difference, the body length detection module uses the integrity detection module to determine whether the seedling sample appears completely in the RGB-D image, and obtains the blind area length when it does not appear completely, so as to more accurately calculate the body length data of the seedling sample; the record report generation module automatically generates a reproduction and release record report based on the net weight data, species data, body length data, sampling time and location, providing strong support for reproduction and release work. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0025] Figure 1 A schematic diagram of the structure provided by the present invention;

[0026] Figure 2 This is a schematic diagram of the seedling classification module provided by the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The embodiment of the present invention discloses an automatic measurement system for seedlings used for proliferation and release, such as Figure 1 As shown, it includes: a data collection module, a data processing module, a PC main control module, and a record report generation module. The data collection module includes a weighing module, an image acquisition module, and a basic data module. The weighing module collects the net weight data of the seedling sample, the image acquisition module collects RGB-D images of the seedling sample at different angles, the basic data module obtains the sampling time and place of the seedling sample, the data collection module sends the net weight data, sampling time and place to the record report generation module through the PC main control module, and sends the image data to the data processing module through the PC main control module; the data processing module obtains the species data and body length data of the seedling sample through the RGB-D image, and sends it to the record report generation module through the PC main control module; the record report generation module generates a reproduction and release record report according to the net weight data, species data, body length data, sampling time and place.

[0029] In a specific embodiment, the generated reproduction and release record report carries an anti-counterfeiting stamp.

[0030] In a specific embodiment, the data processing module includes a species identification module and a body length detection module. The species identification module obtains species data based on the head, trunk and tail images of the seedling sample; the body length detection module calculates the body length pixel value of the seedling sample and further calculates the body length data.

[0031] In a specific embodiment, the category identification module specifically includes:

[0032] The sample segmentation module obtains a complete sample image by splicing RGB-D images from different angles, and then segments the head, trunk and tail images of the seedling sample from the complete sample image;

[0033] A model building module collects RGB-D images of different types of seedling samples from different angles, splices them to obtain complete images of the seedling samples, separates the head, trunk and tail images of the different types of seedling samples from the complete images; builds training samples based on the head, trunk and tail images of the different types of seedling samples, trains the deep convolution classification model using the training samples, and obtains a seedling classification network model when the recognition accuracy of the deep convolution classification model reaches a preset threshold (e.g., 95%);

[0034] The seedling classification module inputs the RGB-D image of the seedling sample to be classified into the seedling classification network model to obtain the type of the seedling sample to be classified.

[0035] In a specific embodiment, the deep convolution classification model uses a classification model of a lightweight deep convolutional neural network ShuffleNetV2, and the feature point extraction algorithm for extracting feature points in the depth image and RGB image at each angle includes a SURF (Speeded Up Robust Features) algorithm based on principal components. The registration algorithm for feature points in the depth image and RGB image at each angle here includes an iterative closest point (Iterative Closest Point, ICP) algorithm based on OpenMP technology.

[0036] In a specific embodiment, obtaining complete sample image data in the sample segmentation module specifically includes: extracting multiple feature points of RGB-D images at different angles, and calculating the histogram of the neighborhood geometric information of the multiple feature points with the multiple feature points as the center points; using the histogram similarity to perform a two-way check of the feature points, and deleting the feature points with a similarity less than a certain value; using a random sampling consistency algorithm to align the feature points between RGB-D images at different angles; performing spatial transformation and splicing on the RGB-D images at different angles to form complete sample image data; using depth and color information to find the edges of the seedling samples to be classified, separating the seedling samples to be classified from the background, and using depth, color and shape information to separate the head, trunk and tail images from the seedling samples to be classified.

[0037] In this step, multiple feature points of RGB-D images at different angles are extracted, and the histograms of the neighborhood geometric information of the multiple feature points are calculated with the multiple feature points as the center points. The feature points are checked bidirectionally using the histogram similarity, and the feature points with a similarity less than a certain value (for example, 90%) are deleted. The RANSAC algorithm is used to align the feature points between RGB-D images at different angles. The RGB-D images at different angles are spatially transformed and spliced ​​to form a complete image of the seedling sample to be classified. The depth and color information are used to find the edges of the seedling sample to be classified, and the seedling sample to be classified is separated from the background. The head, trunk and tail images are separated from the seedling sample to be classified using the depth, color and shape information.

[0038] In a specific embodiment, Figure 2 As shown in the figure, the seedling classification module specifically includes:

[0039] The first confidence module uses the seedling classification network model to identify the type of the head of the seedling sample, and identifies the type to which the seedling sample to be classified belongs and the first confidence corresponding to the type;

[0040] The second confidence module uses the seedling classification network model to identify the type of the trunk of the seedling sample, and identifies the type to which the seedling sample to be classified belongs and the second confidence corresponding to the type;

[0041] The third confidence module uses the seedling classification network model to identify the type of the tail of the seedling sample, and identifies the type to which the seedling sample to be classified belongs and the third confidence corresponding to the type;

[0042] The weighted fusion module calculates the confidence of identifying the seedling sample to be classified as a species based on the first confidence, the second confidence and the third confidence corresponding to the same species; and selects the category with the highest confidence as the species of the seedling sample to be classified.

[0043] Among them, a deep convolutional neural network model is used to identify the type of the head, and the type to which the seedling sample to be classified belongs and the first confidence Q1 corresponding to the type are identified. The trunk of the seedling sample to be classified is selected and the deep convolutional neural network model is used to identify the type of the trunk, and the type to which the seedling sample to be classified belongs and the second confidence Q2 corresponding to the type are identified. The tail of the seedling sample to be classified is selected and the deep convolutional neural network model is used to identify the type of the tail, and the type to which the seedling sample to be classified belongs and the third confidence Q3 corresponding to the type are identified. Based on the first confidence Q1, the second confidence Q2 and the third confidence Q3 corresponding to the same type, the confidence Q of identifying the seedling sample to be classified as the corresponding type is calculated respectively. In some embodiments, the calculation formula for calculating the confidence Q of identifying the seedling sample to be classified as the corresponding type is:

[0044]

[0045] In a specific embodiment, the body length detection module specifically includes: taking the tail of the seedling sample in the RGB-D image as the image boundary, calculating the pixel difference between the mouth and the image end position of the mouth corresponding to the seedling sample in the RGB-D image, and judging whether the pixel difference is less than or equal to a determined pixel difference threshold; when it is judged that the pixel difference is less than or equal to the pixel difference threshold, calculating the length pixel value of the seedling sample in the RGB-D image; and calculating the body length data of the seedling sample according to the length pixel value.

[0046] In the embodiment of the present invention, the tail of the seedling sample in the RGB-D image is taken as the image boundary, i.e., the bottom, and the mouth position of the seedling sample in the RGB-D image is taken as the top. Optionally, based on the top of the RGB-D image of the seedling sample and the bottom of the image of the seedling sample, the length pixel value of the RGB-D image of the seedling sample in the image of the seedling sample is calculated, specifically: the first longitudinal pixel value in the middle of the mouth of the RGB-D image of the seedling sample and the second longitudinal pixel value at the bottom of the image of the seedling sample are obtained, and the longitudinal pixel value difference is obtained by subtracting the second longitudinal pixel value from the first longitudinal pixel value, and the longitudinal pixel value difference is determined as the length pixel value of the RGB-D image of the seedling sample in the image of the seedling sample.

[0047] In an embodiment of the present invention, optionally, body length data corresponding to the length pixel value is determined from a determined information database as the body length data of the seedling sample, wherein the information database stores length information corresponding to pixels of different lengths; or, based on the actual length of the unit represented by the determined unit pixel, the length corresponding to the length pixel value is calculated as the body length data of the seedling sample.

[0048] In this optional embodiment, an information database is established, in which blind spot lengths corresponding to different horizontal distances are stored. When the horizontal distance is obtained, the blind spot length corresponding to the horizontal distance can be directly obtained from the information database as the blind spot length of the seedling sample; or, based on the determined unit blind spot length corresponding to the unit horizontal distance, the length corresponding to the horizontal distance is calculated as the blind spot length of the seedling sample.

[0049] It can be seen that this optional embodiment obtains the horizontal distance between the current image acquisition position of the seedling sample and the position of the image acquisition device, and obtains the length corresponding to the horizontal distance as the blind spot length of the seedling sample, thereby improving the user's accuracy and efficiency in obtaining the blind spot length.

[0050] In a specific embodiment, before the body length detection module calculates the pixel difference, it also includes a completeness detection module: judging whether the seedling sample completely appears in the RGB-D image according to the RGB-D image of the seedling sample; when it is judged that it completely appears in the RGB-D image, calculating the body length data of the seedling sample according to the length pixel value; when it is judged that it does not completely appear in the RGB-D image, obtaining the blind area length of the seedling sample, and the blind area length of the seedling sample is used to represent the actual length corresponding to the part of the image that does not appear in the RGB-D image of the seedling sample;

[0051] Among them, calculating the body length data of the seedling sample according to the length pixel value includes: determining the length matching the length pixel value, and calculating the sum of the length matching the length pixel value and the blind area length of the seedling sample as the body length data of the seedling sample.

[0052] In a specific embodiment, obtaining the blind area length of the seedling sample in the integrity detection module specifically includes: obtaining the RGB-D image acquisition position where the seedling sample is currently located, and determining the horizontal distance between the image acquisition position and the position of the image acquisition module based on the RGB-D image acquisition position, and determining the blind area length of the seedling sample based on the horizontal distance.

[0053] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0054] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic measurement system for seedlings used for proliferation and release, characterized in that: include: A data collection module, a data processing module, a PC main control module, and a record report generation module. The data collection module includes a weighing module, an image acquisition module, and a basic data module. The weighing module collects the net weight data of seedling samples, the image acquisition module collects RGB-D images of seedling samples at different angles, and the basic data module obtains the sampling time and location of the seedling samples. The data collection module sends the net weight data, sampling time and location to the record report generation module through the PC main control module, and sends the image data to the data processing module through the PC main control module; the data processing module obtains the species data and body length data of the seedling samples through the RGB-D image, and sends them to the record report generation module through the PC main control module; the record report generation module generates a reproduction and release record report based on the net weight data, species data, body length data, sampling time and location.

2. The automatic measurement system for seedlings for proliferation and release according to claim 1, characterized in that: The data processing module includes a species identification module and a body length detection module. The species identification module obtains species data based on the head, trunk and tail images of the seedling sample; the body length detection module calculates the body length pixel value of the seedling sample and further calculates the body length data.

3. The automatic measurement system for seedlings for proliferation and release according to claim 2, characterized in that: The type identification module specifically includes: A sample segmentation module, which obtains a complete sample image by splicing the RGB-D images at different angles, and segments the head, trunk and tail images of the seedling sample from the complete sample image; A model building module collects RGB-D images of different types of seedling samples from different angles, splices them to obtain complete images of the seedling samples, separates the head, trunk and tail images of the different types of seedling samples from the complete images; builds training samples based on the head, trunk and tail images of the different types of seedling samples, trains the deep convolution classification model using the training samples, and obtains a seedling classification network model when the recognition accuracy of the deep convolution classification model reaches a preset threshold; The seedling classification module inputs the RGB-D image of the seedling sample to be classified into the seedling classification network model to obtain the type of the seedling sample to be classified.

4. The automatic seed measurement system for proliferation and release according to claim 3 is characterized in that: The sample segmentation module obtains complete sample image data specifically including: extracting multiple feature points of the RGB-D image at different angles, and calculating the histogram of the neighborhood geometric information of the multiple feature points with the multiple feature points as the center points; using the histogram similarity to perform a bidirectional check on the feature points, and deleting the feature points whose similarity is less than a certain value; using a random sampling consensus algorithm to align the feature points between the RGB-D images at different angles; performing spatial transformation and splicing on the RGB-D images at different angles to form the complete sample image data; using depth and color information to find the edges of the seedling samples to be classified, separating the seedling samples to be classified from the background, and using depth, color and shape information to separate the head, trunk and tail images from the seedling samples to be classified.

5. The automatic seed measurement system for proliferation and release according to claim 3 is characterized in that: The seedling classification module specifically includes: A first confidence module, using the seedling classification network model to identify the type of the head of the seedling sample, and identify the type to which the seedling sample to be classified belongs and a first confidence corresponding to the type; A second confidence module, using the seedling classification network model to identify the type of the trunk of the seedling sample, to identify the type to which the seedling sample to be classified belongs and a second confidence corresponding to the type; A third confidence module, using the seedling classification network model to identify the type of the tail of the seedling sample, to identify the type to which the seedling sample to be classified belongs and a third confidence corresponding to the type; A weighted fusion module calculates the confidence of identifying the seedling sample to be classified as the category based on the first confidence, the second confidence and the third confidence corresponding to the same category through weighted fusion; and selects the category with the highest confidence as the category of the seedling sample to be classified.

6. The automatic seed measurement system for proliferation and release according to claim 2 is characterized in that: The body length detection module specifically includes: taking the tail of the seedling sample in the RGB-D image as the image boundary, calculating the pixel difference between the mouth and the image end position of the mouth corresponding to the seedling sample in the RGB-D image, and judging whether the pixel difference is less than or equal to a determined pixel difference threshold; when it is judged that the pixel difference is less than or equal to the pixel difference threshold, calculating the length pixel value of the seedling sample in the RGB-D image; and calculating the body length data of the seedling sample according to the length pixel value.

7. The automatic measurement system for seedlings for proliferation and release according to claim 6, characterized in that: Before the body length detection module calculates the pixel difference, it also includes a completeness detection module: judging whether the seedling sample completely appears in the RGB-D image according to the RGB-D image of the seedling sample; When it is determined that the seedling sample completely appears in the RGB-D image, the body length data of the seedling sample is calculated according to the length pixel value; when it is determined that the seedling sample does not completely appear in the RGB-D image, the blind area length of the seedling sample is obtained, and the blind area length of the seedling sample is used to represent the actual length corresponding to the partial image of the seedling sample that does not appear in the RGB-D image; Among them, calculating the body length data of the seedling sample according to the length pixel value includes: determining the length matching the length pixel value, and calculating the sum of the length matching the length pixel value and the blind area length of the seedling sample as the body length data of the seedling sample.

8. The automatic measurement system for seedlings for proliferation and release according to claim 7, characterized in that: Obtaining the blind area length of the seedling sample in the integrity detection module specifically includes: obtaining the RGB-D image acquisition position where the seedling sample is currently located, and determining the horizontal distance between the image acquisition position and the position of the image acquisition module based on the RGB-D image acquisition position, and determining the blind area length of the seedling sample based on the horizontal distance.

Images