Machine vision-based blue crab sorting method and system

Abstract

The invention discloses a blue crab sorting method and system based on machine vision, and relates to the technical field of machine vision. Comprising the following steps: collecting belly shell texture image information of a plurality of sex-known blue crabs; establishing a sex identification model of the blue crabs, and training the sex identification model of the blue crabs by taking the sample images as input data of the identification model and taking the sex marks in the sample images as labels; the method comprises the following steps: collecting images of belly shell textures and carapace contours of to-be-sorted blue crabs, preprocessing the images, and obtaining weight information of the blue crabs at the same time; inputting the preprocessed belly shell texture images into the trained model to obtain gender information of the to-be-sorted blue crabs, and calculating the areas of the carapace of the blue crabs based on the preprocessed carapace contour images of the to-be-sorted blue crabs; generating a comprehensive evaluation coefficient based on the sex information of the blue crabs, the area size of the carapace of the blue crabs and the weight information of the blue crabs, and grading the blue crabs according to the comprehensive evaluation coefficient.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method and system for sorting blue crabs based on machine vision. Background Art

[0002] In China, blue crabs are mainly distributed in coastal areas such as the South China Sea, the East China Sea and the Yellow Sea. They are also widely farmed in Southeast Asia and other tropical and subtropical regions. Water temperature, salinity, nutritional environment, bait types and other factors in different origins can significantly affect the growth rate, appearance characteristics and nutritional flavor of blue crabs. However, due to the complexity of the supply chain, the origin information of blue crabs is easily confused or missing during the circulation process, especially in cross-regional trade, it is difficult to ensure the accuracy of the origin information. Although some large farms and brands have begun to use blockchain, QR code and other technologies for product traceability, the overall traceability system of the blue crab industry is not perfect, especially among small farmers, the application of traceability technology is more limited. The traditional method of collecting origin information relies on manual records, which is prone to human errors. In addition, the lack of unified traceability standards and regulatory supervision makes it difficult to eliminate the problem of missing or tampering of information in the process of blue crab breeding, transportation and sales. With the increase in market demand for blue crabs, some unscrupulous vendors use forged origins or adulteration to increase profits.

[0003] However, the quality classification of blue crabs mostly relies on manual operations. In the market or wholesale links, workers usually judge the quality of blue crabs based on subjective experience such as appearance and feel. However, due to the large differences in individual size and morphology of blue crabs, manual sorting is time-consuming and labor-intensive, and the sorting results are easily affected by human subjective factors, resulting in inconsistent classification results, which in turn affects market prices and consumer trust. In addition, manual sorting is inefficient and cannot meet the needs of efficient processing when blue crabs are put on the market on a large scale. Especially when the supply chain pressure is high during peak periods, it is easy to cause unsalable or storage problems. With the rise in labor costs, the cost of relying on manual classification of blue crabs has gradually increased. Especially for large-scale breeding and sales companies, this model appears to be even more unsustainable. The key to solving the problem of time-consuming and labor-intensive manual quality classification is to promote the standardization and automation of blue crab quality classification.

[0004] Therefore, it is urgent to develop an automatic and efficient blue crab sorting method to improve sorting efficiency, avoid subjective errors that may occur during manual sorting, ensure that the quality classification of blue crabs is more accurate and consistent, help form a unified market standard, and reduce dependence on manpower.

[0005] In the prior art, the publication number CN116740473B discloses a method and system for automatic sorting of fish based on machine vision, which obtains individual weight information of fish and performs preliminary classification according to the individual weight information of fish; obtains image information of fish after preliminary classification, performs fish category analysis according to the image information of fish, and reclassifies the fish according to the category information of fish; obtains infrared sensor information of reclassified fish and image information of reclassified fish, performs life feature analysis of the fish; obtains appearance image information of fish after life feature classification, performs appearance state analysis of fish and quality grade evaluation according to the appearance image information of fish. The efficiency and accuracy of fish sorting are improved, and the quality grade of fish is evaluated at the same time. However, in this method, if the training data is not diverse enough, the model may not be able to generalize well to new and unseen fish, especially samples of different types, sizes and appearances. When the weight of fish objects is similar, preliminary classification only by weight may lead to misclassification. Different types of catches may overlap in weight, and cannot be effectively classified based on this feature alone. At the same time, the processing process is relatively complicated, which may affect the sorting effect and reduce the real-time and effectiveness of the sorting system.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The object of the present invention is to provide a method and system for sorting blue crabs based on machine vision to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for sorting blue crabs based on machine vision, the specific steps comprising:

[0010] Collecting plastron texture image information of several blue crabs of known gender, marking the gender of the blue crab in each plastron texture image based on manual marking, and recording the marked plastron texture image as a sample image, wherein the gender of the blue crab includes female and male;

[0011] A blue crab gender recognition model is established, sample images are used as input data of the recognition model, and the gender mark in the sample image is used as a label to train the blue crab gender recognition model, and the input is the blue crab plastron texture image, and the output is the recognition model of the blue crab gender in the image;

[0012] The blue crabs to be sorted are passed through a transparent conveyor belt with their plastrons facing upwards into the information collection area, and image information of the plastron texture and carapace outline of the blue crabs to be sorted are collected by a high-definition camera in the information collection area, and the collected images are pre-processed by image enhancement and noise reduction. At the same time, a weighing device is set in the information collection area to obtain the weight information of the blue crabs passing through the information collection area;

[0013] The pre-processed plastron texture image of the blue crab to be sorted is input into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, and based on the pre-processed carapace contour image of the blue crab to be sorted, the edge contour feature information of the carapace contour image of the blue crab is extracted to calculate the area size of the carapace of the blue crab;

[0014] A comprehensive evaluation coefficient is generated based on the gender information, the area of ​​the blue crab's carapace and the blue crab's weight information. The generated comprehensive evaluation coefficient is compared with the pre-set grade threshold, and the blue crabs are graded according to different comparison results. The grade information label is generated and automatically bound and then sent to the corresponding packaging box.

[0015] Furthermore, the specific steps for marking the sex of the blue crab in each plastron texture image by manual marking include: selecting the labeling tool LabelImg, setting the classification in the labeling tool, including male and female, opening the plastron texture images one by one through the labeling tool, and marking the corresponding positions using the labeling tool LabelImg according to the known sex information of the blue crab in each image.

[0016] Furthermore, a blue crab gender recognition model was established based on a convolutional neural network, wherein the convolutional neural network consisted of an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer, wherein the activation function in the convolutional layer was a Tanh function, and for the fully connected layer, the number of neurons in the fully connected layer was set to 32, the initial neural network learning rate was set to 0.001, and the number of training rounds was 100.

[0017] Furthermore, the information collection area includes two high-definition cameras and a weighing device. A high-definition camera 1 is installed above the information collection area to collect information about the blue crab's plastron, and a high-definition camera 2 is installed at the bottom to collect and measure information about the blue crab's carapace. At the same time, a weighing device is set in the information collection area to obtain the weight information of the blue crab passing through the information collection area and calibrate G i is the weight of the i-th blue crab to be sorted, where i is the index of the blue crab, i=1,2,…,n, and n represents the number of blue crabs currently to be sorted.

[0018] Furthermore, the collected plastron texture images of the blue crabs to be sorted are input into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, and the gender information of the blue crabs is mapped and encoded, and the gender information of the female blue crabs is marked as -1, and the gender information of the male blue crabs is marked as 1;

[0019] Based on the carapace contour image of the blue crab to be sorted, the method for extracting the edge contour feature information of the blue crab carapace contour image is as follows: using the Canny edge detection algorithm, find the place with the largest brightness change in the image to form an edge, optimize the edge detection result, and ensure that only the edge of the blue crab carapace is extracted; use the contour detection method to extract the external contour in the image, that is, the outer contour of the blue crab carapace, and convert it into a binary image, the image pixels in the contour area are 255, and the background image pixels outside the contour area are 0, that is, the image in the contour area is white, and the background image outside the contour area is black; based on the converted binary image, count the number of white pixels in the contour area, use the pixel counting method to calculate the contour area, and use this contour area to characterize the area size of the blue crab carapace, wherein the formula based on which the contour area is calculated using the pixel counting method is:

[0020] M i =E i *α

[0021] Where M i represents the carapace area of ​​the i-th crab to be sorted, E i It represents the number of pixels with a pixel value of 255 in the carapace image of the i-th mud crab to be sorted, and α represents the actual area of ​​each pixel.

[0022] Furthermore, a comprehensive evaluation coefficient is generated based on the sex information of the blue crab, the area of ​​the blue crab carapace and the weight information of the blue crab, wherein the formula for calculating the comprehensive evaluation coefficient is:

[0023]

[0024] In the formula, ZH i represents the comprehensive evaluation coefficient of the i-th mud crab to be sorted, It is used to indicate the fatness of the i-th crab to be sorted, H i represents the gender code of the i-th crab to be sorted, where the gender information of female crabs is coded as -1, and the gender information of male crabs is coded as 1, ω 1 ,ω 2 and ω 3 are the weight coefficients of the weight, carapace area and fatness of the i-th crab to be sorted, respectively, where ω 1 ,ω 2 and ω 3 are greater than 0 and ω 1 >ω 2>ω 3 .

[0025] Furthermore, the logic for classifying the mud crabs into different grades according to different comparison results is based on the comparison of the generated comprehensive evaluation coefficient with the pre-set grade threshold value as follows: the pre-set grade threshold value is calibrated as YZ, where YZ is greater than 0;

[0026] If YZ*-1.0≥ZH i When the crab is classified as female crab of first-grade quality;

[0027] If YZ*-1.0 <ZH i When ≤YZ*-0.6, the blue crab is classified as female crab of second-grade quality;

[0028] If YZ*-0.6<ZH i When ≤YZ*-0.2, the blue crab is classified as female crab of grade 3 quality;

[0029] If YZ*0.2 <ZH i When ≤YZ*0.6, the blue crab is classified as male crab of grade 3 quality;

[0030] If YZ*0.6<ZH i When ≤YZ*1.0, the blue crab is classified as male crab of second-grade quality;

[0031] If ZH i >YZ*1.0, the blue crab is classified as male crab of first-grade quality;

[0032] If|ZH i When |≤YZ*0.2, the blue crab is marked as unqualified;

[0033] According to the grade of blue crabs, the grade information tags are generated and automatically bound and then sent to the corresponding packaging boxes.

[0034] The present invention also provides a blue crab sorting system based on machine vision, which is used to execute the blue crab sorting method based on machine vision, and comprises:

[0035] A historical data collection module is used to collect plastron texture image information of a number of blue crabs of known gender, mark the gender of the blue crab in each plastron texture image based on manual marking, and record the marked plastron texture image as a sample image, wherein the gender of the blue crab includes female and male;

[0036] The recognition model training module is used to establish a blue crab gender recognition model, using the sample image as the input data of the recognition model and the gender mark in the sample image as the label to train the blue crab gender recognition model, and obtain a recognition model whose input is the blue crab plastron texture image and output is the blue crab gender in the image;

[0037] A real-time data acquisition module is used to pass the blue crabs to be sorted with their plastrons facing upwards through a transparent conveyor belt into an information collection area, collect image information of the plastron texture and carapace outline of the blue crabs to be sorted through a high-definition camera in the information collection area, perform image enhancement and noise reduction preprocessing on the collected images, and at the same time, a weighing device is set in the information collection area to obtain weight information of the blue crabs passing through the information collection area;

[0038] The feature information acquisition module is used to input the pre-processed plastron texture image of the blue crab to be sorted into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, extract the edge contour feature information of the blue crab carapace contour image based on the pre-processed carapace contour image of the blue crab to be sorted, and calculate the area size of the blue crab carapace;

[0039] The blue crab grade sorting module is used to generate a comprehensive evaluation coefficient based on the gender information, the area of ​​the blue crab carapace and the weight information of the blue crab, compare the generated comprehensive evaluation coefficient with the pre-set grade threshold, and classify the blue crabs according to different comparison results. The grade information label is generated and automatically bound and then transmitted to the corresponding packaging box.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] By introducing modern sorting technology, including computer vision technology and machine learning, it is possible to automatically collect characteristic information such as the plastron, shell pattern, size, weight, and fatness of blue crabs, and automatically classify blue crabs based on this information. This not only greatly improves sorting efficiency, but also avoids subjective errors that may occur during manual sorting. The application of this technology can also ensure that the quality classification of blue crabs is more accurate and consistent, which helps to form a unified market standard and reduce dependence on manpower. At the same time, the traceability system can simplify information management in the supply chain and help companies achieve real-time monitoring of product circulation, thereby reducing waste and unsalable products in the middle links and reducing operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0043] Figure 2 It is a schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0045] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0046] Example:

[0047] See also Figure 1 , the present invention provides a technical solution:

[0048] A method for sorting blue crabs based on machine vision, the specific steps comprising:

[0049] Step 1: Collect plastron texture image information of several blue crabs of known gender, mark the gender of the blue crab in each plastron texture image based on manual marking, and record the marked plastron texture image as a sample image. The gender of the blue crab includes female and male;

[0050] The specific steps for marking the sex of blue crabs in each plastron texture image by manual marking include: selecting the labeling tool LabelImg, opening the terminal or command line, and entering labelImg to start the labeling tool; setting the classification in the labeling tool, including male and female, and in the LabelImg interface, selecting the "Set DefaultLabel" option under the "View" menu, and setting the classification labels to "male" and "female"; opening the plastron texture images one by one through the labeling tool, and clicking the "Open" button according to the known sex information of blue crabs in each image, selecting the folder containing the plastron texture images of blue crabs, and opening the images to be marked one by one, and using the mouse to select the plastron texture area according to the known sex information of blue crabs, and selecting the corresponding label (male or female) on the selected area.

[0051] Step 2: Establish a blue crab gender recognition model, use the sample image as the input data of the recognition model, and use the gender mark in the sample image as the label to train the blue crab gender recognition model, and obtain a recognition model whose input is the blue crab plastron texture image and output is the blue crab gender in the image;

[0052] A blue crab sex recognition model is established based on a convolutional neural network, wherein the convolutional neural network consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer, wherein the activation function in the convolutional layer is the Tanh function, and for the fully connected layer, the number of neurons in the fully connected layer is set to 32, the initial neural network learning rate is set to 0.001, and the number of training rounds is 100.

[0053] The specific expression of the Tanh function is:

[0054]

[0055] In the formula, f(r) represents the Tanh function, and the independent variable r represents the weighted sum of the neuron's input, that is, the result of the weighted summation of the input received by the neuron from the previous layer;

[0056] The trained blue crab gender recognition model takes the blue crab plastron texture image as input and outputs the blue crab gender in the image.

[0057] Step 3: The blue crabs to be sorted are passed through a transparent conveyor belt with their plastrons facing upwards into the information collection area. In the information collection area, image information of the plastron texture and carapace outline of the blue crabs to be sorted is collected by a high-definition camera. The collected images are pre-processed for image enhancement and noise reduction. At the same time, a weighing device is set in the information collection area to obtain the weight information of the blue crabs passing through the information collection area;

[0058] The information collection area includes two high-definition cameras and a weighing device. A high-definition camera 1 is installed directly above the information collection area to collect information about the blue crab's plastron, and a high-definition camera 2 is installed at the bottom to collect and measure information about the blue crab's carapace. A weighing device is also provided in the information collection area to obtain the weight information of the blue crab passing through the information collection area and calibrate G. i is the weight of the i-th blue crab to be sorted, where i is the index of the blue crab, i=1,2,…,n, and n represents the number of blue crabs currently to be sorted.

[0059] The method for performing noise reduction and enhancement processing on the image of the plastron texture and carapace contour of the mud crab to be sorted is: using the wavelet transform denoising method to denoise the image after distortion correction, and the specific steps of the wavelet transform denoising method include: decomposing the image after distortion correction by wavelet transform to obtain the wavelet coefficients of the image at different scales and directions; performing threshold processing on the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; performing inverse transformation on the wavelet coefficients after threshold processing, reconstructing the processed coefficients into an image, and completing the image denoising processing;

[0060] Bilateral filtering is used to enhance the details of the images of the plastron texture and carapace contour of the mud crab to be sorted. The formula based on the specific filtering transformation is:

[0061]

[0062] Where y is the coordinate vector in the image coordinate system, I y is the gray value at the coordinate vector y, B y is the gray value I y The gray value after bilateral filtering transformation, G d and G r are all Gaussian functions, where G d and G r The formula is:

[0063]

[0064]

[0065] Where x is the coordinate vector in the image coordinate system, I x is the gray value at the coordinate vector x, σ d and σ r G d and G r The standard deviation of .

[0066] Step 4: Input the pre-processed plastron texture image of the blue crab to be sorted into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, extract the edge contour feature information of the blue crab carapace contour image based on the pre-processed carapace contour image of the blue crab to be sorted, and calculate the area of ​​the blue crab carapace;

[0067] Input the collected plastron texture images of the blue crabs to be sorted into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, map and encode the gender information of the blue crabs, mark the gender information of female blue crabs as -1, and mark the gender information of male blue crabs as 1;

[0068] Based on the carapace contour image of the blue crab to be sorted, the method for extracting the edge contour feature information of the blue crab carapace contour image is as follows: using the Canny edge detection algorithm, find the place with the largest brightness change in the image to form an edge, optimize the edge detection result, and ensure that only the edge of the blue crab carapace is extracted; use the contour detection method to extract the external contour in the image, that is, the outer contour of the blue crab carapace, and convert it into a binary image, the image pixels in the contour area are 255, and the background image pixels outside the contour area are 0, that is, the image in the contour area is white, and the background image outside the contour area is black; based on the converted binary image, count the number of white pixels in the contour area, use the pixel counting method to calculate the contour area, and use this contour area to characterize the area size of the blue crab carapace, wherein the formula based on which the contour area is calculated using the pixel counting method is:

[0069] M i =E i *α

[0070] Where M i represents the carapace area of ​​the i-th crab to be sorted, E i represents the number of pixels with a pixel value of 255 in the carapace image of the i-th blue crab to be sorted, and α represents the actual area of ​​each pixel, wherein the actual area α represented by each pixel is the same.

[0071] The Sobel operator is used to extract edge features. The specific steps include: using the Sobel operator to calculate the gradient of the image in the horizontal direction and the vertical direction. The formula for calculating the gradient of the image in the horizontal direction and the vertical direction is:

[0072]

[0073]

[0074] Where, L x is the horizontal gradient of the image, L y is the vertical gradient of the image, represents the convolution operation, I is the input image; the formula for calculating the gradient magnitude and direction is:

[0075]

[0076]

[0077] Where L is the gradient amplitude and θ is the direction;

[0078] Non-maximum suppression is performed along the gradient direction, local maxima are retained, non-edge points are suppressed, high thresholds and low thresholds are used to distinguish strong edges from weak edges, and edge detection is completed by connecting weak edges and strong edges.

[0079] Step 5: Generate a comprehensive evaluation coefficient based on the sex information, carapace area and weight information of the blue crab. Compare the generated comprehensive evaluation coefficient with the pre-set grade threshold, classify the blue crabs according to different comparison results, generate and automatically bind grade information labels, and then send them to the corresponding packaging boxes.

[0080] The comprehensive evaluation coefficient is generated based on the sex information, carapace area and weight information of the blue crab. The formula for calculating the comprehensive evaluation coefficient is:

[0081]

[0082] In the formula, ZH irepresents the comprehensive evaluation coefficient of the i-th mud crab to be sorted, It is used to indicate the fatness of the i-th crab to be sorted, H i represents the gender code of the i-th crab to be sorted, where the gender information of female crabs is coded as -1, and the gender information of male crabs is coded as 1, ω 1 ,ω 2 and ω 3 are the weight coefficients of the weight, carapace area and fatness of the i-th crab to be sorted, respectively, where ω 1 ,ω 2 and ω 3 Both are greater than 0. Since the quality of blue crabs has the greatest impact on the quality of blue crabs, the carapace area represents the size of blue crabs. The size of blue crabs has a lower impact on the quality of blue crabs than the quality. The fatness of blue crabs has a slightly smaller impact on the quality of blue crabs than the size of blue crabs. Therefore, ω is set. 1 >ω 2 >ω 3 .

[0083] According to the comparison between the generated comprehensive evaluation coefficient and the pre-set grade threshold, the logic for classifying the blue crabs into grades according to different comparison results is as follows: the pre-set grade threshold is calibrated as YZ, where YZ is greater than 0;

[0084] If YZ*-1.0≥ZH i When the crab is classified as female crab of first-grade quality;

[0085] If YZ*-1.0<ZH i When ≤YZ*-0.6, the blue crab is classified as female crab of second-grade quality;

[0086] If YZ*-0.6<ZH i When ≤YZ*-0.2, the blue crab is classified as female crab of grade 3 quality;

[0087] If YZ*0.2 <ZH i When ≤YZ*0.6, the blue crab is classified as male crab of grade 3 quality;

[0088] If YZ*0.6<ZH i When ≤YZ*1.0, the blue crab is classified as male crab of second-grade quality;

[0089] If ZH i >YZ*1.0, the blue crab is classified as male crab of first-grade quality;

[0090] If|ZH i When |≤YZ*0.2, the blue crab is marked as unqualified;

[0091] According to the grade of blue crabs, the grade information tags are generated and automatically bound and then sent to the corresponding packaging boxes.

[0092] See also Figure 2 The present invention also provides a blue crab sorting system based on machine vision, and the blue crab sorting system based on machine vision is used to execute the blue crab sorting method based on machine vision, comprising:

[0093] A historical data collection module is used to collect plastron texture image information of a number of blue crabs of known gender, mark the gender of the blue crab in each plastron texture image based on manual marking, and record the marked plastron texture image as a sample image, wherein the gender of the blue crab includes female and male;

[0094] The recognition model training module is used to establish a blue crab gender recognition model, using the sample image as the input data of the recognition model and the gender mark in the sample image as the label to train the blue crab gender recognition model, and obtain a recognition model whose input is the blue crab plastron texture image and output is the blue crab gender in the image;

[0095] A real-time data acquisition module is used to pass the blue crabs to be sorted with their plastrons facing upwards through a transparent conveyor belt into an information collection area, collect image information of the plastron texture and carapace outline of the blue crabs to be sorted through a high-definition camera in the information collection area, perform image enhancement and noise reduction preprocessing on the collected images, and at the same time, a weighing device is set in the information collection area to obtain weight information of the blue crabs passing through the information collection area;

[0096] The feature information acquisition module is used to input the pre-processed plastron texture image of the blue crab to be sorted into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, extract the edge contour feature information of the blue crab carapace contour image based on the pre-processed carapace contour image of the blue crab to be sorted, and calculate the area size of the blue crab carapace;

[0097] The blue crab grade sorting module is used to generate a comprehensive evaluation coefficient based on the gender information, the area of ​​the blue crab carapace and the weight information of the blue crab, compare the generated comprehensive evaluation coefficient with the pre-set grade threshold, and classify the blue crabs according to different comparison results. The grade information label is generated and automatically bound and then transmitted to the corresponding packaging box.

[0098] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0099] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for sorting blue crabs based on machine vision, characterized in that: The specific steps include: Collecting plastron texture image information of several blue crabs of known gender, marking the gender of the blue crab in each plastron texture image based on manual marking, and recording the marked plastron texture image as a sample image, wherein the gender of the blue crab includes female and male; A blue crab gender recognition model is established, sample images are used as input data of the recognition model, and the gender mark in the sample image is used as a label to train the blue crab gender recognition model, and the input is the blue crab plastron texture image, and the output is the recognition model of the blue crab gender in the image; The blue crabs to be sorted are passed through a transparent conveyor belt with their plastrons facing upwards into the information collection area, and image information of the plastron texture and carapace outline of the blue crabs to be sorted are collected by a high-definition camera in the information collection area, and the collected images are pre-processed by image enhancement and noise reduction. At the same time, a weighing device is set in the information collection area to obtain the weight information of the blue crabs passing through the information collection area; The pre-processed plastron texture image of the blue crab to be sorted is input into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, and based on the pre-processed carapace contour image of the blue crab to be sorted, the edge contour feature information of the carapace contour image of the blue crab is extracted to calculate the area size of the carapace of the blue crab; A comprehensive evaluation coefficient is generated based on the gender information, the area of ​​the blue crab's carapace and the blue crab's weight information. The generated comprehensive evaluation coefficient is compared with the pre-set grade threshold, and the blue crabs are graded according to different comparison results. The grade information label is generated and automatically bound and then sent to the corresponding packaging box.

2. The method for sorting blue crabs based on machine vision according to claim 1, characterized in that: The specific steps for marking the sex of blue crabs in each plastron texture image by manual marking include: selecting the labeling tool LabelImg, setting the classification in the labeling tool, including male and female, opening the plastron texture images one by one through the labeling tool, and marking the corresponding positions using the labeling tool LabelImg according to the known sex information of blue crabs in each image.

3. The method for sorting blue crabs based on machine vision according to claim 2, characterized in that: A blue crab sex recognition model is established based on a convolutional neural network, wherein the convolutional neural network consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer, wherein the activation function in the convolutional layer is the Tanh function, and for the fully connected layer, the number of neurons in the fully connected layer is set to 32, the initial neural network learning rate is set to 0.001, and the number of training rounds is 100.

4. The method for sorting blue crabs based on machine vision according to claim 1, characterized in that: The information collection area includes two high-definition cameras and a weighing device. A high-definition camera 1 is installed directly above the information collection area to collect information about the blue crab's plastron, and a high-definition camera 2 is installed at the bottom to collect and measure information about the blue crab's carapace. A weighing device is also provided in the information collection area to obtain the weight information of the blue crab passing through the information collection area and calibrate G. i is the weight of the i-th blue crab to be sorted, where i is the index of the blue crab, i=1,2,…,n, and n represents the number of blue crabs currently to be sorted.

5. The method for sorting blue crabs based on machine vision according to claim 4, characterized in that: Input the collected plastron texture images of the blue crabs to be sorted into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, map and encode the gender information of the blue crabs, mark the gender information of female blue crabs as -1, and mark the gender information of male blue crabs as 1; Based on the carapace contour image of the blue crab to be sorted, the method for extracting the edge contour feature information of the blue crab carapace contour image is as follows: using the Canny edge detection algorithm, find the place with the largest brightness change in the image to form an edge, optimize the edge detection result, and ensure that only the edge of the blue crab carapace is extracted; use the contour detection method to extract the external contour in the image, that is, the outer contour of the blue crab carapace, and convert it into a binary image, the image pixels in the contour area are 255, and the background image pixels outside the contour area are 0, that is, the image in the contour area is white, and the background image outside the contour area is black; based on the converted binary image, count the number of white pixels in the contour area, use the pixel counting method to calculate the contour area, and use this contour area to characterize the area size of the blue crab carapace, wherein the formula based on which the contour area is calculated using the pixel counting method is: M i =E i *a Where M i represents the carapace area of ​​the i-th crab to be sorted, E i It represents the number of pixels with a pixel value of 255 in the carapace image of the i-th mud crab to be sorted, and α represents the actual area of ​​each pixel.

6. The method for sorting blue crabs based on machine vision according to claim 5, characterized in that: The comprehensive evaluation coefficient is generated based on the sex information, carapace area and weight information of the blue crab. The formula for calculating the comprehensive evaluation coefficient is: In the formula, ZH i represents the comprehensive evaluation coefficient of the i-th mud crab to be sorted, It is used to indicate the fatness of the i-th crab to be sorted, H i represents the gender code of the i-th mud crab to be sorted, where the gender information of female mud crabs is encoded as -1, and the gender information of male mud crabs is encoded as 1. ω1, ω2 and ω3 are the weight coefficients of the weight, carapace area and fatness of the i-th mud crab to be sorted, respectively. Among them, ω1, ω2 and ω3 are all greater than 0 and ω1>ω2>ω3.

7. The method for sorting blue crabs based on machine vision according to claim 6, characterized in that: According to the comparison between the generated comprehensive evaluation coefficient and the pre-set grade threshold, the logic for classifying the blue crabs into grades according to different comparison results is as follows: the pre-set grade threshold is calibrated as YZ, where YZ is greater than 0; If YZ*-1.0≥ZH i When the crab is classified as female crab of first-grade quality; If YZ*-1.0 <ZH i When ≤YZ*-0.6, the blue crab is classified as female crab of second-grade quality; If YZ*-0.6 <ZH i When ≤YZ*-0.2, the blue crab is classified as female crab of grade 3 quality; If YZ*0.2 <ZH i When ≤YZ*0.6, the blue crab is classified as male crab of grade 3 quality; If YZ*0.6 <ZH i When ≤YZ*1.0, the blue crab is classified as male crab of second-grade quality; If ZH i When >YZ*1.0, the blue crab is classified as male crab of first-grade quality; If|ZH i When |≤YZ*0.2, the blue crab is marked as unqualified; According to the grade of blue crabs, the grade information tags are generated and automatically bound and then sent to the corresponding packaging boxes.

8. A mud crab sorting system based on machine vision, characterized in that: The blue crab sorting system based on machine vision is used to execute the blue crab sorting method based on machine vision according to any one of claims 1 to 7, comprising: A historical data collection module is used to collect plastron texture image information of a number of blue crabs of known gender, mark the gender of the blue crab in each plastron texture image based on manual marking, and record the marked plastron texture image as a sample image, wherein the gender of the blue crab includes female and male; The recognition model training module is used to establish a blue crab gender recognition model, using the sample image as the input data of the recognition model and the gender mark in the sample image as the label to train the blue crab gender recognition model, and obtain a recognition model whose input is the blue crab plastron texture image and output is the blue crab gender in the image; A real-time data acquisition module is used to pass the blue crabs to be sorted with their plastrons facing upwards through a transparent conveyor belt into an information collection area, collect image information of the plastron texture and carapace outline of the blue crabs to be sorted through a high-definition camera in the information collection area, perform image enhancement and noise reduction preprocessing on the collected images, and at the same time, a weighing device is set in the information collection area to obtain weight information of the blue crabs passing through the information collection area; The feature information acquisition module is used to input the pre-processed plastron texture image of the blue crab to be sorted into the trained blue crab gender recognition model to obtain the gender information of each blue crab to be sorted, extract the edge contour feature information of the blue crab carapace contour image based on the pre-processed carapace contour image of the blue crab to be sorted, and calculate the area size of the blue crab carapace; The blue crab grade sorting module is used to generate a comprehensive evaluation coefficient based on the gender information, the area of ​​the blue crab carapace and the weight information of the blue crab, compare the generated comprehensive evaluation coefficient with the pre-set grade threshold, and classify the blue crabs according to different comparison results. The grade information label is generated and automatically bound and then transmitted to the corresponding packaging box.

Images