Computer Vision-Based Monitoring Method for the Health Status of Silkworms

Abstract

The present invention relates to the technical field of image segmentation, and particularly to a method for monitoring the health status of silkworms based on computer vision. The method includes: acquiring images of silkworm trays at different growth stages; determining a normal silkworm morphological connected domain and an abnormal silkworm morphological connected domain according to the morphological distribution characteristics of the edge lines in the silkworm tray images; evaluating the possibility that the abnormal edge region in the abnormal silkworm morphological connected domain belongs to another silkworm according to the difference in the bending conditions of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm morphological connected domain, and determining an overlapping silkworm morphological region; screening a reference connected domain in combination with the similarity of the shape distributions of the edge lines between the abnormal edge region and the surrounding normal silkworm morphological connected domains in the overlapping silkworm morphological region; re-dividing the overlapping silkworm morphological region according to the position distributions of the abnormal edge region and its reference connected domain; and evaluating the growth status of the silkworms in combination with the final division result. The present invention improves the accuracy of the monitoring result of the growth status of silkworms.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and particularly to a method for monitoring the health status of silkworms based on computer vision. Background Art

[0002] During the breeding process, silkworms are vulnerable to various diseases, which may lead to abnormal growth states of silkworms and thus affect the quality of cocoons. Therefore, it is crucial to monitor the health status of silkworms during the breeding process. Traditional monitoring of the health status of silkworms mainly relies on manual observation, and this method has many drawbacks. With the continuous development of computer technology and image processing technology, computer vision technology has been increasingly widely applied in the agricultural field.

[0003] Currently, during the process of monitoring the health status of silkworms based on computer vision, due to the gregarious characteristics of silkworms, they usually like to gather together. Therefore, in the visual images of silkworms obtained, there are usually stacks among silkworms. As a result, the silkworms located below may be blocked, leading to the computer vision system being unable to obtain the complete and accurate growth characteristic information of these silkworms, affecting the monitoring results of the health status of silkworms, and making the accuracy of the evaluation results of the growth status of silkworms relatively low. Summary of the Invention

[0004] In order to solve the problem that the existing method cannot accurately and completely extract the growth characteristic information of silkworms, the purpose of the present invention is to provide a method for monitoring the health status of silkworms based on computer vision, and the specific technical solutions adopted are as follows:

[0005] The present invention provides a method for monitoring the health status of silkworms based on computer vision, and this method includes the following steps:

[0006] Obtain silkworm tray images at different growth stages;

[0007] According to the morphological distribution characteristics of the edge lines in the silkworm tray images, determine the normal silkworm morphological connected regions and the abnormal silkworm morphological connected regions; according to the difference in the bending conditions of the edge lines between the abnormal edge regions and the normal edge regions in the abnormal silkworm morphological connected regions, evaluate the possibility that the abnormal edge regions in the abnormal silkworm morphological connected regions belong to another silkworm, and based on this possibility, judge whether the abnormal silkworm morphological connected regions belong to the overlapping silkworm morphological regions; the abnormal edge regions and the normal edge regions are obtained by clustering the convex hull vertices in the connected regions.

[0008] Combined with the similarity of the shape distribution of the edge lines between the abnormal edge regions in the overlapping silkworm morphological regions and the surrounding normal silkworm morphological connected regions, screen the reference connected regions; according to the relative position distribution between the abnormal edge regions and their reference connected regions, re-divide the overlapping silkworm morphological regions.

[0009] Evaluate the growth state of silkworms by combining the segmentation results of silkworm tray images at different growth stages.

[0010] Preferably, determining the normal silkworm morphological connected regions and abnormal silkworm morphological connected regions according to the morphological distribution characteristics of the edge lines in the silkworm tray image includes:

[0011] Extract the connected regions in the silkworm tray image, and perform convex hull detection on the connected regions to obtain the convex hull vertices;

[0012] For any connected region, obtain the minimum bounding rectangle of the connected region, record the longest side of the minimum bounding rectangle as the body length direction of the silkworm, and record the two pixel points with the farthest distance in the body length direction of the edge line of each connected region as the suspected silkworm head point and the suspected silkworm tail point respectively; According to the difference between the number of convex hull vertices in the connected region and the preset number of vertices, and the distances between the convex hull vertices and the suspected silkworm head point and the suspected silkworm tail point, obtain the silkworm morphological validity of the connected region.

[0013] If the silkworm morphological validity is greater than or equal to the preset validity threshold, determine that the corresponding connected region is a normal silkworm morphological connected region; if the silkworm morphological validity is less than the preset validity threshold, determine that the corresponding connected region is an abnormal silkworm morphological connected region.

[0014] Preferably, obtaining the abnormal edge region and the normal edge region includes:

[0015] For any convex hull vertex in any connected region, respectively obtain the minimum value among the Euclidean distances between the any convex hull vertex and the suspected silkworm head point and the Euclidean distance between the any convex hull vertex and the suspected silkworm tail point; if the normalized value of the minimum value among the Euclidean distances is greater than the preset distance threshold, determine that the any convex hull vertex is a first type of point; if the normalized value of the minimum value among the Euclidean distances is less than or equal to the preset distance threshold, determine that the any convex hull vertex is a second type of point;

[0016] Cluster each type of point in the silkworm tray image respectively, and use the convex hull region corresponding to the clustering cluster obtained by clustering the first type of points as the abnormal edge region, and use the convex hull region corresponding to the clustering cluster obtained by clustering the second type of points as the normal edge region.

[0017] Preferably, the evaluating the growth state of silkworms by combining the segmentation results of silkworm tray images at different growth stages includes:

[0018] Evaluate the stoutness of silkworms according to the area distribution of all connected regions in the silkworm tray image; evaluate the color of silkworms according to the overall gray information of all connected regions in the silkworm tray image; evaluate the length of silkworms according to the size of all connected regions in the silkworm tray image.

[0019] Determine the validity of the growth state of the silkworms in the silkworm tray by combining the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages, where the validity of the growth state is used to characterize the growth state of the silkworms; among them, the characteristics of the silkworms include the stoutness, color, and length of the silkworms.

[0020] Preferably, the determining the validity of the growth state of the silkworms in the silkworm tray by combining the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages includes:

[0021] Calculate the change amount of each characteristic of the silkworms in the silkworm tray images at two adjacent growth stages according to the characteristic values of the same characteristic of the silkworms in the silkworm tray images at two adjacent growth stages.

[0022] Obtain the validity of the growth state of the silkworms in the silkworm tray according to the change amounts of all the characteristics of the silkworms in the silkworm tray images at two adjacent growth stages, and there is a positive correlation between the change amount and the validity of the growth state.

[0023] Preferably, the evaluating the possibility that the abnormal edge area in the abnormal silkworm morphology connected domain belongs to another silkworm according to the difference in the bending conditions of the edge lines between the abnormal edge area and the normal edge area in the abnormal silkworm morphology connected domain, and judging whether the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area based on the possibility includes:

[0024] For any abnormal silkworm morphology connected domain:

[0025] Calculate the first average curvature value of all pixel points on the edge line of each normal edge area in the any abnormal silkworm morphology connected domain respectively, and obtain the minimum value of the difference between the first average curvature value corresponding to each normal edge area in the any abnormal silkworm morphology connected domain and the average curvature value of all pixel points on each abnormal edge area in the any abnormal silkworm morphology connected domain; determine the possibility that the corresponding abnormal edge area in the any abnormal silkworm morphology connected domain belongs to another silkworm according to the minimum value of the difference between the average curvature values, and there is a negative correlation between the minimum value of the difference between the average curvature values and the possibility of belonging to another silkworm;

[0026] If the possibility is greater than or equal to the preset possibility threshold, it is determined that the corresponding abnormal edge area belongs to the silkworm morphology edge area;

[0027] Judge whether the any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area according to the proportion of the number of silkworm morphology edge areas in the any abnormal silkworm morphology connected domain.

[0028] Preferably, the judging whether the any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area according to the proportion of the number of silkworm morphology edge areas in the any abnormal silkworm morphology connected domain includes:

[0029] The ratio between the number of the silkworm form edge regions in any one of the abnormal silkworm form connected regions and the number of the abnormal edge regions in any one of the abnormal silkworm form connected regions is determined as the proportion of the number corresponding to any one of the abnormal silkworm form connected regions.

[0030] If the proportion is greater than or equal to the proportion threshold, it is determined that any one of the abnormal silkworm form connected regions belongs to the overlapping silkworm form region.

[0031] Preferably, screening the reference connected region by combining the shape distribution similarity of the abnormal edge region in the overlapping silkworm form region and the edge line of the surrounding normal silkworm form connected region includes:

[0032] Taking a preset number of normal silkworm form connected regions closest to the region to be analyzed as candidate connected regions;

[0033] For the region to be analyzed and the candidate connected regions, draw a straight line parallel to the longest side of the minimum circumscribed rectangle of the region to be analyzed through the center point of the region. The straight line divides the edge line of the region into two curve segments; respectively fit each curve segment to obtain the corresponding change trend, and extend each end point position of the curve segment based on the change trend to obtain the extended line; where the number of extended pixel points is the preset pixel point value; connect the extended lines, and take the obtained closed region as the corresponding extended region.

[0034] According to the correlation between the pixel positions in the extended region corresponding to the region to be analyzed and the extended regions corresponding to each candidate connected region, screen the reference connected region of the abnormal edge region.

[0035] The region to be analyzed is any abnormal edge region in the overlapping silkworm form region.

[0036] Preferably, screening the reference connected region of the abnormal edge region according to the correlation between the pixel positions in the extended region corresponding to the region to be analyzed and the extended regions corresponding to each candidate connected region includes:

[0037] The correlation coefficient between the pixel coordinate sets in the extended region corresponding to the region to be analyzed and the pixel coordinate sets in the extended regions corresponding to each candidate connected region is recorded as the membership factor of each candidate connected region.

[0038] The candidate connected regions with membership factors greater than the preset membership threshold are determined as the reference connected regions of the region to be analyzed.

[0039] Preferably, re - dividing the overlapping silkworm form region according to the relative position distribution of the abnormal edge region and its reference connected region includes:

[0040] Divide the pixel points in the overlapping part of the extended region corresponding to the region to be analyzed and the extended region corresponding to the reference connected domain into the normal silkworm morphology connected domain where the reference connected domain is located.

[0041] The present invention has at least the following beneficial effects:

[0042] According to the shape characteristics of the connected domains corresponding to each silkworm in each silkworm tray image, the present invention divides all the connected domains into normal silkworm morphology connected domains and abnormal silkworm morphology connected domains. Then, by analyzing the difference in the bending situation of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm morphology connected domain, the possibility that the abnormal edge region in the abnormal silkworm morphology connected domain belongs to another silkworm is obtained, and then the overlapping silkworm morphology region is found. Then, by combining the similarity of the shape distribution of the edge lines between the abnormal edge region and the surrounding normal silkworm morphology connected domains in the overlapping silkworm morphology region, the extra area in the overlapping silkworm morphology connected domain is divided to the corresponding connected domains around it, so as to obtain a more accurate connected domain division result in each silkworm tray image. Finally, by combining the division results of the silkworm tray images in different growth stages, the growth state of the silkworms is accurately evaluated, which ensures the accuracy of the growth characteristic extraction results of different silkworms in the silkworm tray images and improves the accuracy of the silkworm growth state monitoring results. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of a method for monitoring the health status of silkworms based on computer vision provided by an embodiment of the present invention;

[0045] Figure 2 It is a structural block diagram of a system for monitoring the health status of silkworms based on computer vision provided by an embodiment of the present invention. Detailed Embodiments

[0046] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following will be a detailed description of a method for monitoring the health status of silkworms based on computer vision proposed according to the present invention in combination with the drawings and preferred embodiments.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0048] The following specifically describes the specific solution of a method for monitoring the health status of silkworms based on computer vision in conjunction with the accompanying drawings.

[0049] Embodiment of the method for monitoring the health status of silkworms based on computer vision:

[0050] The specific scenario targeted by this embodiment is as follows: During the process of detecting the growth status of silkworms using computer vision methods, images of silkworm trays at different growth stages are collected. By means of the morphological features and edge morphological features of the connected regions corresponding to each silkworm in each silkworm tray image, the silkworm tray image is divided into multiple connected regions. Combining the characteristics presented by the connected regions in the silkworm tray images at different growth stages, the growth status of silkworms is evaluated.

[0051] This embodiment proposes a method for monitoring the health status of silkworms based on computer vision. As Figure 1 shown, the method for monitoring the health status of silkworms based on computer vision in this embodiment includes the following steps:

[0052] Step S1, obtain silkworm tray images at different growth stages.

[0053] Silkworms will go through four stages in the growth process: eggs, larvae, pupae, and adults. Among them, the larval stage will go through multiple instars, and the body length and weight of silkworms will increase in each instar. Therefore, the monitoring of the growth status of silkworm larvae is relatively important. In the actual application of modern large-scale sericulture, the method of using multi-layer silkworm trays for breeding is usually adopted. Therefore, first, directly above the silkworm tray, a high-speed camera is used to take images of the silkworm trays at different growth stages during the growth process of silkworms; it should be noted that: there are silkworms in the growth process placed on the silkworm tray, and the silkworms in the same silkworm tray are produced from the eggs of the same batch. In this embodiment, one silkworm tray is taken as an example for illustration, and the method provided in this embodiment can be used to process other silkworm trays. Then, the collected images of the silkworm trays are subjected to grayscale processing, and the images after grayscale processing are recorded as silkworm tray images. Image grayscale processing is a prior art and will not be elaborated here.

[0054] So far, this embodiment has collected silkworm tray images at different growth stages.

[0055] Step S2, determine the normal silkworm morphological connected region and the abnormal silkworm morphological connected region according to the morphological distribution characteristics of the edge lines in the silkworm tray image; evaluate the possibility that the abnormal edge region in the abnormal silkworm morphological connected region belongs to another silkworm based on the difference in the bending of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm morphological connected region, and judge whether the abnormal silkworm morphological connected region belongs to the overlapping silkworm morphological region based on the possibility; the abnormal edge region and the normal edge region are obtained by clustering the convex hull vertices in the connected region.

[0056] Use the marker - based connected component detection algorithm to perform connected component detection on each silkworm tray image, and extract multiple connected components in each silkworm tray image. The marker - based connected component detection algorithm is an existing technology and will not be elaborated here.

[0057] Since there is an overlapping situation of silkworms in each silkworm tray image, some edges of the silkworms in the obtained silkworm tray images will be blocked or overlapped, so it is necessary to optimize the edge contours of the silkworms in each silkworm tray image.

[0058] Since the movement of silkworms is small and they show a slender or arc - shaped form, when there is no occlusion, overlap, etc., there is no convex hull in each connected component, and the edge of the connected component is relatively smooth. Therefore, each connected component will only bulge at the head or tail. Therefore, first use the convex hull detection algorithm to perform convex hull detection on each connected component in the silkworm tray image, and determine whether there is an overlap or other situation in the silkworm edge corresponding to each connected component according to the convex hull detection result. Among them, the convex hull detection algorithm for convex hull detection is an existing technology and will not be elaborated here. After the convex hull detection is completed, the convex hull vertex set of each connected component can be obtained.

[0059] The fewer the number of vertices of the convex hull within the connected component, and the closer the convex hull vertices are to the head or tail of the silkworm, the greater the possibility that they are the head and tail of the silkworm. Based on this feature, analyze the morphological characteristics of each connected component in the silkworm tray image through the convex hull detection result to determine the silkworm morphological validity of each connected component.

[0060] Next, this embodiment will take one connected component as an example for illustration, and the method provided in this embodiment can be used to process other connected components.

[0061] Specifically, for any connected component in the silkworm tray image, obtain the minimum bounding rectangle of the connected component, record the longest side of the minimum bounding rectangle as the body length direction of the silkworm, and record the two pixel points with the farthest distance in the body length direction of the silkworm among the edge lines of each connected component as the suspected silkworm head point and the suspected silkworm tail point respectively; according to the difference between the number of convex hull vertices in the connected component and the preset number of vertices, and the distances between the convex hull vertices and the suspected silkworm head point and the suspected silkworm tail point, obtain the silkworm morphological validity of the connected component.

[0062] In this embodiment, a specific calculation formula for the silkworm morphological validity is given. The silkworm morphological validity of the i - th connected component in the m - th silkworm tray image can be expressed as:

[0063]

[0064] In the formula, represents the silkworm morphological validity of the i - th connected component in the m - th silkworm tray image, represents the number of convex hull vertices in the i - th connected component in the m - th silkworm tray image, represents the preset number of vertices, represents calculating the Euclidean distance, represents the coordinates of the j-th convex hull vertex in the i-th connected component in the m-th silkworm tray image, represents the coordinates of the suspected silkworm head point in the minimum bounding rectangle of the i-th connected component in the m-th silkworm tray image, represents the coordinates of the suspected silkworm tail point in the minimum bounding rectangle of the i-th connected component in the m-th silkworm tray image, represents and the Euclidean distance between, represents and the Euclidean distance between, represents the normalization function, represents the function of taking the minimum value, represents taking and the minimum value of the two Euclidean distances.

[0065] In this embodiment, the preset number of vertices is 4. In specific applications, the implementer can set it according to specific circumstances. In this embodiment, adding a constant 1 to the two denominators of the calculation formula for the silkworm form effectiveness is to avoid the denominator being 0.

[0066] The smaller the value of, the closer the number of convex hull vertices in the i-th connected component in the m-th silkworm tray image is to normal. The smaller the value, the closer the convex hull vertices in the m-th silkworm tray image are to the head or tail of the silkworm. Since it is not clear whether the j-th convex hull vertex is closer to the head or the tail, in this embodiment, the Euclidean distance between the j-th convex hull vertex and the suspected silkworm head point and the Euclidean distance between the j-th convex hull vertex and the suspected silkworm tail point are calculated, and the minimum distance between the two is selected to participate in the calculation of the silkworm form effectiveness. When the difference between the number of convex hull vertices in the i-th connected component and the preset number of vertices is larger, and the minimum value of the Euclidean distance between the j-th convex hull vertex and the suspected silkworm head point and the Euclidean distance between the j-th convex hull vertex and the suspected silkworm tail point is larger, the value of the silkworm form effectiveness is smaller.

[0067] By using the above method, the silkworm form effectiveness of each connected component in the silkworm tray image can be obtained.

[0068] If the silkworm form validity is greater than or equal to the preset validity threshold, the corresponding connected region is determined as a normal silkworm form connected region; if the silkworm form validity is less than the preset validity threshold, the corresponding connected region is determined as an abnormal silkworm form connected region, and the abnormal silkworm form connected region may be the region where the silkworm forms overlap. In this embodiment, the preset validity threshold is 0.7, and in specific applications, the implementer can set it according to specific circumstances. By using this method, all the connected regions in the silkworm tray image are divided into two categories, namely the normal silkworm form connected region and the abnormal silkworm form connected region.

[0069] Then, by analyzing the distribution relationship between the abnormal silkworm form connected region and the surrounding normal silkworm form connected regions, it is determined whether there is an overlapping abnormality in the abnormal silkworm form connected region, and the overlapping region is separated.

[0070] Specifically, for any convex hull vertex in any connected region, the minimum value between the Euclidean distance between this convex hull vertex and the suspected silkworm head point and the Euclidean distance between this convex hull vertex and the suspected silkworm tail point is obtained respectively; if the normalized value of this minimum value is greater than the preset distance threshold, this convex hull vertex is determined as a first type of point; if the normalized value of this minimum value is less than or equal to the preset distance threshold, this convex hull vertex is determined as a second type of point, and the second type of point is more likely to be the point of the silkworm head or the silkworm tail. The data normalization method is a prior art and will not be elaborated here too much. In this embodiment, the preset distance threshold is 0.5, and in specific applications, the implementer can set it according to specific circumstances.

[0071] The K-means algorithm is used to cluster each type of point in the silkworm tray image respectively, that is, to cluster the first type of points and the second type of points in the silkworm tray image respectively. The convex hull region corresponding to the clustering cluster obtained by clustering the first type of points is used as the abnormal edge region, and the convex hull region corresponding to the clustering cluster obtained by clustering the second type of points is used as the normal edge region. When using the K-means algorithm for clustering, the value of k is determined by the silhouette coefficient method, which is a prior art. As other implementation manners, the value of k can also be set manually according to specific circumstances. The K-means algorithm is a prior art and will not be elaborated here too much.

[0072] For any abnormal silkworm form connected region:

[0073] Calculate the average curvature value of all pixel points on the edge line of each normal edge area within the connected area of the abnormal silkworm morphology, and record this average curvature value as the first average curvature value. It should be noted that: there is a corresponding first average curvature value for each normal edge area; obtain the minimum value of the difference between the first average curvature value corresponding to each normal edge area within the connected area of the abnormal silkworm morphology and the average curvature value of all pixel points on each abnormal edge area within the connected area of the abnormal silkworm morphology; determine the possibility that the corresponding abnormal edge area within the connected area of the abnormal silkworm morphology belongs to another silkworm according to the minimum value of the difference between the average curvature values. The minimum value of the difference between the average curvature values has a negative correlation with the possibility of belonging to another silkworm.

[0074] Among them, the negative correlation means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases. It can be a subtraction relationship, a division relationship, etc., which is determined by the actual application.

[0075] In this embodiment, a specific calculation formula for the possibility that the abnormal edge area belongs to another silkworm is given. The possibility that the k-th abnormal edge area in the n-th connected area of the abnormal silkworm morphology belongs to another silkworm can be expressed as:

[0076]

[0077] In the formula, represents the possibility that the k-th abnormal edge area in the n-th connected area of the abnormal silkworm morphology belongs to another silkworm, represents the minimum value of the difference between the average curvature value of all pixel points on the edge line of the normal edge area in the n-th connected area of the abnormal silkworm morphology and the average curvature value of all pixel points on the edge line of the k-th abnormal edge area in the n-th connected area of the abnormal silkworm morphology.

[0078] Adding a constant 1 to the denominator of the calculation formula for the possibility that the abnormal edge area belongs to another silkworm is to prevent the denominator from being 0. In specific applications, the implementer can set it according to the specific situation. It should be noted that: the specific calculation method for the difference between the average curvature value of all pixel points on the edge line of the normal edge area and the average curvature value of all pixel points on the edge line of the abnormal edge area is: calculate the absolute value of the difference between the average curvature value of all pixel points on the edge line of the normal edge area and the average curvature value of all pixel points on the edge line of the abnormal edge area, and use this absolute value as the difference between the two average curvature values.

[0079] The larger the value of

[0080] By using the above method, it is possible to obtain the possibility that each abnormal edge region within the connected region of the abnormal silkworm morphology belongs to the head or tail of another silkworm. If the possibility is greater than or equal to the preset possibility threshold, it is determined that the corresponding abnormal edge region belongs to the edge region of the silkworm morphology; if the possibility is less than the preset possibility threshold, it is determined that the corresponding abnormal edge region does not belong to the edge region of the silkworm morphology, that is, it belongs to the non-silkworm morphology edge region. In this embodiment, the preset possibility threshold is 0.6. In specific applications, the implementer can set it according to the specific situation.

[0081] The ratio between the number of silkworm morphology edge regions and the number of abnormal edge regions in the connected region of the abnormal silkworm morphology is determined as the quantity proportion corresponding to the connected region of the abnormal silkworm morphology. The larger this quantity proportion, the greater the possibility of the abnormal silkworm morphology in the connected region of the abnormal silkworm morphology. Therefore, if this quantity proportion is greater than or equal to the preset proportion threshold, it is determined that the connected region of the abnormal silkworm morphology belongs to the overlapping silkworm morphology region; if this quantity proportion is less than the preset proportion threshold, it is determined that the connected region of the abnormal silkworm morphology does not belong to the overlapping silkworm morphology region, that is, the connected region of the abnormal silkworm morphology belongs to the non-overlapping silkworm morphology region. In this embodiment, the preset proportion threshold is 0.5. In specific applications, the implementer can set it according to the specific situation.

[0082] By using the above method, the connected regions of the abnormal silkworm morphology are classified, and multiple overlapping silkworm morphology regions are screened out. Next, these regions will be re-divided to avoid the influence of these connected regions with a greater possibility of abnormal silkworm morphology on the analysis result when extracting the growth characteristics of silkworms subsequently.

[0083] Step S3: Combine the similarity of the shape distributions of the edges of the abnormal edge regions and the normal silkworm morphology connected regions around them in the overlapping silkworm morphology region to screen the reference connected regions; re-divide the overlapping silkworm morphology region according to the relative position distribution between the abnormal edge regions and their reference connected regions.

[0084] For any overlapping silkworm morphology region, when different silkworms overlap, there may be a lack of silkworm morphology in the silkworm morphologies corresponding to the connected regions around this overlapping silkworm morphology region.

[0085] Next, this embodiment takes an overlapping silkworm morphology region as an example for illustration. The methods provided in this embodiment can be used to process other overlapping silkworm morphology regions.

[0086] Specifically, any abnormal edge region in the overlapping silkworm morphology region is denoted as the region to be analyzed, and a preset number of normal silkworm morphology connected regions closest to the region to be analyzed are used as candidate connected regions. In this embodiment, the preset number is 4. In specific applications, the implementer can set it according to specific circumstances. For any region in the region to be analyzed and each candidate connected region, a straight line parallel to the longest side of the minimum circumscribed rectangle of the region to be analyzed is drawn through the center point of the region, and the straight line divides the edge line of the region into two curve segments. The corresponding change trends are obtained by fitting each curve segment respectively, and the extension lines are obtained by extending each endpoint position of the curve segment based on the change trends. Among them, the number of extended pixel points is the preset pixel point value. The obtained closed region is used as the corresponding extended region by connecting the extension lines with a straight line. In this embodiment, the preset pixel point value is 50. In specific applications, the implementer can set it according to specific circumstances. It should be noted that if the extension line is not closed after extension, the last extension point is directly connected to form a closed region.

[0087] The Jaccard correlation coefficients between the pixel coordinate sets in the extended region corresponding to the region to be analyzed and the pixel coordinate sets in the extended regions corresponding to each candidate connected region are calculated respectively, and this correlation coefficient is denoted as the membership factor of each candidate connected region. Each candidate connected region has a corresponding membership factor. The larger the correlation coefficient, that is, the larger the membership factor, the more similar the pixel distributions in the extended region corresponding to the region to be analyzed and the extended region corresponding to the candidate connected region are, and the greater the possibility that the corresponding candidate connected region belongs to the region to be analyzed. Therefore, the candidate connected regions with membership factors greater than the preset membership threshold are determined as the reference connected regions of the region to be analyzed. In this example, the preset membership threshold is 0.5. In specific applications, the implementer can set it according to specific circumstances. After screening out the reference connected regions of the region to be analyzed, the pixel points in the overlapping part of the extended region corresponding to the region to be analyzed and the extended region corresponding to the reference connected region are divided into the normal silkworm morphology connected region where the reference connected region is located, and multiple connected regions after division are obtained.

[0088] So far, using the method provided in this embodiment, the silkworm tray image is divided into multiple sub-regions.

[0089] Step S4, combining the division results of the silkworm tray images in different growth stages, evaluate the growth state of the silkworms.

[0090] After the silkworm tray image is divided into multiple connected regions, the various characteristics of the silkworms can be evaluated by combining the distribution characteristics of all the connected regions in the silkworm tray images in different growth stages.

[0091] In this embodiment, the growth status of the silkworms is evaluated by combining the distribution characteristics of all the connected regions in the silkworm tray images in different growth stages.

[0092] At the same growth stage, the stoutness, body length characteristics, color characteristics, etc. of silkworms are relatively similar. In this embodiment, the characteristics of silkworms at different growth stages are unified to evaluate the growth state of silkworms.

[0093] Specifically, the stoutness of silkworms is evaluated according to the area distribution of all connected regions in the silkworm tray image; the color of silkworms is evaluated according to the overall gray level information of all connected regions in the silkworm tray image; the length of silkworms is evaluated according to the size of all connected regions in the silkworm tray image.

[0094] For any silkworm tray image, the specific calculation formula for the stoutness of silkworms in the silkworm tray image is:

[0095]

[0096] In the formula, represents the stoutness characteristic value of silkworms in the silkworm tray image, represents the variance value of the areas of all connected regions in the silkworm tray image, represents the average value of the areas of all connected regions in the silkworm tray image, represents the average value of the average areas of all regions in the silkworm tray images of all growth stages.

[0097] In this embodiment, adding a constant 1 to the two denominators of the calculation formula for the stoutness characteristic value of silkworms is to avoid a denominator of 0. The calculation process of is as follows. For each silkworm tray image, after calculating the average value of the areas of all connected regions in each image, an averaging operation is performed on the average values corresponding to all silkworm tray images, that is,

[0098] is obtained. The smaller the variance value of the areas of all connected regions in the silkworm tray image, the more similar the areas of all connected regions in the silkworm tray image are, that is, the more similar the shapes of silkworms are. The smaller the value of

[0099] is, the more similar the areas of all connected regions in the silkworm tray image are to the areas of connected regions in other silkworm tray images, that is, the more similar the stoutness of silkworms in this silkworm tray is to the stoutness of silkworms in other silkworm trays, and the greater the effectiveness of the stoutness of silkworms in the corresponding silkworm tray.

[0100] For a silkworm tray image, calculate the average gray value of the pixel points in each connected region of the silkworm tray image, take this average gray value as the color evaluation value of the corresponding connected region, and take the average value of the color evaluation values of all connected regions in the silkworm tray image as the color feature value of the silkworms in the silkworm tray image.

[0101] After determining the characteristics of the thickness, color, and length of the silkworms in each silkworm tray image, next, combine the changes in these characteristics of the silkworms in the silkworm tray images at different growth stages to determine the growth state effectiveness of the silkworms in the silkworm tray, where the growth state effectiveness is used to characterize the growth state of the silkworms.

[0102] Specifically, according to the characteristic values of the same characteristic of the silkworms in the silkworm tray images of two adjacent growth stages, calculate the change amount of each characteristic of the silkworms in the silkworm tray images of two adjacent growth stages; according to the change amounts of all characteristics of the silkworms in the silkworm tray images of two adjacent growth stages, obtain the growth state effectiveness of the silkworms in the silkworm tray, and there is a positive correlation between the change amount and the growth state effectiveness.

[0103] Among them, the positive correlation means that the dependent variable will increase as the independent variable increases, and the dependent variable will decrease as the independent variable decreases. It can be an additive relationship, a multiplicative relationship, etc., which is determined by the actual application.

[0104] In this embodiment, a specific calculation formula for the growth state effectiveness of the silkworms is given, specifically:

[0105]

[0106] In the formula, represents the growth state effectiveness of the silkworms, represents the number of types of characteristics of the silkworms, represents the normalization function, represents the characteristic value of the x-th characteristic of the silkworms in the silkworm tray image of the current growth stage, represents the characteristic value of the x-th characteristic of the silkworms in the silkworm tray image of the previous growth stage of the current growth stage. represents the change amount of the x-th characteristic of the silkworms.

[0107] Since all characteristics of the silkworms gradually increase as the growth state progresses. Therefore, the larger the value of the growth state effectiveness of the silkworms, the greater the growth state effectiveness of the silkworms in the current growth stage, that is, the better the growth state of the silkworms.

[0108] It should be noted that after extracting various characteristics of the silkworms, the breeding personnel can also evaluate the health status of the silkworms in combination with these characteristics, and make timely treatment according to the evaluation results, so as to maintain the healthy growth of the silkworms.

[0109] So far, the method provided by this embodiment has completed the accurate evaluation of the growth status of silkworms.

[0110] In this embodiment, according to the shape characteristics of the connected regions corresponding to each silkworm in each silkworm tray image, all the connected regions are divided into normal silkworm shape connected regions and abnormal silkworm shape connected regions. Then, by analyzing the difference in the bending of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm shape connected region, the possibility that the abnormal edge region in the abnormal silkworm shape connected region belongs to another silkworm is obtained, and then the overlapping silkworm shape region is found. Then, by combining the similarity of the shape distribution of the edge lines between the abnormal edge region and the surrounding normal silkworm shape connected regions in the overlapping silkworm shape region, the extra area in the overlapping silkworm shape connected region is divided to the corresponding connected regions around it, so as to obtain a more accurate connected region division result for each silkworm tray image. Finally, by combining the division results of the silkworm tray images in different growth stages, the growth status of the silkworms is accurately evaluated, which ensures the accuracy of the growth characteristic extraction results of different silkworms in the silkworm tray images and improves the accuracy of the silkworm growth status monitoring results.

[0111] Embodiment of a silkworm health status monitoring system based on computer vision:

[0112] Refer to Figure 2 , which shows the structural block diagram of a silkworm health status monitoring system based on computer vision provided by an embodiment of the present invention. The system may include an image acquisition module, a screening module, a division module, and an evaluation module.

[0113] Among them, the image acquisition module is used to obtain silkworm tray images in different growth stages;

[0114] The screening module is used to determine normal silkworm shape connected regions and abnormal silkworm shape connected regions according to the morphological distribution characteristics of the edge lines in the silkworm tray images; evaluate the possibility that the abnormal edge region in the abnormal silkworm shape connected region belongs to another silkworm according to the difference in the bending of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm shape connected region, and judge whether the abnormal silkworm shape connected region belongs to an overlapping silkworm shape region based on the possibility; the abnormal edge region and the normal edge region are obtained by clustering the convex hull vertices in the connected region;

[0115] The division module is used to screen reference connected regions by combining the similarity of the shape distribution of the edge lines between the abnormal edge region and the surrounding normal silkworm shape connected regions in the overlapping silkworm shape region; re-divide the overlapping silkworm shape region according to the relative position distribution between the abnormal edge region and its reference connected region;

[0116] The evaluation module is used to evaluate the growth status of the silkworms by combining the division results of the silkworm tray images in different growth stages.

[0117] It should be understood thatFigure 2 The block diagram of the computer vision-based silkworm health status monitoring system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented through hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art can understand that the above methods and systems can be implemented using computer-executable instructions and / or included in processor control code. For example, such code is provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The systems and modules of this specification can be implemented not only by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips and transistors, or programmable hardware devices such as field-programmable gate arrays and programmable logic devices, but also by software executed by various types of processors, or by a combination of the above hardware circuits and software (e.g., firmware).

[0118] For more details about each of the above modules, reference can be made to other parts of this specification and will not be elaborated here.

[0119] In other embodiments, a computer vision-based silkworm health status monitoring device is also provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the above-mentioned computer vision-based silkworm health status monitoring method. The device can specifically be a chip, component, or module. The chip can include a connected processor and memory; among them, the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the computer vision-based silkworm health status monitoring method provided in the above embodiments.

[0120] In other embodiments, a computer program product is also provided. When the computer program product runs on a computer, the computer is caused to execute the above-related steps to implement the computer vision-based silkworm health status monitoring method provided in the above embodiments.

[0121] In other embodiments, a computer-readable storage medium is also provided. The computer-readable storage medium stores computer program code. When the computer program code runs on a computer, the computer is caused to execute the above-related method steps to implement the computer vision-based silkworm health status monitoring method provided in the above embodiments.

[0122] Among them, the provided system, electronic device, computer program product, and computer-readable storage medium are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be elaborated here.

[0123] It should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for monitoring the health status of silkworms based on computer vision, characterized in that: The method comprises the following steps: Acquire images of silkworm discs at different growth stages; According to the morphological distribution characteristics of the edge lines in the silkworm disk image, the normal silkworm morphological connected domain and the abnormal silkworm morphological connected domain are determined, including: extracting the connected domain in the silkworm disk image, and performing convex hull detection on the connected domain to obtain the convex hull vertices; for any connected domain, obtaining the minimum circumscribed rectangle of the connected domain, recording the longest side of the minimum circumscribed rectangle as the body length direction of the silkworm, and recording the two pixel points with the farthest distance in the body length direction of the silkworm in the edge line of each connected domain as the suspected silkworm head point and the suspected silkworm tail point respectively; according to the difference between the number of convex hull vertices in the connected domain and the preset number of vertices, and the distances between the convex hull vertices and the suspected silkworm head point and the suspected silkworm tail point, the silkworm morphological validity of the connected domain is obtained; if the silkworm morphological validity is greater than or equal to the preset validity threshold, the corresponding connected domain is determined to be the normal silkworm morphological connected domain; if the silkworm morphological validity is less than the preset validity threshold, the corresponding connected domain is determined to be the abnormal silkworm morphological connected domain; According to the difference in the curvature of the edge lines between the abnormal edge area and the normal edge area in the abnormal silkworm morphology connected domain, the possibility that the abnormal edge area in the abnormal silkworm morphology connected domain belongs to another silkworm is evaluated, and based on the possibility, whether the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region is judged; the abnormal edge area and the normal edge area are obtained by clustering the convex hull vertices in the connected domain; A preset number of normal silkworm morphology connected domains closest to the area to be analyzed are used as candidate connected domains; for the area to be analyzed and the candidate connected domains, a reference connected domain of the abnormal edge area is screened according to the correlation between the extended area corresponding to the area to be analyzed and the positions of the pixel points in the extended area corresponding to each candidate connected domain; the area to be analyzed is any abnormal edge area in the overlapping silkworm morphology area; Divide the overlapping pixel points in the extended area corresponding to the area to be analyzed and the extended area corresponding to the reference connected domain into the normal silkworm morphology connected domain where the reference connected domain is located; The growth status of silkworms is evaluated by combining the segmentation results of silkworm disc images at different growth stages.

2. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: Acquisition of abnormal edge areas and normal edge areas, including: For any convex hull vertex in any connected domain, obtain the minimum value of the Euclidean distance between any convex hull vertex and the suspected silkworm head point and the minimum value of the Euclidean distance between any convex hull vertex and the suspected silkworm tail point; if the normalized value of the minimum value in the Euclidean distance is greater than the preset distance threshold, then determine that any convex hull vertex is a first-class point; if the normalized value of the minimum value in the Euclidean distance is less than or equal to the preset distance threshold, then determine that any convex hull vertex is a second-class point; Each type of point in the silkworm disk image is clustered separately, and the convex hull area corresponding to the cluster obtained by clustering the first type of points is used as the abnormal edge area, and the convex hull area corresponding to the cluster obtained by clustering the second type of points is used as the normal edge area.

3. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: Combined with the segmentation results of silkworm disc images at different growth stages, the growth status of silkworms is evaluated, including: The thickness of the silkworm is evaluated according to the area distribution of all connected domains in the silkworm disc image; the color of the silkworm is evaluated according to the overall grayscale information of all connected domains in the silkworm disc image; the length of the silkworm is evaluated according to the size of all connected domains in the silkworm disc image; Combined with the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages, the growth status validity of the silkworms in the silkworm tray is determined, and the growth status validity is used to characterize the growth status of the silkworms; the silkworm characteristics include the thickness, color and length of the silkworms.

4. The method for monitoring silkworm health status based on computer vision according to claim 3, characterized in that: Combined with the changes in the characteristics of silkworms in the silkworm tray images at different growth stages, the validity of the growth status of the silkworms in the silkworm tray is determined, including: According to the feature values ​​of the same feature of the silkworm in the silkworm disk images of two adjacent growth stages, the change amount of each feature of the silkworm in the silkworm disk images of two adjacent growth stages is calculated; According to the variation of all the characteristics of the silkworm in the silkworm tray images of two adjacent growth stages, the growth status validity of the silkworm in the silkworm tray is obtained, and the variation is positively correlated with the growth status validity.

5. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: According to the difference in curvature of the edge lines between the abnormal edge area and the normal edge area in the abnormal silkworm morphology connected domain, the possibility that the abnormal edge area in the abnormal silkworm morphology connected domain belongs to another silkworm is evaluated, and whether the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region is judged based on the possibility, including: For any abnormal silkworm morphological connected domain: Calculate the first average curvature value of all pixel points on the edge line of each normal edge area in any abnormal silkworm morphological connected domain respectively, and obtain the minimum value of the difference between the first average curvature value corresponding to each normal edge area in any abnormal silkworm morphological connected domain and the average curvature value of all pixel points on each abnormal edge area in any abnormal silkworm morphological connected domain; determine the possibility that the corresponding abnormal edge area in any abnormal silkworm morphological connected domain belongs to another silkworm according to the minimum value of the difference between the average curvature values, and the minimum value of the difference between the average curvature values ​​is negatively correlated with the possibility of belonging to another silkworm; If the probability is greater than or equal to the preset probability threshold, the corresponding abnormal edge region is determined to belong to the silkworm morphology edge region; According to the proportion of the number of silkworm morphology edge regions in any abnormal silkworm morphology connected domain, it is judged whether any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region.

6. The method for monitoring silkworm health status based on computer vision according to claim 5, characterized in that: According to the proportion of the number of silkworm morphology edge regions in any abnormal silkworm morphology connected domain, it is judged whether any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region, including: The ratio between the number of silkworm morphological edge regions in any abnormal silkworm morphological connected domain and the number of abnormal edge regions in any abnormal silkworm morphological connected domain is determined as the number ratio corresponding to any abnormal silkworm morphological connected domain; If the quantity ratio is greater than or equal to the ratio threshold, it is determined that any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area.

7. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: According to the extended area corresponding to the area to be analyzed and the extended area corresponding to each candidate connected domain, it includes: For the area to be analyzed and the candidate connected domain, a straight line parallel to the longest side of the minimum circumscribed rectangle of the area to be analyzed is drawn through the center point of the area, and the straight line divides the edge line of the area into two curve segments; each curve segment is fitted to obtain the corresponding change trend, and each endpoint position of the curve segment is extended based on the change trend to obtain an extension line; wherein the number of extended pixel points is the preset pixel point value; the extended lines are connected, and the obtained closed area is used as the corresponding extended area.

8. The method for monitoring silkworm health status based on computer vision according to claim 7, characterized in that: According to the correlation between the positions of the pixels in the extended area corresponding to the area to be analyzed and the extended area corresponding to each candidate connected domain, the reference connected domain of the abnormal edge area is screened, including: The correlation coefficient between the pixel point coordinate set in the extended area corresponding to the area to be analyzed and the pixel point coordinate set in the extended area corresponding to each candidate connected domain is recorded as the membership factor of each candidate connected domain; The candidate connected domains whose membership factors are greater than the preset membership threshold are determined as the reference connected domains of the area to be analyzed.

Images