Silkworm health state monitoring method based on computer vision

Abstract

The invention relates to the technical field of image segmentation, in particular to a silkworm health state monitoring method based on computer vision. The method comprises the following steps: acquiring silkworm tray images at different growth stages; determining a normal silkworm form connected domain and an abnormal silkworm form connected domain according to form distribution characteristics of edge lines in the silkworm tray image; according to the difference of the bending conditions of the edge lines of the abnormal edge region and the normal edge region in the abnormal silkworm form connected domain, the possibility that the abnormal edge region in the abnormal silkworm form connected domain belongs to another silkworm is evaluated, and an overlapped silkworm form region is determined; screening a reference connected domain by combining the similar condition of shape distribution of edge lines of an abnormal edge area in the overlapped silkworm form area and a normal silkworm form connected domain around the abnormal edge area; according to the position distribution of the abnormal edge area and the reference connected domain thereof, re-dividing the overlapped silkworm form area; and evaluating the growth state of the silkworms in combination with a final division result. The accuracy of the silkworm growth state monitoring result is improved.

Description

Technical Field

[0001] The invention relates to the technical field of image segmentation, and in particular to a method for monitoring the health status of silkworms based on computer vision. Background Art

[0002] Silkworms are susceptible to various diseases during the breeding process, which leads to abnormal growth of silkworms and affects the quality of cocoons. Therefore, it is very important to monitor the health of silkworms during the breeding process. Traditional monitoring of silkworm health mainly relies on manual observation, which has many disadvantages. With the continuous development of computer technology and image processing technology, computer vision technology is more and more widely used in the agricultural field.

[0003] At present, in the process of monitoring the health status of silkworms based on computer vision, due to the gregarious characteristics of silkworms, they usually like to gather together. Therefore, in the visual images of silkworms obtained, there is usually overlap between the silkworms, which may cause the silkworms located below to be blocked, resulting in the computer vision system being unable to obtain complete and accurate growth characteristic information of these silkworms, affecting the monitoring results of the silkworm health status and making the assessment results of the silkworm growth status less accurate. Summary of the invention

[0004] In order to solve the problem that the existing methods cannot accurately and completely extract the growth characteristic information of silkworms, the purpose of the present invention is to provide a method for monitoring the health status of silkworms based on computer vision. The technical scheme adopted is as follows: The present invention provides a method for monitoring the health status of silkworms based on computer vision, the method comprising the following steps: Acquire images of silkworm discs at different growth stages; According to the morphological distribution characteristics of the edge lines in the silkworm disc image, a normal silkworm morphological connected domain and an abnormal silkworm morphological connected domain are determined; according to the difference in the curvature of the edge lines between the abnormal edge area and the normal edge area in the abnormal silkworm morphological connected domain, the possibility that the abnormal edge area in the abnormal silkworm morphological connected domain belongs to another silkworm is evaluated, and based on the possibility, whether the abnormal silkworm morphological connected domain belongs to an overlapping silkworm morphological region is judged; the abnormal edge area and the normal edge area are obtained by clustering the convex hull vertices in the connected domain; Based on the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domain, the reference connected domain is screened; according to the relative position distribution of the abnormal edge region and its reference connected domain, the overlapping silkworm morphology region is re-divided; The growth status of silkworms is evaluated by combining the segmentation results of silkworm disc images at different growth stages.

[0005] Preferably, the determining of the normal silkworm morphological connected domain and the abnormal silkworm morphological connected domain according to the morphological distribution characteristics of the edge lines in the silkworm disc image comprises: Extract the connected domain in the silkworm disk image, and perform convex hull detection on the connected domain to obtain the convex hull vertices; For any connected domain, the minimum circumscribed rectangle of the connected domain is obtained, the longest side of the minimum circumscribed rectangle is recorded as the body length direction of the silkworm, and the two pixel points with the farthest distance in the body length direction of the silkworm in the edge line of each connected domain are recorded as the suspected silkworm head point and the suspected silkworm tail point respectively; according to the difference between the number of convex hull vertices in the connected domain and the preset number of vertices, and the distances between the convex hull vertices and the suspected silkworm head point and the suspected silkworm tail point, the silkworm morphological validity of the connected domain is obtained; If the silkworm morphology validity is greater than or equal to a preset validity threshold, the corresponding connected domain is determined to be a normal silkworm morphology connected domain; if the silkworm morphology validity is less than the preset validity threshold, the corresponding connected domain is determined to be an abnormal silkworm morphology connected domain.

[0006] Preferably, obtaining the abnormal edge region and the normal edge region includes: For any convex hull vertex in any connected domain, respectively obtain the minimum value of the Euclidean distance between any convex hull vertex and the suspected silkworm head point and the minimum value of the Euclidean distance between any convex hull vertex and the suspected silkworm tail point; if the normalized value of the minimum value in the Euclidean distance is greater than a preset distance threshold, then determine that any convex hull vertex is a first-class point; if the normalized value of the minimum value in the Euclidean distance is less than or equal to the preset distance threshold, then determine that any convex hull vertex is a second-class point; Each type of point in the silkworm disk image is clustered separately, and the convex hull area corresponding to the cluster obtained by clustering the first type of points is used as the abnormal edge area, and the convex hull area corresponding to the cluster obtained by clustering the second type of points is used as the normal edge area.

[0007] Preferably, the step of evaluating the growth status of silkworms by combining the segmentation results of the silkworm disc images at different growth stages includes: The thickness of the silkworm is evaluated according to the area distribution of all connected domains in the silkworm disc image; the color of the silkworm is evaluated according to the overall grayscale information of all connected domains in the silkworm disc image; the length of the silkworm is evaluated according to the size of all connected domains in the silkworm disc image; Combined with the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages, the growth status validity of the silkworms in the silkworm tray is determined, and the growth status validity is used to characterize the growth status of the silkworms; wherein the characteristics of the silkworms include the thickness, color and length of the silkworms.

[0008] Preferably, the determination of the validity of the growth status of the silkworms in the silkworm tray based on the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages includes: According to the feature values ​​of the same feature of the silkworm in the silkworm disk images of two adjacent growth stages, the change amount of each feature of the silkworm in the silkworm disk images of two adjacent growth stages is calculated; The growth status validity of the silkworms in the silkworm tray is obtained according to the variation of all the features of the silkworms in the silkworm tray images of two adjacent growth stages, and the variation is positively correlated with the growth status validity.

[0009] Preferably, the method of evaluating the possibility that the abnormal edge region in the abnormal silkworm morphology connected domain belongs to another silkworm according to the difference in curvature of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm morphology connected domain, and judging whether the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region based on the possibility includes: For any abnormal silkworm morphological connected domain: Respectively calculating the first average curvature value of all pixel points on the edge line of each normal edge region in any abnormal silkworm morphological connected domain, obtaining the minimum value of the difference between the first average curvature value corresponding to each normal edge region in any abnormal silkworm morphological connected domain and the average curvature value of all pixel points on each abnormal edge region in any abnormal silkworm morphological connected domain; determining the possibility that the corresponding abnormal edge region in any abnormal silkworm morphological connected domain belongs to another silkworm according to the minimum value of the difference between the average curvature values, and the minimum value of the difference between the average curvature values ​​is negatively correlated with the possibility of belonging to another silkworm; If the possibility is greater than or equal to a preset possibility threshold, it is determined that the corresponding abnormal edge region belongs to the silkworm morphology edge region; According to the proportion of the number of silkworm morphology edge regions in any abnormal silkworm morphology connected domain, it is determined whether any abnormal silkworm morphology connected domain belongs to an overlapping silkworm morphology region.

[0010] Preferably, judging whether any abnormal silkworm morphology connected domain belongs to an overlapping silkworm morphology region according to the proportion of the number of silkworm morphology edge regions in any abnormal silkworm morphology connected domain comprises: Determine the ratio between the number of silkworm morphological edge regions in any abnormal silkworm morphological connected domain and the number of abnormal edge regions in any abnormal silkworm morphological connected domain as the number ratio corresponding to any abnormal silkworm morphological connected domain; If the quantity proportion is greater than or equal to the proportion threshold, it is determined that any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area.

[0011] Preferably, the method of combining the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domain to screen the reference connected domain comprises: A preset number of connected domains of normal silkworm morphology closest to the area to be analyzed are used as candidate connected domains; For the region to be analyzed and the candidate connected domain, a straight line is drawn through the center point of the region and is parallel to the longest side of the minimum circumscribed rectangle of the region to be analyzed, and the straight line divides the edge line of the region into two curve segments; each curve segment is fitted to obtain a corresponding change trend, and each end point position of the curve segment is extended based on the change trend to obtain an extended line; wherein the number of extended pixel points is a preset pixel point value; the extended lines are connected, and the obtained closed area is used as the corresponding extended area; According to the correlation between the positions of the pixels in the extended area corresponding to the area to be analyzed and the extended area corresponding to each candidate connected area, the reference connected area of ​​the abnormal edge area is screened; The area to be analyzed is any abnormal edge area in the overlapping silkworm morphology area.

[0012] Preferably, the step of screening the reference connected domain of the abnormal edge region according to the correlation between the positions of the pixels in the extended region corresponding to the region to be analyzed and the extended region corresponding to each candidate connected domain comprises: The correlation coefficient between the pixel point coordinate set in the extended area corresponding to the area to be analyzed and the pixel point coordinate set in the extended area corresponding to each candidate connected domain is recorded as the membership factor of each candidate connected domain; The candidate connected domains whose membership factors are greater than the preset membership threshold are determined as the reference connected domains of the area to be analyzed.

[0013] Preferably, the re-dividing the overlapping silkworm morphology regions according to the relative position distribution between the abnormal edge regions and their reference connected domains comprises: The pixel points of the overlapped part in the extended area corresponding to the area to be analyzed and the extended area corresponding to the reference connected domain are divided into the normal silkworm morphology connected domain where the reference connected domain is located.

[0014] The present invention has at least the following beneficial effects: The present invention divides all connected domains into normal silkworm morphology connected domains and abnormal silkworm morphology connected domains according to shape features of connected domains corresponding to each silkworm in each silkworm disc image, then obtains the possibility that the abnormal edge region in the abnormal silkworm morphology connected domain belongs to another silkworm by analyzing the difference in bending conditions of edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm morphology connected domain, and then finds out the overlapping silkworm morphology region; then, by combining the similarity of shape distribution of edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domains, the redundant area in the overlapping silkworm morphology connected domain is divided to the surrounding corresponding connected domains, so as to obtain a more accurate connected domain division result in each silkworm disc image; finally, by combining the division results of silkworm disc images at different growth stages, the growth state of the silkworm is accurately evaluated, so as to ensure the accuracy of the growth feature extraction results of different silkworms in the silkworm disc image, and improve the accuracy of the silkworm growth state monitoring result. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of a method for monitoring the health status of silkworms based on computer vision provided by an embodiment of the present invention; Figure 2 A structural block diagram of a silkworm health status monitoring system based on computer vision provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, a method for monitoring the health status of silkworms based on computer vision proposed in accordance with the present invention is described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] The following is a detailed description of a method for monitoring the health status of silkworms based on computer vision provided by the present invention in conjunction with the accompanying drawings.

[0020] Embodiment of the method for monitoring the health status of silkworms based on computer vision: The specific scenario targeted by this embodiment is: in the process of detecting the growth status of silkworms based on computer vision methods, silkworm disc images at different growth stages are collected, and the silkworm disc images are divided into multiple connected domains through the morphological characteristics and edge morphological characteristics of the connected domain corresponding to each silkworm in each silkworm disc image. The growth status of the silkworm is evaluated based on the characteristics presented by the connected domains in the silkworm disc images at different growth stages.

[0021] This embodiment proposes a method for monitoring the health status of silkworms based on computer vision, such as Figure 1 As shown, the method for monitoring the health status of silkworms based on computer vision in this embodiment includes the following steps: Step S1, acquiring images of silkworm discs at different growth stages.

[0022] Silkworms go through four stages during their growth: eggs, larvae, pupae, and adults. The larval stage goes through multiple instars, and the length and weight of silkworms increase at each instar. Therefore, it is important to monitor the growth status of silkworms during the larval stage. In the practical application of modern large-scale silkworm breeding, multi-layer silkworm trays are usually used for breeding. Therefore, first, a high-speed camera is used to capture images of silkworm trays at different growth stages during the growth of silkworms directly above the silkworm trays; it should be noted that silkworms in the growth process are placed on the silkworm trays, and the silkworms in the same silkworm tray are produced by the same batch of eggs. In this embodiment, a silkworm tray is used as an example for explanation, and the method provided in this embodiment can be used to process other silkworm trays. Then, the collected image of the silkworm tray is grayed, and the grayed image is recorded as the silkworm tray image. Image graying is a prior art and will not be described in detail here.

[0023] So far, this embodiment has collected images of silkworm discs at different growth stages.

[0024] Step S2, determining a normal silkworm morphological connected domain and an abnormal silkworm morphological connected domain according to the morphological distribution characteristics of edge lines in the silkworm disc image; evaluating the possibility that the abnormal edge region in the abnormal silkworm morphological connected domain belongs to another silkworm according to the difference in the curvature of the edge lines between the abnormal edge region and the normal edge region in the abnormal silkworm morphological connected domain, and judging whether the abnormal silkworm morphological connected domain belongs to an overlapping silkworm morphological region based on the possibility; the abnormal edge region and the normal edge region are obtained by clustering the convex hull vertices in the connected domain.

[0025] A connected domain detection algorithm based on a marker is used to perform connected domain detection on each silkworm tray image, and multiple connected domains in each silkworm tray image are extracted. The connected domain detection algorithm based on a marker is a prior art and will not be described in detail here.

[0026] Since the silkworms in each silkworm tray image overlap, the edges of some silkworms in the acquired silkworm tray image will be blocked or overlapped, and the edge contour of the silkworm in each silkworm tray image needs to be optimized.

[0027] Since the movement of silkworms is small, they present an elongated or arc-shaped shape. Therefore, when there is no occlusion, overlap, etc., each connected domain does not have a convex hull, and the edge of the connected domain is relatively smooth, so each connected domain will only have a convex situation at the head or tail. Therefore, first use the convex hull detection algorithm to perform convex hull detection on each connected domain in the silkworm disk image, and determine whether the silkworm edge corresponding to each connected domain overlaps based on the convex hull detection result. Among them, the convex hull detection algorithm performs convex hull detection as a prior art, which will not be repeated here. After the convex hull detection is completed, the convex hull vertex set of each connected domain can be obtained.

[0028] The fewer the vertices of the convex hull in the connected domain, and the closer the convex hull vertex is to the head or tail of the silkworm, the greater the possibility that it is the head and tail of the silkworm. Based on this feature, the morphological features of each connected domain in the silkworm disk image are analyzed through the convex hull detection results to determine the silkworm morphological validity of each connected domain.

[0029] Next, this embodiment is described by taking a connected domain as an example, and other connected domains can be processed by the method provided in this embodiment.

[0030] Specifically, for any connected domain in the silkworm disk image, the minimum circumscribed rectangle of the connected domain is obtained, the longest side of the minimum circumscribed rectangle is recorded as the body length direction of the silkworm, and the two pixel points with the farthest distance in the body length direction of the silkworm in the edge line of each connected domain are recorded as the suspected silkworm head point and the suspected silkworm tail point respectively; according to the difference between the number of convex hull vertices in the connected domain and the preset number of vertices, and the distances between the convex hull vertices and the suspected silkworm head point and the suspected silkworm tail point, the silkworm morphological validity of the connected domain is obtained.

[0031] In this embodiment, a specific calculation formula for the silkworm morphological validity is given, and the silkworm morphological validity of the i-th connected domain in the m-th silkworm disc image can be expressed as: In the formula, represents the silkworm morphological validity of the ith connected domain in the mth silkworm disk image, represents the number of convex hull vertices in the i-th connected domain in the m-th silkworm disk image, Indicates the preset number of vertices. Indicates the Euclidean distance. represents the coordinates of the jth convex hull vertex in the ith connected domain in the mth silkworm disk image, represents the coordinates of the suspected silkworm head point in the minimum circumscribed rectangle of the ith connected domain in the mth silkworm disk image, represents the coordinates of the suspected silkworm tail point in the minimum circumscribed rectangle of the ith connected domain in the mth silkworm disk image, express and The Euclidean distance between express and The Euclidean distance between represents the normalization function, represents the minimum value function, Indicates taking and The minimum of two Euclidean distances.

[0032] In this embodiment, the preset number of vertices is 4. In specific applications, the implementer can set it according to specific circumstances. In this embodiment, the constant 1 is added to the two denominators of the calculation formula of the silkworm morphological validity in order to avoid the denominator being zero.

[0033] The smaller the value of is, the more normal the number of convex hull vertices in the i-th connected domain in the m-th silkworm disk image is. The smaller the value, the closer the convex hull vertex in the mth silkworm disk image is to the head or tail of the silkworm. Since it is unclear whether the jth convex hull vertex is closer to the head or the tail, this embodiment calculates the Euclidean distance between the jth convex hull vertex and the suspected silkworm head point and the Euclidean distance between the jth convex hull vertex and the suspected silkworm tail point, and selects the minimum distance from the two to participate in the calculation of the silkworm morphological validity. The greater the difference between the number of convex hull vertices in the i-th connected domain and the preset number of vertices, and the greater the minimum value of the Euclidean distance between the jth convex hull vertex and the suspected silkworm head point and the Euclidean distance between the jth convex hull vertex and the suspected silkworm tail point, the smaller the value of the silkworm morphological validity.

[0034] By adopting the above method, the silkworm morphological validity of each connected domain in the silkworm disc image can be obtained.

[0035] If the silkworm morphology validity is greater than or equal to the preset validity threshold, the corresponding connected domain is determined to be a normal silkworm morphology connected domain; if the silkworm morphology validity is less than the preset validity threshold, the corresponding connected domain is determined to be an abnormal silkworm morphology connected domain, and the abnormal silkworm morphology connected domain may be an area where the silkworm morphology overlaps. In this embodiment, the preset validity threshold is 0.7. In specific applications, the implementer can set it according to specific circumstances. Using this method, all connected domains in the silkworm disk image are divided into two categories, namely, normal silkworm morphology connected domains and abnormal silkworm morphology connected domains.

[0036] Then, by analyzing the distribution relationship between the abnormal silkworm morphological connected domain and the surrounding normal silkworm morphological connected domain, it is determined whether the abnormal silkworm morphological connected domain has overlapping abnormalities, and the overlapping areas are separated.

[0037] Specifically, for any convex hull vertex in any connected domain, the minimum value of the Euclidean distance between the convex hull vertex and the suspected silkworm head point and the Euclidean distance between the convex hull vertex and the suspected silkworm tail point is obtained respectively; if the normalized value of the minimum value is greater than the preset distance threshold, the convex hull vertex is determined to be a first-class point; if the normalized value of the minimum value is less than or equal to the preset distance threshold, the convex hull vertex is determined to be a second-class point, and the second-class point is more likely to be a point on the silkworm head or the silkworm tail. The data normalization method is a prior art and will not be described in detail here. In this embodiment, the preset distance threshold is 0.5. In specific applications, the implementer can set it according to the specific situation.

[0038] The K-means algorithm is used to cluster each type of point in the silkworm disk image, that is, the first type of points and the second type of points in the silkworm disk image are clustered respectively, and the convex hull area corresponding to the cluster obtained by clustering the first type of points is used as the abnormal edge area, and the convex hull area corresponding to the cluster obtained by clustering the second type of points is used as the normal edge area. When the K-means algorithm is used for clustering, the value of k is determined by the silhouette coefficient method, which is a prior art. As other implementation methods, the value of k can also be set manually according to specific circumstances. The K-means algorithm is a prior art and will not be described in detail here.

[0039] For any abnormal silkworm morphological connected domain: The average curvature value of all pixel points on the edge line of each normal edge area in the abnormal silkworm morphological connected domain is calculated respectively, and the average curvature value is recorded as the first average curvature value. It should be noted that: each normal edge area has a corresponding first average curvature value; the minimum value of the difference between the first average curvature value corresponding to each normal edge area in the abnormal silkworm morphological connected domain and the average curvature value of all pixel points on each abnormal edge area in the abnormal silkworm morphological connected domain is obtained; the possibility that the corresponding abnormal edge area in the abnormal silkworm morphological connected domain belongs to another silkworm is determined according to the minimum value of the difference between the average curvature values, and the minimum value of the difference between the average curvature values ​​is negatively correlated with the possibility of belonging to another silkworm.

[0040] Among them, the negative correlation relationship means that the dependent variable will decrease as the independent variable increases, and the dependent variable will increase as the independent variable decreases. It can be a subtraction relationship, a division relationship, etc., which is determined by practical applications.

[0041] In this embodiment, a specific calculation formula for the possibility that the abnormal edge region belongs to another silkworm is given. The possibility that the kth abnormal edge region in the nth abnormal silkworm morphological connected domain belongs to another silkworm can be expressed as: In the formula, It indicates the possibility that the kth abnormal edge region in the nth abnormal silkworm morphological connected domain belongs to another silkworm. It represents the minimum value of the difference between the average curvature value of all pixels on the edge line of the normal edge area in the nth abnormal Silkworm morphological connected domain and the average curvature value of all pixels on the edge line of the kth abnormal edge area in the nth abnormal Silkworm morphological connected domain.

[0042] The constant 1 is added to the denominator of the calculation formula of the possibility that the abnormal edge area belongs to another silkworm in order to prevent the denominator from being 0. In specific applications, the implementer can set it according to specific circumstances. It should be noted that the specific calculation method of the difference between the average curvature value of all pixels on the edge line of the normal edge area and the average curvature value of all pixels on the edge line of the abnormal edge area is: calculate the absolute value of the difference between the average curvature value of all pixels on the edge line of the normal edge area and the average curvature value of all pixels on the edge line of the abnormal edge area, and use the absolute value as the difference between the two average curvature values.

[0043] The larger the value of is, the greater the possibility that the kth abnormal edge region in the nth abnormal silkworm morphological connected domain belongs to the head or tail of another silkworm.

[0044] By adopting the above method, the possibility that each abnormal edge region in the abnormal silkworm morphology connected domain belongs to the head or tail of another silkworm can be obtained. If the possibility is greater than or equal to the preset possibility threshold, the corresponding abnormal edge region is determined to belong to the silkworm morphology edge region; if the possibility is less than the preset possibility threshold, the corresponding abnormal edge region is determined not to belong to the silkworm morphology edge region, that is, it belongs to the non-silkworm morphology edge region. In this embodiment, the preset possibility threshold is 0.6. In specific applications, the implementer can set it according to specific circumstances.

[0045] The ratio between the number of silkworm morphology edge regions in the abnormal silkworm morphology connected domain and the number of abnormal edge regions in the abnormal silkworm morphology connected domain is determined as the number proportion corresponding to the abnormal silkworm morphology connected domain. The larger the number proportion, the greater the possibility of abnormal silkworm morphology in the abnormal silkworm morphology connected domain. Therefore, if the number proportion is greater than or equal to the preset proportion threshold, it is determined that the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region; if the number proportion is less than the preset proportion threshold, it is determined that the abnormal silkworm morphology connected domain does not belong to the overlapping silkworm morphology region, that is, the abnormal silkworm morphology connected domain belongs to the non-overlapping silkworm morphology region. In this embodiment, the preset proportion threshold is 0.5. In specific applications, the implementer can set it according to the specific situation.

[0046] The above method was used to classify the connected domains of abnormal silkworm morphology, and multiple overlapping silkworm morphology regions were screened out. These regions will be re-divided next to avoid the influence of these connected domains with a high possibility of abnormal silkworm morphology on the analysis results when the silkworm growth characteristics are subsequently extracted.

[0047] Step S3, combining the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domain, screening the reference connected domain; and re-dividing the overlapping silkworm morphology region according to the relative position distribution of the abnormal edge region and its reference connected domain.

[0048] For any overlapping silkworm morphology region, when different silkworms overlap, the silkworm morphology corresponding to the connected domain around the overlapping silkworm morphology region may be missing.

[0049] Next, this embodiment is described by taking an overlapping silkworm morphology region as an example, and other overlapping silkworm morphology regions can be processed by the method provided in this embodiment.

[0050] Specifically, any abnormal edge area in the overlapping silkworm morphology area is recorded as the area to be analyzed, and the preset number of normal silkworm morphology connected domains closest to the area to be analyzed are used as candidate connected domains; in this embodiment, the preset number is 4, and in specific applications, the implementer can set it according to specific circumstances. For the area to be analyzed and any area in each candidate connected domain, a straight line parallel to the longest side of the minimum circumscribed rectangle of the area to be analyzed is drawn through the center point of the area, and the straight line divides the edge line of the area into two curve segments; each curve segment is fitted to obtain the corresponding change trend, and each end point position of the curve segment is extended based on the change trend to obtain an extension line; wherein the number of extended pixel points is the preset pixel point value; the extended lines are connected by straight lines, and the obtained closed area is used as the corresponding extended area. In this embodiment, the preset pixel point value is 50, and in specific applications, the implementer can set it according to specific circumstances. It should be noted that if the extended line is not closed after extension, the last extended point is directly connected to form a closed area.

[0051] The Jaccard correlation coefficient between the pixel point coordinate set in the extended area corresponding to the area to be analyzed and the pixel point coordinate set in the extended area corresponding to each candidate connected domain is calculated respectively, and the correlation coefficient is recorded as the membership factor of each candidate connected domain, and each candidate connected domain has a corresponding membership factor. The larger the correlation coefficient, that is, the larger the membership factor, the more similar the pixel point distribution in the extended area corresponding to the area to be analyzed and the extended area corresponding to the candidate connected domain is, and the greater the possibility that the corresponding candidate connected domain belongs to the area to be analyzed. Therefore, the candidate connected domain whose membership value is greater than the preset membership threshold is determined as the reference connected domain of the area to be analyzed. In this example, the preset membership threshold is 0.5. In specific applications, the implementer can set it according to specific circumstances. After screening out the reference connected domain of the area to be analyzed, the pixel points of the overlapping part in the extended area corresponding to the area to be analyzed and the extended area corresponding to the reference connected domain are divided into the normal silkworm morphology connected domain where the reference connected domain is located, and multiple connected domains after the division are obtained.

[0052] So far, the method provided in this embodiment is used to divide the silkworm disk image into multiple sub-areas.

[0053] Step S4, evaluating the growth status of the silkworms in combination with the segmentation results of the silkworm disc images at different growth stages.

[0054] After the silkworm disk image is divided into multiple connected domains, the various characteristics of the silkworm can be evaluated by combining the distribution characteristics of all connected domains in the silkworm disk images at different growth stages.

[0055] In this embodiment, the growth status of the silkworm is evaluated by combining the distribution characteristics of all connected domains in the silkworm disk images at different growth stages.

[0056] The stoutness, body length, color characteristics, etc. of silkworms at the same growth stage are relatively similar. In this embodiment, these characteristics of silkworms at different growth stages are used to evaluate the growth status of the silkworms.

[0057] Specifically, the thickness of the silkworm is evaluated according to the area distribution of all connected domains in the silkworm disc image; the color of the silkworm is evaluated according to the overall grayscale information of all connected domains in the silkworm disc image; and the length of the silkworm is evaluated according to the size of all connected domains in the silkworm disc image.

[0058] For any silkworm disc image, the specific calculation formula for the thickness of the silkworm in the silkworm disc image is: In the formula, Indicates the characteristic value of the thickness of the silkworm in the silkworm disc image, represents the variance of the area of ​​all connected domains in the silkworm disk image, represents the average area of ​​all connected domains in the silkworm disk image, Represents the mean of the area averages of all regions in the silkworm disc images of all growth stages.

[0059] In this embodiment, the constant 1 is added to the two denominators of the calculation formula of the silkworm's robustness characteristic value in order to avoid the denominator being 0. The calculation process is as follows: for each silkworm disk image, after calculating the average area of ​​all connected domains in each image, the average area of ​​all silkworm disk images is averaged to obtain .

[0060] The smaller the variance value of the area of ​​all connected domains in the silkworm disk image is, the more similar the areas of all connected domains in the silkworm disk image are, that is, the more similar the morphologies of the silkworms are. The smaller the value is, the more similar the area of ​​all connected domains in the silkworm tray image is to the area of ​​connected domains in other silkworm tray images, that is, the more similar the sturdiness of the silkworms in this silkworm tray is to the sturdiness of the silkworms in other silkworm trays, the greater the effectiveness of the sturdiness of the silkworms in the corresponding silkworm tray.

[0061] Then, the edge line length of the connected domain corresponding to all the silkworms in each silkworm tray image is recorded as the length of the silkworm in each silkworm tray, the average length of all the silkworms in a silkworm tray image is taken as the body length characteristic value of the silkworm in the silkworm tray image, and the mean grayscale value of all the pixels in the connected domain is recorded as the color characteristic value of the silkworm in each silkworm tray.

[0062] For a silkworm tray image, the average grayscale value of the pixels in each connected domain in the silkworm tray image is calculated, and the average grayscale value is used as the color evaluation value of the corresponding connected domain. The average value of the color evaluation values ​​of all connected domains in the silkworm tray image is used as the color feature value of the silkworm in the silkworm tray image.

[0063] After determining the characteristics of the thickness, color, and length of the silkworm in each silkworm disc image, the growth status validity of the silkworm in the silkworm disc is determined based on the changes in these characteristics of the silkworm in the silkworm disc images at different growth stages, wherein the growth status validity is used to characterize the growth status of the silkworm.

[0064] Specifically, according to the feature value of the same feature of the silkworm in the silkworm disc images of two adjacent growth stages, the change amount of each feature of the silkworm in the silkworm disc images of two adjacent growth stages is calculated; according to the change amount of all features of the silkworm in the silkworm disc images of two adjacent growth stages, the growth status validity of the silkworm in the silkworm disc is obtained, and the change amount is positively correlated with the growth status validity.

[0065] Among them, the positive correlation means that the dependent variable will increase as the independent variable increases, and the dependent variable will decrease as the independent variable decreases. It can be an additive relationship, a multiplicative relationship, etc., which is determined by practical applications.

[0066] In this embodiment, a specific calculation formula for the effectiveness of the growth state of silkworms is given, specifically: In the formula, Indicates the effectiveness of the growth status of silkworms, The number of species that represent the characteristics of silkworms, represents the normalization function, represents the feature value of the xth feature of the silkworm in the silkworm disc image at the current growth stage, Represents the feature value of the x-th feature of the silkworm in the silkworm disc image of the previous growth stage of the current growth stage. Represents the change in the x-th characteristic of the silkworm.

[0067] As the growth state progresses, the characteristics of the silkworm gradually increase. Therefore, the greater the value of the effectiveness of the silkworm's growth state, the greater the effectiveness of the silkworm's growth state at the current growth stage, that is, the better the silkworm's growth state.

[0068] It should be noted that after extracting various characteristics of silkworms, breeders can also evaluate the health status of silkworms based on these characteristics and take timely measures according to the evaluation results, so as to maintain the healthy growth of silkworms.

[0069] So far, the growth status of silkworms has been accurately evaluated using the method provided in this embodiment.

[0070] This embodiment divides all connected domains into normal silkworm morphology connected domains and abnormal silkworm morphology connected domains according to the shape features of the connected domains corresponding to each silkworm in each silkworm disc image, and then obtains the possibility that the abnormal edge region in the abnormal silkworm morphology connected domain belongs to another silkworm by analyzing the difference in the bending conditions of the edge lines of the abnormal edge region and the normal edge region in the abnormal silkworm morphology connected domain, and then finds the overlapping silkworm morphology region; then, by combining the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domains, the redundant area in the overlapping silkworm morphology connected domain is divided to the surrounding corresponding connected domains, so as to obtain a more accurate connected domain division result in each silkworm disc image; finally, by combining the division results of silkworm disc images at different growth stages, the growth status of the silkworm is accurately evaluated, thereby ensuring the accuracy of the growth feature extraction results of different silkworms in the silkworm disc image, and improving the accuracy of the silkworm growth status monitoring results.

[0071] Embodiment of the silkworm health status monitoring system based on computer vision: See also Figure 2 , which shows a structural block diagram of a silkworm health status monitoring system based on computer vision provided by an embodiment of the present invention. The system may include an image acquisition module, a screening module, a division module and an evaluation module.

[0072] Among them, the image acquisition module is used to obtain the images of silkworm discs at different growth stages; The screening module is used to determine the normal silkworm morphology connected domain and the abnormal silkworm morphology connected domain according to the morphological distribution characteristics of the edge lines in the silkworm disc image; evaluate the possibility that the abnormal edge area in the abnormal silkworm morphology connected domain belongs to another silkworm according to the difference in the curvature of the edge lines between the abnormal edge area and the normal edge area in the abnormal silkworm morphology connected domain, and judge whether the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area based on the possibility; the abnormal edge area and the normal edge area are obtained by clustering the convex hull vertices in the connected domain; A division module is used to screen the reference connected domain by combining the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and its surrounding normal silkworm morphology connected domain; and re-dividing the overlapping silkworm morphology region according to the relative position distribution of the abnormal edge region and its reference connected domain; The evaluation module is used to evaluate the growth status of silkworms by combining the segmentation results of silkworm disc images at different growth stages.

[0073] It should be understood that Figure 2The structural block diagram of the silkworm health status monitoring system based on computer vision and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art can understand that the above-mentioned method and system can be implemented using computer executable instructions and / or included in a processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of this specification can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software (for example, firmware).

[0074] For more details about the above modules, please refer to other places in this manual and will not be repeated here.

[0075] In other embodiments, a device for monitoring the health status of silkworms based on computer vision is also provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the above-mentioned method for monitoring the health status of silkworms based on computer vision. The device can be specifically a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the method for monitoring the health status of silkworms based on computer vision provided in the above-mentioned embodiment.

[0076] In other embodiments, a computer program product is also provided. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method for monitoring the health status of silkworms based on computer vision provided in the above-mentioned embodiment.

[0077] In other embodiments, a computer-readable storage medium is also provided, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement the silkworm health status monitoring method based on computer vision provided in the above-mentioned embodiment.

[0078] Among them, the provided system, electronic device, computer program product, and computer-readable storage medium are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0079] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring the health status of silkworms based on computer vision, characterized in that: The method comprises the following steps: Acquire images of silkworm discs at different growth stages; According to the morphological distribution characteristics of the edge lines in the silkworm disc image, a normal silkworm morphological connected domain and an abnormal silkworm morphological connected domain are determined; according to the difference in the curvature of the edge lines between the abnormal edge area and the normal edge area in the abnormal silkworm morphological connected domain, the possibility that the abnormal edge area in the abnormal silkworm morphological connected domain belongs to another silkworm is evaluated, and based on the possibility, whether the abnormal silkworm morphological connected domain belongs to an overlapping silkworm morphological region is judged; the abnormal edge area and the normal edge area are obtained by clustering the convex hull vertices in the connected domain; Based on the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domain, the reference connected domain is screened; according to the relative position distribution of the abnormal edge region and its reference connected domain, the overlapping silkworm morphology region is re-divided; The growth status of silkworms is evaluated by combining the segmentation results of silkworm disc images at different growth stages.

2. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: The method of determining the normal silkworm morphology connected domain and the abnormal silkworm morphology connected domain according to the morphological distribution characteristics of the edge lines in the silkworm disc image comprises: Extract the connected domain in the silkworm disk image, and perform convex hull detection on the connected domain to obtain the convex hull vertices; For any connected domain, the minimum circumscribed rectangle of the connected domain is obtained, the longest side of the minimum circumscribed rectangle is recorded as the body length direction of the silkworm, and the two pixel points with the farthest distance in the body length direction of the silkworm in the edge line of each connected domain are recorded as the suspected silkworm head point and the suspected silkworm tail point respectively; according to the difference between the number of convex hull vertices in the connected domain and the preset number of vertices, and the distances between the convex hull vertices and the suspected silkworm head point and the suspected silkworm tail point, the silkworm morphological validity of the connected domain is obtained; If the silkworm morphology validity is greater than or equal to a preset validity threshold, the corresponding connected domain is determined to be a normal silkworm morphology connected domain; if the silkworm morphology validity is less than the preset validity threshold, the corresponding connected domain is determined to be an abnormal silkworm morphology connected domain.

3. The method for monitoring silkworm health status based on computer vision according to claim 2, characterized in that: Acquisition of abnormal edge areas and normal edge areas, including: For any convex hull vertex in any connected domain, respectively obtain the minimum value of the Euclidean distance between any convex hull vertex and the suspected silkworm head point and the minimum value of the Euclidean distance between any convex hull vertex and the suspected silkworm tail point; if the normalized value of the minimum value in the Euclidean distance is greater than a preset distance threshold, then determine that any convex hull vertex is a first-class point; if the normalized value of the minimum value in the Euclidean distance is less than or equal to the preset distance threshold, then determine that any convex hull vertex is a second-class point; Each type of point in the silkworm disk image is clustered separately, and the convex hull area corresponding to the cluster obtained by clustering the first type of points is used as the abnormal edge area, and the convex hull area corresponding to the cluster obtained by clustering the second type of points is used as the normal edge area.

4. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: The step of evaluating the growth status of silkworms by combining the segmentation results of the silkworm disc images at different growth stages includes: The thickness of the silkworm is evaluated according to the area distribution of all connected domains in the silkworm disc image; the color of the silkworm is evaluated according to the overall grayscale information of all connected domains in the silkworm disc image; the length of the silkworm is evaluated according to the size of all connected domains in the silkworm disc image; Combined with the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages, the growth status validity of the silkworms in the silkworm tray is determined, and the growth status validity is used to characterize the growth status of the silkworms; wherein the characteristics of the silkworms include the thickness, color and length of the silkworms.

5. The method for monitoring silkworm health status based on computer vision according to claim 4, characterized in that: The method of determining the validity of the growth state of the silkworms in the silkworm tray in combination with the changes in the characteristics of the silkworms in the silkworm tray images at different growth stages includes: According to the feature values ​​of the same feature of the silkworm in the silkworm disk images of two adjacent growth stages, the change amount of each feature of the silkworm in the silkworm disk images of two adjacent growth stages is calculated; The growth status validity of the silkworms in the silkworm tray is obtained according to the variation of all the features of the silkworms in the silkworm tray images of two adjacent growth stages, and the variation is positively correlated with the growth status validity.

6. The method for monitoring silkworm health status based on computer vision according to claim 1, characterized in that: The method of evaluating the possibility that the abnormal edge region in the abnormal silkworm morphology connected domain belongs to another silkworm according to the difference in curvature between the edge regions in the abnormal silkworm morphology connected domain and the normal edge region, and judging whether the abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology region based on the possibility, comprises: For any abnormal silkworm morphological connected domain: Respectively calculating the first average curvature value of all pixel points on the edge line of each normal edge region in any abnormal silkworm morphological connected domain, obtaining the minimum value of the difference between the first average curvature value corresponding to each normal edge region in any abnormal silkworm morphological connected domain and the average curvature value of all pixel points on each abnormal edge region in any abnormal silkworm morphological connected domain; determining the possibility that the corresponding abnormal edge region in any abnormal silkworm morphological connected domain belongs to another silkworm according to the minimum value of the difference between the average curvature values, and the minimum value of the difference between the average curvature values ​​is negatively correlated with the possibility of belonging to another silkworm; If the possibility is greater than or equal to a preset possibility threshold, it is determined that the corresponding abnormal edge region belongs to the silkworm morphology edge region; According to the proportion of the number of silkworm morphology edge regions in any abnormal silkworm morphology connected domain, it is determined whether any abnormal silkworm morphology connected domain belongs to an overlapping silkworm morphology region.

7. The method for monitoring silkworm health status based on computer vision according to claim 6, characterized in that: The determining, based on the proportion of the number of silkworm morphology edge regions in any abnormal silkworm morphology connected domain, whether any abnormal silkworm morphology connected domain belongs to an overlapping silkworm morphology region comprises: Determine the ratio between the number of silkworm morphological edge regions in any abnormal silkworm morphological connected domain and the number of abnormal edge regions in any abnormal silkworm morphological connected domain as the number ratio corresponding to any abnormal silkworm morphological connected domain; If the quantity proportion is greater than or equal to the proportion threshold, it is determined that any abnormal silkworm morphology connected domain belongs to the overlapping silkworm morphology area.

8. The method for monitoring silkworm health status based on computer vision according to claim 2, characterized in that: The method combines the similarity of the shape distribution of the edge lines of the abnormal edge region in the overlapping silkworm morphology region and the surrounding normal silkworm morphology connected domain to screen the reference connected domain, including: A preset number of connected domains of normal silkworm morphology closest to the area to be analyzed are used as candidate connected domains; For the region to be analyzed and the candidate connected domain, a straight line is drawn through the center point of the region and is parallel to the longest side of the minimum circumscribed rectangle of the region to be analyzed, and the straight line divides the edge line of the region into two curve segments; each curve segment is fitted to obtain a corresponding change trend, and each end point position of the curve segment is extended based on the change trend to obtain an extended line; wherein the number of extended pixel points is a preset pixel point value; the extended lines are connected, and the obtained closed area is used as the corresponding extended area; According to the correlation between the positions of the pixels in the extended area corresponding to the area to be analyzed and the extended area corresponding to each candidate connected area, the reference connected area of ​​the abnormal edge area is screened; The area to be analyzed is any abnormal edge area in the overlapping silkworm morphology area.

9. The method for monitoring silkworm health status based on computer vision according to claim 8, characterized in that: The method of screening the reference connected domain of the abnormal edge region according to the correlation between the positions of the pixel points in the extended region corresponding to the region to be analyzed and the extended region corresponding to each candidate connected domain comprises: The correlation coefficient between the pixel point coordinate set in the extended area corresponding to the area to be analyzed and the pixel point coordinate set in the extended area corresponding to each candidate connected domain is recorded as the membership factor of each candidate connected domain; The candidate connected domains whose membership factors are greater than the preset membership threshold are determined as the reference connected domains of the area to be analyzed.

10. The method for monitoring silkworm health status based on computer vision according to claim 8, characterized in that: The method of re-dividing the overlapping silkworm morphology region according to the relative position distribution between the abnormal edge region and its reference connected domain comprises: The pixel points of the overlapped part in the extended area corresponding to the area to be analyzed and the extended area corresponding to the reference connected domain are divided into the normal silkworm morphology connected domain where the reference connected domain is located.

Images