Real-time monitoring method and system for stiff pupae based on deep learning, and storage medium

Abstract

The invention relates to the technical field of silkworm chrysalis breeding, and discloses a dead pupa real-time monitoring method and system based on deep learning and a storage medium. The method comprises the following steps: shooting a first image of a target area, and carrying out gridding segmentation on the first image to obtain a plurality of second images; integrating the second images into an image sequence, extracting a silkworm chrysalis area in the image sequence based on a deep learning model, and generating a corresponding silkworm chrysalis contour based on the silkworm chrysalis area; the same silkworm chrysalis contours are integrated into a contour sequence, and the contour sequence is analyzed to obtain individual features of each silkworm chrysalis; dividing silkworm chrysalis into rigid silkworm chrysalis and common silkworm chrysalis based on individual characteristics; and comparing the actual images of the stiff silkworm chrysalis and the common silkworm chrysalis with a preset image, and determining the stiff stage of the stiff silkworm chrysalis and the physiological state of the common silkworm chrysalis. According to the invention, the stiffness stage of silkworm chrysalis can be monitored in real time, and the abnormal cause of death of stiff silkworm chrysalis can be determined, so that the automation degree of breeding is greatly improved.

Description

Technical Field

[0001] The present application relates to the technical field of silkworm pupa breeding, and in particular to a real-time monitoring method, system and storage medium for silkworm pupa based on deep learning. Background Art

[0002] Dead pupa refers to the process of artificially inoculating silkworms by spraying Beauveria bassiana onto the surface of silkworms or feeding them on mulberry leaves when the silkworms grow to 4 to 5 years old. The Beauveria bassiana infects the silkworms and rapidly reproduces in their bodies, causing the silkworms to die. After that, the mycelium in their bodies continues to grow and reproduce, and gradually fills the silkworm body, causing the silkworm's corpse to become rigid. The rigidified silkworm's body is filled with mycelium and spores of Beauveria bassiana, forming dead pupa. In order to facilitate storage and transportation, the dead pupae are usually dried in the sun. Because they are white in color, they are called dead pupae.

[0003] The traditional breeding method is to estimate the formation time of stiff pupae through manual measurement, and then when the estimated time arrives, go to the silkworm seat to pick out the stiffened silkworm pupae for drying. However, since the death time of silkworms is different, it is necessary to go to the silkworm seat many times to pick out the stiff pupae. When carrying out large-scale breeding, this method will consume a lot of manpower. Therefore, in the large-scale breeding process, the following method is introduced in the prior art to monitor the breeding process of silkworms. For example, Chinese patent document with publication number CN114972480A discloses a method and system for measuring the area of ​​silkworms in the silkworm seat. The method first takes an image of the silkworm seat, and then removes the mulberry leaf area in the silkworm seat image through image recognition to obtain the area where the silkworms are located. Then, the behavior and state of the silkworms are tracked to automatically determine the timing of feeding the silkworms.

[0004] Although the above method can be applied to the death monitoring of silkworms, during the cultivation of pupae, it is still necessary to determine whether the death of the silkworms is caused by infection with Beauveria bassiana or infection with a disease. The prior art does not provide a solution to this problem. Summary of the invention

[0005] In order to solve the problems raised in the above background technology, the present application provides a real-time monitoring method, system and storage medium for zombie pupae based on deep learning.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention proposes a real-time monitoring method for pupae based on deep learning, comprising:

[0007] Taking a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images;

[0008] Integrate the second images of the same area into an image sequence, extract the silkworm pupa region in each of the second images in the image sequence based on the deep learning model, and generate a corresponding silkworm pupa contour based on the silkworm pupa region;

[0009] Mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours;

[0010] Integrate the silkworm pupa contours with the same number into a contour sequence, and analyze the contour sequence to obtain individual features of each silkworm pupa, wherein the individual features include individual movement features and surface texture features;

[0011] Based on the individual characteristics, the silkworm pupae are divided into rigid silkworm pupae and common silkworm pupae, and a silkworm pupae image library is established, wherein the silkworm pupae image library includes a plurality of preset images;

[0012] The actual images of the rigid silkworm pupae and the common silkworm pupae are compared with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

[0013] Furthermore, mapping the silkworm pupa contours to each other comprises the following steps:

[0014] Positioning the second images of adjacent time sequence in the image sequence and defining them as preceding images and succeeding images respectively, defining the silkworm pupa outline in the preceding image as a body outline, positioning the first endpoint and the second endpoint of each of the body outlines in the preceding image, generating an endpoint connecting line for connecting the first endpoint and the second endpoint within the body outline, generating a first circular area with the center point of each of the endpoint connecting lines as the center of the circle, and the radius of the first circular area is determined based on the preset time interval;

[0015] The first circular area is mapped from the preceding image to the succeeding image, the silkworm pupa outline located in the first circular area in the succeeding image is defined as an alternative outline, a first similarity between each of the alternative outlines and the corresponding main outline is calculated, and the alternative outline whose first similarity is greater than a first threshold is determined as a mapping of the main outline in the succeeding image.

[0016] Further, calculating the first similarity includes the following steps:

[0017] Taking the center point as the center of the circle, a second circular area is generated in the main body contour and the alternative contour respectively, the parts of the main body contour and the alternative contour within the second circular area are located as the body contour, a plurality of distance measuring lines are drawn at intervals with the center point as the starting point and the boundary of the body contour and the second circular area as the end point, and a first curve corresponding to the change in length of the distance measuring lines of the main body contour and a second curve corresponding to the change in length of the distance measuring lines of the alternative contour are drawn in the coordinate system;

[0018] The first curve is respectively aligned with different second curves, and a plurality of comparison points with the same horizontal coordinates are located in the aligned first curve and the second curve, and the overlap degree between the first curve and the second curve is calculated based on the comparison points, and mapping data is obtained after the overlap degree is normalized, and the mapping data is used as the first similarity.

[0019] Furthermore, comparing and analyzing the actual image with the preset image includes the following steps:

[0020] Marking an image label for each of the preset images, locating the silkworm pupa regions in the actual image and the preset image, defining them as a first region and a second region respectively, locating the endpoint connection lines in the first region and the second region, and splitting the first region and the second region into a plurality of sub-regions along the endpoint connection lines;

[0021] The sub-similarity of the sub-regions corresponding to the first region and the second region is calculated, an average value of the sub-similarity is used as the second similarity, and the image label corresponding to the preset image with the largest second similarity is assigned to the actual image to determine the rigidification stage or the physiological state of the actual image.

[0022] Further, calculating the sub-similarity comprises the following steps:

[0023] A pixel value of each pixel in the sub-region is obtained, a color distribution histogram of the sub-region is constructed based on the pixel value, and the sub-similarity between the two sub-regions is calculated based on the color distribution histogram.

[0024] Further, calculating the individual movement feature comprises the following steps:

[0025] The endpoint connection line is used as the first skeleton diagram and the second skeleton diagram of the body contour in the preceding image and the succeeding image respectively, a first movement amount is calculated based on the position of the center point in the preceding image and the succeeding image, the first skeleton diagram is superimposed on the second skeleton diagram with the center point as a reference point, the distance between the first endpoints and the second endpoints after superposition is used as the second movement amount, and the sum of the first movement amount and the second movement amount is defined as the individual movement feature.

[0026] Furthermore, the rigidification stage includes a starting stage and a completion stage, the physiological state includes a feeding state, a dormant state, a death state and an abnormal state, and a first reminder is generated when the number of rigidified silkworm pupae in the completion stage reaches a second threshold, and a second reminder is generated when the number of ordinary silkworm pupae in the death state or the abnormal state reaches a third threshold.

[0027] Furthermore, the deep learning model is a U-NET model.

[0028] The present application provides a real-time monitoring system for pupae based on deep learning, which is used to implement the above-mentioned real-time monitoring method for pupae based on deep learning. The system includes:

[0029] A shooting module, shooting a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images;

[0030] A recognition module, integrating the second images of the same area into an image sequence, extracting a silkworm pupa region in each of the second images in the image sequence based on a deep learning model, generating a corresponding silkworm pupa contour based on the silkworm pupa region, mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours;

[0031] An analysis module integrates the silkworm pupa contours with the same number into a contour sequence, and analyzes the contour sequence to obtain individual features of each silkworm pupa, wherein the individual features include individual movement features and surface texture features;

[0032] The judgment module divides the silkworm pupae into rigid silkworm pupae and common silkworm pupae based on the individual characteristics, establishes a silkworm pupae image library, and the silkworm pupae image library includes a plurality of preset images. The actual images of the rigid silkworm pupae and the common silkworm pupae are compared with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

[0033] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, a real-time monitoring method for zombie pupae based on deep learning as described above is implemented.

[0034] Beneficial effects:

[0035] The present invention performs grid segmentation on the first image, and each grid area is converted into an independent second image. These images can be processed in parallel later, which significantly improves the speed of system recognition. Through the contour segmentation technology based on the deep learning model, the silkworm pupa area in the image sequence can be accurately extracted, and then its contour is extracted. After that, the silkworm pupa contours belonging to the same silkworm pupa are integrated into an image sequence, and the individual characteristics of the silkworm pupae are obtained by analyzing the image sequence, so as to identify the survival of the silkworm pupae and whether they are infected with Beauveria bassiana, and then find and distinguish between ordinary silkworm pupae and rigid silkworm pupae. Finally, through the comparison of the image library, the rigid stage of the rigid silkworm pupae can be further determined, and the cause of death can be judged according to the surface texture characteristics, so that the breeder can monitor the state of the silkworm pupae remotely, reducing the labor pressure of the breeder. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the real-time monitoring method of pupae based on deep learning in this application;

[0037] Figure 2 Schematic diagram of generating a sequence of contours for this application;

[0038] Figure 3 A schematic diagram of a first circular area is generated for this application;

[0039] Figure 4 A schematic diagram of a first curve and a second curve is drawn for this application;

[0040] Figure 5 A schematic diagram comparing the first skeleton diagram and the second skeleton diagram for this application;

[0041] Figure 6 This is a structural diagram of the real-time monitoring system for zombie pupae based on deep learning in this application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0044] like Figure 1 As shown, a real-time monitoring method for pupae based on deep learning includes:

[0045] S1: capturing a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images.

[0046] The preset resolution of this embodiment is 4096×2160, and the preset time interval is 10s. In other embodiments, it can also be 1min. The target area is the silkworm seat area. Each first image is then gridded according to 500x500 pixels. Each first image will be divided into multiple second images, and each second image contains a grid area in the original image. Compared with the sliding window method, this method can process the second image in parallel in the subsequent recognition process, thereby speeding up the system recognition speed.

[0047] S2: Integrate the second images of the same area into an image sequence, extract the silkworm pupa area in each second image in the image sequence based on the deep learning model, and generate the corresponding silkworm pupa outline based on the silkworm pupa area.

[0048] In this embodiment, the deep learning model is a U-NET model.

[0049] The second image of the same area obtained each time is integrated into an image sequence in chronological order, such as Figure 2 As shown, in Stp1, the second image E at the same position in the first image is cut out in the time sequence and combined into an image sequence, as shown in Stp2, and then each second image in the image sequence is semantically segmented based on the U-NET model to obtain the area where the silkworm pupa is located, that is, the silkworm pupa area. Then the boundary of the silkworm pupa area is extracted to obtain the silkworm pupa outline. In the case of a small number of initial data sets, the U-NET model can be constructed using the transfer learning method.

[0050] S3: mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours.

[0051] S4: The contours of silkworm pupae with the same number are integrated into a contour sequence, and the contour sequence is analyzed to obtain the individual characteristics of each silkworm pupa, wherein the individual characteristics include individual movement characteristics and surface texture characteristics.

[0052] By mapping the images to each other, the outlines of the silkworm pupae belonging to the same silkworm pupae have the same number in each second image. The specific mapping method will be introduced later. Figure 2 After the mapping is completed, the contours of the silkworm pupae belonging to the same number are cut out from each second image and integrated into the contour sequence shown in Step 3. Then the contour sequence is analyzed to obtain the individual movement characteristics and surface texture characteristics of the silkworm pupae within a certain period of time. The individual movement characteristics can be used to determine whether the silkworm is still alive, and the surface texture characteristics can be used to determine whether Beauveria bassiana appears on the silkworm body, or pathological patterns appear.

[0053] S5: The silkworm pupae are divided into rigid silkworm pupae and common silkworm pupae based on individual characteristics, and a silkworm pupae image library is established. The silkworm pupae image library includes a variety of preset images.

[0054] S6: Compare the actual images of the rigid silkworm pupae and the common silkworm pupae with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

[0055] In this embodiment, the rigidification stage includes the starting stage and the completion stage, the physiological state includes the eating state, the dormant state, the dead state and the abnormal state, and the first reminder is generated when the number of rigid silkworm pupae in the completion stage reaches the second threshold, and the second reminder is generated when the number of ordinary silkworm pupae in the dead state or the abnormal state reaches the third threshold.

[0056] When it is determined based on individual movement characteristics that the silkworm has not moved for a long time, it can be determined that the silkworm has died, or has not moved for a certain period of time, and is judged to be in a dormant state. When only the head shows the movement characteristics of chewing mulberry leaves, it is judged to be in a feeding state. If the silkworm pupa is determined to be dead based on individual movement characteristics, and the silkworm pupa is determined to have Beauveria based on surface texture characteristics, it is classified as a rigid silkworm pupa. For silkworm pupae that are dead but do not have Beauveria, the cause of death is further determined. The image of the rigid silkworm pupa is compared with the preset image in the silkworm pupa image library to determine the density of Beauveria on the body of the rigid silkworm pupa, so as to determine the rigid stage of the silkworm pupa. By comparing the image of the ordinary silkworm pupa with the preset image, it can be determined what texture appears on the body of the silkworm pupa, thereby further determining the cause of death of the silkworm pupa, such as gray muscardine disease and aspergillosis.

[0057] In particular, if the number of rigid silkworm pupae in the completed stage reaches the second threshold, a first reminder is generated to remind the breeder that a large number of rigid silkworm pupae have been formed and they should be taken out and dried in time. If the number of silkworm pupae in the dead state or abnormal state reaches the third threshold, a second reminder is generated to remind the breeder that a large number of silkworm pupae have died abnormally and attention should be paid.

[0058] The present invention performs grid segmentation on the first image, and each grid area is converted into an independent second image. These images can be processed in parallel later, which significantly improves the speed of system recognition. Through the contour segmentation technology based on the deep learning model, the silkworm pupa area in the image sequence can be accurately extracted, and then its contour is extracted. After that, the silkworm pupa contours belonging to the same silkworm pupa are integrated into an image sequence, and the individual characteristics of the silkworm pupae are obtained by analyzing the image sequence, so as to identify the survival of the silkworm pupae and whether they are infected with Beauveria bassiana, and then find and distinguish between ordinary silkworm pupae and rigid silkworm pupae. Finally, by comparing the image library, the rigid stage of rigid silkworm pupae can be further determined, and the cause of death can be judged according to the surface texture characteristics, which enables breeders to monitor the state of silkworm pupae remotely, reducing the labor pressure of breeders.

[0059] It is particularly noteworthy that the present invention can monitor the rigidification stage of silkworm pupae in real time and determine the abnormal reasons for the death of rigid pupae, thereby greatly improving the automation level of breeding.

[0060] In this embodiment, mapping the silkworm pupa contours to each other includes the following steps:

[0061] The second images of adjacent time sequences in the image sequence are located and defined as preceding and succeeding images respectively, the outline of the silkworm pupa in the preceding image is defined as the main body outline, the first endpoint and the second endpoint of each main body outline in the preceding image are located, an endpoint connecting line for connecting the first endpoint and the second endpoint is generated within the main body outline, a first circular area is generated with the center point of each endpoint connecting line as the center of the circle, and the radius of the first circular area is determined based on a preset time interval.

[0062] Here, we take the first image and the second image in the image sequence as an example. The first image is defined as the preceding image, and the second image is defined as the succeeding image. The preceding image may include multiple silkworm pupa contours. All silkworm pupa contours included in the preceding image are defined as the main body contours. The head and tail of each main body contour, i.e., the first endpoint and the second endpoint, are located, and the endpoint connection line is generated inside the silkworm pupa contour, such as Figure 3 As shown by the connecting line between A and B in P1, a first circular area is generated with the center point C of the connecting line between the first endpoint A and the second endpoint B as the center of the circle. The larger the preset time interval, the larger the radius of the generated first circular area. The first circular area is the possible movement range of the body contour within the preset time interval.

[0063] The first circular area is mapped from the preceding image to the succeeding image, the outline of the silkworm pupa located in the first circular area in the succeeding image is defined as an alternative outline, the first similarity between each alternative outline and the corresponding main outline is calculated, and the alternative outline whose first similarity is greater than a first threshold is determined as the mapping of the main outline in the succeeding image.

[0064] According to the position of the first circular area in the preceding image, it is mapped to the subsequent image accordingly, as shown in P2. In the subsequent image, other surviving silkworm pupae may move to the first circular area, resulting in the presence of multiple silkworm pupae outlines in the first circular area. Therefore, the silkworm pupae outlines in P2 may all be the result of the main body outline moving, and they are all identified as candidate outlines. Afterwards, each candidate outline is compared with the main body outline, and which candidate outline is the result of the main body outline moving is determined based on the first similarity obtained by the comparison. After this step, the second image and the third image are continuously mapped until the end of the image sequence is reached.

[0065] In this embodiment, calculating the first similarity includes the following steps:

[0066] Generate a second circular area in the main body contour and the alternative contour respectively with the center point as the center of the circle, locate the parts of the main body contour and the alternative contour within the second circular area as the body contour, draw a plurality of distance measurement lines with the center point as the starting point and the boundary of the body contour and the second circular area as the end point, and draw a first curve corresponding to the change of the length of the distance measurement line of the main body contour and a second curve corresponding to the change of the length of the distance measurement line of the alternative contour in the coordinate system;

[0067] The first curve is respectively aligned with different second curves, and a plurality of comparison points with the same horizontal coordinates are located in the aligned first and second curves, and the overlap degree of the first curve and the second curve is calculated based on the comparison points, and mapping data is obtained after the overlap degree is normalized, and the mapping data is used as the first similarity.

[0068] like Figure 4 As shown, taking the body contour as an example, in the body contour, a second circular area is generated with the center point as the center of the circle, and the radius of the second circular area is smaller than the first circular area. Then, the part of the body contour within the second circular area is intercepted as the body contour, as shown in Figure 4 As shown by the dotted line in , multiple radial distance measuring lines are then drawn with the center point as the center of the circle. It can be seen from the figure that since the starting points of the distance measuring lines are the same but the end points are different, their lengths are also different. With the number of the distance measuring line as the X-axis and the length value as the Y-axis, the first curve L1 of the distance measuring line length change is drawn. The first curve drawn according to the distance measuring line length will also show a certain fluctuation trend. Similarly, the second curve L2 of the distance measuring line length change in the alternative contour can be drawn.

[0069] In order to eliminate the influence of the rotation direction of the silkworm pupa, it is necessary to align it before comparison. Figure 4 It can be seen that the end point of some distance measurement lines is on the boundary of the second circular area, and the length of the distance measurement lines of this part is the same. There will be some straight lines in the first curve and the second curve, so the part of the straight line interval between the two curves can be used as a reference for alignment. After alignment, multiple comparison points are selected on the first curve and the second curve at intervals, such as d1 and D1, and then the overlap is calculated using the first formula, which is: , where R is the overlap degree, and are the ordinates of the nth comparison points on the first curve and the second curve, respectively, and m is the number of comparison points. The closer the trends of the two curves are, the greater the calculated overlap. Finally, after calculating the overlap between the first curve and all the second curves, the values ​​are mapped to between 0 and 1 through normalization, thereby obtaining the first similarity.

[0070] In this embodiment, the comparative analysis of the actual image and the preset image includes the following steps:

[0071] Mark an image label for each preset image, locate the silkworm pupa area in the actual image and the preset image, define them as the first area and the second area respectively, locate the endpoint connection line in the first area and the second area, and split the first area and the second area into multiple sub-areas along the endpoint connection line.

[0072] The sub-similarity of the corresponding sub-regions in the first region and the second region is calculated, the average of the sub-similarity is used as the second similarity, and the image label corresponding to the preset image with the largest second similarity is assigned to the actual image to determine the rigidification stage or physiological state of the actual image.

[0073] Specifically, the image labels include the starting stage, the completion stage, the eating state, the dormant state, the dead state and the abnormal state, as well as the labels of various diseases. The silkworm pupa regions to be compared are cut out from the actual image and the preset image, and are defined as the first region and the second region respectively. Then, the first region is split into three sub-regions along the endpoint connection line, and the second region is split into three sub-regions. The three sub-regions are defined as the head region, the body region and the tail region respectively. The actual image and the preset image can be compared in a more detailed manner through splitting. Then, the head region, the body region and the tail region are compared to obtain three sub-similarity, and the average value of the sub-similarity is used as the second similarity between the actual image and the preset image. Through comparison, it is found that the preset image with the image label of the completion stage has the highest similarity with the actual image, so the state of the silkworm pupa in the actual image is marked as a stiff pupa, which is in the completion stage.

[0074] Calculating sub-similarity includes the following steps:

[0075] The pixel value of each pixel in the sub-region is obtained, a color distribution histogram of the sub-region is constructed based on the pixel value, and the sub-similarity of the two sub-regions is calculated based on the color distribution histogram.

[0076] When calculating the sub-similarity, we first construct a histogram representing the distribution of these colors by counting the number of occurrences of each color. For example, if sub-region 1 contains 50 pixels, 20 of which are red, 15 are green, 10 are blue, and 5 are other colors, then the color distribution histogram of sub-region 1 will have this information. Then, the histogram intersection method is used to calculate the similarity between the two histograms, that is, the sub-similarity. The specific calculation process of the histogram intersection method is prior art and will not be repeated here.

[0077] In this embodiment, calculating the individual movement amount includes the following steps:

[0078] The endpoint connection line is used as the first skeleton map and the second skeleton map of the body contour in the previous image and the subsequent image respectively, the first movement amount is calculated based on the position of the center point in the previous image and the subsequent image, the first skeleton map is superimposed on the second skeleton map with the center point as the reference point, the distance between the first endpoints and the second endpoints after superposition is used as the second movement amount, and the sum of the first movement amount and the second movement amount is defined as the individual movement feature.

[0079] After determining the position of the body contour in the preceding image and the succeeding image, the first movement amount is calculated according to the position of the center point in the preceding image and the succeeding image. The first movement amount reflects the distance that the position of the silkworm pupa as a whole has moved. Then, the first skeleton image is superimposed on the second skeleton image, as shown in FIG. Figure 5 As shown, the center point is used as the reference point during superposition. After superposition, the sum of the distance between the first endpoints A1 and A2 and the distance between the second endpoints B1 and B2 is the second movement amount. The second movement amount reflects the changes in the head and tail of the silkworm pupa. Finally, the sum of the first movement amount and the second movement amount is used as the individual movement feature.

[0080] In particular, when superimposing, the second skeleton graph is rotated to calculate a plurality of second movement amounts, and the second movement amount with the smallest value is output as the final result.

[0081] like Figure 6 As shown, the present application provides a real-time monitoring system for pupae based on deep learning, which is used to implement the above-mentioned real-time monitoring method for pupae based on deep learning. The system includes:

[0082] A shooting module, shooting a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images;

[0083] A recognition module integrates the second images of the same area into an image sequence, extracts the silkworm pupa region in each second image in the image sequence based on the deep learning model, generates corresponding silkworm pupa contours based on the silkworm pupa region, maps the silkworm pupa contours in the image sequence to each other, and numbers the mapped silkworm pupa contours;

[0084] The analysis module integrates the contours of silkworm pupae with the same number into a contour sequence, and analyzes the contour sequence to obtain the individual characteristics of each silkworm pupa, wherein the individual characteristics include individual movement characteristics and surface texture characteristics;

[0085] The judgment module divides silkworm pupae into rigid silkworm pupae and common silkworm pupae based on individual characteristics, establishes a silkworm pupae image library, and the silkworm pupae image library includes a variety of preset images. The actual images of rigid silkworm pupae and common silkworm pupae are compared with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

[0086] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, a real-time monitoring method for zombie pupae based on deep learning as described above is implemented.

[0087] It should be understood that the various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time monitoring method for pupae based on deep learning, characterized in that: Taking a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images; Integrate the second images of the same area into an image sequence, extract the silkworm pupa region in each of the second images in the image sequence based on the deep learning model, and generate a corresponding silkworm pupa contour based on the silkworm pupa region; Mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours; Integrate the silkworm pupa contours with the same number into a contour sequence, and analyze the contour sequence to obtain individual features of each silkworm pupa, wherein the individual features include individual movement features and surface texture features; Based on the individual characteristics, the silkworm pupae are divided into rigid silkworm pupae and common silkworm pupae, and a silkworm pupae image library is established, wherein the silkworm pupae image library includes a plurality of preset images; The actual images of the rigid silkworm pupae and the common silkworm pupae are compared with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

2. The real-time monitoring method of pupae based on deep learning according to claim 1 is characterized in that: The mutual mapping of the silkworm chrysalis contours comprises the following steps: Positioning the second images of adjacent time sequence in the image sequence and defining them as preceding images and succeeding images respectively, defining the silkworm pupa outline in the preceding image as a body outline, positioning the first endpoint and the second endpoint of each of the body outlines in the preceding image, generating an endpoint connecting line for connecting the first endpoint and the second endpoint within the body outline, generating a first circular area with the center point of each of the endpoint connecting lines as the center of the circle, and the radius of the first circular area is determined based on the preset time interval; The first circular area is mapped from the preceding image to the succeeding image, the silkworm pupa outline located in the first circular area in the succeeding image is defined as an alternative outline, a first similarity between each of the alternative outlines and the corresponding main outline is calculated, and the alternative outline whose first similarity is greater than a first threshold is determined as a mapping of the main outline in the succeeding image.

3. The real-time monitoring method of pupae based on deep learning according to claim 2 is characterized in that: Calculating the first similarity comprises the following steps: Taking the center point as the center of the circle, a second circular area is generated in the main body contour and the alternative contour respectively, the parts of the main body contour and the alternative contour within the second circular area are located as the body contour, a plurality of distance measuring lines are drawn at intervals with the center point as the starting point and the boundary of the body contour and the second circular area as the end point, and a first curve corresponding to the change in length of the distance measuring lines of the main body contour and a second curve corresponding to the change in length of the distance measuring lines of the alternative contour are drawn in the coordinate system; The first curve is respectively aligned with different second curves, and a plurality of comparison points with the same horizontal coordinates are located in the aligned first curve and the second curve, and the overlap degree between the first curve and the second curve is calculated based on the comparison points, and mapping data is obtained after the overlap degree is normalized, and the mapping data is used as the first similarity.

4. The real-time monitoring method of pupae based on deep learning according to claim 2 is characterized in that: Comparing and analyzing the actual image with the preset image includes the following steps: Marking an image label for each of the preset images, locating the silkworm pupa regions in the actual image and the preset image, defining them as a first region and a second region respectively, locating the endpoint connection lines in the first region and the second region, and splitting the first region and the second region into a plurality of sub-regions along the endpoint connection lines; The sub-similarity of the sub-regions corresponding to the first region and the second region is calculated, an average value of the sub-similarity is used as the second similarity, and the image label corresponding to the preset image with the largest second similarity is assigned to the actual image to determine the rigidification stage or the physiological state of the actual image.

5. The method for real-time monitoring of pupae based on deep learning according to claim 4 is characterized in that: Calculating the sub-similarity comprises the following steps: A pixel value of each pixel in the sub-region is obtained, a color distribution histogram of the sub-region is constructed based on the pixel value, and the sub-similarity between the two sub-regions is calculated based on the color distribution histogram.

6. The real-time monitoring method of pupae based on deep learning according to claim 2 is characterized in that: Calculating the individual movement characteristics comprises the following steps: The endpoint connection line is used as the first skeleton diagram and the second skeleton diagram of the body contour in the preceding image and the succeeding image respectively, a first movement amount is calculated based on the position of the center point in the preceding image and the succeeding image, the first skeleton diagram is superimposed on the second skeleton diagram with the center point as a reference point, the distance between the first endpoints and the second endpoints after superposition is used as the second movement amount, and the sum of the first movement amount and the second movement amount is defined as the individual movement feature.

7. The real-time monitoring method of pupae based on deep learning according to claim 1 is characterized in that: The rigidification stage includes a starting stage and a completion stage, and the physiological state includes a feeding state, a dormant state, a death state and an abnormal state. When the number of rigid silkworm pupae in the completion stage reaches a second threshold, a first reminder is generated, and when the number of ordinary silkworm pupae in the death state or the abnormal state reaches a third threshold, a second reminder is generated.

8. The real-time monitoring method of pupae based on deep learning according to claim 1 is characterized in that: The deep learning model is a U-NET model.

9. A real-time monitoring system for pupae based on deep learning, used to implement a real-time monitoring method for pupae based on deep learning as claimed in any one of claims 1 to 8, characterized in that: A shooting module, shooting a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images; A recognition module, integrating the second images of the same area into an image sequence, extracting a silkworm pupa region in each of the second images in the image sequence based on a deep learning model, generating a corresponding silkworm pupa contour based on the silkworm pupa region, mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours; An analysis module integrates the silkworm pupa contours with the same number into a contour sequence, and analyzes the contour sequence to obtain individual features of each silkworm pupa, wherein the individual features include individual movement features and surface texture features; The judgment module divides the silkworm pupae into rigid silkworm pupae and common silkworm pupae based on the individual characteristics, establishes a silkworm pupae image library, and the silkworm pupae image library includes a plurality of preset images. The actual images of the rigid silkworm pupae and the common silkworm pupae are compared with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, a real-time monitoring method for pupae based on deep learning as described in any one of claims 1 to 8 is implemented.

Images