Deep learning-based real-time monitoring method, system and storage medium for rigid pupa

Abstract

The present application relates to the technical field of silkworm pupa breeding, and discloses a real-time monitoring method, system and storage medium for rigid silkworm pupae based on deep learning. The method includes: taking a first image of a target area, performing grid segmentation on the first image to obtain multiple second images; integrating the second images into an image sequence, extracting the silkworm pupa areas in the image sequence based on a deep learning model, and generating corresponding silkworm pupa contours based on the silkworm pupa areas; integrating the same silkworm pupa contours into a contour sequence, analyzing the contour sequence to obtain the individual characteristics of each silkworm pupa; classifying the silkworm pupae into rigid silkworm pupae and ordinary silkworm pupae based on the individual characteristics; comparing the actual images of the rigid silkworm pupae and ordinary silkworm pupae with preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the ordinary silkworm pupae. The present invention can monitor the rigidification stage of silkworm pupae in real time and determine the abnormal reasons for the death of rigid silkworm pupae, thereby greatly improving the degree of automation of breeding.

Description

Technical Field

[0001] This application relates to the technical field of silkworm pupa breeding, and particularly to a real-time monitoring method, system and storage medium for rigid pupae based on deep learning. Background Art

[0002] A rigid pupa refers to when a silkworm grows to the 4th to 5th instar, Beauveria bassiana is sprayed onto the surface of the silkworm or fed on mulberry leaves through artificial inoculation, causing Beauveria bassiana to infect the silkworm and rapidly reproduce in its body, resulting in the death of the silkworm. Subsequently, the hyphae in its body continue to grow and reproduce, gradually filling the silkworm body, making the silkworm corpse rigidify. After rigidification, the silkworm body is filled with hyphae and spores of Beauveria bassiana, forming a rigid pupa. For easy storage and transportation, the rigid pupa is usually dried by sunning. Because it is white, it is called a rigid pupa.

[0003] The traditional breeding method is to estimate the formation time of the rigid pupa through manual calculation, and then when the estimated time arrives, go to the silkworm bed to pick out the rigidified silkworm pupae for sunning. However, due to the inconsistent death times of silkworms, it is necessary to go to the silkworm bed multiple times to pick out the rigid pupae. In large-scale breeding, this method will consume a large amount of manpower. Therefore, in the process of large-scale breeding, the following methods are introduced in the prior art to monitor the breeding process of silkworms. For example, the Chinese patent document with the publication number CN114972480A discloses a method and system for calculating the area of silkworms in a silkworm bed. This method first takes an image of the silkworm bed, then removes the mulberry leaf area in the silkworm bed image through image recognition to obtain the area where the silkworms are located, and then tracks the behavior and state of the silkworms to automatically determine the timing of feeding silkworms.

[0004] Although the above method can be applied to the death monitoring of silkworms, in the process of cultivating rigid pupae, it is also necessary to determine whether the death of the silkworm is caused by Beauveria bassiana infection or disease infection. The prior art does not give a solution to this problem. Summary of the Invention

[0005] To solve the problems raised in the above background art, this application provides a real-time monitoring method, system and storage medium for rigid pupae based on deep learning.

[0006] To achieve the above invention purpose, the present invention proposes a real-time monitoring method for rigid pupae based on deep learning, including:

[0007] Taking a first image of the target area at a preset resolution and preset time interval, and performing grid segmentation on the first image to obtain multiple second images;

[0008] Integrating the second images of the same area into an image sequence, extracting the silkworm pupa area in each second image in the image sequence based on a deep learning model, and generating a corresponding silkworm pupa contour based on the silkworm pupa area;

[0009] Mutually map the silkworm pupa contours in the image sequence, and number the mapped silkworm pupa contours;

[0010] Integrate the silkworm pupa contours with the same number into a contour sequence, and analyze the contour sequence to obtain the individual characteristics of each silkworm pupa. The individual characteristics include individual movement characteristics and surface texture characteristics;

[0011] Divide the silkworm pupae into rigid silkworm pupae and ordinary silkworm pupae based on the individual characteristics, and establish a silkworm pupa image library. The silkworm pupa image library includes a variety of preset images;

[0012] Compare the actual images of the rigid silkworm pupae and the ordinary silkworm pupae with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the ordinary silkworm pupae.

[0013] Further, mutually mapping the silkworm pupa contours includes the following steps:

[0014] Locate the second images at adjacent time sequences in the image sequence and define them as the pre-order image and the post-order image respectively. The silkworm pupa contour in the pre-order image is defined as the body contour. Locate the first endpoint and the second endpoint of each body contour in the pre-order image, generate an endpoint connection line for connecting the first endpoint and the second endpoint within the body contour, and generate a first circular area with the center point of each endpoint connection line as the center. The radius of the first circular area is determined based on the preset time interval;

[0015] Map the first circular area from the pre-order image to the post-order image, define the silkworm pupa contour in the post-order image located within the first circular area as the alternative contour, calculate the first similarity between each alternative contour and the corresponding body contour, and determine the mapping of the body contour in the post-order image for the alternative contours with the first similarity greater than the first threshold.

[0016] Further, calculating the first similarity includes the following steps:

[0017] Generate second circular areas with the center point as the center in the body contour and the alternative contour respectively, locate the parts of the body contour and the alternative contour within the second circular area as the body outlines, draw multiple ranging lines at intervals with the center point as the starting point and the boundaries of the body outlines and the second circular area as the ending points, and draw a first curve corresponding to the change in the length of the ranging lines of the body contour and a second curve corresponding to the change in the length of the ranging lines of the alternative contour in the coordinate system;

[0018] Align the first curve with different second curves respectively, locate a plurality of comparison points with the same abscissa in the aligned first curve and second curve, calculate the coincidence degree of the first curve and the second curve based on the comparison points, normalize the coincidence degree to obtain mapping data, and use the mapping data as the first similarity.

[0019] Further, the comparative analysis of the actual image and the preset image includes the following steps:

[0020] Mark image labels for each preset image, locate the silkworm pupa regions in the actual image and the preset image, which are respectively defined as the first region and the second region, locate the connecting lines of the endpoints in the first region and the second region, and split the first region and the second region into a plurality of sub-regions along the connecting lines of the endpoints;

[0021] Calculate the sub-similarities of the corresponding sub-regions in the first region and the second region, take the average value of the sub-similarities as the second similarity, and assign the image label corresponding to the preset image with the maximum second similarity to the actual image to determine the rigidification stage or the physiological state of the actual image.

[0022] Further, calculating the sub-similarity includes the following steps:

[0023] Obtain the pixel values of each pixel point in the sub-region, construct a color distribution histogram of the sub-region based on the pixel values, and calculate the sub-similarity of the two sub-regions based on the color distribution histogram.

[0024] Further, calculating the individual movement feature includes the following steps:

[0025] Use the connecting line of the endpoints as the first skeleton diagram and the second skeleton diagram of the body contour in the previous image and the subsequent image respectively, calculate the first movement amount based on the positions of the center points in the previous image and the subsequent image, superimpose the first skeleton diagram on the second skeleton diagram with the center point as the reference point, take the distances between the first endpoints and between the second endpoints after superimposition as the second movement amount, and define the sum of the first movement amount and the second movement amount as the individual movement feature.

[0026] Further, the rigidification stage includes a starting stage and a completion stage, the physiological state includes a feeding state, a dormant state, a death state, and an abnormal state. When the number of rigidified silkworm pupae reaches a second threshold in the completion stage, a first reminder is generated. When the number of ordinary silkworm pupae reaches a third threshold in the death state or the abnormal state, a second reminder is generated.

[0027] Furthermore, the deep learning model is a U-NET model.

[0028] The present application provides a real-time monitoring system for pupae based on deep learning, which is used to implement the above-mentioned real-time monitoring method for pupae based on deep learning. The system includes:

[0029] A shooting module, shooting a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images;

[0030] A recognition module, integrating the second images of the same area into an image sequence, extracting a silkworm pupa region in each of the second images in the image sequence based on a deep learning model, generating a corresponding silkworm pupa contour based on the silkworm pupa region, mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours;

[0031] An analysis module integrates the silkworm pupa contours with the same number into a contour sequence, and analyzes the contour sequence to obtain individual features of each silkworm pupa, wherein the individual features include individual movement features and surface texture features;

[0032] The judgment module divides the silkworm pupae into rigid silkworm pupae and common silkworm pupae based on the individual characteristics, establishes a silkworm pupae image library, and the silkworm pupae image library includes a plurality of preset images. The actual images of the rigid silkworm pupae and the common silkworm pupae are compared with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the common silkworm pupae.

[0033] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, a real-time monitoring method for zombie pupae based on deep learning as described above is implemented.

[0034] Beneficial effects:

[0035] The present invention performs grid segmentation on the first image, and each grid area is converted into an independent second image. These images can be processed in parallel later, which significantly improves the speed of system recognition. Through the contour segmentation technology based on the deep learning model, the silkworm pupa area in the image sequence can be accurately extracted, and then its contour is extracted. After that, the silkworm pupa contours belonging to the same silkworm pupa are integrated into an image sequence, and the individual characteristics of the silkworm pupae are obtained by analyzing the image sequence, so as to identify the survival of the silkworm pupae and whether they are infected with Beauveria bassiana, and then find and distinguish between ordinary silkworm pupae and rigid silkworm pupae. Finally, through the comparison of the image library, the rigid stage of the rigid silkworm pupae can be further determined, and the cause of death can be judged according to the surface texture characteristics, so that the breeder can monitor the state of the silkworm pupae remotely, reducing the labor pressure of the breeder. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the real-time monitoring method of pupae based on deep learning in this application;

[0037] Figure 2 Schematic diagram of generating a sequence of contours for this application;

[0038] Figure 3 A schematic diagram of a first circular area is generated for this application;

[0039] Figure 4 A schematic diagram of a first curve and a second curve is drawn for this application;

[0040] Figure 5 A schematic diagram comparing the first skeleton diagram and the second skeleton diagram for this application;

[0041] Figure 6 This is a structural diagram of the real-time monitoring system for zombie pupae based on deep learning in this application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0044] like Figure 1 As shown, a real-time monitoring method for pupae based on deep learning includes:

[0045] S1: capturing a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images.

[0046] The preset resolution of this embodiment is 4096×2160, and the preset time interval is 10s. In other embodiments, it can also be 1min. The target area is the silkworm seat area. Each first image is then gridded according to 500x500 pixels. Each first image will be divided into multiple second images, and each second image contains a grid area in the original image. Compared with the sliding window method, this method can process the second image in parallel in the subsequent recognition process, thereby speeding up the system recognition speed.

[0047] S2: Integrate the second images of the same area into an image sequence, extract the silkworm pupa areas in each second image in the image sequence based on a deep learning model, and generate corresponding silkworm pupa contours based on the silkworm pupa areas.

[0048] In this embodiment, the deep learning model is a U-NET model.

[0049] Integrate the second images of the same area obtained from each shooting into an image sequence in chronological order. As Figure 2 shown, in Stp1, the second images E at the same position in the first image are intercepted and combined into an image sequence according to the shooting order in time. As shown in Stp2, then, based on the U-NET model, semantic segmentation is performed on each second image in the image sequence to obtain the area where the silkworm pupa is located, that is, the silkworm pupa area. Then, the boundary of the silkworm pupa area is extracted to obtain the silkworm pupa contour. In the case of a small initial dataset, the method of transfer learning can be used to construct the U-NET model.

[0050] S3: Mutually map the silkworm pupa contours in the image sequence, and number the mapped silkworm pupa contours.

[0051] S4: Integrate the silkworm pupa contours with the same number into a contour sequence, analyze the contour sequence to obtain the individual characteristics of each silkworm pupa, and the individual characteristics include individual movement characteristics and surface texture characteristics.

[0052] By mutually mapping the images, the silkworm pupa contours belonging to the same silkworm pupa have the same number in each second image. The specific mapping method will be introduced later. Continuing to refer to Figure 2 , after the mapping is completed, the silkworm pupa contours belonging to the same number are intercepted from each second image and integrated into the contour sequence shown in Step3. Then, the contour sequence is analyzed to obtain the individual movement characteristics and surface texture characteristics of the silkworm pupa within a certain period of time. Through the individual movement characteristics, it can be known whether the silkworm is still alive, and through the surface texture characteristics, it can be known whether white muscardine fungi appear on the silkworm body or whether pathological patterns appear.

[0053] S5: Classify the silkworm pupae into rigid silkworm pupae and ordinary silkworm pupae based on the individual characteristics, and establish a silkworm pupa image library, which includes a variety of preset images.

[0054] S6: Compare the actual images of the rigid silkworm pupae and ordinary silkworm pupae with the preset images to determine the rigid stage of the rigid silkworm pupae and the physiological state of the ordinary silkworm pupae.

[0055] In this embodiment, the rigidification stage includes a starting stage and a completion stage, and the physiological states include a feeding state, a dormant state, a dead state, and an abnormal state. When the number of rigidified silkworm pupae in the completion stage reaches a second threshold, a first reminder is generated. When the number of ordinary silkworm pupae in the dead state or the abnormal state reaches a third threshold, a second reminder is generated.

[0056] When it is determined based on the individual movement characteristics that the silkworm has not moved for a long time, it can be determined that the silkworm has died, or if it has not moved within a certain period of time, it is determined to be in the dormant state. When only the head shows the movement characteristics of eating mulberry leaves, it is judged to be in the feeding state. If it is determined based on the individual movement characteristics that the silkworm pupa has died, and it is determined based on the surface texture characteristics that Beauveria bassiana has appeared on the silkworm pupa, then it is classified as a rigidified silkworm pupa. For the silkworm pupae that are dead but without the appearance of Beauveria bassiana, the cause of their death is continuously judged. The picture of the rigidified silkworm pupa is compared with the preset pictures in the silkworm pupa image library, so as to determine the density of Beauveria bassiana on the body of the rigidified silkworm pupa, and to determine the rigidification stage of the silkworm pupa. By comparing the picture of the ordinary silkworm pupa with the preset picture, it can be determined what texture appears on the body of the silkworm pupa, and thus the cause of the death of the silkworm pupa, such as gray muscardine disease and aspergillosis, can be further determined.

[0057] Specifically, when the number of rigidified silkworm pupae in the completion stage reaches the second threshold, a first reminder is generated to remind the breeder that a large number of rigid pupae have formed, and they should be taken out in time for drying. When the number in the dead state or the abnormal state reaches the third threshold, a second reminder is generated to remind the breeder that a large number of silkworm pupae have died abnormally and need attention.

[0058] In the present invention, the first image is segmented into grids, and each grid area is transformed into an independent second image, which can be processed in parallel subsequently, significantly improving the system recognition speed. Through the contour segmentation technology based on the deep learning model, the silkworm pupa area in the image sequence can be accurately extracted, and then its contour can be extracted. Then the silkworm pupa contours belonging to the same silkworm pupa are integrated into an image sequence, and the individual characteristics of the silkworm pupa are obtained by analyzing the image sequence, so as to identify the survival situation of the silkworm pupa and whether it is infected with Beauveria bassiana, and further discover and distinguish ordinary silkworm pupae and rigidified silkworm pupae. Finally, through the comparison with the image library, the rigidification stage of the rigidified silkworm pupa can be further determined, and the cause of death can be judged according to the surface texture characteristics, which enables the breeder to monitor the state of the silkworm pupa remotely and reduces the labor pressure of the breeder.

[0059] It should be particularly noted that the present invention can monitor the rigidification stage of the silkworm pupa in real time and determine the abnormal cause of the death of the rigid pupa, thus greatly improving the automation degree of breeding.

[0060] In this embodiment, the mutual mapping of the silkworm pupa contours includes the following steps:

[0061] Locate the second image in adjacent time series of the image sequence and define them as the pre-image and the post-image respectively. The silkworm pupa contour located in the pre-image is defined as the body contour. Locate the first endpoint and the second endpoint of each body contour in the pre-image, generate an endpoint connection line inside the body contour for connecting the first endpoint and the second endpoint, generate a first circular region with the center point of each endpoint connection line as the center, and the radius of the first circular region is determined based on a preset time interval.

[0062] Here, the first image and the second image in the image sequence are taken as examples for illustration. The first image is defined as the pre-image, and the second image is defined as the post-image. The pre-image may include multiple silkworm pupa contours, and all the silkworm pupa contours included in the pre-image are defined as body contours. Locate the head and the tail of each body contour, that is, the first endpoint and the second endpoint, and generate an endpoint connection line inside the silkworm pupa contour, as shown by the connection line between A and B in P1 in Figure 3 Generate a first circular region with the center point C of the endpoint connection line between the first endpoint A and the second endpoint B as the center. The larger the preset time interval is, the larger the radius of the generated first circular region is. The first circular region is the possible movement range of the body contour within the preset time interval.

[0063] Map the first circular region from the pre-image to the post-image, define the silkworm pupa contour located in the first circular region in the post-image as the alternative contour, calculate the first similarity between each alternative contour and the corresponding body contour, and determine the mapping of the body contour in the post-image for the alternative contour with the first similarity greater than the first threshold.

[0064] According to the position of the first circular region in the pre-image, map it to the post-image correspondingly. As shown in P2, in the subsequent images, other surviving silkworm pupae may move into the first circular region, resulting in multiple silkworm pupa contours in the first circular region. Therefore, the silkworm pupa contours in P2 may all be the results of the movement of the body contour. All of them are identified as alternative contours. Then, by comparing each alternative contour with the body contour, determine which alternative contour is the result of the movement of the body contour according to the first similarity obtained from the comparison. After this step, continue to map the second image and the third image until the end of the image sequence is reached.

[0065] In this embodiment, calculating the first similarity includes the following steps:

[0066] Generate second circular regions centered at the center point within the body contour and the alternative contour respectively, locate the parts of the body contour and the alternative contour within the second circular regions as the body outline, draw multiple ranging lines at intervals with the center point as the starting point and the boundary of the body outline and the second circular region as the ending points, and draw a first curve of the length change of the ranging lines corresponding to the body contour and a second curve of the length change of the ranging lines corresponding to the alternative contour in the coordinate system;

[0067] Align the first curve with different second curves respectively, locate multiple comparison points with the same abscissa in the aligned first curve and second curve, calculate the coincidence degree of the first curve and the second curve based on the comparison points, normalize the coincidence degree to obtain mapping data, and use the mapping data as the first similarity.

[0068] As Figure 4 shown, taking the body contour as an example, within the body contour, generate a second circular region centered at the center point, the radius of the second circular region is smaller than that of the first circular region, and then intercept the part of the body contour within the second circular region as the body outline, as shown by the dotted line in Figure 4 . Then draw multiple radioactive ranging lines centered at the center point. It can be seen from the figure that since the starting points of the ranging lines are the same but the ending points are different, their lengths are also different. Taking the number of the ranging line as the X-axis and the length value as the Y-axis, draw the first curve L1 of the length change of the ranging lines. Then the first curve drawn according to the length of the ranging lines will also show a certain fluctuating trend. Similarly, the second curve L2 of the length change of the ranging lines in the alternative contour can be drawn.

[0069] To eliminate the influence brought by the rotation direction of the silkworm pupa body, it needs to be aligned before comparison. It can be seen from Figure 4 that the ending points of some ranging lines are on the boundary of the second circular region, and the lengths of these ranging lines are the same. There will be a straight line interval in part of the first curve and the second curve. Then the parts of the two curves where the straight line intervals appear can be used as a reference for alignment. After alignment, select multiple comparison points at intervals on the first curve and the second curve, such as d1 and D1, and then calculate the coincidence degree through the first formula. The first formula is: , where R is the coincidence degree, and are the ordinates of the nth comparison point on the first curve and the second curve respectively, and m is the number of comparison points. The closer the trends of the two curves are, the greater the calculated coincidence degree. Finally, after calculating the coincidence degrees of the first curve and all the second curves, map the values to between 0 and 1 through normalization to obtain the first similarity.

[0070] In this embodiment, the comparative analysis of the actual image and the preset image includes the following steps:

[0071] Label the image tags for each preset image, locate the silkworm pupa regions in the actual image and the preset image, which are defined as the first region and the second region respectively, locate the connecting line of the endpoints in the first region and the second region, and split the first region and the second region into multiple sub-regions along the connecting line of the endpoints.

[0072] Calculate the sub-similarity of the corresponding sub-regions in the first region and the second region, take the average value of the sub-similarity as the second similarity, and assign the image tag corresponding to the preset image with the maximum second similarity to the actual image to determine the rigidification stage or physiological state of the actual image.

[0073] Specifically, the image tags include the starting stage, the completion stage, the feeding state, the dormant state, the death state, the abnormal state, and the tags of various diseases. Cut out the silkworm pupa regions to be compared from the actual image and the preset image, which are defined as the first region and the second region respectively. Then split the first region into 3 sub-regions and the second region into 3 sub-regions along the connecting line of the endpoints. The three sub-regions are respectively defined as the head region, the body region, and the tail region. By splitting, the comparison between the actual image and the preset image can be made more refined. Then compare the head region, the body region, and the tail region to obtain three sub-similarities, and take the average value of the sub-similarities as the second similarity between the actual image and the preset image. Through comparison, it is found that the preset image with the image tag of the completion stage has the highest similarity with the actual image, so the state of the silkworm pupa in the actual image is marked as a rigid pupa, in the completion stage.

[0074] Calculating the sub-similarity includes the following steps:

[0075] Obtain the pixel values of each pixel point in the sub-region, construct the color distribution histogram of the sub-region based on the pixel values, and calculate the sub-similarity of the two sub-regions based on the color distribution histogram.

[0076] When calculating the sub-similarity, first construct a histogram representing the color distribution by counting the occurrence times of each color. For example, if sub-region 1 contains 50 pixels, among which 20 are red, 15 are green, 10 are blue, and 5 are other colors, then the color distribution histogram of sub-region 1 will have this information. Then use the histogram intersection method to calculate the similarity between the two histograms, that is, the sub-similarity. The specific calculation process of the histogram intersection method is prior art and will not be elaborated here.

[0077] In this embodiment, calculating the individual movement amount includes the following steps:

[0078] Use the endpoint connection line as the body contour in the first skeleton diagram and the second skeleton diagram of the pre-image and the post-image respectively. Calculate the first movement amount based on the positions of the center points in the pre-image and the post-image. Superimpose the first skeleton diagram on the second skeleton diagram with the center point as the reference point. Take the distances between the first endpoints and the second endpoints after superimposition as the second movement amount, and define the sum of the first movement amount and the second movement amount as the individual movement feature.

[0079] After determining the positions of the body contour in the pre-image and the post-image, calculate the first movement amount according to the positions of the center points in the pre-image and the post-image. The first movement amount reflects the distance that the silkworm pupa has moved as a whole. Then superimpose the first skeleton diagram and the second skeleton diagram. As Figure 5 shown, when superimposing, use the center point as the reference point. After superimposition, the sum of the distances between the first endpoints A1 and A2 and the distances between the second endpoints B1 and B2 is the second movement amount. The second movement amount reflects the changes in the head and tail of the silkworm pupa. Finally, take the sum of the first movement amount and the second movement amount as the individual movement feature.

[0080] Specifically, when superimposing, rotate the second skeleton diagram to calculate multiple second movement amounts, and output the second movement amount with the smallest value as the final result.

[0081] As Figure 6 shown, this application provides a real-time monitoring system for rigid pupae based on deep learning, which is used to implement the above-mentioned real-time monitoring method for rigid pupae based on deep learning. The system includes:

[0082] A shooting module that shoots the first image of the target area at a preset resolution and a preset time interval, and performs grid segmentation on the first image to obtain multiple second images;

[0083] An identification module that integrates the second images of the same area into an image sequence, extracts the silkworm pupa area in each second image in the image sequence based on a deep learning model, generates a corresponding silkworm pupa contour based on the silkworm pupa area, performs mutual mapping on the silkworm pupa contours in the image sequence, and numbers the mapped silkworm pupa contours;

[0084] An analysis module that integrates the silkworm pupa contours with the same number into a contour sequence, analyzes the contour sequence to obtain the individual characteristics of each silkworm pupa, and the individual characteristics include individual movement characteristics and surface texture characteristics;

[0085] A judgment module that classifies the silkworm pupae into rigid silkworm pupae and ordinary silkworm pupae based on the individual characteristics, establishes a silkworm pupa image library, the silkworm pupa image library includes a variety of preset images, compares the actual images of the rigid silkworm pupae and the ordinary silkworm pupae with the preset images, and determines the rigidification stage of the rigid silkworm pupae and the physiological state of the ordinary silkworm pupae.

[0086] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, a real-time monitoring method for zombie pupae based on deep learning as described above is implemented.

[0087] It should be understood that the various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time monitoring method for pupae based on deep learning, characterized in that: Taking a first image of a target area at a preset resolution and a preset time interval, and performing grid segmentation on the first image to obtain a plurality of second images; The second images of the same area obtained by each shooting are integrated into an image sequence in chronological order, the silkworm pupa region in each second image in the image sequence is extracted based on the deep learning model, and the corresponding silkworm pupa contour is generated based on the silkworm pupa region; Mapping the silkworm pupa contours in the image sequence to each other, and numbering the mapped silkworm pupa contours; The contours of silkworm pupae with the same number are integrated into a contour sequence, and the contour sequence is analyzed to obtain the individual characteristics of each silkworm pupa, which include individual movement characteristics and surface texture characteristics; The individual movement characteristics can be used to determine whether the silkworm is still alive, and the texture characteristics can be used to determine whether the silkworm has white bassiana or pathological patterns. The silkworm pupae are divided into rigid silkworm pupae and common silkworm pupae based on individual characteristics, and a silkworm pupae image library is established, which includes a variety of preset images; Comparing the actual images of the fossilized silkworm pupae and the common silkworm pupae with the preset images, determining the fossilization stage of the fossilized silkworm pupae and the physiological state of the common silkworm pupae; The mutual mapping of silkworm chrysalis contours includes the following steps: Positioning the second image of the adjacent time sequence in the image sequence and defining them as the preceding image and the succeeding image respectively, defining the outline of the silkworm pupa in the preceding image as the main body outline, positioning the first end point and the second end point of each main body outline in the preceding image, generating an end point connecting line for connecting the first end point and the second end point within the main body outline, generating a first circular area with the center point of each end point connecting line as the center of the circle, and the radius of the first circular area is determined based on a preset time interval; Mapping the first circular area from the preceding image to the succeeding image, defining the silkworm pupa contour located in the first circular area in the succeeding image as a candidate contour, calculating a first similarity between each candidate contour and the corresponding main contour, and determining the candidate contour whose first similarity is greater than a first threshold as a mapping of the main contour in the succeeding image; Calculating the first similarity includes the following steps: Generate a second circular area in the main body contour and the alternative contour respectively with the center point as the center of the circle, locate the parts of the main body contour and the alternative contour within the second circular area as the body contour, draw a plurality of distance measurement lines with the center point as the starting point and the boundary of the body contour and the second circular area as the end point, and draw a first curve corresponding to the change of the length of the distance measurement line of the main body contour and a second curve corresponding to the change of the length of the distance measurement line of the alternative contour in the coordinate system; Aligning the first curve with different second curves respectively, and locating a plurality of comparison points with the same horizontal coordinates in the aligned first curve and the second curve, calculating the overlap degree of the first curve and the second curve based on the comparison points, normalizing the overlap degree to obtain mapping data, and using the mapping data as the first similarity; The comparative analysis of the actual image and the preset image includes the following steps: Label the image tags for each preset image. The image tags include the starting stage, the completion stage, the feeding state, the dormant state, the death state, the abnormal state, and the tags for various diseases. Locate the silkworm pupa regions in the actual image and the preset image, which are respectively defined as the first region and the second region. Locate the connecting line of the endpoints in the first region and the second region, and split the first region and the second region into multiple sub-regions along the connecting line of the endpoints; Calculate the sub-similarity of the corresponding sub-regions in the first region and the second region, take the average value of the sub-similarities as the second similarity, and assign the image tag corresponding to the preset image with the maximum second similarity to the actual image to determine the rigidification stage or physiological state of the actual image; Calculating the sub-similarity includes the following steps: Obtain the pixel values of each pixel point in the sub-region, construct the color distribution histogram of the sub-region based on the pixel values, and calculate the sub-similarity of the two sub-regions based on the color distribution histogram.

2. The real-time monitoring method for stiff pupa based on deep learning according to claim 1, characterized in that, Calculating the individual movement feature includes the following steps: Use the connecting line of the endpoints as the first skeleton diagram and the second skeleton diagram of the body contour in the previous image and the subsequent image respectively. Calculate the first movement amount based on the positions of the center points in the previous image and the subsequent image. Superimpose the first skeleton diagram on the second skeleton diagram with the center point as the reference point, and take the distances between the first endpoints and the second endpoints after superposition as the second movement amount. Define the sum of the first movement amount and the second movement amount as the individual movement feature.

3. The real-time monitoring method of pupae based on deep learning according to claim 1 is characterized in that: The rigidification stage includes the starting stage and the completion stage. The physiological state includes the feeding state, the dormant state, the death state, and the abnormal state. Generate a first reminder when the number of rigid silkworm pupae in the completion stage reaches the second threshold, and generate a second reminder when the number of ordinary silkworm pupae in the death state or abnormal state reaches the third threshold.

4. The real-time monitoring method for stiff pupae based on deep learning according to claim 1, characterized in that The deep learning model is a U-NET model.

5. A real-time monitoring system for rigid silkworm pupae based on deep learning, which is used to implement a real-time monitoring method for rigid silkworm pupae based on deep learning as described in any one of claims 1-4, characterized in that, The shooting module takes the first image of the target area at a preset resolution and a preset time interval, and performs grid segmentation on the first image to obtain multiple second images; The recognition module integrates the second images of the same region into an image sequence, extracts the silkworm pupa regions in each second image in the image sequence based on the deep learning model, generates corresponding silkworm pupa contours based on the silkworm pupa regions, performs mutual mapping on the silkworm pupa contours in the image sequence, and numbers the mapped silkworm pupa contours; The analysis module integrates the silkworm pupa contours with the same number into a contour sequence, analyzes the contour sequence to obtain the individual characteristics of each silkworm pupa, and the individual characteristics include individual movement characteristics and surface texture characteristics; The judgment module divides the silkworm pupae into rigid silkworm pupae and ordinary silkworm pupae based on the individual characteristics, establishes a silkworm pupa image library, and the silkworm pupa image library includes a variety of preset images. Compare the actual images of the rigid silkworm pupae and ordinary silkworm pupae with the preset images to determine the rigidification stage of the rigid silkworm pupae and the physiological state of the ordinary silkworm pupae.

6. A computer-readable storage medium having instructions stored thereon, characterized in that, When the instruction is executed by the processor, it implements a real-time monitoring method for rigid silkworm pupae as described in any one of claims 1-4.

Images