Method for fully automatically monitoring physiological status of young silkworms

Through the method of fully automatic monitoring of the physiological state of small silkworms, the improved convolutional neural network model and mobile platform is used to achieve rapid and accurate monitoring of the physiological state of small silkworms, solving the problem of difficulty in identifying diseased silkworms in mulberry silkworm breeding, and improving the Breeding management efficiency and economic benefits.

CN118918607APending Publication Date: 2024-11-08SICHUAN UNIVERSITY OF SCIENCE AND ENGINEERING
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410978215.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing mulberry silkworm breeding lacks a system that fully automatically monitors the physiological status of small silkworms, which makes it difficult for silkworms to accurately judge the types and management of diseased silkworms, affecting the economic benefits of the mulberry silkworm industry.

Method used

The method of fully automatic monitoring of the physiological state of small silkworms is adopted, including the research on the physiological state mechanism of small silkworms, image collection and preprocessing, data set production, improved convolutional neural network model deployment to Android devices, and the construction of X and Y axis mobile platforms and websites. Realize automatic photography and identification of the physiological status of the silkworm, and upload the recognition results to the website.

Benefits of technology

It has achieved rapid and accurate monitoring of the physiological status of small silkworms, improved the management ability of mulberry silkworm breeders, reduced the occurrence of diseases, and protected the economic interests of the mulberry silkworm industry.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918607A_ABST
    Figure CN118918607A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of silkworm breeding, and provides a method for fully automatically monitoring the physiological status of young silkworms, which comprises the following steps: 1, studying the physiological status mechanism of young silkworms; 2, collecting young silkworm pictures in different physiological states; 3, preprocessing the picture; 4, making a data set; 5, constructing and improving a convolutional neural network model; 6, training an optimal model and packaging; 7, deploying the model to an Android device; 8, constructing an X-axis moving platform and a Y-axis moving platform; 9, establishing a website for monitoring the physiological status of young silkworms; and 10, automatically photographing and identifying the physiological status of the young silkworms through Android equipment, and uploading the physiological status to a website at the same time. According to the invention, the physiological state of young silkworms can be fully automatically monitored.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of silkworm breeding, and in particular to a method for fully automatically monitoring the physiological state of young silkworms. Background Art

[0002] Silkworm breeding has become an important economic pillar for many rural families to survive, and has played a significant role in leading the take-off of the agricultural economy. In order to inject more vitality into the silkworm industry and ensure its sustained and rapid development, we need to actively explore and implement a series of innovative strategies to optimize the existing production model, thereby significantly improving production efficiency and promoting the silkworm breeding industry to a higher stage of development.

[0003] In summer and autumn, infectious silkworm diseases caused by pathogenic microorganisms are common and frequently-occurring diseases in the silkworm breeding process, accounting for about 70% of the losses caused by silkworm diseases. If silkworm farmers cannot accurately identify the types of diseased silkworms and take corresponding measures, the economic benefits of silkworms will be affected. Therefore, a fully automatic monitoring system for the physiological status of young silkworms is needed, including monitoring the silkworm age and identifying diseased silkworms.

[0004] There is little research on the existing fully automatic monitoring system for the physiological state of young silkworms. Therefore, the research on the fully automatic monitoring system for the physiological state of young silkworms is conducive to enabling farmers to observe the physiological state of young silkworms in real time and accurately, helping silkworm breeders to better manage them, and strengthening the preventive early warning of the physiological state of young silkworms, thereby protecting the people's treasure trove of wealth. Summary of the invention

[0005] The present invention provides a method for fully automatically monitoring the physiological state of young silkworms, which can overcome certain defects of the prior art.

[0006] A method for fully automatically monitoring the physiological state of young silkworms according to the present invention comprises the following steps:

[0007] Step 1: Study on the mechanism of physiological state of silkworms;

[0008] Step 2: Collect pictures of young silkworms in different physiological states;

[0009] Step 3: Image preprocessing;

[0010] Step 4: Create a dataset;

[0011] Step 5: Build and improve the convolutional neural network model;

[0012] Step 6: Train the optimal model and package;

[0013] Step 7: Deploy the model to the Android device;

[0014] Step 8: Build the X and Y axis moving platform;

[0015] Step 9: Establish a website to monitor the physiological status of silkworms;

[0016] Step 10: Use the Android device to automatically take photos and identify the physiological status of the silkworms, and upload them to the website.

[0017] Preferably, in step 1, the research on the mechanism of the physiological state of young silkworms includes the research on the mechanism of diseased silkworms and the research on the mechanism of silkworm age. In the research on the mechanism of diseased silkworms, silkworm diseases include bacterial diseases, fungal diseases, pus diseases and myiasis; in the research on the mechanism of silkworm age, silkworm ages include one-instar silkworms, two-instar silkworms and three-instar silkworms.

[0018] Preferably, in step 2, 100,000 pictures of silkworms are taken under different light sources, at different times, in different spaces, and at different angles, including one-instar silkworms, two-instar silkworms, three-instar silkworms, and diseased silkworms, in a ratio of 1:1:1:1 to produce a data set.

[0019] Preferably, in step 3, firstly, the collected images with poor quality are deleted, and then data enhancement operation is performed; data enhancement includes geometric transformation of the image, adjustment of HSV contrast, median filtering of the image, and salt and pepper noise of the image, and finally a data set of 500,000 images is obtained.

[0020] Preferably, in step 4, the preprocessed data set is annotated using the labelimg tool, and the annotated data set is divided into a training set, a validation set, and a test set in a ratio of 92:4:4.

[0021] Preferably, in step 5, a YOLOv5 neural network is used as a basic network, and the network uses CNN to perform end-to-end detection on the target. The network structure of YOLOv5 is divided into four parts: input end, Backbone, Neck, and Head;

[0022] The SPD-Conv module is used for improvement to improve the performance when processing low-resolution images and small objects. The module first converts the original feature map into an intermediate feature map with feature identification information through the SPD module, and then converts the intermediate feature map into the final output feature map through the non-strided convolution Conv module.

[0023] The CA attention mechanism is selected to embed the position information into the channel attention, so that the lightweight network can pay attention to a larger area. The specific process is as follows:

[0024] 1) The input feature map size is C×H×W, where C is the number of channels, H and W are the height and width respectively;

[0025] 2) Perform global average pooling along the width W on each channel to obtain a C×H×1 feature map that captures information in the height direction;

[0026] 3) Perform global average pooling along the height H on each channel to obtain a C×1×W feature map that captures information in the width direction;

[0027] 4) The two feature maps are then encoded into two attention maps, each of which can capture the long-range dependency of the input feature map along a spatial direction; finally, the two attention maps are weighted fused with the original input feature map to obtain an enhanced feature map with position information;

[0028] The CA attention mechanism implements global average pooling operations in the two dimensions X and Y, respectively, thereby generating two sets of independent convolutional feature maps, as described in Equations (1) and (2). Taking the feature map X of size W×H×C as an example, the CA attention mechanism uses the pooling kernels of (H, 1) and (1, H) to encode each channel along the horizontal and vertical coordinate directions of the feature map X, and outputs the c-th channel information with a height of h after encoding. and the cth channel information with width w

[0029]

[0030] Where W and H are the width and height of the input feature map, x c (h,i) is the value of the input feature map at the position (h,i). These channel information contain rich global receptive fields and precise location information. and Perform feature fusion to obtain two direction-aware feature maps Z h and Z w .

[0031] In the attention generation module, the CA attention mechanism embeds the coordinate information into the attention feature Z obtained by the module. h and Z w Through concatenation and convolution, the feature map f is obtained after batch normalization and nonlinear activation. Then f is cut in the spatial dimension to obtain two separate tensors f h 、f w .f h 、f w After two 1×1 convolutions F h and F w Get a feature map with the same number of channels as the input feature map X, and then use the σ activation function to act on the feature map to get g h , g w , the calculation formula is:

[0032] g h =σ(F h (f h )) (8)

[0033] g w =σ(F w (f w )) (9)

[0034] Then for g h and g w Expand and get the attention weight of the input feature map X. Finally, the CA attention mechanism module obtains the output feature map Y containing internal weights and feature information. c (i, j) is the value of the output feature map at position (i, j), and the calculation formula is:

[0035]

[0036] Preferably, in step 6, after improving the YOLOv5 model, use PyTorch-GPU1.12+cu113 to import the processed data set into the deep learning framework for training, set certain hyperparameters, learning rate, cosine annealing, learning rate momentum and weight decay coefficient, and obtain the optimal model after setting and training. After adjusting the parameters, select the optimal model and save it as a .pt file.

[0037] Preferably, in step 7, first convert the .pt file format to the onnx file format, then use the model conversion tool to input the onnx file and output the NCNN file; use Android studio to load the project of the ncnn framework and modify the parameter file, load the weights and parameters of the model, read the processed network input, create a network extractor and obtain the network output; then add the permission to call the camera and save files, add the code for the camera to automatically take pictures and identify, output the number of target categories and save them to local files; finally, package it into apk, and test the stability and responsiveness of its APP on multiple mobile phones; after the test is completed, an APP that can automatically take pictures, identify the physiological state of silkworms and save them locally is obtained.

[0038] Preferably, in step 8, the silkworm tray is divided into 9 equal parts, and the X-axis and Y-axis moving platforms cooperate with the APP to automatically take 9 pictures continuously and save them locally after recognition.

[0039] As a preferred method, in step 9, the front-end website is established using the VUE framework, and VS code is used as the IDE; the back-end uses Spring Boot to achieve front-end and back-end separation;

[0040] The specific steps are:

[0041] (1) Create a RESTful API in Spring Boot and use the @RestController annotation to mark the API;

[0042] (2) Define the processing logic of HTTP requests in the API, obtain data from the database and return it to the front end;

[0043] (3) Use the axios library in Vue to send HTTP requests and obtain backend data;

[0044] (4) Render the returned data on the page in the Vue component.

[0045] The beneficial effects of the present invention are as follows:

[0046] According to the research on the mechanism of the physiological state of young silkworms, the present invention is divided into the research on the mechanism of diseased silkworms and the research on the mechanism of silkworm age. The research on the physiological state of young silkworms in the present invention is the most comprehensive, covering the physiological states of most young silkworms. The convolutional neural network model is constructed and improved by shooting and preprocessing the pictures of young silkworms. Among them, the method of improving the convolutional neural network model adopts the SPD-Conv module for improvement to improve the performance when processing low-resolution images and small objects; the CA attention mechanism is added, and the position information is embedded into the channel attention, so that the lightweight network can pay attention to a larger area.

[0047] 2. The present invention realizes full-automatic monitoring of the physiological state of silkworms. It is fully automatic in that, except for pressing the power button, no other operations are required, and the mobile phone will automatically cooperate with the X and Y axis mobile platform to shoot 9 pictures, and the mobile phone's built-in processor can automatically identify these 9 pictures, and the data after its identification is automatically uploaded to the server, so that the user can view its physiological state of silkworms in real time. Its steps are summarized as follows: after the present invention constructs and improves the physiological state model of silkworms, because the background in the silkworm seat is complex and the number of silkworms is large, the model is deployed on the Android device based on the NCNN framework (i.e., an APP is made, and a local processor is used for identification) and data can be saved locally. Then the X and Y axis mobile platform is designed, so that after it cooperates with the APP to shoot 9 pictures, the pictures and data are saved locally and uploaded to the server, and finally a website is designed based on the VUE framework, so that users can visualize the operation and monitor the physiological state of silkworms in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flow chart of a method for fully automatically monitoring the physiological state of young silkworms in an embodiment;

[0049] Figure 2 It is a structural diagram of the CA module in the embodiment;

[0050] Figure 3Schematic diagram of the improved YOLOv5 network model in the embodiment;

[0051] Figure 4 It is the operation structure diagram of NCNN in the embodiment;

[0052] Figure 5 It is a three-dimensional modeling diagram of the X-axis and Y-axis moving platforms in the embodiment;

[0053] Figure 6 Schematic diagram of the moving direction of the X-axis slider in the embodiment;

[0054] Figure 7 This is a flow chart of uploading Android app data to a server in an embodiment. DETAILED DESCRIPTION

[0055] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0056] Example

[0057] like Figure 1 As shown, this embodiment provides a method for fully automatically monitoring the physiological state of silkworms, which includes the following steps:

[0058] Step 1: Study on the mechanism of physiological state of silkworms;

[0059] The research on the physiological state mechanism of young silkworms includes the research on the mechanism of diseased silkworms and the research on the mechanism of silkworm age. In the research on the mechanism of diseased silkworms, silkworm diseases include bacterial diseases, fungal diseases, pus diseases and myiasis. In order to accurately detect these silkworm diseases, we need to classify the diseased silkworms according to the cause. However, since the final confirmation of silkworm diseases requires the dissection of silkworm bodies, it may not be possible to accurately determine the specific type of silkworm diseases based on image judgment alone. Therefore, when making a data set, it is necessary to distinguish them under the guidance of silkworm experts. The standard judgment table for silkworm diseases is shown in Table 1. In the research on the mechanism of silkworm age, their appearance and body shape are quite different, so convolutional neural networks can be used to distinguish them. The physiological characteristics of young silkworms at different ages are shown in Table 2. Silkworm ages include one-instar silkworms, two-instar silkworms and three-instar silkworms.

[0060] Table 1 Silkworm disease standard judgment table

[0061]

[0062] Table 2 Physiological characteristics of silkworms at different ages

[0063] Age Appearance Dormancy First instar silkworm Black turns to dark brown 8-12 hours Second instar silkworm Increase in size and lighten in color 8-12 hours Third instar silkworm Increases in size and turns light brown in color 10-14 hours

[0064] Step 2: Collect pictures of young silkworms in different physiological states;

[0065] About 100,000 images of young silkworms were taken under different light sources, at different times, in different spaces, and at different angles. The various shooting methods are designed to make the images obtained by using the camera to shoot a variety of different backgrounds more diverse, thereby enhancing the robustness of the convolutional network. The dataset includes first-instar silkworms, second-instar silkworms, third-instar silkworms, and diseased silkworms in a 1:1:1:1 ratio.

[0066] Step 3: Image preprocessing;

[0067] First, delete the images with poor quality, such as those with severe blur or severe exposure, and then perform data enhancement. The use of data enhancement technology can greatly reduce the impact of insufficient data, improve the robustness of the model, increase the model's ability to resist overfitting, and accelerate the training process. These include four data enhancement methods: geometric transformation of images, adjustment of HSV contrast, median filtering of images, and salt and pepper noise of images. Therefore, a data set of about 500,000 images was finally obtained.

[0068] Step 4: Create a dataset;

[0069] The preprocessed data set is annotated using the labelimg tool, and the annotated data set is divided into training set, validation set, and test set in a ratio of 92:4:4, making the trained model more realistic and reliable.

[0070] Step 5: Build and improve the convolutional neural network model;

[0071] The YOLOv5 neural network is used as the basic network. The network uses CNN to perform end-to-end detection of the target. The network structure of YOLOv5 is divided into four parts: input, backbone, neck, and head.

[0072] Since the detection object is a small silkworm, the image background is complex and the target is small, the SPD-Conv module is used for improvement to improve the performance when processing low-resolution images and small objects; the SPD-Conv module can completely replace the convolution stride and pooling layer. SPD-Conv is a space-to-depth layer followed by a non-stride convolution layer. The SPD layer downsamples the feature map X, but retains all information in the channel dimension, so there is no information loss. The module first converts the original feature map into an intermediate feature map with feature identification information through the SPD module, and then converts the intermediate feature map into the final output feature map through the non-stride convolution Conv module.

[0073] Considering that the deployed device is an Android device, the model is more lightweight, so the CA (CoordinateAttention) attention mechanism is selected and placed in the P8 layer of the original model backbone. By embedding the position information into the channel attention, the lightweight network can pay attention to a larger area. The specific process is:

[0074] 1) The input feature map size is C×H×W, where C is the number of channels, H and W are the height and width respectively;

[0075] 2) Perform global average pooling along the width W on each channel to obtain a C×H×1 feature map that captures information in the height direction;

[0076] 3) Perform global average pooling along the height H on each channel to obtain a C×1×W feature map that captures information in the width direction;

[0077] 4) The two feature maps are then encoded into two attention maps, each of which can capture the long-range dependency of the input feature map along a spatial direction; finally, the two attention maps are weighted fused with the original input feature map to obtain an enhanced feature map with position information;

[0078] The CA module encodes channel relationships and long-range dependencies through precise location information. It is divided into two steps: coordinate information embedding and coordinate attention generation. Its specific structure is as follows: Figure 2 As shown. In order to maintain the integrity of spatial information, the CA attention mechanism implements global average pooling operations in the X and Y dimensions respectively, thereby generating two sets of independent convolutional feature maps, as described in Equations (1) and (2); Subsequently, the two sets of feature maps are merged after dimension conversion, and the dependency relationship between them is constructed through the convolution layer; Next, in order to obtain accurate weights in two dimensions and retain detailed location information, the merged feature information undergoes normalization and nonlinear activation processing; Finally, by applying these weights to the original feature map, the CA attention mechanism can output feature maps with attention weights in both the X and Y directions, as shown in Equation (3);

[0079]

[0080] Among them, x c is the output feature map; and are the position information of the input feature image along the horizontal direction X and the vertical direction Y respectively; c (h,i) represents the eigenvalue of the i-th position at channel c and height h; x c (j,w) represents the feature value of the jth position on channel c and width w; and is the weight in the X and Y directions; c is the output feature map.

[0081] The improved YOLOv5 model structure is as follows Figure 3 shown.

[0082] Step 6: Train the optimal model and package;

[0083] After improving the YOLOv5 model, use PyTorch-GPU1.12+cu113 to import the processed data set into the deep learning framework for training. Set certain hyperparameters, including learning rate, cosine annealing, learning rate momentum, and weight decay coefficient. After setting and training, obtain the optimal model. After adjusting the parameters, select the optimal model and save it as a .pt file.

[0084] Step 7: Deploy the model to the Android device;

[0085] Use Tencent's open source ncnn framework, which is a high-performance neural network forward computing framework that is extremely optimized for mobile phones. It can port deep learning algorithms to mobile phones for efficient execution. First, convert the .pt file format to the onnx file format, and then use the model conversion tool to input the onnx file and output the NCNN file (.param and .bin). Use Androidstudio to load the ncnn framework project and modify the parameter file, load the model's weights and parameters, read the processed network input, create a network extractor and get the network output. Among them, the operating structure diagram of NCNN is as follows Figure 4 shown.

[0086] To fully automatically monitor the physiological status of silkworms, no other operations must be performed except pressing the start button. Therefore, the permissions for accessing the camera and saving files are added, and the code for outputting the number of target categories and saving them to local files after the camera automatically takes pictures and identifies them is added. Finally, it is packaged into an apk, and the stability and responsiveness of the APP are tested on multiple mobile phones. After the test, an APP that can automatically take pictures, identify the physiological status of silkworms, and save them locally is obtained. The APP can change the interval between pictures so that the X-axis and Y-axis mobile platforms can work together.

[0087] Step 8: Build the X and Y axis moving platform;

[0088] After getting this APP through step 7, if the entire silkworm tray is directly identified, the large number of small silkworms will lead to low accuracy and slow processing response. Therefore, it is chosen to build an X- and Y-axis mobile platform. The various motion modes of the entire mechanism are realized through the transmission of a belt. This transmission method can effectively reduce vibration and avoid the burden of installing a lead screw and moving a stepper motor, making the overall structure simpler and the movement more flexible, which can improve the working efficiency of the mechanism to a certain extent. The purpose of dividing the silkworm tray into 9 equal parts is to make the X- and Y-axis mobile platforms cooperate with the APP, and after taking 9 pictures continuously and automatically and identifying them, the number of target categories is output and saved locally. This not only improves the accuracy of image recognition, but also helps to improve the speed of mobile phone recognition of pictures.

[0089] in, Figure 5 This is the 3D modeling diagram of the X-axis and Y-axis mobile platforms. 1: X-axis idler connection seat; 2: LCF6-2040-1000 aluminum profile; 3: L-shaped iron angle; 4: idler, idler seat; 5: LCFB6-2020-350 aluminum profile; 6: synchronous belt 1; 7: synchronous belt 2; 8: LCF6-2020-1000 aluminum profile; 9: European standard T-nut; 10: 42 stepper motor M1; 11: X-axis slider; 12: expandable corexy platform; 13: optical axis; 14: mobile phone holder; 15: X-axis fixed seat.

[0090] Among them, LCF6-2020-1000 aluminum profile is assembled with LCFB6-2020-350 aluminum profile and LCF6-2040-1000 aluminum profile to support the X and Y axis platforms; European standard T-nuts and L-shaped iron angles are assembled between the idler, idler seat and LCF6-2040-1000 aluminum profile; 42 stepper motor M1 drives synchronous belt 1 and synchronous belt 2 to slide on the idler and idler seat; the optical axis is assembled with the X-axis idler connecting seat and the X-axis fixing seat, and the mobile phone holder and the expandable corexy platform are installed on the X-axis slider. When the synchronous belt 2 rotates, its X-axis slider will move the X-axis according to the direction of rotation; when the synchronous belt 1 rotates, the X-axis idler connecting seat will move the Y-axis according to the direction of rotation.

[0091] The 45-degree movement of the XY axis is not considered in this slider movement. Synchronous belt 1 and synchronous belt 2 rotate around the four idler wheels and idler wheel seats respectively, and the two 42 stepper motors M1 rotate forward and reverse at the same time, so that synchronous belt 1 and synchronous belt 2 rotate in the same direction, completing the X-axis movement of the X-axis slider. The two 42 stepper motors M1 rotate forward and reverse at the same time, so that synchronous belt 1 and synchronous belt 2 rotate in opposite directions, completing the Y-axis movement of the optical axis. Finally, put the mobile phone on the mobile phone holder and use the programmed motor to move in the X and Y axis directions. The movement direction of the X-axis slider is Figure 6 shown.

[0092] The working principle is: the relationship between the mechanism motion displacement ΔX, ΔY and the linear displacement ΔA, ΔB of the stepper motors M1 and M2 is shown in equations (4) and (5):

[0093]

[0094]

[0095] When the stepper motors M1 and M2 rotate clockwise or counterclockwise at the same speed, the slider of the mechanism will move horizontally along the X-axis, and the magnitude and direction of ΔA and ΔB are the same; at this time:

[0096]

[0097] When the stepper motors M1 and M2 rotate at the same speed and in opposite directions, the slider of the mechanism will move along the Y-axis. At this time, ΔA and ΔB are the same in magnitude and opposite in direction. At this time, we have:

[0098]

[0099] When the stepper motor M1 rotates and M2 is stationary, the slider of the mechanism will move along the Y=X direction. At this time, we have:

[0100]

[0101] When the stepper motor M2 rotates and M1 is stationary, the slider of the mechanism will move along the Y=-X direction. At this time, we have:

[0102]

[0103] Step 9: Establish a website to monitor the physiological status of silkworms;

[0104] Use VUE framework to build the front-end website, and use VS code as IDE. Spring Boot is used for the back-end to separate the front-end and back-end. The advantages of front-end and back-end separation development include: 1. Improve development efficiency, 2. Improve maintainability, 3. Improve scalability, 4. Improve user experience.

[0105] The specific steps are:

[0106] (1) Create a RESTful API in Spring Boot and use the @RestController annotation to mark the API;

[0107] (2) Define the processing logic of HTTP requests in the API, obtain data from the database and return it to the front end;

[0108] (3) Use the axios library in Vue to send HTTP requests and obtain backend data;

[0109] (4) Render the returned data on the page in the Vue component.

[0110] Step 10: Use the Android device to automatically take photos and identify the physiological status of the silkworms, and upload them to the website.

[0111] Upload the pictures and data of the silkworms before and after identification obtained in step 7 to this website. The website can not only update in real time, but also save the previous data for viewing. The overall process steps of uploading Android APP data to the server are as follows: Figure 7 shown.

[0112] The present invention can quickly and accurately identify the physiological state of young silkworms, and deploy the model to an Android device. The Android device is controlled by an X-axis and Y-axis mobile platform to take photos continuously, and the data is saved locally and uploaded to a website for real-time viewing by silkworm breeders. If diseased young silkworms appear, the breeders can take corresponding measures. The model of the present invention has high detection accuracy and detection speed, and can fully automatically display the recognition results on the website system.

[0113] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A method for fully automatic monitoring of the physiological state of young silkworms, characterized in that: The following steps are involved: Step 1: Study on the mechanism of physiological state of silkworms; Step 2: Collect pictures of young silkworms in different physiological states; Step 3: Image preprocessing; Step 4: Create a dataset; Step 5: Build and improve the convolutional neural network model; Step 6: Train the optimal model and package; Step 7: Deploy the model to the Android device; Step 8: Build the X and Y axis moving platform; Step 9: Establish a website to monitor the physiological status of silkworms; Step 10: Use the Android device to automatically take photos and identify the physiological status of the silkworms, and upload them to the website.

2. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 1, characterized in that: In step 1, the research on the mechanism of the physiological state of young silkworms includes the research on the mechanism of diseased silkworms and the research on the mechanism of silkworm age. In the research on the mechanism of diseased silkworms, silkworm diseases include bacterial diseases, fungal diseases, pus diseases and myiasis; in the research on the mechanism of silkworm age, silkworm ages include one-instar silkworms, two-instar silkworms and three-instar silkworms.

3. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 2, characterized in that: In step 2, 100,000 images of silkworms were taken under different light sources, at different times, in different spaces, and at different angles, including first-instar silkworms, second-instar silkworms, third-instar silkworms, and diseased silkworms, in a ratio of 1:1:1:1 to create a data set.

4. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 3, characterized in that: In step 3, the collected images with poor quality are first deleted, and then data enhancement is performed; data enhancement includes geometric transformation of the image, adjustment of HSV contrast, median filtering of the image, and salt and pepper noise of the image, and finally a data set of 500,000 images is obtained.

5. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 4, characterized in that: In step 4, the preprocessed data set is annotated using the labelimg tool, and the annotated data set is divided into a training set, a validation set, and a test set in a ratio of 92:4:

4.

6. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 5, characterized in that: In step 5, the YOLOv5 neural network is used as the basic network. The network uses CNN to perform end-to-end detection of the target. The network structure of YOLOv5 is divided into four parts: input end, Backbone, Neck, and Head. The SPD-Conv module is used for improvement to improve the performance when processing low-resolution images and small objects. The module first converts the original feature map into an intermediate feature map with feature identification information through the SPD module, and then converts the intermediate feature map into the final output feature map through the non-strided convolution Conv module. The CA attention mechanism is selected to embed the position information into the channel attention, so that the lightweight network can pay attention to a larger area. The specific process is as follows: 1) The input feature map size is C×H×W, where C is the number of channels, H and W are the height and width respectively; 2) Perform global average pooling along the width W on each channel to obtain a C×H×1 feature map that captures information in the height direction; 3) Perform global average pooling along the height H on each channel to obtain a C×1×W feature map that captures information in the width direction; 4) These two feature maps are then encoded into two attention maps, each of which can capture the long-range dependencies of the input feature maps along a spatial direction; Finally, these two attention maps are weighted fused with the original input feature map to obtain an enhanced feature map with position information; The CA attention mechanism implements global average pooling operations in the two dimensions of X and Y, respectively, thereby generating two sets of independent convolutional feature maps, as described in Equations (1) and (2). Taking the feature map X of size W×H×C as an example, the CA attention mechanism uses the pooling kernels of (H, 1) and (1, H) to encode each channel along the horizontal coordinate direction and vertical coordinate direction of the feature map X, respectively, and outputs the cth channel information with a height of h after encoding. and the cth channel information with width w Where W and H are the width and height of the input feature map, x c (h,i) is the value of the input feature map at the position (h,i). These channel information contain rich global receptive fields and precise location information. and Perform feature fusion to obtain two direction-aware feature maps Z h and Z w . In the attention generation module, the CA attention mechanism embeds the coordinate information into the attention feature Z obtained by the module. h and Z w Through concatenation and convolution, the feature map f is obtained after batch normalization and nonlinear activation. Then f is cut in the spatial dimension to obtain two separate tensors f h 、f w .f h 、f w After two 1×1 convolutions F h and F w Get a feature map with the same number of channels as the input feature map X, and then use the σ activation function to act on the feature map to get g h , g w , the calculation formula is: g h =σ(F h (f h )) (3) g w =σ(F w (f w )) (4) Then for g h and g w Expand and get the attention weight of the input feature map X. Finally, the CA attention mechanism module obtains the output feature map Y containing internal weights and feature information. c (i, j) is the value of the output feature map at position (i, j), and the calculation formula is:

7. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 6, characterized in that: In step 6, after improving the YOLOv5 model, use PyTorch-GPU1.12+cu113 to import the processed data set into the deep learning framework for training, set certain hyperparameters, learning rate, cosine annealing, learning rate momentum and weight decay coefficient, and obtain the optimal model after setting and training. After adjusting the parameters, select the optimal model and save it as a .pt file.

8. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 7, characterized in that: In step 7, first convert the .pt file format to the onnx file format, then use the model conversion tool to input the onnx file and output the NCNN file; Use Android studio to load the ncnn framework project and modify the parameter file, load the model's weights and parameters, read the processed network input, create a network extractor and get the network output; then add permissions to access the camera and save files, add the camera to automatically take pictures and identify, output the number of target categories and save them to local files; finally, package it into apk, and test the stability and responsiveness of the APP on multiple mobile phones; after the test, you will get an APP that can automatically take pictures, identify the physiological state of silkworms and save them locally.

9. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 8, characterized in that: In step 8, the silkworm tray is divided into 9 equal parts, and the X-axis and Y-axis moving platforms cooperate with the APP to automatically take 9 pictures continuously and save them locally after recognition.

10. The method for fully automatic monitoring of the physiological state of young silkworms according to claim 9, characterized in that: In step 9, the VUE framework is used to build the front-end website, and VS code is used as the IDE; Spring Boot is used for the back-end to achieve front-end and back-end separation; The specific steps are: (1) Create a RESTful API in Spring Boot and use the @RestController annotation to mark the API; (2) Define the processing logic of HTTP requests in the API, obtain data from the database and return it to the front end; (3) Use the axios library in Vue to send HTTP requests and obtain backend data; (4) Render the returned data on the page in the Vue component.

Citation Information

Patent Citations

  • Gesture recognition method and device, computer equipment and storage medium

    CN117275086A

  • Rice planthopper target detection method, system and equipment based on improved Yolov5 and medium

    CN117351471A

  • Knowledge management system and method for optimizing information technology service process

    CN117745224A

  • Method and system for detecting seeding effect of seedling raising tray of air vibration type precision seeder

    CN117975345A

  • Bombyx mori breeding sign recognition method based on Internet of Things

    CN118230143A