Insect hierarchical classification method and system based on multi-task convolutional neural network

Abstract

The application provides an insect hierarchical classification method and system based on a multi-task convolutional neural network, relates to the technical field of insect identification, and collects insect images, performs greying and HSV color space conversion, and generates two groups of identification images. Edge features are extracted by using canny edge detection, edge complexity and circularity are calculated to generate edge feature indexes, hue, saturation and brightness information is extracted from the HSV image to generate image vividness. The edge feature indexes and image vividness are spliced to form a comprehensive feature vector, and a feature library containing different insect species and their life stages is constructed. Finally, an insect hierarchical classification model is constructed based on a multi-task convolutional neural network, and the comprehensive feature vector of the to-be-identified insect image is input into the trained model to determine the insect species and life stage.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of insect recognition, in particular to an insect hierarchical classification method and system based on a multi-task convolutional neural network. BACKGROUND

[0002] Insect hierarchical classification is of great significance in the fields of biodiversity research, agricultural pest control, and ecological environment monitoring. Traditional insect hierarchical classification methods usually rely on manual observation and identification, which is not only time-consuming and labor-intensive, but also susceptible to human factors, resulting in low accuracy and consistency of the results. In recent years, with the development of image processing technology and machine learning technology, image-based automatic classification methods have gradually attracted attention. However, existing insect classification methods are often limited to single features, such as color features, texture features, or morphological features, which cannot fully express the complex information of insect images. In addition, many classification methods ignore the feature changes of insects at different life stages, resulting in insufficient accuracy and refinement of hierarchical classification. Therefore, there is an urgent need to develop an efficient, accurate, and applicable insect classification and hierarchical classification method for different life stages.

[0003] In the prior art, the publication number CN116343182A discloses an insect species identification method and system and an electronic device. An infrared thermal imager is used to collect the current frame thermal imaging image of the target area at a first collection period. The current frame infrared thermal imaging image is processed and identified. A target search box is established and processed, and a dot matrix is marked. The total gray value of each marker point in the marker dot matrix is calculated. The actual difference between the total gray value of each marker point in the marker dot matrix and the total gray value of each sample point in the sample dot matrix of various insect sample points in the sample database is calculated. According to the actual difference, a signal related to the insect species judgment result is output.

[0004] The main problem of the above-mentioned scheme is that the feature extraction only relies on the infrared thermal imaging method to extract the temperature distribution feature of the insect, the dimension of the feature extraction is relatively single, which leads to insufficient recognition accuracy. Moreover, the infrared thermal imager is easily affected by environmental temperature, humidity and other factors, and has poor applicability.

[0005] The above information disclosed in the background section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The present application aims to provide an insect hierarchical classification method and system based on a multi-task convolutional neural network to solve the problems raised in the background.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0008] An insect hierarchical classification method based on a multi-task convolutional neural network, the specific steps comprising:

[0009] Step 1: Collect the insect image to be classified, scale the insect image to 224x224 size and copy it into two identical groups, one group is grayed to generate a first recognition image, and the other group is converted from RGB color space to HSV color space to generate a second recognition image;

[0010] Step 2: Extract the edge pixel points of the first recognition image based on the Canny edge detection to generate a third recognition image, record the number of edge pixel points, the area and perimeter of the region surrounded by the edge pixel points in the third recognition image, generate the edge complexity based on the number of edge pixel points and the area of the region surrounded by the edge pixel points, generate the edge circularity based on the perimeter and area of the region surrounded by the edge pixel points, and generate the edge feature index based on the edge complexity and the edge circularity;

[0011] Step 3: Extract the hue, saturation and brightness of the pixel points in the second recognition image, generate the hue index, saturation index and brightness index of the second recognition image based on the average values of the hue, saturation and brightness of all pixel points, and generate the image vividness based on the hue index, saturation index and brightness index;

[0012] Step 4: Concatenate the edge feature index and the image vividness to generate a comprehensive feature vector, and calculate the comprehensive feature vectors of different species of insects at different life stages according to the methods of steps 1-3, and construct an insect feature library containing comprehensive feature vectors of different species of insects at different life stages;

[0013] Step 5: Construct a model based on a multi-task convolutional neural network, use the comprehensive feature vectors in the insect feature library as input, and use the species and life stage of the insect as label, train the insect hierarchical classification model;

[0014] Step 6: Input the comprehensive feature vector of the insect image to be classified into the trained insect hierarchical classification model, and output the species and life stage of the insect.

[0015] Further, the principle for generating the third recognition image is:

[0016] For each pixel point in the first recognition image, respectively convolve the pixel point and its eight surrounding neighborhood pixel points with the horizontal direction template and the vertical direction template of the Prewitt operator to generate the gray difference of the pixel point in the horizontal direction and the vertical direction, and the formula is:

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] wherein, represents a horizontal direction template of the Prewitt operator, represents a vertical direction template of the Prewitt operator, represents a horizontal direction difference of a pixel point, represents a vertical direction difference of a pixel point, represents a coordinate of a pixel point;

[0022] According to the horizontal direction and the vertical direction gray scale differences, the gradient amplitude of each pixel point is generated, and the formula is:

[0023] ;

[0024] wherein, represents the gradient amplitude of the pixel point with the coordinate , represents a horizontal direction difference of a pixel point, represents a vertical direction difference of a pixel point;

[0025] A preset edge threshold is set, and when the gradient amplitude of the pixel point is higher than the edge pixel point, the pixel point is marked as an edge pixel point.

[0026] Further, the principle for generating the edge feature index is:

[0027] The formula for generating the edge complexity is:

[0028] ;

[0029] wherein, represents the edge complexity, represents the number of edge pixel points, represents the area of the region surrounded by the edge pixel points;

[0030] The formula for generating the edge circularity is:

[0031] ;

[0032] wherein, represents the edge circularity, represents the edge perimeter;

[0033] The formula for generating the edge feature index is:

[0034] ;

[0035] wherein, represents the edge feature index.

[0036] Further, the principle according to which the image vividness is generated is:

[0037] The formula according to which the hue index, the saturation index and the brightness index are generated is:

[0038] ;

[0039] ;

[0040] ;

[0041] wherein, represents the hue index, represents the index of the pixel point, and , represents the number of pixel points, represents the hue of the i-th pixel point, represents the saturation index, represents the saturation of the i-th pixel point, represents the brightness index of the pixel point, represents the brightness of the i-th pixel point; The formula according to which the image vividness is generated is:

[0042]

[0043] ;

[0044] ;

[0045] wherein, represents the image vividness, represents the correction function of the hue to the vividness;

[0046] Further, the formula according to which the comprehensive feature vector is generated is:

[0047] ;

[0048] wherein, represents the comprehensive feature vector, represents the edge feature index, represents the image vividness.

[0049] ​​​The application further provides an insect hierarchical classification system based on a multi-task convolutional neural network, which is used to implement the insect hierarchical classification method based on the multi-task convolutional neural network.

[0050] An image acquisition module is configured to acquire an insect image to be classified, copy the insect image into two groups after scaling the insect image to 224x224, generate a first recognition image after performing grayscale processing on one group, and generate a second recognition image by converting the image from an RGB color space to an HSV color space.

[0051] An edge extraction module is configured to extract edge pixel points of the first recognition image based on a canny edge detection to generate a third recognition image, record the number of edge pixel points, the area and the perimeter of a region surrounded by the edge pixel points in the third recognition image, generate an edge complexity based on the number of edge pixel points and the area of the region surrounded by the edge pixel points, generate an edge circularity based on the perimeter and the area of the region surrounded by the edge pixel points, and generate an edge feature index based on the edge complexity and the edge circularity.

[0052] A color extraction module is configured to extract hue, saturation and brightness of pixel points in the second recognition image, generate a hue index, a saturation index and a brightness index of the second recognition image based on average values of the hue, the saturation and the brightness of all pixel points, and generate an image vividness based on the hue index, the saturation index and the brightness index.

[0053] A feature library construction module is configured to splice the edge feature index and the image vividness to generate a comprehensive feature vector, and calculate comprehensive feature vectors corresponding to different species of insects in different life stages according to the method of the above module to construct an insect feature library containing comprehensive feature vectors of different species of insects in different life stages.

[0054] A model training module is configured to construct a model based on a multi-task convolutional neural network, take the comprehensive feature vectors in the insect feature library as input, take the species and the life stage of the insect as a label, and train an insect hierarchical classification model.

[0055] A judgment output module is configured to input the comprehensive feature vector of the insect image to be classified into the trained insect hierarchical classification model, and output the species and the life stage of the insect.

[0056] Compared with the prior art, the application has the following beneficial effects:

[0057] The application improves the recognition accuracy of the classification model for different kinds of insects and different life stages of the insects by extracting the edge features and color features of the images respectively, and constructing a more comprehensive feature vector, and provides more abundant insect shape features; the color features of the images are comprehensively analyzed by extracting hue, saturation and brightness, the hue index, saturation index and brightness index are calculated by the average value of the pixel points, the problem of uneven color distribution of the images caused by light, shooting angle or other interference factors is effectively alleviated, and the stability of feature extraction and the applicability of the scheme are enhanced. Quantifying the brightness of the insect body surface color by image brightness helps to identify insects with significant color differences.

[0058] The application also generates a comprehensive feature vector by splicing the edge feature index and image brightness, fully utilizes the complementarity between different features, covers different kinds and life stages of insects, fully reflects the diversity of insects, and helps to improve the generalization ability of the training model; the insect hierarchical classification model is trained based on the multi-task convolutional neural network, the model can not only quickly adapt to new data, but also can be classified in detail for specific insect species and life stages, and ensure the accuracy and efficiency of the whole scheme. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 The figure is a method flowchart of an embodiment of the application;

[0060] Figure 2 The figure is a gray image of insects in the same growth stage in an embodiment of the application;

[0061] Figure 3 The figure is an HSV image of insects in the same growth stage in an embodiment of the application;

[0062] Figure 4 The figure is a gray image of insects in different growth stages in an embodiment of the application;

[0063] Figure 5 The figure is an HSV image of insects in different growth stages in an embodiment of the application;

[0064] Figure 6 The figure is a system module schematic diagram of an embodiment of the application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages of the application clearer, the application is further described in detail below with specific embodiments.

[0066] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the meanings that can be commonly understood by a person with ordinary skill in the art to which this application belongs. The terms "first", "second", and similar terms used in the present application do not denote any order, quantity, or importance, but are used to distinguish different components. The terms "include" or "contain" and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are used only to represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships can also change accordingly.

[0067] Embodiment:

[0068] Please refer to Figures 1 to 5 , the present application provides a technical solution:

[0069] A method for classifying insects based on a multi-task convolutional neural network, the specific steps comprising:

[0070] Step 1: Collect the insect image to be classified, and copy the insect image into two groups of the same size after scaling to 224x224. One group is processed to generate a first identification image, and the other group is converted from an RGB color space to an HSV color space to generate a second identification image.

[0071] In this embodiment, the formula for generating the first identification image by grayscale processing is:

[0072] ;

[0073] Among them, represents the gray value of the pixel point, represents the red channel value of the pixel point, represents the green channel value of the pixel point, represents the blue channel value of the pixel point;

[0074] The principle for generating the second identification image is:

[0075] For any pixel point in the image, its RGB color space is , and its HSV color space is First, convert the values of R, G, and B to the range of , where R, G, and B correspond to :

[0076] ;

[0077] ;

[0078] ;

[0079] Based on H, S, V are generated, and the formula is:

[0080] ;

[0081] ;

[0082] ;

[0083] H, S, V represent hue, saturation and lightness respectively.

[0084] Step 2: Based on the canny edge detection, the edge pixel points of the first recognition image are extracted, the third recognition image is generated, the number of edge pixel points in the third recognition image, the area and perimeter of the region surrounded by the edge pixel points are recorded, the edge complexity is generated based on the number of edge pixel points and the area of the region surrounded by the edge pixel points, the edge circularity is generated based on the perimeter and area of the region surrounded by the edge pixel points, and the edge feature index is generated based on the edge complexity and the edge circularity;

[0085] In this embodiment, the principle for generating the third recognition image is:

[0086] For each pixel point in the first recognition image, the pixel point and its eight surrounding neighborhood pixel points are respectively convolved with the horizontal direction template and the vertical direction template of the Prewitt operator to generate the gray difference of the pixel point in the horizontal direction and the vertical direction, and the formula is:

[0087] ;

[0088] ;

[0089] ;

[0090] ;

[0091] H, S, V represent hue, saturation and lightness respectively. represents the horizontal direction template of the Prewitt operator, represents the vertical direction template of the Prewitt operator, represents the horizontal direction difference of the pixel point, represents the vertical direction difference of the pixel point, represents the coordinates of the pixel point;

[0092] According to the gray difference in the horizontal direction and the vertical direction, the gradient amplitude of each pixel point is generated, and the formula is:

[0093] ;

[0094] wherein, represents the gradient amplitude of the pixel point with coordinates represents the horizontal direction difference of the pixel point, represents the vertical direction difference of the pixel point;

[0095] A preset edge threshold is set, and when the gradient amplitude of the pixel point is higher than the edge pixel point, the pixel point is marked as the edge pixel point.

[0096] The gradient amplitude reflects the rate of the gray value change of the image at a certain pixel point. The higher the gradient amplitude of the pixel point, the more intense the gray value change of the image at the pixel point, and the more likely the pixel point is an edge pixel point. Therefore, the edge threshold is set to compare with the gradient amplitude. When the gradient amplitude is less than the edge threshold, it indicates that the gray value change of the pixel point is not significant, and it is not considered as an edge pixel point, which may be a smooth area or noise of the image. When the gradient amplitude is greater than the edge threshold, it indicates that the gray value change of the pixel point is significant, which may be an edge area of the image. The median value of the pixel gray value is selected as the initial gradient amplitude, and the extracted edge pixel point is determined according to the initial gradient amplitude. Whether the extracted edge pixel point can reflect the contour features of the insect is judged. If the edge pixel point is too sparse, the edge threshold is reduced. If the edge pixel point is too complex, the edge threshold is increased.

[0097] The principle for generating the edge feature index is:

[0098] The formula for generating the edge complexity is:

[0099] ;

[0100] wherein, represents the edge complexity, represents the number of edge pixel points, represents the area of the region surrounded by the edge pixel points;

[0101] The edge complexity reflects the complexity of the edge in the third identification image. The higher the edge complexity, the more detailed the shape structure of the insect, which is used to distinguish the insect species with complex and simple shapes, and the different complexity of the shape corresponding to the insect at different stages. The edge complexity is proportional to the number of edge pixel points and inversely proportional to the area of the region surrounded by the edge pixel points. The insect with higher edge complexity corresponds to more complex shape structure.

[0102] ​The formula for generating the edge circularity is:

[0103]

[0104] wherein, denotes the edge circularity, denotes the edge perimeter;

[0105] The edge circularity reflects whether the contour shape of the region surrounded by the edge pixel points is close to a circle. The higher the edge circularity, the closer the region surrounded by the edge pixel points is to a circle, and the closer the shape of the insect is to a circle, such as a beetle or an insect in the larva or pupa stage.

[0106] The formula for generating the edge feature index is:

[0107]

[0108] wherein, denotes the edge feature index.

[0109] The edge complexity is used to measure the degree of detail of the edge. The higher the edge complexity, the more complex the edge shape is. The edge circularity reflects the regularity of the shape of the region surrounded by the edge. The closer the edge circularity is to 1, the closer the shape is to a circle. The smaller the value, the more irregular the shape is. The edge feature index generated based on the edge complexity and the edge circularity reflects the degree of detail and the contour shape of the edge. The more complex the edge is, the lower the general circularity is, and the higher the edge feature index is. The simpler the edge is, the higher the general circularity is, and the lower the edge feature index is. The edge feature index is directly proportional to the edge complexity and inversely proportional to the edge circularity.

[0110] Step 3: Extract the hue, saturation, and brightness of the pixel points in the second identification image, generate the hue index, saturation index, and brightness index of the second identification image based on the average values of the hue, saturation, and brightness of all the pixel points, and generate the image vividness based on the hue index, saturation index, and brightness index;

[0111] In this embodiment, the principle for generating the image vividness is:

[0112] The formula for generating the hue index, saturation index, and brightness index is:

[0113]

[0114]

[0115]

[0116] wherein, denotes the hue index, denotes the index of the pixel point, and​​​​​ , represents the number of pixel points, represents the hue of the th pixel point, represents the saturation index, represents the saturation of the th pixel point, represents the brightness index of the pixel point, represents the brightness of the th pixel point;

[0117] The formula for generating image vividness is:

[0118] ;

[0119] ;

[0120] wherein, represents the image vividness, represents the correction function of hue on vividness;

[0121] Saturation reflects the purity of color, indicating how much gray component is in the color. The higher the saturation, the purer the color, and the more vivid it appears visually. When the saturation is 0, the color appears gray, and when the saturation is 1, the color is in the purest state. Therefore, saturation has the greatest impact on image vividness, and image vividness is directly proportional to saturation. Brightness reflects the lightness and darkness of the color, represents that when the brightness is close to the median value 0.5, the vividness is the largest, and when the brightness becomes higher or lower, the vividness gradually decreases. Hue reflects the type of color and does not directly affect saturation, but different hues have different visual expressions. Warm colors are more vivid than cold colors, so different weights are set for different hues to adjust the difference between warm and cold colors.

[0122] Step 4: splice the edge feature index and the image vividness to generate a comprehensive feature vector, and calculate the comprehensive feature vectors corresponding to different species of insects at different life stages according to the methods of steps 1-3, to construct an insect feature library containing comprehensive feature vectors of different species of insects at different life stages;

[0123] In this embodiment, the formula for generating a comprehensive feature vector is:

[0124] ;

[0125] wherein, represents the comprehensive feature vector, represents the edge feature index, represents the image vividness.

[0126] The comprehensive feature vector reflects the morphological features and color features of the insects, different insects have differences in shape, size and color, and the comprehensive feature vector combines the complexity of the contour edge and the uniqueness of the color distribution, and can distinguish different insect species; the body features of the insects change significantly at different life stages, including larvae, pupae and adult insects, the morphological changes are captured by the edge feature index, such as the appearance of wings or changes in body shape, and the image brightness is used to reflect the brightness of the color, such as the single color of the insects at the larva stage and the bright color of the adult insects; the edge feature index and the image brightness of the insects of known species and life stages are calculated to generate the comprehensive feature vector, and an insect feature library is constructed, which includes the comprehensive feature vectors of different insects at different life stages.

[0127] Step 5: constructing a model based on a multi-task convolutional neural network, taking the comprehensive feature vector in the insect feature library as input, and taking the species and life stage of the insect as label, training an insect classification model;

[0128] In this embodiment, the structure of the model based on the multi-task convolutional neural network is as follows:

[0129] Input layer: containing 1 neuron, used for inputting the comprehensive feature vector;

[0130] First hidden layer: containing 64 neurons, activated by ReLU function;

[0131] Second hidden layer: containing 32 neurons, activated by ReLU function;

[0132] Output layer: containing 2 neurons, used for outputting the species and life stage of the insect.

[0133] Step 6: inputting the comprehensive feature vector of the insect image to be classified into the trained insect classification model, and outputting the species and life stage of the insect.

[0134] Please refer to Figure 6 The application also provides an insect classification system based on a multi-task convolutional neural network, which is used to implement the insect classification method based on the multi-task convolutional neural network, and specifically comprises:

[0135] An image acquisition module is used to acquire the insect image to be classified, and the insect image is scaled to 224x224 and then copied into two identical groups, one group is processed to generate a first recognition image, and the other group is converted from RGB color space to HSV color space to generate a second recognition image;

[0136] an edge extraction module configured to extract edge pixels of the first recognition image based on a Canny edge detection to generate a third recognition image, record a number of edge pixels in the third recognition image, an area and a perimeter of a region surrounded by the edge pixels, generate an edge complexity based on the number of edge pixels and the area of the region surrounded by the edge pixels, generate an edge circularity based on the perimeter and the area of the region surrounded by the edge pixels, and generate an edge feature index based on the edge complexity and the edge circularity;

[0137] a color extraction module configured to extract hue, saturation and brightness of pixels in the second recognition image, generate a hue index, a saturation index and a brightness index of the second recognition image based on average values of hue, saturation and brightness of all the pixels, and generate an image vividness based on the hue index, the saturation index and the brightness index;

[0138] a feature library construction module configured to splice the edge feature index and the image vividness to generate a comprehensive feature vector, and calculate comprehensive feature vectors corresponding to different kinds of insects in different life stages according to methods of the above modules to construct an insect feature library containing comprehensive feature vectors of different kinds of insects in different life stages;

[0139] a model training module configured to construct a model based on a multi-task convolutional neural network, take the comprehensive feature vectors in the insect feature library as input, take the kinds and life stages of the insects as labels, and train an insect hierarchical classification model;

[0140] a judgment output module configured to input the comprehensive feature vector of an insect image to be hierarchically classified into the trained insect hierarchical classification model, and output the kind and the life stage of the insect.

[0141] The above formulas are all dimensionless numerical calculations, and the formulas are obtained by software simulation of a large amount of data to obtain a formula closest to the real situation. The preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0142] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. A person skilled in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on specific application and design constraints of the technical solutions.

[0143] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, and may be located in one place, or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0144] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for hierarchical classification of insects based on a multi-task convolutional neural network, characterized by, The specific steps include: Step 1: Collect the insect image to be classified, copy the insect image into two groups of the same size after scaling to 224x224, one group is processed to generate a first identification image, and the other group is converted from an RGB color space to an HSV color space to generate a second identification image; Step 2: Extract the edge pixel points of the first identification image based on the Canny edge detection to generate a third identification image, record the number of edge pixel points, the area and perimeter of the region surrounded by the edge pixel points in the third identification image, generate the edge complexity based on the number of edge pixel points and the area of the region surrounded by the edge pixel points, generate the edge circularity based on the perimeter and area of the region surrounded by the edge pixel points, and generate the edge feature index based on the edge complexity and the edge circularity; Step 3: Extract the hue, saturation and brightness of the pixel points in the second identification image, generate the hue index, saturation index and brightness index of the second identification image based on the average values of the hue, saturation and brightness of all pixel points, and generate the image vividness based on the hue index, saturation index and brightness index; Step 4: Concatenate the edge feature index and the image vividness to generate a comprehensive feature vector, and calculate the comprehensive feature vectors of different types of insects in different life stages according to the methods of steps 1-3 to construct an insect feature library containing comprehensive feature vectors of different types of insects in different life stages; Step 5: Construct a model based on a multi-task convolutional neural network, use the comprehensive feature vectors in the insect feature library as input, and use the types and life stages of insects as labels to train an insect classification model; Step 6: Input the comprehensive feature vector of the insect image to be classified into the trained insect classification model, and output the type and life stage of the insect; The principle for generating the edge feature index is: The formula for generating the edge complexity is: Where C represents the edge complexity, N represents the number of edge pixel points, and A represents the area of the region surrounded by the edge pixel points; The formula for generating the edge circularity is: Where R represents the edge circularity, and L represents the edge perimeter; The formula for generating the edge feature index is: Where X represents the edge feature index; The principle for generating the image vividness is: The formula for generating the hue index, saturation index and brightness index is: wherein H represents a hue index, i represents an index of a pixel point, and i ∈ (1, 2, …, M), M represents a number of pixel points, H i represents a hue of the i-th pixel point, S represents a saturation index, S i represents a saturation of the i-th pixel point, V represents a lightness index of the pixel point, V i represents a lightness of the i-th pixel point; The formula for generating the image vividness is: K=S·(1-|V-0.5|)·f(H) Where K represents the image vividness, and f(H) represents a correction function of the hue on the vividness.

2. The insect hierarchical classification method based on the multi-task convolutional neural network according to claim 1, characterized in that: In step 2, the principle for generating the third identification image is: For each pixel point in the first identification image, respectively convolve the pixel point and its eight surrounding neighborhood pixel points with the horizontal direction template and the vertical direction template of the Prewitt operator to generate the gray difference of the pixel point in the horizontal direction and the vertical direction, and the formula is: wherein P X represents a horizontal direction template of the Prewitt operator, P Y represents a vertical direction template of the Prewitt operator, G x represents a horizontal direction difference of a pixel point, G y represents a vertical direction difference of a pixel point, (x, y) represents a coordinate of a pixel point; According to the gray difference in the horizontal direction and the vertical direction, generate the gradient amplitude of each pixel point, and the formula is: wherein G(x, y) represents a gradient amplitude of a pixel point with coordinates (x, y), G x represents a horizontal direction difference of a pixel point, and G y represents a vertical direction difference of a pixel point. A preset edge threshold is set, and when the gradient amplitude of the pixel point is higher than the edge pixel point, the pixel point is marked as an edge pixel point.

3. The insect hierarchical classification method based on the multi-task convolutional neural network according to claim 1, characterized in that: The formula for generating the comprehensive feature vector in step 4 is: Z=(X,K) Wherein, Z represents the comprehensive feature vector, X represents the edge feature index, and K represents the image brightness.

4. An insect hierarchical classification system based on a multi-task convolutional neural network, characterized in that: The system is used to execute the multi-task convolutional neural network-based insect hierarchical classification method of any one of claims 1-3, and specifically comprises: An image acquisition module, configured to acquire an insect image to be classified, copy the insect image into two groups of the same size after scaling the insect image to 224x224, generate a first recognition image after performing grayscale processing on one group, and generate a second recognition image by converting the image from an RGB color space to an HSV color space; An edge extraction module, configured to extract edge pixel points of the first recognition image based on canny edge detection to generate a third recognition image, record the number of edge pixel points, the area and the perimeter of the region surrounded by the edge pixel points in the third recognition image, generate an edge complexity based on the number of edge pixel points and the area of the region surrounded by the edge pixel points, generate an edge circularity based on the perimeter and the area of the region surrounded by the edge pixel points, and generate an edge feature index based on the edge complexity and the edge circularity; A color extraction module, configured to extract hue, saturation and brightness of pixel points in the second recognition image, generate a hue index, a saturation index and a brightness index of the second recognition image based on the average values of the hue, the saturation and the brightness of all pixel points, and generate an image brightness based on the hue index, the saturation index and the brightness index; A feature library construction module, configured to splice the edge feature index and the image brightness to generate a comprehensive feature vector, and calculate comprehensive feature vectors corresponding to different life stages of different types of insects according to the method of the above module, to construct an insect feature library containing comprehensive feature vectors of different types of insects in different life stages; A model training module, configured to construct a model based on a multi-task convolutional neural network, take the comprehensive feature vectors in the insect feature library as input, and take the types and life stages of insects as labels, to train an insect hierarchical classification model; A judgment output module, configured to input the comprehensive feature vector of the insect image to be classified into the trained insect hierarchical classification model, and output the types and life stages of insects.

Images