Crop leaf phenotype detection method and device based on deep learning

Abstract

The invention discloses a crop leaf phenotype detection method and device based on deep learning. The method comprises the following steps: acquiring a front image and a back image of a crop leaf; generating leaf feature image data according to the front image and the back image; a leaf phenotype detection model is adopted to detect the leaf feature image data to obtain leaf surface parameters, and the surface parameters comprise leaf area, leaf width, SPAD value and leaf freshness quality; wherein the leaf phenotype detection model is provided with an inverse residual module, and the inverse residual module is provided with an attention mechanism; according to the invention, the leaf phenotype monitoring model based on deep learning is adopted to detect the leaf of the crop, an attention mechanism is added in the inverse residual module, and the leaf can be detected by using the spatial position, texture and other features of the leaf, so that the detection precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop leaf phenotype detection, and in particular relates to a crop leaf phenotype detection method and device based on deep learning. Background Art

[0002] The accuracy and abundance of crop phenotypic data collection directly determine the number of dimensions and accuracy of subsequent multi-omics big data association analysis, and its importance is self-evident. Traditional manual phenotypic measurement methods are time-consuming and labor-intensive, and are subject to human subjectivity and prone to errors. Plant leaf phenotyping technology based on spectral technology has high accuracy, but the cost is high and the process is relatively complicated. It is only suitable for small-scale use in laboratories and is not suitable for large-scale promotion.

[0003] The method of crop phenotype detection by extracting image features has achieved high accuracy, but only considers the correlation between features such as the number of image pixels and image color components and the measured values, without considering features such as spatial position and texture in the image, resulting in unsatisfactory detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a crop leaf phenotype detection method and device based on deep learning, which uses the spatial position and texture characteristics in the leaf image to perform parameter detection, thereby improving the accuracy of crop leaf phenotype detection.

[0005] The present invention adopts the following technical solution: a crop leaf phenotype detection method based on deep learning, characterized in that it includes:

[0006] Acquire front and back images of crop leaves;

[0007] Generate leaf feature image data according to the front image and the back image;

[0008] The leaf phenotype detection model is used to detect the leaf feature image data to obtain the phenotypic parameters of the leaf, including leaf area, leaf width, SPAD value and leaf freshness quality;

[0009] Among them, the leaf phenotype detection model has an inverted residual module, and the inverted residual module has an attention mechanism.

[0010] Preferably, the convolution operation in the inverted residual module is performed using a grouped convolution method.

[0011] Preferably, in the group convolution, the leaf feature image data corresponding to the RGB three channels of the front image are grouped into one group, and the leaf feature image data corresponding to the RGB three channels of the back image are grouped into another group.

[0012] Preferably, the leaf phenotype detection model comprises a first convolutional layer, an inverted residual module group, a second convolutional layer, an average pooling layer, a third convolutional layer and a fourth convolutional layer connected in sequence;

[0013] The inverted residual module group includes several inverted residual module subgroups, each of which includes a first inverted residual module or a second inverted residual module. The first inverted residual module does not include an attention mechanism, and the second inverted residual module includes an attention mechanism.

[0014] Preferably, the activation functions of the second inverted residual modules in different inverted residual module subgroups are different.

[0015] Preferably, the loss function of the leaf phenotype detection model consists of cross entropy loss and content prediction loss.

[0016] Preferably, the cross entropy loss is:

[0017]

[0018] in, represents the cross entropy loss of the i-th leaf phenotypic parameter, represents the actual value of the i-th leaf phenotypic parameter, represents the predicted value of the i-th leaf phenotypic parameter;

[0019] The content prediction loss is:

[0020]

[0021] in, represents the content prediction loss of the i-th leaf phenotypic parameter, ε represents the adjustment factor, which is a constant.

[0022] Preferably, generating the blade feature image data according to the front image and the back image includes:

[0023] Flip the back image / front image horizontally and align the pixels with the front image / back image;

[0024] The RGB channel information of the front image and the back image after pixel alignment are combined to obtain the leaf feature image data.

[0025] Another technical solution of the present invention: a crop leaf phenotype detection device based on deep learning, comprising:

[0026] An acquisition module, used for acquiring a front image and a back image of a crop leaf;

[0027] A generating module, used for generating leaf feature image data according to the front image and the back image;

[0028] A detection module is used to detect leaf feature image data using a leaf phenotype detection model to obtain phenotypic parameters of the leaf, including leaf area, leaf width, SPAD value and leaf freshness quality;

[0029] Among them, the leaf phenotype detection model has an inverted residual module, and the inverted residual module has an attention mechanism.

[0030] Another technical solution of the present invention: a crop leaf phenotype detection device based on deep learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above method is implemented when the processor executes the computer program.

[0031] The beneficial effect of the present invention is that the present invention detects crop leaves by adopting a leaf phenotype monitoring model based on deep learning, and at the same time adds an attention mechanism in the inverse residual module, so as to detect the leaves using features such as the spatial position and texture of the leaves, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the process of establishing a leaf phenotype detection model in an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the main growth period of wheat in an embodiment of the present invention;

[0034] Figure 3 The front image and the back image of the blade taken in the embodiment of the present invention;

[0035] Figure 4 This is a comparison diagram of the principles of grouped convolution and ordinary convolution in an embodiment of the present invention;

[0036] Figure 5 It is a schematic diagram of the structure of the attention mechanism SE module in an embodiment of the present invention;

[0037] Figure 6 Schematic diagram of the structure of the inverted residual module in an embodiment of the present invention;

[0038] Figure 7 Schematic diagram for comparing different function curves in the embodiments of the present invention;

[0039] Figure 8 Schematic diagram of correlation coefficients obtained by combining different tasks in an embodiment of the present invention;

[0040] Fig. 9 Schematic diagram of determination coefficients obtained by different task combinations in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] In recent years, with the continuous development of science and technology and computing power, artificial intelligence (AI) has been widely used in large-scale data analysis to solve complex problems in computer vision (CV), natural language processing (NLP), autonomous driving, knowledge graph, protein structure and other fields.

[0043] In this context, the field of agricultural breeding has also ushered in the "breeding 4.0 era" marked by the integration and development of technologies such as gene editing, synthetic biology, and artificial intelligence. The essence of breeding is to change the composition of genes to obtain excellent traits. It is necessary to closely organize multidisciplinary cross-collaboration such as information science, life science, and agronomy, obtain phenotype group-genome-environment big data through collaborative innovation systems, and systematically explore the intrinsic association between phenotype-genotype-environment type from the omics level, comprehensively analyze the formation mechanism of important traits, and create intelligent combination of excellent allele breeding design schemes, and finally realize the intelligent, efficient and targeted breeding of new crop varieties. Therefore, we need to build an end-to-end AI breeding prediction model solution driven by multi-omics big data, and its overall process is: 1) Acquisition of multi-omics big data; 2) Analysis of multi-omics big data; 3) Construction of intelligent breeding prediction model.

[0044] Plant leaves are the most important organs in the growth and development of crops. Leaf area, leaf width, fresh weight and chlorophyll SPAD value and other phenotypic parameters are important data for evaluating the nutritional status, growth and development of crops and yield prediction. Therefore, these leaf phenotypes are the focus of research. Using convolutional neural networks to perform regression detection on them is a new trend in medium and high precision detection technology for crop leaf phenotypes.

[0045] The present invention discloses a crop leaf phenotype detection method based on deep learning, comprising: acquiring a front image and a back image of a crop leaf; generating leaf feature image data according to the front image and the back image; using a leaf phenotype detection model to detect the leaf feature image data to obtain phenotypic parameters of the leaf, the phenotypic parameters including leaf area, leaf width, SPAD value and leaf freshness quality; wherein the leaf phenotype detection model has an inverted residual module, and the inverted residual module has an attention mechanism.

[0046] The present invention detects crop leaves by adopting a leaf phenotype monitoring model based on deep learning, and at the same time adds an attention mechanism in the inverted residual module, so that the leaves can be detected by using features such as spatial position and texture, thereby improving the detection accuracy.

[0047] The characteristic image data of each leaf on the crop contains two sets of strongly correlated data subsets on the front and back of the leaf, which doubles the original input data from the traditional three channels to six channels (i.e., three channels on the front and three channels on the back) in the same spatial position. This not only greatly enhances the integrity of the model data, but also improves the quality of the training data without increasing the data acquisition cost or even at a lower cost.

[0048] like Figure 1 As shown, the implementation of the leaf phenotype detection model of the present invention follows the AI ​​development process, and the following content will explain the detailed process of each stage one by one according to this process.

[0049] In agricultural production, according to the growth characteristics of wheat at different stages, in order to facilitate cultivation management and research, the whole life cycle of wheat is generally divided into 12 growth stages, such as Figure 2 As shown, they are respectively emergence, three leaves, tillering, overwintering, greening, standing up, jointing, booting, heading, flowering, filling, and maturity. In the present invention, leaves of wheat of the variety Huaimai No. 40 at the three-leaf stage were selected as the research object for data collection and recording.

[0050] The collected data includes feature image data and corresponding label data, and the method is as follows:

[0051] Special leaf photo frame: the size is 10cm*20cm, which is large enough to completely hold a wheat leaf; it uses double-layer high-transparency thin glass with a glass interlayer gap of 1.5mm, and the front and back of the photo frame are clear and transparent; it also comes with a base that can rotate 180°, and the leaf photo frame can be easily inserted on it.

[0052] Feature image acquisition method: Use an industrial camera with the lens 60 cm away from the leaf frame; each leaf clip is placed into the frame and fixed in horizontal position by rotating the base, such as Figure 3 As shown in the figure, two sets of images of the front and back sides of the blade are taken before and after the rotation of 180°, where: Figure 3 (a) is the front image, Figure 3 (b) is the back image, and it is important to store them separately so that the back image can be further processed later.

[0053] Label data collection method: Leaf area and leaf width were measured using the Topyunnong TOP-1300 leaf area index meter; SPAD value was measured using the SPAD-502Plus meter produced by KONICAMINOLTA. This instrument is a point detection instrument, so 2 points each at the tip, middle and tail of the leaf were selected to avoid the veins for measurement. Each part was measured twice, and the total average value was taken as the SPAD value of the leaf; the fresh weight of the leaf was measured using an electronic scale with an accuracy of 0.01g.

[0054] There is a certain amount of noise and useless background in the image capture process, so the image needs to be denoised by using median filtering. Median filtering is a nonlinear filtering method that sets the grayscale of a certain pixel to the median of the grayscale values ​​of the neighboring points, which can effectively eliminate noise. The image background obtained under the experimental conditions is relatively simple. The image is binarized after the three channels are separated to obtain the leaf outline, and the part outside the leaf outline is set to 0 to obtain a leaf image without background.

[0055] In the present invention, generating leaf characteristic image data according to the front image and the back image includes: horizontally flipping the back image / front image and aligning the pixels with the front image / back image; combining the RGB channel information of the front image and the back image after pixel alignment to obtain the leaf characteristic image data.

[0056] The back image / front image of the blade is flipped horizontally and aligned with the front image / back image at the pixel level to ensure that the channel information of the same object is multiplied at the same spatial position. However, if the manufacturing process precision of the photo frame and the rotating base is high enough, this step can be omitted.

[0057] At this time, two images of the front and back sides of the same leaf are obtained. By constructing a six-channel zero matrix with unchanged length and width, the front and back image information are written into the matrix in sequence (and the grayscale value of each pixel in each channel in the image is filled into the matrix of the corresponding channel), and the final input leaf feature image data is obtained.

[0058] During the training process, in order to increase the sample size of training data and thus improve the robustness and generalization ability of the neural network model, as well as to avoid overfitting problems, the image generator in the keras framework is used to perform simple transformations such as random horizontal / vertical flipping, rotation, shifting, and scaling on the images to generate new images to expand the training samples that will not cause large shape differences.

[0059] In the embodiment of the present invention, each pair of data contains six-channel feature image data of the front and back sides and corresponding label data, and the data set contains 520 pairs of data in total, including 20 pairs of test sets and 500 pairs of training sets and validation sets (divided in a ratio of 8:2).

[0060] In the present invention, as shown in Table 1 below, the leaf phenotype detection model includes a first convolutional layer, an inverted residual module group, a second convolutional layer, an average pooling layer, a third convolutional layer and a fourth convolutional layer connected in sequence;

[0061] The inverted residual module group includes several inverted residual module subgroups, each of which includes a first inverted residual module or a second inverted residual module. The first inverted residual module does not include an attention mechanism, and the second inverted residual module includes an attention mechanism.

[0062] Table 1 Network structure of leaf phenotype detection model

[0063]

[0064]

[0065] Among them, Input Shape represents the input shape; Operator represents the operation; t represents the expansion factor; c represents the output feature matrix depth Channel; n represents the number of repetitions of block; s represents the step size (only for the first layer, the others are 1); √ represents the attention mechanism module SE; NL represents the nonlinear activation function; RE represents the Relu activation function, HS represents the h-switch activation function; NBN represents not using the Batch Normalization layer (Batch Normalization); Since the recurrent neural network needs to output continuous eigenvalues, the last layer of the fully connected layer uses a linear activation function (linear).

[0066] Taking into account that this scheme is to perform regression detection on leaf phenotypic characteristics such as leaf area, leaf width, fresh weight, and chlorophyll SPAD value, and accuracy is the only indicator to be considered, this scheme plans to build a convolutional neural network with more than 50 layers, and apply customized modules such as grouped convolution, attention mechanism, residual structure, loss function, and activation function to achieve the purpose of accurate detection of wheat leaf phenotype.

[0067] It can be seen from Table 1 above that the activation functions of the second inverted residual modules in different inverted residual module subgroups are different.

[0068] In a specific implementation, the convolution operation in the inverted residual module is performed by group convolution. In the group convolution, the leaf feature image data corresponding to the RGB three channels of the front image are grouped into one group, and the leaf feature image data corresponding to the RGB three channels of the back image are grouped into another group.

[0069] Specifically, Figure 4As shown in the figure, W×H×C represents the input feature map size, k×k represents the width and height of the convolution kernel, and g represents the number of groups. From the figure, we can see that group convolution mainly groups the input feature map, and then performs convolution on each group separately.

[0070] If the input feature map size is C×H×W, the number of output feature maps is N. If it is set to be divided into G groups, the number of input feature maps in each group is C / G, and the number of output feature maps in each group is N / G. The size of each convolution kernel is C / G×K×K, the total number of convolution kernels is still N, and the number of convolution kernels in each group is N / G. The convolution kernel only convolves with the input map of the same group. The total number of convolution kernel parameters is N×C / G×K×K. It is easy to obtain the total number of parameters reduced to 1 / G of the original.

[0071] Therefore, grouped convolution has the following advantages: 1) Grouped convolution can reduce the number of parameters; 2) Grouped convolution can be regarded as a sparse operation, which can sometimes achieve better results with fewer parameters (equivalent to regularization operation); 3) When the number of groups is equal to the number of input feature maps, the number of output feature maps is also equal to the number of input feature maps. At this time, grouped convolution becomes Depthwise convolution, which can further reduce the number of parameters.

[0072] In this scheme, the input feature map of the grouped convolution is actually a six-channel feature image data formed by merging two groups of three-channel feature image data of the front and back sides of the leaf, so the parameters here can be set to C=6, G=2.

[0073] Regarding the attention mechanism, SENet is a channel attention mechanism with a structure like Figure 5 As shown in Figure 2, the SE attention module consists of three parts: Squeeze operation, Excitation operation, and Scale operation.

[0074] Squeeze operation: Perform global average pooling on the input feature map to reduce the feature value of each channel to a global vector. This step aims to capture the global information of each channel.

[0075] Excitation operation: It consists of two full connections, a ReLU activation function, and a Softmax activation function. It first reduces the dimension and then increases the dimension. Finally, the weight vector is generated through the sigmoid function to ensure that their sum is 1.

[0076] Scale operation: Multiply the channel attention weight obtained in the previous step by the original input feature map. This step is used to adjust the feature value of each channel, emphasize the information of important channels, and suppress the information of unimportant channels.

[0077] In the present invention, the SE module will be added and applied to the second inverted residual module, and the activation function therein is replaced. Different activation functions (such as: ReLU, ReLU6, Switch, h-Switch, sigmoid, h-sigmoid and other non-linear functions) can be selected in different modules as needed.

[0078] Regarding the inverted residual module, it is generally believed that each layer of the neural network corresponds to extracting feature information at different levels, including low, middle and high layers. The deeper the network, the more information at different levels will be extracted, and the more combinations of information between different levels will be. However, as the depth of the network increases, gradient disappearance and gradient explosion may occur, resulting in more and more training errors, which is described as network degradation.

[0079] Residual Networks (ResNets) can solve the above problems through designs such as shortcut connections and identity mapping. Even if the network is deeper, the training performance is still very good. It helps solve the gradient vanishing and gradient exploding problems, allowing us to train deeper networks while maintaining good information.

[0080] In this invention, we combine the experience of some excellent convolutional network models and believe that the number of layers constituting the inverted residual module block ≥ 3 will have a better effect, and in this case, the number of network layers to construct the overall model is preferably greater than 50. Finally, r (i.e., the dimensionality reduction ratio) is set to 4. In addition, in order to avoid the ReLu function causing lower information loss for feature images with a lower number of channels, the output layer of the block is replaced with a Linear function. Figure 6 As shown, there is a shortcut connection only when stride=1. Figure 6 (a) is a schematic diagram of the structure of the first inverted residual module. Figure 6 (b) is a schematic diagram of the structure of the second inverted residual module.

[0081] The model training process includes two loss functions, namely, the loss function of the leaf phenotype detection model consists of cross entropy loss and content prediction loss. The first is the classification prediction loss function, which calculates the cross entropy between the label value and the predicted value:

[0082]

[0083] in, represents the cross entropy loss of the i-th leaf phenotypic parameter, represents the actual value of the i-th leaf phenotypic parameter, represents the predicted value of the i-th leaf phenotypic parameter;

[0084] The second is the content prediction loss, which calculates the content prediction loss of the predicted value and the label value, but because the absolute value difference between the two variables is not differentiable, the content prediction loss is:

[0085]

[0086] in, represents the content prediction loss of the i-th leaf phenotypic parameter, ε represents the adjustment factor, which is a constant and is taken as 10e in the experiment. -5 . The derivative of this loss function is:

[0087]

[0088] In order to make the model prediction more accurate, the above two loss functions are combined in the training process of this solution, which are specifically manifested as follows:

[0089]

[0090] Among them, w1 is a constant weight, which is set to 0.8 during training. This loss function can calculate the pixel difference between the predicted image and the true value image, and the loss generated by the calculation is propagated back to each node in the network through the back propagation method.

[0091] Regarding the activation function, in this model, a nonlinearity called swish is introduced, which can significantly improve the accuracy of the neural network when used as a direct replacement for ReLU. The nonlinear function is defined as:

[0092] Swish(x)=x.0(x),

[0093]

[0094] Although this nonlinearity improves accuracy, it has a non-zero cost in an embedded system environment because the cost of computing the sigmoid function on a mobile device is much higher. The present invention solves this problem in the following two ways.

[0095]

[0096] ReLU6(x)=min(max(x,0),6),

[0097] Among them, different activation function curves are compared as follows Figure 7 As shown, according to Figure 7It can be seen that the computational cost of the above two loss functions is much lower, and there is no obvious decrease in accuracy.

[0098] In the embodiment of the present invention, the hardware platform configuration required for training / implementing the leaf phenotype detection model is as follows: computing node (CPU intel i73.4GHz, GPU NVIDIA TASLA T40M, with 64-bit LinuxUbuntu 18 installed); the storage node adopts the self-developed distributed parallel file system storage product (OrcaFS) and provides high-performance NFS file services.

[0099] Software platform configuration: The present invention uses the Keras deep learning framework based on the TensorFlow platform, which can simplify many complex mathematical functions; it is implemented in the Python3.8 programming language.

[0100] Model training: Data and model are fitted through code, and the number of deep learning is adjusted according to the Epochs parameter. After the machine runs the code, it will repeatedly calculate as much as possible, and the model will find an answer from the data so that it can meet the shortest path. The model training Batch_size in the present invention is set to 8, the training Epochs is set to 300, and the optimal model weight of the validation set is saved.

[0101] After training, the above model is evaluated. The evaluation function includes the correlation coefficient r and the determination coefficient R 2 The correlation coefficient r can describe the degree of correlation between two predicted values ​​and actual values. The closer the r value is to 1, the stronger the correlation between the two variables.

[0102] Coefficient of determination R 2 It can indicate the degree of fit of the regression equation of two variables, R 2 The closer it is to 1, the better the regression fit between the two variables. Therefore, the correlation coefficient r and the determination coefficient R are used. 2 The prediction accuracy of the model can be evaluated on the test set.

[0103] In this experiment, the weight parameters of the network model obtained by training different combination tasks were used to infer and predict the test set, and the correlation coefficient r and determination coefficient R were used. 2 To evaluate the performance of the model on the test set and test the generalization ability of the model.

[0104] Depend on Figure 8It can be seen that Task 1 represents the correlation coefficient calculated after training and testing using only the leaf area parameter, Task 2 represents the correlation coefficient calculated after training and testing using only the fresh weight parameter, Task 3 represents the correlation coefficient calculated after training and testing using only the leaf width parameter, and Task 4 represents the correlation coefficient calculated after training and testing using only the SPAD parameter. Tasks 5-10 represent the correlation coefficients calculated after training and testing using a combination of two parameters, Tasks 11-14 represent the correlation coefficients calculated after training and testing using a combination of three parameters, and Task 15 represents the correlation coefficient calculated after training and testing using a combination of four parameters. According to Figure 8 It can be seen that the present invention has the highest accuracy when four parameters are used.

[0105] In addition, the correlation coefficient r of leaf area in each task group on the test set reached 0.998. The fresh weight had the highest correlation coefficient r in tasks 11 and 15, which was 0.995. The leaf width had the highest correlation coefficient r in task 15, which was 0.993. The SPAD had the highest correlation coefficient r in task 15, which was 0.992.

[0106] Depend on Fig. 9 It can be seen that each task combination and Figure 8 The task combination is the same, the leaf area in each task grouping, the determination coefficient R 2 All reached 0.998, and the freshness quality was R in tasks 5, 8, 11, 14 and 15. 2 Reaching 0.987, Ye Kuan scored R in Task 15 2 The highest is 0.998, SPAD in task 15 R 2 The highest, reaching 0.997.

[0107] The above results intuitively show the prediction accuracy of each leaf phenotypic parameter prediction in different combination tasks, which not only shows that different leaf phenotypic parameter combinations have a direct impact on the model prediction accuracy, but also find that the more relevant phenotypic parameters there are in the combination task, the higher the prediction accuracy of the model. According to the experimental results and comparison, the prediction accuracy of the customized convolutional neural network model of the present invention not only meets the requirements of practical applications, but also exceeds the CNN network model in the existing similar technical solutions.

[0108] In summary, the data acquisition method of the present invention not only improves the effectiveness and quality of the input feature images used for training, but also reduces the acquisition cost. Specifically, the input feature image acquisition of each blade includes two sets of strongly correlated data subsets on the front and back of the blade, which doubles the original input feature image from the traditional three channels to six channels in the same spatial position. Without increasing the data acquisition cost or even at a lower cost, the integrity of the input feature image data is greatly enhanced, the quality of the training data is improved, and the scale of the training data is effectively expanded.

[0109] The customized neural network model of the present invention has the highest prediction accuracy and higher efficiency (i.e., fewer parameters and less computation), and is more suitable for deployment in mobile devices or embedded systems; moreover, the network model can be easily expanded to more than 100 layers, and network degradation problems such as gradient disappearance and gradient explosion will not occur.

[0110] The present invention also discloses a crop leaf phenotype detection device based on deep learning, comprising: an acquisition module, used to acquire the front image and the back image of the crop leaf; a generation module, used to generate leaf feature image data according to the front image and the back image; a detection module, used to detect the leaf feature image data using a leaf phenotype detection model to obtain phenotypic parameters of the leaf, the phenotypic parameters including leaf area, leaf width, SPAD value and leaf freshness quality; wherein the leaf phenotype detection model has an inverted residual module, and the inverted residual module has an attention mechanism.

[0111] The present invention also discloses a crop leaf phenotype detection device based on deep learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above method is implemented when the processor executes the computer program.

[0112] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0113] The present invention also provides a computer program product. When the computer program product runs on a data storage device, the data storage device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0114] If the integrated unit module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a storage device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0117] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0118] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims

1. A crop leaf phenotype detection method based on deep learning, characterized in that: include: Acquire front and back images of crop leaves; Generate blade feature image data according to the front image and the back image; Using a leaf phenotype detection model to detect the leaf feature image data, and obtaining the phenotypic parameters of the leaf, wherein the phenotypic parameters include leaf area, leaf width, SPAD value and leaf freshness quality; Among them, the leaf phenotype detection model has an inverted residual module, and the inverted residual module has an attention mechanism.

2. A crop leaf phenotype detection method based on deep learning as claimed in claim 1, characterized in that: The convolution operation in the inverted residual module is performed using a grouped convolution method.

3. A crop leaf phenotype detection method based on deep learning as claimed in claim 2, characterized in that: In the grouped convolution, the leaf feature image data corresponding to the RGB three channels of the front image are grouped into one group, and the leaf feature image data corresponding to the RGB three channels of the back image are grouped into another group.

4. A crop leaf phenotype detection method based on deep learning as claimed in claim 2 or 3, characterized in that: The leaf phenotype detection model comprises a first convolutional layer, an inverted residual module group, a second convolutional layer, an average pooling layer, a third convolutional layer and a fourth convolutional layer connected in sequence; The inverted residual module group includes several inverted residual module subgroups, each inverted residual module subgroup includes a first inverted residual module or a second inverted residual module, the first inverted residual module does not include an attention mechanism, and the second inverted residual module includes an attention mechanism.

5. A crop leaf phenotype detection method based on deep learning as claimed in claim 4, characterized in that: The activation functions of the second inverted residual modules in different inverted residual module subgroups are different.

6. A crop leaf phenotype detection method based on deep learning as claimed in claim 4, characterized in that: The loss function of the leaf phenotype detection model consists of cross entropy loss and content prediction loss.

7. A crop leaf phenotype detection method based on deep learning as claimed in claim 4, characterized in that: The cross entropy loss is: in, represents the cross entropy loss of the i-th leaf phenotypic parameter, represents the actual value of the i-th leaf phenotypic parameter, represents the predicted value of the i-th leaf phenotypic parameter; The content prediction loss is: in, represents the content prediction loss of the i-th leaf phenotypic parameter, ε represents the adjustment factor, which is a constant.

8. A crop leaf phenotype detection method based on deep learning as claimed in claim 7, characterized in that: Generating blade feature image data according to the front image and the back image comprises: Flipping the back image / front image horizontally and aligning the pixels with the front image / back image; The RGB channel information of the front image and the back image after pixel alignment are combined to obtain leaf feature image data.

9. A crop leaf phenotype detection device based on deep learning, characterized in that: include: An acquisition module, used for acquiring a front image and a back image of a crop leaf; A generating module, used for generating blade characteristic image data according to the front image and the back image; A detection module, for detecting the leaf characteristic image data using a leaf phenotype detection model to obtain the phenotypic parameters of the leaf, wherein the phenotypic parameters include leaf area, leaf width, SPAD value and leaf freshness quality; Among them, the leaf phenotype detection model has an inverted residual module, and the inverted residual module has an attention mechanism.

10. A crop leaf phenotype detection device based on deep learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images