A cross-domain field boll recognition method based on a deep convolutional neural network

Abstract

The application discloses a kind of based on deep convolutional neural network's cross-domain field cotton boll identification method, the method includes the following steps: with field cotton boll image as processing object, construct MDLNet network model based on deep convolutional neural network, MDLNet network model includes feature extraction module, classification module and discrimination module;Initialize model parameter, cotton boll image in source domain is input to MDLNet network model and is adjusted parameter;Cotton boll image in target domain is input to the MDLNet network model trained, and the classification accuracy and loss result are calculated to verify the effectiveness and convergence of model;Real-time acquisition of the cotton boll image to be identified is input to the MDLNet network model trained, and the field cotton boll state identification result is output.The MDLNet network model of the application can identify field cotton boll image, automatically extract features from field cotton boll image, and determine the category of target field cotton boll state using the extracted image features, with high accuracy and strong robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural computer vision technology, and in particular to a cross-domain field cotton boll identification method based on a deep convolutional neural network. BACKGROUND

[0002] Cotton is one of the world's important economic crops and an important source of fiber and feed. The growth status of cotton bolls is a decisive factor affecting cotton yield and quality. Timely control of the growth status of cotton bolls can prevent premature or overripe rotting of cotton bolls, increase cotton yield, and improve cotton quality. Due to differences in cotton growing environment, growing year, and growing region, there are certain difficulties in identifying the status of cotton bolls in different fields. Therefore, it is of great significance to use a certain amount of cotton boll dataset to complete the research on the identification of cotton boll status in different fields.

[0003] The development of transfer learning technology improves the generalization ability in the field of agricultural applications. Sofia Marino et al. published a paper in 2020 entitled "Unsupervised adversarial deep domain adaptation method for potato defects classification", which uses the strategy of adversarial learning to train the classifier using pseudo-label loss, and is verified on the constructed potato dataset. Li Yanan et al. published a paper in 2020 entitled "Unsupervised domain adaptation for in-field cotton boll status identification", which first applies unsupervised domain adaptation to field cotton boll status identification. To achieve self-adaptation from the source domain to the target domain, a deep neural network is used to extract features, a nearest component analysis method is used to generate a feature transformation matrix, and finally a target label is obtained to realize the identification of the cotton boll status in the target domain. Compared with the early method based on hand-crafted features, the classification accuracy is greatly improved.

[0004] However, these transfer learning methods do not start from the perspective of feature extraction, resulting in a large number of domain-specific features in the extracted features. Therefore, it is necessary to start from the perspective of feature extraction and explore how to obtain more accurate domain-invariant features to solve the cross-domain cotton boll status identification problem. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a cross-domain field cotton boll identification method based on a deep convolutional neural network in view of the defects in the prior art.

[0006] The technical scheme adopted by the present application to solve its technical problem is:

[0007] The application provides a cross-domain field cotton boll identification method based on a deep convolutional neural network, which comprises the following steps:

[0008] Step 1, a field cotton boll image dataset is constructed and divided into a source domain and a target domain;

[0009] Step 2, a MDLNet network model based on a deep convolutional neural network is constructed, the MDLNet network model comprises a feature extraction module, a classification module and a discrimination module; wherein: the feature extraction module is used for extracting the deep features of the image, the classification module is used for supervising the image classification, and the discrimination module is used for determining whether the image comes from the source domain or the target domain;

[0010] Step 3, model training: initializing the model parameters, inputting the cotton boll images in the source domain into the MDLNet network model for parameter adjustment;

[0011] Step 4, model verification: inputting the cotton boll images in the target domain into the trained MDLNet network model, calculating the classification accuracy and loss results to verify the effectiveness and convergence of the model;

[0012] Step 5, inputting the real-time acquired cotton boll images to be identified into the trained MDLNet network model, and outputting the field cotton boll state identification result.

[0013] Further, the method of step 1 of the application is specifically:

[0014] Part of the field cotton boll image dataset is shot in a test field, and the cotton bolls in different planting years and different planting regions are selected; another part is intercepted from the network cotton boll images; after screening and classification, the field cotton boll image dataset is constructed, which comprises a source domain and a target domain.

[0015] Further, the feature extraction module of step 2 of the application is specifically:

[0016] The feature extraction module of the MDLNet network model is composed of a ResNet basic module; ResNet50 is used as a feature extractor, and ResNet50 comprises one layer of 7x7 convolution and four convolution blocks; after the input cotton boll images in the source domain pass through the four convolution blocks, four feature images are outputted;

[0017] Four difference blocks identical with the convolution blocks are respectively constructed, and after the input cotton boll images in the target domain pass through the four difference blocks, four feature images are outputted;

[0018] A disparity module connected with the outputs of the convolution blocks and the difference blocks is constructed, and the intermediate results c l of each convolution block are used in the disparity module l to obtain domain-invariant features r l, four intermediate outputs are converted into the same size and dimension using upsampling and convolution; four results are added to obtain the final result g, called "disparity", which is interpreted as the unique feature of each layer source domain or target domain; g is defined as follows:

[0019] r l =c l -g l

[0020]

[0021] Wherein, the up-sampling operation is ip, in order to better obtain domain invariant feature, reduce the influence of domain unique feature on domain migration, minimize g; the feature extraction loss Is defined as follows:

[0022]

[0023] Wherein, N s , N y It is the number of source domain and target domain samples; C is the number of categories; L is the number of backbone network layers; g is the unique feature of the i-th sample, and the target of the feature extraction loss is to minimize the domain unique feature, and better domain invariant feature is obtained in the feature extraction stage.

[0024] Further, the classification module of the step 2 of the present application is specifically:

[0025] The classification module of the MDLNet network model uses the linear layer of ResNet50 as a classifier, the input is the output of the feature extraction module, and the classification result is obtained by mapping through the average pooling layer and the softmax function; the classification module is supervised by the cross-entropy loss to ensure the accuracy of classification, and the classification loss Is defined as follows:

[0026]

[0027] Wherein, L ce It is the cross-entropy loss, It is the output of the feature extraction module, It is the real label; the cross-entropy loss measures the difference between the predicted result and the real result, and the smaller the cross-entropy loss, the higher the classification performance.

[0028] Further, the discrimination module of the step 2 of the present application is specifically:

[0029] The discrimination module of the MDLNet network model includes a linear layer, a dropout layer and a sigmod layer, which is used to judge the domain label of the picture, i.e. the picture comes from which domain; the discrimination module and the feature extraction module are used to define the adversarial loss, which is specifically as follows:

[0030]

[0031] Wherein, D is a discrimination module, through the adversarial loss, the discrimination module cannot distinguish the picture from which domain, the feature extraction module is easier to extract the domain invariant feature, and the robustness of cross-domain cotton boll state recognition is improved.

[0032] Further, the method for verifying the model in the step 4 of the application is specifically:

[0033] The cotton boll image in the target domain is input into the trained MDLNet network model, and the classification accuracy and loss result are calculated to verify the effectiveness and convergence of the model; the classification accuracy of the cotton boll image is obtained by calculating the ratio of the number of cotton boll images predicted correctly by the MDLNet network model to the total number of cotton boll images.

[0034] Further, the feature extraction module of the application uses the pre-trained ResNet50 model parameters in the feature extraction process to ensure the quality of the generated feature image.

[0035] The application has the following beneficial effects:

[0036] The application provides a cross-domain field cotton boll recognition method based on a deep convolutional neural network, takes field cotton boll images as processing objects, constructs a deep convolutional neural network model, automatically extracts features of the field cotton boll images, and uses a domain adaptation method of adversarial learning to obtain domain invariant features input into a classification module, so that the cotton boll images from different domains can be better recognized; the application obtains the classification results of the cotton boll states in the cotton boll data set through training and iterative feedback, and has the characteristics of high accuracy and good robustness. BRIEF DESCRIPTION OF DRAWINGS

[0037] The application will be further described below in combination with the drawings and examples, and the drawings are as follows:

[0038] Figure 1 It is the overall flowchart of the embodiment of the application;

[0039] Figure 2 It is the MDLNet network structure schematic diagram of the embodiment of the application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application, and are not used to limit the application.

[0041] Example 1

[0042] The cross-domain field cotton boll identification method based on deep convolutional neural networks according to embodiments of the present invention includes the following steps:

[0043] Step 1: Construct a dataset of cotton boll images in the field and divide it into a source domain and a target domain;

[0044] Images of cotton bolls in the field from different planting regions and years were obtained. After screening and classification, a dataset of cotton boll images in the field was constructed, which specifically includes the source domain (training set) and the target domain (test set).

[0045] Step 2: Construct an MDLNet network model based on a deep convolutional neural network. The MDLNet network model includes a feature extraction module, a classification module, and a discriminant module. The feature extraction module is used to extract the deep features of the image, the classification module is used to supervise image classification, and the discriminant module is used to determine whether the image comes from the source domain or the target domain.

[0046] Step 3: Train the model: Initialize the model parameters by inputting the cotton boll image from the source domain into the MDLNet network model for parameter adjustment;

[0047] Step 4: Validate the model: Input the cotton boll image in the target domain into the trained MDLNet network model, calculate the classification accuracy and loss results to validate the effectiveness and convergence of the model;

[0048] Step 5: Input the real-time acquired images of the cotton bolls to be identified into the trained MDLNet network model, and output the results of the cotton boll status identification in the field.

[0049] Example 2

[0050] The cross-domain field cotton boll identification method based on deep convolutional neural networks according to embodiments of the present invention includes the following steps:

[0051] 1. Obtain images of cotton bolls in the field, and construct a field cotton boll image dataset after screening and classification, specifically including the source domain and the target domain;

[0052] The dataset includes cotton boll images, some taken from experimental fields in Xinjiang, taking into account different planting years and regional conditions; and others extracted from online cotton boll images. A total of 1400 images were selected from the collected cotton boll image data, representing five different planting years and three different planting regions, with each region or year folder containing 200 cotton boll images. A total of 1000 online images were also included, all depicting four growth stages of the cotton boll.

[0053] 2. Construct an MDLNet network model based on a deep convolutional neural network;

[0054] The MDLNet network model is composed of a feature extraction module, a classification module and a discrimination module. The feature extraction module is used to extract the depth features of the image; the classification module is used to supervise the image classification, and the discrimination module is used to determine whether the image comes from the source domain or the target domain.

[0055] The feature extraction module of MDLNet is composed of a ResNet basic module. ResNet50 is used as a feature extractor, and ResNet50 includes 1 layer of 7x7 convolution and four convolution block groups. The input image (size HxWx3) is processed. After four convolution blocks, the size of the feature image is (56x56x256), (28x28x512), (14x14x1024) and (7x7x2048) respectively. Then four difference blocks are constructed, which are the same as the convolution blocks, and the same operation is performed on the feature extraction module. At the same time, the intermediate results c l of each convolution block are used to subtract the difference block g l to obtain the domain invariant feature r l . The four intermediate outputs are converted to the same size and dimension using upsampling and convolution. Finally, the four results are added to obtain the final result g, which is called "parallax" and can be explained as the unique feature of each layer of the source domain or the target domain. g is defined as follows:

[0056] r l =c l -g l

[0057]

[0058] where ip is the upsampling operation. In order to better obtain the domain invariant feature and reduce the influence of the domain unique feature on the domain transfer, g needs to be minimized. Specifically, the feature extraction loss is defined as follows:

[0059]

[0060] where N s , N t is the number of source domain and target domain samples; C is the number of categories; L is the number of backbone network layers, and L=4 at this time; g is the unique feature of the i-th sample, and the goal of the feature extraction loss is to minimize the domain unique feature and obtain better domain invariant features in the feature extraction stage.

[0061] In addition, in the feature extraction process, the pre-trained ResNet50 model parameters are used to ensure the quality of the generated feature image.

[0062] The linear layer of ResNet50 used by the classification module of MDLNet is used as a classifier, the input is the output of the feature extraction module, the size is (7x7x2048), and the classification result is obtained by mapping through the average pooling layer and the softmax function. In this method, the number of classifications is 4. The classification module is supervised by the cross-entropy loss to ensure the accuracy of the classification. The classification loss is defined as follows:

[0063]

[0064] where L ce is the cross-entropy loss, is the output of the feature extraction module, is the true label. The cross-entropy loss measures the difference between the predicted result and the true result. The smaller the cross-entropy loss, the higher the classification performance.

[0065] The discrimination module of MDLNet is composed of a linear layer, a dropout layer and a sigmod layer, which mainly judges the domain label of the picture, i.e. which domain the picture comes from. Through the discrimination module and the feature extraction module, the adversarial loss is defined, which is as follows:

[0066]

[0067] where D is the discrimination module. Through the adversarial loss, the discrimination module cannot distinguish which domain the picture comes from, and the feature extraction module can more easily extract domain-invariant features, achieving the improvement of the robustness of cross-domain cotton boll state recognition.

[0068] 3. Train the model.

[0069] Initialize the model parameters, input the source domain cotton boll image into the MDLNet network model for parameter adjustment; wherein the size of the cotton boll image needs to be modified to 224*224 before inputting into the MDLNet network model; the parallel learning number (batchsize) is set to 16 during training, the training iteration number (iteration) is 4000, and the learning rate is set to 0.001.

[0070] 4. Verify the model.

[0071] Input the cotton boll image in the target domain into the MDLNet network model to calculate the classification accuracy and loss result to verify the effectiveness and convergence of the model. The accuracy of the cotton boll image is calculated by calculating the percentage of the number of cotton boll images predicted correctly by the network to the total number of cotton boll images.

[0072] 5. Input the real-time acquired cotton boll image into the trained MDLNet network model, and the output result is the field cotton boll state recognition result;

[0073] The application automatically extracts features of cotton boll images in the field, and uses a domain adaptation method of adversarial learning to obtain domain invariant features input into a classification module, so that cotton boll images from different domains can be better recognized.

[0074] It should be understood that, for those skilled in the art, improvements or changes can be made according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A deep convolutional neural network-based cross-domain field boll recognition method, characterized in that, The method comprises the following steps: Step 1, constructing a field cotton boll image dataset and dividing it into a source domain and a target domain; Step 2, constructing a MDLNet network model based on a deep convolutional neural network, the MDLNet network model comprising a feature extraction module, a classification module and a discrimination module; wherein: the feature extraction module is used for extracting the deep features of the image, the classification module is used for supervising the image classification, and the discrimination module is used for determining whether the image comes from the source domain or the target domain; wherein: The feature extraction module of the MDLNet network model is composed of a ResNet basic module; ResNet50 is used as the feature extractor, and ResNet50 comprises one layer of 7x7 convolution and four convolution block groups; after the input cotton boll image in the source domain passes through the four convolution block groups, four feature images are output; The classification module of the MDLNet network model uses the linear layer of ResNet50 as the classifier, the input is the output of the feature extraction module, and the classification result is obtained through the average pooling layer and the softmax function; the classification module is supervised by the cross-entropy loss to ensure the accuracy of the classification; The discrimination module of the MDLNet network model comprises a linear layer, a dropout layer and a sigmod layer, and is used for judging the domain label of the image, i.e. whether the image comes from which domain; the discrimination module is used for the confrontation with the feature extraction module; Step 3, training the model: initializing the model parameters, inputting the cotton boll image in the source domain into the MDLNet network model for parameter adjustment; Step 4, verifying the model: inputting the cotton boll image in the target domain into the trained MDLNet network model to calculate the classification accuracy and the loss result to verify the effectiveness and convergence of the model; Step 5, inputting the real-time acquired cotton boll image to be identified into the trained MDLNet network model to output the field cotton boll state recognition result. 2.The deep convolutional neural network based cross-domain field boll recognition method according to claim 1, characterized in that, The method of step 1 is specifically: Part of the field cotton boll image dataset is shot in the test field, and the cotton bolls under different planting years and different planting regional conditions are selected; another part is intercepted from the network cotton boll image; after screening and classification, the field cotton boll image dataset is constructed, specifically including the source domain and the target domain. 3.The deep convolutional neural network based cross-domain field boll recognition method according to claim 1, characterized in that, The feature extraction module of step 2 is specifically: Four difference blocks identical to the convolution block are constructed respectively, and after the input cotton boll image in the target domain passes through the four difference blocks, four feature images are output; A disparity module is constructed which is connected to the outputs of the convolutional blocks and the difference block, and uses the intermediate results of each convolutional block Subtracting the result of the difference block Obtaining domain-invariant features The four intermediate outputs are converted to the same size and dimension using upsampling and convolution; the four results are added to obtain the final result , called "disparity", which is interpreted as the unique features of each layer of the source domain or the target domain; is defined as follows: wherein, is an up-sampling operation, to better obtain domain-invariant features, reduce the influence of domain-specific features on domain migration, minimize g; feature extraction loss is defined as follows: wherein, , is the number of source domain, target domain samples; C is the number of categories; L is the number of backbone network layers; is the unique feature of the th sample, and the target of the feature extraction loss is to minimize the domain unique feature to obtain better domain invariant features in the feature extraction stage. 4.The deep convolutional neural network based cross-domain field boll recognition method according to claim 3, characterized in that, The classification module of step 2 is specifically: Classification loss is defined as follows: wherein, is a cross-entropy loss, is an output of the feature extraction module, is a true label; the cross-entropy loss measures the difference between the predicted result and the true result, and the smaller the cross-entropy loss, the higher the classification performance.

5. The deep convolutional neural network-based cross-domain field boll recognition method according to claim 4, characterized in that, The discrimination module of step 2 is specifically: The adversarial loss is defined, which is specifically as follows: wherein, As the discriminant module cannot distinguish the picture from which domain through the adversarial loss, the feature extraction module is more likely to extract domain-invariant features, thereby improving the robustness of cross-domain cotton boll state recognition. 6.The deep convolutional neural network based cross-domain field boll recognition method according to claim 1, characterized in that, The method for verifying the model in step 4 is specifically: The cotton boll image in the target domain is input into the trained MDLNet network model to calculate the classification accuracy and the loss result to verify the effectiveness and convergence of the model; the classification accuracy of the cotton boll image is obtained by calculating the ratio of the number of cotton boll images predicted correctly by the MDLNet network model to the total number of cotton boll images.

7. The deep convolutional neural network-based cross-domain field boll recognition method according to claim 3, characterized in that, In the feature extraction process, the pre-trained ResNet50 model parameters are used to ensure the quality of the generated feature images.

Images