Citrus Huanglongbing image acquisition and recognition method, storage medium and system

Abstract

The invention relates to a citrus Huanglongbing image acquisition and recognition method, a storage medium and a system thereof. The method comprises the following steps: introducing evaluation indicators F1 value, P value and R value, and training an existing image data set of citrus Huanglongbing symptoms through a convolutional neural network model; when the number of pictures acquired by an acquisition terminal is greater than a preset value, adding the acquired image data set to an image data set, and then training the convolutional neural network model, evaluating the updated recognition model to obtain updated data values ​​of the evaluation indicators F1 value, P value and R value, comparing the previous and next data values, and using the updated recognition model as the final recognition model if verified, otherwise keeping the old model unchanged, and finally continuously updating the recognition model to improve the accuracy of the recognition model, thereby improving the recognition efficiency and accuracy of citrus Huanglongbing, and realizing automatic updating of the recognition model.

Description

Technical Field

[0001] The present invention relates to the technical field of citrus Huanglongbing identification, and in particular to a citrus Huanglongbing image acquisition and identification method, a storage medium and a system thereof. Background Art

[0002] Citrus is the most cultivated fruit tree in southern China. Citrus Huanglongbing (Huanglongbing) is a devastating disease in citrus production. Fruit on affected plants exhibits symptoms such as smaller, deformed, green, and red-nosed fruit, leading to rapid decline in vigor, reduced yield, and poor quality. Currently, no medication can cure HLB. Foliar application of additional nutrients and plant growth regulators (such as the auxin plant hormone derivative 2,4-D and gibberellins) can alleviate HLB symptoms, but this does not reduce the incidence of HLB in orchards and may even increase the risk of spread.

[0003] To ensure healthy production in citrus orchards, strict vector control through pesticide spraying and removal of infected plants remains the best long-term measure in orchards affected by Huanglongbing. Early detection of infected plants is crucial for successful implementation. However, infected plants often exhibit a variety of symptoms, including leaf mottling, uniform yellowing, zinc deficiency mosaic, and red-nosed fruit, greatly complicating diagnosis.

[0004] Currently, the most common methods for identifying Huanglongbing (Huanglongbing) are field morphological classification and laboratory PCR testing. Symptoms of Huanglongbing-infected plants vary, and field morphological classification relies on extensive practical experience, background knowledge, and a thorough understanding of the underlying conditions of the orchards suspected of being affected. However, the results are susceptible to subjective factors, and the misdiagnosis rate can exceed 30%. PCR testing is a more reliable method, but it requires specialized equipment and requires a high level of operator expertise. The process is cumbersome, time-consuming, and inefficient in rapidly identifying citrus Huanglongbing-infected plants. Therefore, accurate field diagnosis of citrus Huanglongbing is a crucial skill for citrus producers.

[0005] More information related to the above technical solutions can be found in the following documents:

[0006] In the invention patent with patent publication number CN104008285A, a citrus Huanglongbing diagnosis method based on MINI-PAM acquisition parameters is disclosed, and the steps are as follows: S1, MINI-PAM collects various fluorescence parameters of citrus leaves and transmits them to a computer; S2, the computer removes singular samples and preliminarily processes the original data collected by MINI-PAM; S3, establishes a probabilistic neural network sub-classifier, and performs classification training on various disease type data of citrus to obtain a disease classification model for citrus; S4, inputs the fluorescence parameter data of various samples collected by MINI-PAM into the disease classification model, and classifies them through the disease classification model; S5, executes a probability algorithm based on the classification output results of the disease classification model to obtain the judgment probability of the corresponding various disease types, and then determines whether the citrus is suffering from Huanglongbing based on the probability results.

[0007] In the invention patent with patent publication number CN114169397A, an electronic certificate intelligent classification model based on a convolutional neural network is disclosed, which relates to the field of deep learning technology. It includes: a data module, which is used to obtain electronic certificate data and store it in a certificate library, and is also used to convert the acquired data into data that can be used by the convolutional neural network, and randomly divide it into a training set and a validation set; a construction and training module, which is used to construct an electronic certificate intelligent classification model based on a convolutional neural network, and is also used to train and optimize the electronic certificate intelligent classification model using data from the training set and the validation set; a testing module, which is used to use data in the certificate library that can be used by the convolutional neural network to input the optimal electronic certificate intelligent classification model, and the optimal electronic certificate intelligent classification model outputs the classification results of the subject to which the certificate belongs. The advantages and disadvantages of the electronic certificate intelligent classification model are evaluated by calculating the precision, recall and accuracy of the output results.

[0008] In the process of implementing the present invention, the inventors found that the prior art has the following problems:

[0009] Existing technologies use image recognition algorithms to detect Huanglongbing. However, because deep learning relies on large datasets, and currently few open plant disease and insect pest datasets exist, researchers typically use different network models to find the optimal solution. However, the sample collection environments of some studies do not align with actual field applications, resulting in inconsistent accuracy. A method for collecting and recognizing citrus Huanglongbing images that can automatically update the recognition model is lacking evaluation metrics. Summary of the Invention

[0010] In view of the above problems, the present application provides a citrus Huanglongbing image acquisition and recognition method, storage medium and system thereof, which are used to solve the technical problems of the existing citrus Huanglongbing image acquisition and recognition method in terms of unstable accuracy, lack of evaluation indicators, and automatic updating of the recognition model.

[0011] To achieve the above objectives, in a first aspect, the inventors provide a method for collecting and identifying citrus Huanglongbing images, comprising the following steps:

[0012] A dataset of images of citrus Huanglongbing symptoms is pre-established and divided into a training set, a validation set, and a test set according to a preset ratio.

[0013] Use the Yolo convolutional neural network model for training to obtain the initial recognition model, and use the test set to evaluate the initial recognition model to obtain the initial values ​​of the evaluation indicators F1 value, P value, and R value F1-0, P-0, and R-0;

[0014] The collection terminal captures and collects data

[0015] Use the initial recognition model to automatically annotate images taken by the acquisition terminal, annotate the symptoms of citrus Huanglongbing on the images, and manually review whether the automatically annotated symptoms are correct.

[0016] When the number of images collected by the acquisition terminal is greater than the preset value, the collected image data set is added to the above-mentioned training set, validation set and test set according to the above-mentioned preset ratio, and the self-training program is started to train the initial recognition model to obtain an updated recognition model;

[0017] Use the updated test set to evaluate the updated recognition model and obtain the updated evaluation index F1 value, P value, and R value data values ​​F1-1, P-1, and R-1. When F1-1>F1-0, P-1>P-0, and R-1>R-0 are satisfied at the same time, the verification passes and the updated recognition model is used as the final recognition model. Otherwise, the old model remains unchanged.

[0018] Identify samples to be tested

[0019] Place the sample to be tested on the photo background board of the acquisition terminal, take a photo using the photo control key of the acquisition terminal, send the collected photo to the final recognition model for recognition, and feed back the recognition result to the acquisition terminal and display it on the display screen.

[0020] Different from the existing technology, the above technical solution introduces the evaluation index F1 value, P value, and R value, and trains the existing image data set of citrus Huanglongbing symptoms through a convolutional neural network model to obtain the initial values ​​F1-0, P-0, and R-0 of the evaluation index F1 value, P value, and R value; uses the initial recognition model to automatically annotate the images taken by the acquisition terminal, and manually reviews whether the automatically annotated citrus Huanglongbing symptoms are correct to improve the accuracy of the annotation. When the number of pictures collected by the acquisition terminal is greater than the preset value, the collected image data set is added to the image data set, and then trained through the convolutional neural network model, and the updated recognition model is evaluated to obtain the updated evaluation index F1 value, P value, and R value data values ​​F1-1, P-1, and R-1. When F1-1>F1-0, P-1>P-0, and R-1> When R-0 is higher than R-0, the verification is passed and the updated recognition model is used as the final recognition model. Otherwise, the old model remains unchanged. Finally, the recognition model is continuously updated to improve the accuracy of the recognition model, improve the recognition efficiency and accuracy of citrus Huanglongbing, and realize automatic update of the recognition model.

[0021] As an embodiment of the present invention, the recognition model selects three indicators, namely, precision P, recall R, and F1 value, as evaluation criteria. The calculation formulas are as follows:

[0022]

[0023]

[0024]

[0025] Among them, true positive TP refers to symptoms detected by the algorithm as Huanglongbing, and they are actually Huanglongbing symptoms; false positive FP refers to symptoms detected by the algorithm as Huanglongbing symptoms, but they are not actually Huanglongbing symptoms; false negative FN refers to symptoms detected by the algorithm as non-Huanglongbing symptoms, but they are actually Huanglongbing symptoms.

[0026] In this way, by calculating the precision P, recall R and F1 value to evaluate the accuracy of the recognition model, which can be used as the standard for updating the recognition model, the accuracy of the recognition model can be effectively improved.

[0027] As an embodiment of the present invention, there are 5 symptoms of Huanglongbing, corresponding to 5 P, R, and F1 values. The symptoms of Huanglongbing are red nose fruit, mottled yellowing, zinc deficiency, yellowing of leaf veins, and uniform yellowing. The initial values ​​F1-0, P-0, and R-0 are the averages of the 5 symptoms, and the updated evaluation indexes F1 value, P value, and R value data values ​​F1-1, P-1, and R-1 are also the averages of the 5 symptoms.

[0028] In this way, the symptoms of Huanglongbing are red nose fruit, mottled yellowing, zinc deficiency, yellowing of leaf veins, and uniform yellowing. The initial values ​​F1-0, P-0, and R-0 are the averages of the five symptoms, and the updated evaluation indicators F1 value, P value, and R value data values ​​F1-1, P-1, and R-1 are also the averages of the five symptoms, which can effectively improve the accuracy of the five Huanglongbing symptoms.

[0029] As an embodiment of the present invention, the specific steps of capturing and collecting data by the collection terminal are as follows:

[0030] The acquisition terminal takes images of citrus leaves or fruits, and the initial recognition model automatically labels the images photographed by the acquisition terminal, and marks the symptoms of citrus Huanglongbing on the photographed images. The automatically labeled images are manually reviewed to see if the automatically labeled content is correct. If the automatically labeled content is correct, the labeled images are included in the acquisition image data set, and the total number of images in the acquisition image data set is checked to see if it is greater than a preset value. When the total number of images in the acquisition image data set is greater than the preset value, the self-training program is started. After the self-training is completed, an updated recognition model is obtained.

[0031] In this way, the automatically annotated content can be judged through manual review. There is no need to manually annotate each image, only review is required, which can improve the accuracy and efficiency of annotation. When the total number of collected images is greater than the preset value, the preset value here is about 800-1000 photos, then the collected image data set will be added to the above-mentioned training set, verification set and test set according to the above-mentioned preset ratio. The preset ratio here is 8:1:1, and the self-training program is started to train the initial recognition model to obtain the updated recognition model, which can be updated.

[0032] As an embodiment of the present invention, the specific steps of identifying the sample to be tested are as follows:

[0033] Place the sample to be tested on the photo background plate of the collection terminal, and use the negative pressure device of the collection terminal to make the sample flat and adsorbed on the photo background plate. Use the photo control key of the collection terminal to take a photo, and upload the collected photo to the model of the machine learning platform for recognition through the edge computing smart box of the collection terminal. After recognition, the recognition result will be fed back to the collection terminal, and the result will be displayed on the display screen of the collection terminal.

[0034] In this way, the negative pressure device allows the sample to be flatly adsorbed on the background plate for photography, solving the problem of sample tilt and deformation during photography. Especially for leaf samples, the leaf can be fixed without damaging the surface of the leaf, and the entire leaf image information can be clearly and completely photographed, ensuring that the photography results are accurate and fast. The edge computing smart box can perform simple calculations locally.

[0035] As an embodiment of the present invention, the acquisition terminal includes a shooting component, a display screen, a shell, a negative pressure device, a photo background plate and an edge computing smart box. The shooting component is arranged above the shell, the photo background plate is arranged on the shell, the shooting component is arranged toward the photo background plate, and the photo background plate is provided with more than two through holes. The negative pressure device is arranged inside the shell, and the negative pressure device provides negative pressure for the more than two through holes. The shooting component is electrically connected to the edge computing smart box, the edge computing smart box is electrically connected to the negative pressure device, and the edge computing smart box is communicatively connected to an external machine learning platform.

[0036] In this way, by providing two or more through holes on the photographing background plate, and the negative pressure device provides negative pressure for the two or more through holes, the sample can be flatly adsorbed on the photographing background plate, solving the problem of tilt and deformation of the sample during photography. Especially for leaf samples, the leaf can be fixed without damaging the leaf surface.

[0037] As an embodiment of the present invention, the acquisition terminal also includes a lifting bracket, a coarse focus knob and a fine focus knob. The shooting component is arranged on the lifting bracket. The coarse focus knob is used to adjust the distance between the shooting component and the shooting background board over a long distance. The fine focus knob is used to adjust the distance between the shooting component and the shooting background board over a short distance.

[0038] In this way, the shooting component can be adjusted over a long distance through the coarse focus knob, and the focal length can be fine-tuned through the fine focus knob.

[0039] As an embodiment of the present invention, the step of photographing and collecting data by the acquisition terminal is performed simultaneously with the step of identifying the sample to be detected. When the identified sample is greater than a preset value, the acquired image data set and the preset proportion are added to the training set, verification set and test set, and the self-training program is started.

[0040] In this way, the steps of photographing and collecting data by the collection terminal and identifying the samples to be tested are carried out simultaneously. The step of photographing and collecting data by the collection terminal is suitable for maintenance personnel, and the step of identifying the samples to be tested is provided to users. During the user's use, the recognition model can also be optimized to improve the recognition efficiency and accuracy of citrus Huanglongbing disease, and the recognition model can be automatically updated.

[0041] To achieve the above objectives, in a second aspect, the inventors provide a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method steps as described in any one of the above items are implemented.

[0042] Different from the existing technology, the storage medium of the technical solution of this application can improve the accuracy of labeling, continuously update the recognition model to improve the accuracy of the recognition model, improve the recognition efficiency and accuracy of citrus Huanglongbing, and realize automatic update of the recognition model.

[0043] To achieve the above objectives, in a third aspect, the inventors provide a citrus Huanglongbing image acquisition and recognition system, comprising a processor and a storage medium, wherein the storage medium is the storage medium described above;

[0044] The processor is configured to execute the computer program stored in the storage medium to implement the above method steps.

[0045] Different from the existing technology, the system of the technical solution of this application can improve the accuracy of labeling, continuously update the recognition model to improve the accuracy of the recognition model, improve the recognition efficiency and accuracy of citrus Huanglongbing, and realize automatic updating of the recognition model.

[0046] The above-mentioned records related to the content of the invention are only an overview of the technical solution of this application. In order to enable ordinary technicians in this field to understand the technical solution of this application more clearly, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned purposes and other purposes, features and advantages of this application easier to understand, the following is an explanation in combination with the specific implementation methods and drawings of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, characteristics and effects of the specific embodiments of this application and other related contents, and are not to be considered as limiting this application.

[0048] In the drawings of the specification:

[0049] Figure 1 This is a flow chart of a method for collecting and identifying citrus Huanglongbing images according to one embodiment of the present application;

[0050] Figure 2 This is a schematic diagram of the structure of a collection terminal according to an embodiment of the present application;

[0051] Figure 3 This is a schematic diagram of a module of an electronic device according to an embodiment of the present application;

[0052] Figure 4 This is a circuit structure diagram of a collection terminal according to an embodiment of the present application;

[0053] Figure 5 These are images of different symptoms of a sample according to an embodiment of the present application.

[0054] The reference numerals in the above drawings are described as follows:

[0055] 10. Electronic equipment;

[0056] 101. Processor;

[0057] 102. Storage media;

[0058] 20. Shooting component;

[0059] 30. Display screen;

[0060] 40. Shell;

[0061] 41. Power button;

[0062] 42. Photo control key;

[0063] 50. Negative pressure device;

[0064] 60. Photo background board;

[0065] 61, through hole;

[0066] 70. Edge computing smart box;

[0067] 80. Lifting bracket;

[0068] 90. Coarse focus knob. DETAILED DESCRIPTION

[0069] In order to explain in detail the possible application scenarios, technical principles, specific solutions that can be implemented, and the purpose and effects of this application, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of this application and are therefore only examples and are not intended to limit the scope of protection of this application.

[0070] References to "embodiments" herein mean that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the word "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the various technical features mentioned in the embodiments can be combined in any manner to form a corresponding implementable technical solution.

[0071] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms herein is only for describing specific embodiments and is not intended to limit this application.

[0072] In the description of this application, the term "and / or" is used to describe a logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and both A and B exist. In addition, the character " / " in this document generally indicates that the objects before and after are in a logical "or" relationship.

[0073] In this application, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, priority or sequence relationship between these entities or operations.

[0074] Without further limitations, in this application, the words "include", "comprise", "have" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those defined elements, but also other elements not explicitly listed, or elements inherent to such process, method or product.

[0075] Consistent with the understanding in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceed" are understood to exclude the number itself; expressions such as "above," "below," and "within" are understood to include the number itself. Furthermore, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this manner, such as "multiple groups," "multiple times," etc., unless otherwise specifically defined.

[0076] Existing technologies use image recognition algorithms to detect Huanglongbing. However, because deep learning relies on large datasets, and currently few open plant disease and insect pest datasets exist, researchers typically use different network models to find the optimal solution. However, the sample collection environments of some studies do not align with actual field applications, resulting in inconsistent accuracy. A method for collecting and recognizing citrus Huanglongbing images that can automatically update the recognition model is lacking evaluation metrics.

[0077] The applicant's research found that the evaluation indicators F1 value, P value, and R value can be introduced to manually review whether the automatically labeled citrus Huanglongbing symptoms are correct, thereby improving the accuracy of the labeling. When the number of pictures collected by the acquisition terminal is greater than the preset value, the collected image data set is added to the image data set, and then trained through the convolutional neural network model to evaluate the updated recognition model to obtain the updated evaluation indicators F1 value, P value, and R value data values ​​F1-1, P-1, and R-1. When F1-1>F1-0, P-1>P-0, and R-1>R-0 are satisfied at the same time, the verification is passed, and the updated recognition model is used as the final recognition model. Otherwise, the old model remains unchanged, and the recognition model is continuously updated to improve the accuracy of the recognition model, improve the recognition efficiency and accuracy of citrus Huanglongbing, and realize automatic updating of the recognition model.

[0078] This embodiment involves a citrus Huanglongbing image acquisition and recognition method, storage medium, and system thereof, which can be applied to various technical fields of citrus Huanglongbing recognition.

[0079] According to some embodiments of this application, please refer to Figures 1 to 5 This embodiment relates to a method for collecting and identifying citrus Huanglongbing images, comprising the following steps:

[0080] A dataset of images of citrus Huanglongbing symptoms is pre-established and divided into a training set, a validation set, and a test set according to a preset ratio.

[0081] Use the Yolo convolutional neural network model for training to obtain the initial recognition model, and use the test set to evaluate the initial recognition model to obtain the initial values ​​of the evaluation indicators F1 value, P value, and R value F1-0, P-0, and R-0;

[0082] The collection terminal captures and collects data

[0083] Use the initial recognition model to automatically annotate images taken by the acquisition terminal, annotate the symptoms of citrus Huanglongbing on the images, and manually review whether the automatically annotated symptoms are correct.

[0084] When the number of images collected by the acquisition terminal is greater than the preset value, the collected image data set is added to the above-mentioned training set, validation set and test set according to the above-mentioned preset ratio, and the self-training program is started to train the initial recognition model to obtain an updated recognition model;

[0085] Use the updated test set to evaluate the updated recognition model and obtain the updated evaluation index F1 value, P value, and R value data values ​​F1-1, P-1, and R-1. When F1-1>F1-0, P-1>P-0, and R-1>R-0 are satisfied at the same time, the verification passes and the updated recognition model is used as the final recognition model. Otherwise, the old model remains unchanged.

[0086] Identify samples to be tested

[0087] The sample to be tested is placed on the photographing background plate 60 of the acquisition terminal, and a photograph is taken using the photographing control key 42 of the acquisition terminal. The collected photograph is sent to the final recognition model for recognition, and the recognition result is fed back to the acquisition terminal and displayed on the display screen 30.

[0088] Different from the existing technology, the above technical solution introduces the evaluation index F1 value, P value, and R value, and trains the existing image data set of citrus Huanglongbing symptoms through a convolutional neural network model to obtain the initial values ​​F1-0, P-0, and R-0 of the evaluation index F1 value, P value, and R value; uses the initial recognition model to automatically annotate the images taken by the acquisition terminal, and manually reviews whether the automatically annotated citrus Huanglongbing symptoms are correct to improve the accuracy of the annotation. When the number of pictures collected by the acquisition terminal is greater than the preset value, the collected image data set is added to the image data set, and then trained through the convolutional neural network model, and the updated recognition model is evaluated to obtain the updated evaluation index F1 value, P value, and R value data values ​​F1-1, P-1, and R-1. When F1-1>F1-0, P-1>P-0, and R-1> When R-0 is higher than R-0, the verification is passed and the updated recognition model is used as the final recognition model. Otherwise, the old model remains unchanged. Finally, the recognition model is continuously updated to improve the accuracy of the recognition model, improve the recognition efficiency and accuracy of citrus Huanglongbing, and realize automatic update of the recognition model.

[0089] According to some embodiments of the present application, the recognition model optionally selects three indicators, namely, precision P, recall R, and F1 value, as evaluation criteria, and the calculation formulas are as shown in formulas (1)-(3), which are expressed as follows:

[0090]

[0091]

[0092]

[0093] Among them, true positive TP refers to symptoms detected by the algorithm as Huanglongbing, and they are actually Huanglongbing symptoms; false positive FP refers to symptoms detected by the algorithm as Huanglongbing symptoms, but they are not actually Huanglongbing symptoms; false negative FN refers to symptoms detected by the algorithm as non-Huanglongbing symptoms, but they are actually Huanglongbing symptoms.

[0094] Among them, the data of F1 value depends on the values ​​of precision P and recall R.

[0095] In this way, by calculating the precision P, recall R and F1 value to evaluate the accuracy of the recognition model, which can be used as the standard for updating the recognition model, the accuracy of the recognition model can be effectively improved.

[0096] According to some embodiments of the present application, optionally, there are 5 symptoms of Huanglongbing, corresponding to 5 P, R, and F1 values. The symptoms of Huanglongbing are red nose fruit, mottled yellowing, zinc deficiency, yellowing of leaf veins, and uniform yellowing. The initial values ​​F1-0, P-0, and R-0 are the averages of the 5 symptoms, and the data values ​​F1-1, P-1, and R-1 of the updated evaluation indicators F1 value, P value, and R value are also the averages of the 5 symptoms.

[0097] In this way, the symptoms of Huanglongbing are red nose fruit, mottled yellowing, zinc deficiency, yellowing of leaf veins, and uniform yellowing. The initial values ​​F1-0, P-0, and R-0 are the averages of the five symptoms, and the updated evaluation indicators F1 value, P value, and R value data values ​​F1-1, P-1, and R-1 are also the averages of the five symptoms, which can effectively improve the accuracy of the five Huanglongbing symptoms.

[0098] According to some embodiments of the present application, optionally, the specific steps of the acquisition terminal shooting and collecting data are as follows:

[0099] The acquisition terminal takes images of citrus leaves or fruits, and the initial recognition model automatically labels the images photographed by the acquisition terminal, and marks the symptoms of citrus Huanglongbing on the photographed images. The automatically labeled images are manually reviewed to see if the automatically labeled content is correct. If the automatically labeled content is correct, the labeled images are included in the acquisition image data set, and the total number of images in the acquisition image data set is checked to see if it is greater than a preset value. When the total number of images in the acquisition image data set is greater than the preset value, the self-training program is started. After the self-training is completed, an updated recognition model is obtained.

[0100] In this way, the automatically annotated content can be judged through manual review. There is no need to manually annotate each image, only review is required, which can improve the accuracy and efficiency of annotation. When the total number of collected images is greater than the preset value, the preset value here is about 800-1000 photos, then the collected image data set will be added to the above-mentioned training set, verification set and test set according to the above-mentioned preset ratio. The preset ratio here is 8:1:1, and the self-training program is started to train the initial recognition model to obtain the updated recognition model, which can be updated.

[0101] According to some embodiments of the present application, optionally, the specific steps of identifying a sample to be tested are as follows:

[0102] The sample to be tested is placed on the photographing background plate 60 of the acquisition terminal. The negative pressure device 50 of the acquisition terminal is used to make the sample flat and adsorbed on the photographing background plate 60. The photograph is taken through the photographing control key 42 of the acquisition terminal. The collected photograph is uploaded to the model of the machine learning platform through the edge computing smart box 70 of the acquisition terminal for recognition. After recognition, the recognition result is fed back to the acquisition terminal, and the result is displayed on the display screen 30 of the acquisition terminal.

[0103] In this way, the negative pressure device 50 is used to make the sample flat and adsorbed on the photographing background plate 60, which solves the problem of sample tilt and deformation during photography. Especially for leaf samples, the leaf can be fixed without damaging the surface of the leaf, and the image information of the entire leaf can be clearly and completely photographed, ensuring that the photographing results are accurate and fast. The edge computing smart box 70 can perform simple calculations locally.

[0104] According to some embodiments of the present application, optionally, Figure 2 and Figure 4 As shown, the acquisition terminal includes a shooting component 20, a display screen 30, a shell 40, a negative pressure device 50, a photo background plate 60 and an edge computing smart box 70. The shooting component 20 is arranged above the shell 40, the photo background plate 60 is arranged on the shell 40, the shooting component 20 is arranged toward the photo background plate 60, and the photo background plate 60 is provided with more than two through holes 61. The negative pressure device 50 is arranged inside the shell 40, and the negative pressure device 50 provides negative pressure for the more than two through holes 61. The shooting component 20 is electrically connected to the edge computing smart box 70, the edge computing smart box 70 is electrically connected to the negative pressure device 50, and the edge computing smart box 70 is communicatively connected to the external machine learning platform.

[0105] In this embodiment, the housing 40 is provided with a power button 41 for the camera assembly 20, which includes a circular aperture. The housing 40 is also provided with a photo control key 42 for controlling the camera assembly 20 to take photos. The edge computing smart box 70 is a conventional technology, and its principles are not detailed here.

[0106] In this embodiment, the external machine learning platform structures the data recognition model on an external server to facilitate the management of the data recognition model.

[0107] In this way, by providing two or more through holes 61 on the photographing background plate 60, the negative pressure device 50 provides negative pressure for the two or more through holes 61, so that the sample can be flatly adsorbed on the photographing background plate 60, solving the problem of sample tilt and deformation during photography. In particular, for leaf samples, the leaf can be fixed without damaging the leaf surface.

[0108] According to some embodiments of the present application, optionally, the acquisition terminal further includes a lifting bracket 80, a coarse focus knob 90 and a fine focus knob. The shooting component 20 is arranged on the lifting bracket 80. The coarse focus knob 90 is used to adjust the distance between the shooting component 20 and the shooting background plate 60 over a long distance, and the fine focus knob is used to adjust the distance between the shooting component 20 and the shooting background plate 60 over a short distance.

[0109] The principles of the coarse focus knob 90 and the fine focus knob refer to the principles of use of a microscope and will not be elaborated on again.

[0110] In this way, the long-distance adjustment of the shooting component 20 can be performed through the coarse focus knob 90, and the focal length can be fine-tuned through the fine focus knob.

[0111] According to some embodiments of the present application, optionally, the step of capturing the collected data by the acquisition terminal is performed simultaneously with the step of identifying the samples to be detected. When the identified samples are greater than a preset value, a preset proportion of the collected image data set is added to the training set, the verification set, and the test set, and the self-training program is started.

[0112] In this way, the steps of photographing and collecting data by the collection terminal and identifying the samples to be tested are carried out simultaneously. The step of photographing and collecting data by the collection terminal is suitable for maintenance personnel, and the step of identifying the samples to be tested is provided to users. During the user's use, the recognition model can also be optimized to improve the recognition efficiency and accuracy of citrus Huanglongbing disease, and the recognition model can be automatically updated.

[0113] In this embodiment, sample materials for pre-establishing an image dataset of citrus Huanglongbing symptoms were collected from 12 citrus plantations in Fuzhou, Ningde, Nanping, Sanming, Zhangzhou and other places in Fujian Province through specific sample collection. The citrus varieties collected include Lugan, Jiaogan, Wenzhou Mandarin, Wogan, Hongmeiren, Shatangju, Sweet Orange, and Shatian Pomelos. From September to December 2018 to 2021, sample images of Huanglongbing, ulcer disease, yellow spot disease, anthracnose, sooty mold, magnesium deficiency, boron deficiency, moss and healthy plants were collected on summer shoots and autumn shoots. The leaves used for image collection are required to have complete leaf shapes and can unfold naturally, and the fruits are required to be intact and the fruit stalks are clearly visible.

[0114] The weather during sampling was sunny, cloudy or rainy. In order to increase the diversity of samples, some samples were photographed indoors with a white or black board as the background (e.g. Figure 5 (as shown in A5 and A6).

[0115] The original images were processed using Photoshop to JPG format with a resolution of 72 pixels per inch and scaled down to 640 × m or m × 640 pixels (m ≤ 640). To improve sample annotation efficiency, we used HyperSpider LabelTool, a self-developed sample annotation software, to annotate the bounding boxes of leaves and fruits with or without Huanglongbing. The annotations included two types of information: bounding box coordinates and category. During annotation, the minimum bounding rectangle of the citrus leaf or fruit was used as the ground truth box to reduce background pixels within the ground truth box. The annotation files were stored in TXT text format with the same name as the image.

[0116] The symptom categories of this disease are classified by plant protection experts. Based on the differences in leaf and fruit symptoms, 14 subcategories are divided into S1-S14 (Table 1). Detailed information is as follows:

[0117]

[0118] like Figure 5 As shown, A1: healthy leaves (S2); A2: healthy fruits (S1); A3: mottled yellowing (S3); A4: red nose fruits (S4); A5: zinc deficiency (S5); A6: vein yellowing (S6); A7: uniform yellowing (S7); A8: magnesium deficiency (S8); A9: boron deficiency (S9); A10-A11: anthracnose (S10); A12: fat spot yellow spot (S11); A13: moss (S14); A14: sooty mold (S13); A15: canker (S12);

[0119] In a second aspect, the present invention further provides a storage medium storing a computer program, which implements the method steps of the first aspect of the present invention when executed by a processor.

[0120] In the third area, see Figure 3 The present invention also provides an electronic device 10, including a processor 101 and a storage medium 102, wherein the storage medium 102 is the storage medium of the second aspect; the processor 101 is used to execute the computer program stored in the storage medium 102 to implement the method steps of the first aspect.

[0121] Different from the existing technology, the storage medium / system of the technical solution of this application can improve the accuracy of labeling, continuously update the recognition model to improve the accuracy of the recognition model, improve the recognition efficiency and accuracy of citrus Huanglongbing, and realize automatic update of the recognition model.

[0122] Those skilled in the art will appreciate that the above embodiments may be provided as methods, devices, or computer program products. These embodiments may take the form of fully hardware embodiments, fully software embodiments, or embodiments combining software and hardware. All or part of the steps in the methods involved in the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a storage medium readable by a computer device for executing all or part of the steps described in the methods of the above embodiments. The computer device includes, but is not limited to, personal computers, servers, general-purpose computers, special-purpose computers, network devices, embedded devices, programmable devices, smart mobile terminals, etc.; the storage medium includes, but is not limited to, RAM, ROM, flash memory, USB flash drives, mobile hard drives, memory cards, memory sticks, network server storage, network cloud storage, etc.

[0123] The above embodiments are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a computer device to generate a machine, so that the instructions executed by the processor of the computer device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions can also be stored in a computer device readable memory that can guide a computer device to work in a specific manner, so that the instructions stored in the computer device readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer device so that a series of operating steps are executed on the computer device to produce a computer-implemented process, whereby the instructions executed on the computer device provide for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0126] It should be noted that although the above embodiments have been described herein, this does not limit the scope of patent protection of the present invention. Therefore, based on the innovative concept of the present invention, changes and modifications to the embodiments described herein, or equivalent structural or equivalent process transformations made using the contents of the present invention's specification and drawings, and direct or indirect application of the above technical solutions to other related technical fields, are all included in the scope of patent protection of the present invention.

Claims

1. A method for collecting and identifying citrus Huanglongbing images, characterized in that: The following steps are involved: A dataset of images of citrus Huanglongbing symptoms is pre-established and divided into a training set, a validation set, and a test set according to a preset ratio. Use the Yolo convolutional neural network model for training to obtain the initial recognition model, and use the test set to evaluate the initial recognition model to obtain the initial values ​​of the evaluation indicators F1 value, precision P value, and recall rate R value F1-0, P-0, and R-0; The collection terminal shoots and collects data; Use the initial recognition model to automatically annotate images taken by the acquisition terminal, annotate the symptoms of citrus Huanglongbing on the images, and manually review whether the automatically annotated symptoms are correct. When the number of images collected by the acquisition terminal is greater than the preset value, the collected image data set is added to the above-mentioned training set, validation set and test set according to the above-mentioned preset ratio, and the self-training program is started to train the initial recognition model to obtain an updated recognition model; Use the updated test set to evaluate the updated recognition model and obtain the updated evaluation index F1 value, P value, and R value data values ​​F1-1, P-1, and R-1. When F1-1>F1-0, P-1>P-0, and R-1>R-0 are satisfied at the same time, the verification passes and the updated recognition model is used as the final recognition model. Otherwise, the old model remains unchanged. Identify samples to be tested; Place the sample to be tested on the photo background board of the acquisition terminal, take a photo using the photo control key of the acquisition terminal, send the collected photo to the final recognition model for recognition, and feed back the recognition result to the acquisition terminal and display it on the display screen.

2. The citrus Huanglongbing image acquisition and identification method according to claim 1, characterized in that: The recognition model selects three indicators, namely, precision P value, recall R value, and F1 value, as evaluation criteria. The calculation formulas are as follows: （1） （2） （3） Among them, true positive TP refers to symptoms detected by the algorithm as Huanglongbing, and they are actually Huanglongbing symptoms; false positive FP refers to symptoms detected by the algorithm as Huanglongbing symptoms, but they are not actually Huanglongbing symptoms; false negative FN refers to symptoms detected by the algorithm as non-Huanglongbing symptoms, but they are actually Huanglongbing symptoms.

3. The citrus Huanglongbing image acquisition and identification method according to claim 2, characterized in that: There are five symptoms of Huanglongbing, corresponding to five P, R, and F1 values. The symptoms of Huanglongbing are red nose fruit, mottled yellowing, zinc deficiency, yellowing of leaf veins, and uniform yellowing. The initial values ​​F1-0, P-0, and R-0 are the averages of the five symptoms. The updated evaluation indicators F1 value, P value, and R value data values ​​F1-1, P-1, and R-1 are also the averages of the five symptoms.

4. The method for collecting and identifying citrus Huanglongbing images according to claim 1, wherein: The specific steps for the collection terminal to capture and collect data are as follows: The acquisition terminal takes images of citrus leaves or fruits, and the initial recognition model automatically labels the images photographed by the acquisition terminal, and marks the symptoms of citrus Huanglongbing on the photographed images. The automatically labeled images are manually reviewed to see if the automatically labeled content is correct. If the automatically labeled content is correct, the labeled images are included in the acquisition image data set, and the total number of images in the acquisition image data set is checked to see if it is greater than a preset value. When the total number of images in the acquisition image data set is greater than the preset value, the self-training program is started. After the self-training is completed, an updated recognition model is obtained.

5. The method for collecting and identifying citrus Huanglongbing images according to claim 1, wherein: The specific steps for identifying samples to be tested are as follows: Place the sample to be tested on the photo background plate of the collection terminal, and use the negative pressure device of the collection terminal to make the sample flat and adsorbed on the photo background plate. Use the photo control key of the collection terminal to take a photo, and upload the collected photo to the model of the machine learning platform for recognition through the edge computing smart box of the collection terminal. After recognition, the recognition result will be fed back to the collection terminal, and the result will be displayed on the display screen of the collection terminal.

6. The method for collecting and identifying citrus Huanglongbing images according to claim 5, characterized in that: The acquisition terminal includes a shooting component, a display screen, a shell, a negative pressure device, a photo background plate and an edge computing smart box. The shooting component is arranged above the shell, the photo background plate is arranged on the shell, the shooting component is arranged toward the photo background plate, and the photo background plate is provided with more than two through holes. The negative pressure device is arranged inside the shell, and the negative pressure device provides negative pressure for the more than two through holes. The shooting component is electrically connected to the edge computing smart box, the edge computing smart box is electrically connected to the negative pressure device, and the edge computing smart box is communicatively connected to an external machine learning platform.

7. The method for collecting and identifying citrus Huanglongbing images according to claim 6, characterized in that: The acquisition terminal also includes a lifting bracket, a coarse focus knob and a fine focus knob. The shooting component is arranged on the lifting bracket. The coarse focus knob is used to adjust the distance between the shooting component and the shooting background plate over a long distance. The fine focus knob is used to adjust the distance between the shooting component and the shooting background plate over a short distance.

8. The method for collecting and identifying citrus Huanglongbing images according to any one of claims 1 to 7, characterized in that: The step of capturing and collecting data by the acquisition terminal is performed simultaneously with the step of identifying the samples to be detected. When the identified samples are greater than a preset value, the acquired image data set and the preset proportion are added to the training set, the verification set and the test set, and the self-training program is started.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 8 are implemented.

10. A citrus Huanglongbing image acquisition and recognition system, characterized in that: comprising a processor and a storage medium, wherein the storage medium is the storage medium according to claim 9; The processor is configured to execute the computer program stored in the storage medium to implement the method steps according to any one of claims 1 to 8.

Images