Crop disease and insect pest identification method and device and prevention and control scheme generation method and device

Abstract

The invention provides a crop disease and insect pest recognition method and device and a prevention and control scheme generation method and device, and belongs to the field of agricultural disease and insect pest prevention and control. Performing pest target segmentation on the to-be-identified pest image based on the large segmentation model to obtain a segmentation mask; and performing pest type identification according to the segmentation mask and the to-be-identified pest image to obtain a pest classification result. According to the method, the contour of a pest and disease target, especially a small target, is efficiently and accurately identified and segmented by segmenting the image segmentation capability of the large model, the corresponding segmentation mask is generated, the type identification is performed according to the obtained segmentation mask result, the position of the pest and disease target is effectively focused in the type identification process, and the identification efficiency is improved. In this way, the image segmentation capability of a segmentation large model and the classification capability of an identification model are combined, the advantages of different models are fully utilized, and the detection effect on a small target is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural pest control, and in particular, to a method for identifying crop pests and diseases, a method for generating a control plan, and a device therefor. Background Art

[0002] In agricultural production, pests and diseases pose extremely serious hazards to crops. Pests and diseases can not only directly damage parts such as the leaves and fruits of crops, affecting the normal growth of crops, but also cause large-scale crop yield reduction or quality decline by spreading diseases. With the expansion of agricultural production scale and the refined requirements of production management, how to identify and control pests and diseases in a timely and accurate manner has become the key to ensuring the efficient and sustainable development of agricultural production.

[0003] With the development of deep learning in computer vision, various models with convolutional neural networks as the core can extract features from a large amount of crop pest and disease image data through automatic feature extraction and data-driven learning methods, identify and classify different types of pests and diseases. However, traditional detection algorithms mostly rely on detection frameworks based on feature pyramids. By identifying appropriate detection frames and then classifying and identifying based on the detection frames, that is, first dividing the image into several regions to identify the regions where target objects exist, or searching for regions where target objects exist through sliding windows of different sizes. However, in the identification of pests and diseases, due to reasons such as the development stage of pests and diseases, shooting distance, and angle, there are often some small pest and disease targets. However, due to changes in environmental conditions (such as light, humidity, etc.) and shooting angles, it is often impossible to obtain sufficiently clear and detailed crop images. When facing small pest and disease targets, existing detection algorithms often cannot extract sufficiently detailed local information from feature maps with low resolution. Therefore, it is difficult to accurately find the detection frames where small pest and disease targets exist, and at the same time, when classifying and identifying small targets within the detection frames, the accuracy of identification cannot be guaranteed, resulting in missed detections or false detections of small target pests and diseases.

[0004] Therefore, the existing technology has the technical problem that small target pests and diseases cannot be accurately identified, and missed detections or false detections are likely to occur, which needs to be improved. Summary of the Invention

[0005] In view of this, it is necessary to provide a method for identifying crop pests and diseases, a method for generating a control plan, and a device therefor, which are used to solve the technical problem in the existing technology that small target pests and diseases cannot be accurately identified, and missed detections or false detections are likely to occur.

[0006] To solve the above technical problems, in a first aspect, the present invention provides a method for identifying crop pests and diseases, including: Input the obtained pest and disease image to be recognized into the trained pest and disease control network, and perform pest and disease target segmentation on the pest and disease image to be recognized based on the segmentation large model to obtain a segmentation mask; Perform pest and disease type recognition based on the segmentation mask and the pest and disease image to be recognized to obtain a pest and disease classification result.

[0007] In a possible implementation, the pest and disease image to be recognized includes a crop image and a prompt message; performing pest and disease target segmentation on the pest and disease image to be recognized based on the segmentation large model to obtain a segmentation mask includes: Perform feature encoding on the crop image and the prompt message to obtain image features and prompt features; Perform attention-enhanced feature decoding on the image features according to the prompt features to obtain a segmentation mask.

[0008] In a possible implementation, performing pest and disease type recognition based on the segmentation mask and the pest and disease image to be recognized to obtain a pest and disease classification result includes: Perform pest and disease target segmentation on the pest and disease image according to the segmentation mask and the pest and disease image to be recognized to obtain a pest and disease segmentation image; Perform multi-scale feature extraction, feature enhancement fusion, and classification prediction on the pest and disease segmentation image in sequence to obtain a pest and disease classification result.

[0009] In a possible implementation, the trained pest and disease control network is obtained by training an initial pest and disease control network. The initial pest and disease control network includes a pest and disease target segmentation module and a pest and disease type recognition module. Training the initial pest and disease control network includes: Determine the target segmentation loss according to the segmentation mask, and determine the classification loss according to the pest and disease classification result; Perform low-rank adaptive parameter fine-tuning on the pest and disease target segmentation module according to the target segmentation loss, and iteratively train the pest and disease type recognition module according to the classification loss.

[0010] In a second aspect, the present invention provides a method for generating a control plan, including: Determine a pest and disease classification result based on the crop pest and disease recognition method, perform corpus retrieval enhancement on the pest and disease classification result based on a preset corpus to obtain an enhanced query statement, and perform question and answer output on the enhanced query statement based on a large language model to obtain a pest and disease control plan output result; Among them, the crop pest and disease recognition method is the crop pest and disease recognition method in any of the above implementations.

[0011] In a possible implementation, performing corpus retrieval enhancement on the pest and disease classification result based on a preset corpus to obtain an enhanced query statement includes: Construct an initial query statement according to the classification results of pests and diseases, and perform query expansion on the initial query statement to obtain an expanded query statement; Perform document similarity matching between the expanded query statement and a preset corpus to obtain relevant documents; Perform sentence similarity matching between the relevant documents and the expanded query statement to obtain the sentence similarity between each sentence in the relevant documents and the expanded query statement; Determine the abstract sampling probability according to the sentence similarity, perform abstract sampling on the relevant documents according to the abstract sampling probability to obtain corresponding compressed abstract information, and splice the compressed abstract information and the initial query statement to obtain an enhanced query statement.

[0012] In a possible implementation manner, perform question-and-answer output on the enhanced query statement based on a large language model to obtain the output result of the pest and disease control plan, including: Perform prompt adjustment on the enhanced query statement to obtain an optimized query statement, and perform question-and-answer output on the optimized query statement based on a large language model to obtain the output result of the pest and disease control plan.

[0013] Thirdly, the present invention also provides a device for identifying crop pests and diseases, including: A target segmentation unit, configured to input the acquired pest and disease image to be identified into a trained pest and disease control network, and perform pest and disease target segmentation on the pest and disease image to be identified based on a segmentation large model to obtain a segmentation mask; A type recognition unit, configured to perform pest and disease type recognition according to the segmentation mask and the pest and disease image to be identified to obtain the classification result of the pests and diseases.

[0014] Fourthly, the present invention also provides a device for intelligent identification of pests and diseases, including a processor and a memory, wherein, The memory is used to store programs; The processor is coupled to the memory and is configured to execute the programs stored in the memory to implement the steps in the method for identifying crop pests and diseases in any of the above implementation manners and / or the steps in the method for generating a control plan in any of the above implementation manners.

[0015] Fifthly, the present invention also provides a computer-readable storage medium, configured to store computer-readable programs or instructions, and when the programs or instructions are executed by a processor, they can implement the steps in the method for identifying crop pests and diseases in any of the above implementation manners and / or the steps in the method for generating a control plan in any of the above implementation manners.

[0016] The beneficial effects of the present invention are as follows: The method for identifying crop pests and diseases provided by the present invention first performs large model segmentation on the pest and disease targets, and can utilize the image segmentation ability of the segmentation large model to efficiently and accurately identify and segment the outlines of pest and disease targets, especially small targets, and generate corresponding segmentation masks. Then, type recognition is performed based on the obtained segmentation mask results. During the type recognition process, the recognition object is effectively focused on the position of the pest and disease targets, effectively improving the detection accuracy of pest and disease targets, especially small targets. By combining the image segmentation ability of the segmentation large model and the classification ability of the recognition model, the advantages of different models are fully utilized, and the detection effect of small targets is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of an embodiment of the method for identifying crop pests and diseases provided by the present invention; Figure 2 For the present invention Figure 1 It is a schematic flowchart of an embodiment of S101 in the present invention; Figure 3 For the present invention Figure 1 It is a schematic flowchart of an embodiment of S102 in the present invention; Figure 4 It is a schematic flowchart of an embodiment of training the initial pest and disease control network in the embodiment of the present invention; Figure 5 It is a schematic flowchart of the corpus retrieval enhancement in the embodiment of the present invention; Figure 6 It is a schematic structural diagram of an embodiment of the crop pest and disease identification device provided by the present invention; Figure 7 It is a schematic structural diagram of an embodiment of the intelligent pest and disease identification device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] In the embodiments of the present invention, the descriptions such as "first", "second", etc. involved are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Therefore, the technical features defined with "first" and "second" may explicitly or implicitly include at least one such feature.

[0022] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present invention. The phrase appearing at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0023] Figure 1 It is a schematic flowchart of an embodiment of the crop pest and disease identification method provided by the present invention. As Figure 1 shown, the crop pest and disease identification method includes: S101. Input the obtained pest and disease image to be recognized into a well-trained pest and disease control network, and perform pest and disease target segmentation on the pest and disease image to be recognized based on the segmentation large model to obtain a segmentation mask; In some embodiments of the present invention, the pest and disease image to be recognized can be an original crop image captured based on an intelligent monitoring camera, or a crop image with prompt information (such as marked points, marked frames, or rough masks) provided by the user. The pest and disease target segmentation module can be built based on the SAM (Segment Anything Model) large model. Compared with the traditional method of searching for target areas with detection frames, the SAM large model has efficient and accurate target segmentation capabilities, can effectively identify the contours of each pest and disease target, especially small targets, and generate a segmentation mask of its location.

[0024] S102. Perform pest and disease type identification based on the segmentation mask and the pest and disease image to be recognized to obtain a pest and disease classification result.

[0025] In some embodiments of the present invention, the pest and disease type recognition module can be improved and built based on YOLOv9. In the embodiments, the bounding box prediction part in the traditional YOLOv9 is removed, and only the network layers related to classification are retained, so as to concentrate computing resources on class prediction, improve classification efficiency and detection accuracy. At the same time, the improved recognition module uses the segmentation mask result obtained by the segmentation module to identify the type of pest and disease targets. During the recognition process, the recognition object is effectively focused on the position of the pest and disease targets, effectively improving the detection accuracy of pest and disease targets, especially small targets, and significantly enhancing the recognition ability of small target pests and diseases under complex backgrounds.

[0026] In summary, the method for identifying crop pests and diseases provided by the present invention can, by first segmenting pest and disease targets with a large model, efficiently and accurately identify and segment the contours of pest and disease targets, especially small targets, by using the image segmentation ability of the segmentation large model, and generate corresponding segmentation masks; then perform type recognition according to the obtained segmentation mask results, and effectively focus the recognition object on the position of the pest and disease targets during the type recognition process, effectively improving the detection accuracy of pest and disease targets, especially small targets; thereby combining the image segmentation ability of the segmentation large model and the classification ability of the recognition model, making full use of the advantages of different models and enhancing the detection effect on small targets.

[0027] In some embodiments of the present invention, as Figure 2 shown, step S101 includes: S201. Perform feature encoding on the crop image and the prompt information to obtain image features and prompt features; S202. Perform attention-enhanced feature decoding on the image features according to the prompt features to obtain a segmentation mask.

[0028] Among them, the SAM large model for building the pest and disease target segmentation module is implemented based on a typical encoder-decoder structure, and its input includes each crop image in the image dataset and the prompt information provided by the user (such as points, boxes or rough masks marked by the user). These prompt information can guide the model to focus on the characteristic regions in the image, so as to complete the segmentation task.

[0029] In the feature encoding stage, for the crop image , in the embodiments, feature extraction is performed through an image encoder to generate a high-dimensional feature representation . These feature representations retain the semantic information of the image, can capture the details of the image structure and target regions, and lay a foundation for subsequent segmentation tasks. For the prompt information , in the embodiments, it is converted into a feature representation through a prompt encoder , which is used to subsequently guide the model to focus on the target area specified by the user. The encoder is prompted to set multiple encoding processing methods to flexibly handle various prompt types, thereby adapting to different segmentation tasks.

[0030] In the feature decoding stage, the image features and the prompt features are input into the mask decoder , generating the final segmentation mask. This process combines the global semantic information of the image and the user's prompt information. The mask generation formula is expressed as:

[0031] Among them, the core task of the mask decoder is to input and into it, apply the prompt information to the image features using the attention mechanism to locate the target area, and generate the final high-resolution segmentation mask through layer-by-layer decoding operations . Among them, the mask is a matrix that defines the attribution relationship of each pixel in the image.

[0032] In some embodiments of the present invention, as Figure 3 shown, step S102 includes: S301. Perform pest and disease target segmentation on the segmentation mask and the pest and disease image to be recognized to obtain the pest and disease segmentation image; S302. Perform multi-scale feature extraction, feature enhancement and fusion, and classification prediction on the pest and disease segmentation image in sequence to obtain the pest and disease classification result.

[0033] Among them, the pest and disease type recognition module is built based on YOLOv9. The traditional design of YOLOv9 includes three parts: feature extraction, bounding box detection, and classification. In the embodiment, the bounding box detection part of YOLOv9 is removed, and only the layers related to classification are retained, so that the computing resources are concentrated on class prediction, significantly improving the classification efficiency.

[0034] In the pest and disease type recognition module, the embodiment first performs preprocessing according to the segmentation mask obtained by the pest and disease target segmentation module. According to the segmentation mask, first extract the minimum bounding rectangle of each pest and disease target, and accurately calculate the boundary information of the target. Subsequently, convert this boundary information into annotation data that meets the format requirements of YOLOv9, including the center point coordinates, width, height of the target, and the corresponding class label, providing a standardized input for the subsequent classification task.

[0035] For the pest and disease segmentation image obtained by preprocessing, it is then successively passed through the backbone network layer, the neck module, and the head module for feature extraction and processing. The backbone network extracts the deep semantic features of the image, the neck module further fuses and enhances the multi-scale features, and the head module classifies and predicts the target area and refines the processing, finally generating the classification result of the pests and diseases.

[0036] In some embodiments of the present invention, the trained pest and disease control network is obtained by training the initial pest and disease control network, and the initial pest and disease control network includes a pest and disease target segmentation module and a pest and disease type recognition module. Figure 4 As shown in the schematic flowchart of an embodiment for training the initial pest and disease control network provided by the present invention, Figure 4 as shown, training the initial pest and disease control network includes: S401. Determine the target segmentation loss according to the segmentation mask, and determine the classification loss according to the pest and disease classification result; S402. Perform low-rank adaptive parameter fine-tuning on the pest and disease target segmentation module according to the target segmentation loss, and iteratively train the pest and disease type recognition module according to the classification loss.

[0037] In the training stage, for the segmentation task, the target segmentation loss of the model is used to measure the generated mask and the ground truth mask The loss function includes cross-entropy loss and Dice loss, and the formula is expressed as:

[0038] where is the total number of pixels, is the predicted value of the th pixel in the generated mask, is the ground truth segmentation label.

[0039] Then the embodiment uses the LoRA fine-tuning technique to train the SAM large model in the segmentation task. Among them, the core idea of LoRA is to reduce the number of fine-tuning parameters through low-rank matrix decomposition. Introduce low-rank decomposition on the weight matrix of the mask decoder, and the update rule is as follows:

[0040] where is the original weight matrix, with a dimension of , is the low-rank matrix, with a dimension of , is the low-rank matrix, with a dimension of , is the rank of the low-rank matrix, which is significantly smaller than the original dimension of the weight matrix.

[0041] Through this decomposition form, the amount of parameter updates can be effectively reduced, thereby reducing the computational cost required for training while ensuring that the model performance is not significantly affected.

[0042] During the fine-tuning process, freeze the image encoder and prompt encoder of the SAM model, and only optimize the newly added low-rank parameters of the mask decoder and This not only retains the generalization ability of the pre-trained model but also enhances the segmentation performance of the mask decoder.

[0043] For the type recognition task, for the modified YOLOv9, the loss function is defined as:

[0044] where represents the number of grids, is the number of classes, is the class probability predicted by the model, is the true label, is the weight parameter of the classification loss.

[0045] This loss function optimizes the classification performance by measuring the squared error between the class probability predicted by the model and the true class label. Compared with the traditional cross-entropy loss, this design can better adapt to the simplified structure of the classification module.

[0046] In addition, considering that in the existing crop pest and disease identification methods, it is often only possible to identify the types of pests and diseases, but it is not possible to intelligently generate corresponding detailed control plans based on the identification results, which is difficult to meet the application requirements of large-scale modern agricultural production management. Therefore, the present invention also provides a control plan generation method to provide an intelligent control plan as a reference for large-scale agricultural production management.

[0047] In some embodiments of the present invention, the control plan generation method includes: Determine the pest and disease classification result based on the crop pest and disease identification method, perform corpus retrieval enhancement on the pest and disease classification result based on a preset corpus to obtain an enhanced query statement, and perform question-and-answer output on the enhanced query statement based on a large language model to obtain the output result of the pest and disease control plan; wherein, the crop pest and disease identification method is the crop pest and disease identification method in any of the above implementation manners.

[0048] Among them, in order to provide a more detailed and accurate prevention and control plan, the embodiment constructs an enhanced query statement for the large language model by means of enhancing corpus retrieval of a preset corpus, and generates an accurate and detailed pest and disease prevention and control plan according to the large language model Q&A output based on the enhanced query statement.

[0049] In order to make the generated prevention and control plan scientific and effective, the preset corpus includes but is not limited to a large number of prevention and control plans, pest and disease characteristics, and crop type information. The embodiment generates a query statement for inputting into the large language model based on the preset corpus and in combination with the method of enhancing corpus retrieval, and can automatically generate accurate and detailed prevention and control suggestions according to the pest type, crop type, and environmental factors, providing an effective pest and disease prevention and control reference plan for the intelligent management of agricultural production.

[0050] In some embodiments of the present invention, Figure 5 is a schematic flowchart of the enhanced corpus retrieval of the embodiment of the present invention, as Figure 5 shown, enhancing the corpus retrieval of the pest and disease classification result based on the preset corpus to obtain an enhanced query statement, including: S501. Construct an initial query statement according to the pest and disease classification result, and perform query expansion on the initial query statement to obtain an expanded query statement; S502. Perform document similarity matching between the expanded query statement and the preset corpus to obtain relevant documents; S503. Perform sentence similarity matching between the relevant documents and the expanded query statement to obtain the sentence similarity between each sentence in the relevant documents and the expanded query statement; S504. Determine the summary sampling probability according to the sentence similarity, perform summary sampling on the relevant documents according to the summary sampling probability to obtain the corresponding compressed summary information, and splice the compressed summary information and the initial query statement to obtain an enhanced query statement.

[0051] In some embodiments of the present invention, performing Q&A output on the enhanced query statement based on the large language model to obtain the pest and disease prevention and control plan output result, including: Performing prompt adjustment on the enhanced query statement to obtain an optimized query statement, and performing Q&A output on the optimized query statement based on the large language model to obtain the pest and disease prevention and control plan output result.

[0052] Among them, in order to automatically generate a corresponding prevention and control plan according to the pest and disease classification result, the embodiment constructs a prevention and control plan generation module based on the large language model. The prevention and control generation module constructs a complete link from the user's question to the final answer by combining enhanced corpus retrieval, the Qwen Qianwen large model, and the Prompt Tuning method, realizing efficient information processing and accurate answer generation.

[0053] Among them, in the embodiment, considering that the results obtained by only performing large language model questions and answers based on the obtained pest and disease classification results are often not detailed enough, a step of corpus retrieval enhancement is designed. First, the embodiment constructs a detailed knowledge base related to pests and diseases, vectorizes it and stores it in a vector database to obtain a preset corpus containing rich prevention and control knowledge.

[0054] Then, an initial query statement is constructed according to the pest and disease classification result After that, through the generation model The initial query statement Is expanded into multiple hypothetical documents to achieve pre-retrieval enhancement of the query statement:

[0055] Among them, Is the user's original query, Is the generation instruction, such as "write a paragraph answering the question", Is the generation model, Is the hypothetical document.

[0056] Then, the expanded hypothetical documents are combined into an expanded query statement And passed to the retrieval stage to improve the coverage and accuracy of matching with the knowledge base.

[0057] In the retrieval stage, first calculate the similarity between the expanded query And each document vector :

[0058] Among them, Is the expanded query vector, Is the document 's embedding vector.

[0059] Then, sort according to the similarity and select the top Documents:

[0060] Among them Is the set of relevant documents retrieved, and the retrieval result Is passed to the post-retrieval enhancement stage to further compress the information content.

[0061] In the post-retrieval enhancement stage, the embodiment calculates the sentence similarity between the sentences in the document and the query sentence one by one:

[0062] Among them, Is the vector representation of the sentence.

[0063] Then, a generation model is used according to the sentence similarity to generate a concise summary :

[0064] where is the probability of the generation model generating a summary under the given document and query conditions

[0065] Then, the compressed summary information obtained by compression is concatenated with the initial query statement and input into the Qwen Qianwen large model for subsequent steps

[0066] In addition, in order to further optimize the output results of the Qianwen large model and improve the accuracy and consistency of the generated results, the embodiment adopts the Prompt Tuning method to optimize the Qianwen large model, specifically as follows By analyzing the knowledge characteristics in the field of crop pests and diseases, a Prompt structure suitable for the pest and disease knowledge Q&A task is designed. The initial Prompt can be expressed as

[0067] where represents the input initial query statement (e.g., "How to control aphids?") is the relevant background information after compression (e.g., "Aphids are common crop pests, and their prevention measures include...")

[0068] The parameters of the Prompt part can be fine-tuned so that the generation model can better understand the context of a specific field. We designed a fine-tuning function for adjusting the representation of the Prompt to optimize the answer generated by the model

[0069] where represents the optimized Prompt representation is the model parameter is the trainable fine-tuning function

[0070] The optimized Prompt is used as the input to the Qianwen large model, and finally the relevant answer is generated , and the generation process can be expressed as

[0071] where is the answer generated by the model is the Qianwen large model function

[0072] To optimize the Prompt representation during Prompt-Tuning, the embodiment defines a loss function for measuring the generated answer and the target answer The gap between them is represented by the following loss function:

[0073] The loss function The optimized form can be expressed as:

[0074] where is the optimized Prompt representation, is the parameter of the optimized Qianwen large model.

[0075] In addition, to enhance the model's generalization across different types, the present invention has higher accuracy and adaptability in identifying diverse pest types. During the data preparation stage, based on the publicly available dataset IP102 and combined with the images taken, the embodiment collected a total of over 80,000 crop pest images. These images cover 102 common pest types, including aphids, spider mites, leaf rollers, etc., to form the dataset IP102-Extended (IP102-E). The dataset is strictly classified according to pest types, and each image is accompanied by detailed label information, including the pest category and the target location (bounding box annotation or segmentation mask). In addition, to improve the model's generalization ability, the embodiment adopted various data augmentation techniques such as rotation, scaling, cropping, color jittering, and blurring on the dataset to generate diverse samples to simulate the pest distribution and changes in real scenarios. Finally, a dataset for training the modified YOLOv9 in the type recognition module was obtained. Further, to fine-tune the SAM segmentation module, based on the aforementioned dataset, in the format of the VOC2007 dataset (a classic dataset in the field of image segmentation), the embodiment performed segmentation processing using three classification models, YOLO, U-2-Net, and mask-CNN, to generate candidate segmentation results. Then, through manual screening and correction, the optimal mask was selected from multiple segmentation results, and all segmentation samples were formatted. Finally, a dataset AAG2024 for training and fine-tuning the SAM segmentation module was constructed. In addition, the embodiment also created a database containing 3,101 diseases, 3,106 pests, and 534 grass pests based on the Agricultural Sub-Center of the China National Knowledge Infrastructure for Engineering Science and Technology, and vectorized it and stored it in the vector database, laying a good foundation for the retrieval enhancement part of the large model in the intelligent pest knowledge Q&A system.

[0076] In summary, in order to accurately identify small target pests and diseases in crop images, the present invention first inputs the obtained pest and disease image to be identified into a well-trained pest and disease control network, and performs pest and disease target segmentation on the pest and disease image to be identified based on a segmentation large model to obtain a segmentation mask; then, according to the segmentation mask and the pest and disease image to be identified, pest and disease type recognition is performed to obtain a pest and disease classification result. By first segmenting the pest and disease targets with a large model, the present invention can efficiently and accurately identify and segment the outlines of pest and disease targets, especially small targets, by using the image segmentation ability of the segmentation large model, and generate corresponding segmentation masks; then, type recognition is performed according to the obtained segmentation mask results, and the recognition object is effectively focused on the position of the pest and disease target during the type recognition process, effectively improving the detection accuracy of pest and disease targets, especially small targets; in this way, by combining the image segmentation ability of the segmentation large model and the classification ability of the recognition model, the advantages of different models are fully utilized, and the detection effect of small targets is enhanced.

[0077] In order to better implement the crop pest and disease recognition method in the embodiments of the present invention, correspondingly, on the basis of the crop pest and disease recognition method, as Figure 6 shown, the embodiments of the present invention further provide a crop pest and disease recognition device. The crop pest and disease recognition device 600 includes: A target segmentation unit 601, configured to input the obtained pest and disease image to be identified into a well-trained pest and disease control network, and perform pest and disease target segmentation on the pest and disease image to be identified based on a segmentation large model to obtain a segmentation mask; A type recognition unit 602, configured to perform pest and disease type recognition according to the segmentation mask and the pest and disease image to be identified to obtain a pest and disease classification result.

[0078] The above-mentioned crop pest and disease recognition device 600 provided in the above embodiments can implement the technical solutions described in the above embodiments of the crop pest and disease recognition method. The specific implementation principles of the above modules or units can be referred to the corresponding content in the above embodiments of the crop pest and disease recognition method, and will not be elaborated here.

[0079] As Figure 7 shown, the present invention also correspondingly provides a pest and disease intelligent recognition device 700. The pest and disease intelligent recognition device 700 includes a processor 701, a memory 702, and a display 703. Figure 7 Only some components of the pest and disease intelligent recognition device 700 are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0080] In some embodiments, the processor 701 may be a central processing unit (CPU), a microprocessor, or other data processing chips, which are used to run the program code stored in the memory 702 or process data, such as the crop pest and disease identification method and / or the prevention and control plan generation method in the present invention.

[0081] In some embodiments, the processor 701 may be a single server or a server group. The server group can be centralized or distributed. In some embodiments, the processor 701 can be local or remote. In some embodiments, the processor 701 can be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination of the above.

[0082] In some embodiments, the memory 702 may be an internal storage unit of the pest and disease intelligent identification device 700, such as the hard disk or memory of the pest and disease intelligent identification device 700. In some other embodiments, the memory 702 may also be an external storage device of the pest and disease intelligent identification device 700, such as a plug-in hard disk equipped on the pest and disease intelligent identification device 700, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0083] Furthermore, the memory 702 may also include both the internal storage unit and the external storage device of the pest and disease intelligent identification device 700. The memory 702 is used to store the application software installed on the pest and disease intelligent identification device 700 and various types of data.

[0084] In some embodiments, the display 703 may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (organic light-emitting diode) toucher, etc. The display 703 is used to display the information of the pest and disease intelligent identification device 700 and to display a visual user interface. The components 701-703 of the pest and disease intelligent identification device 700 communicate with each other through a system bus.

[0085] In one embodiment, when the processor 701 executes the crop pest and disease identification program in the memory 702, the following steps can be achieved: Input the obtained pest and disease image to be identified into a trained pest and disease prevention and control network, and perform pest and disease target segmentation on the pest and disease image to be identified based on a segmentation large model to obtain a segmentation mask; Perform pest and disease type identification based on the segmentation mask and the pest and disease image to be identified to obtain a pest and disease classification result.

[0086] In another embodiment, when the processor 701 executes the prevention and control plan generation program in the memory 702, the following steps can be implemented: Determine the pest classification result based on the crop pest and disease identification method, perform corpus retrieval enhancement on the pest classification result based on a preset corpus to obtain an enhanced query statement, and perform question-and-answer output on the enhanced query statement based on a large language model to obtain the pest control plan output result.

[0087] It should be understood that when the processor 701 executes the crop pest and disease identification program in the memory 702, in addition to the above functions, other functions can also be implemented. For specific details, reference can be made to the description of the corresponding method embodiments above.

[0088] Furthermore, the embodiments of the present invention do not specifically limit the type of the pest and disease intelligent identification device 700 mentioned. The pest and disease intelligent identification device 700 can be a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, or other portable electronic devices. Exemplary embodiments of the portable electronic device include, but are not limited to, portable electronic devices equipped with IOS, android, microsoft, or other operating systems. The above portable electronic devices can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (such as a touch panel). It should also be understood that in some other embodiments of the present invention, the pest and disease intelligent identification device 700 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (such as a touch panel).

[0089] Correspondingly, an embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium is used to store computer-readable programs or instructions. When the programs or instructions are executed by a processor, the steps or functions in the crop pest and disease identification method and / or prevention and control plan generation method provided by the above method embodiments can be implemented.

[0090] Those skilled in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The computer program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a magnetic disk, an optical disk, a read-only memory, or a random access memory, etc.

[0091] The above has introduced in detail the method for identifying crop pests and diseases, the method for generating prevention and control solutions, the device, the intelligent identification device and the storage medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for identifying crop pests and diseases, characterized in that: include: Input the acquired pest and disease image to be identified into a well-trained pest and disease control network, and perform pest and disease target segmentation on the pest and disease image to be identified based on the large segmentation model to obtain a segmentation mask; The pest type is identified based on the segmentation mask and the pest image to be identified to obtain a pest classification result.

2. The method for identifying crop diseases and insect pests according to claim 1, characterized in that: The pest image to be identified includes a crop image and prompt information; the pest target segmentation of the pest image to be identified based on the large segmentation model to obtain a segmentation mask includes: Performing feature encoding on the crop image and the prompt information to obtain image features and prompt features; Attention-enhanced feature decoding is performed on the image features according to the prompt features to obtain a segmentation mask.

3. The method for identifying crop diseases and insect pests according to claim 1, characterized in that: The step of performing pest and disease type identification according to the segmentation mask and the pest and disease image to be identified to obtain a pest and disease classification result includes: Performing pest and disease target segmentation according to the segmentation mask and the pest and disease image to be identified to obtain a pest and disease segmentation image; The pest and disease segmentation image is sequentially subjected to multi-scale feature extraction, feature enhancement fusion and classification prediction to obtain a pest and disease classification result.

4. The method for identifying crop diseases and insect pests according to claim 1, characterized in that: The fully trained pest control network is obtained by training an initial pest control network, wherein the initial pest control network includes a pest target segmentation module and a pest type recognition module, and the trained initial pest control network includes: Determine the target segmentation loss according to the segmentation mask, and determine the classification loss according to the pest classification result; The low-rank adaptive parameter fine-tuning of the pest target segmentation module is performed according to the target segmentation loss, and the pest type recognition module is iteratively trained according to the classification loss.

5. A method for generating a prevention and control plan, characterized in that: include: Determine a pest classification result based on a crop pest identification method, perform corpus retrieval enhancement on the pest classification result based on a preset corpus to obtain an enhanced query statement, and perform question-answering output on the enhanced query statement based on a large language model to obtain a pest control solution output result; Wherein, the crop disease and insect pest identification method is the crop disease and insect pest identification method according to any one of claims 1 to 4.

6. The method for generating a prevention and control plan according to claim 5, characterized in that: The enhanced query statement obtained by performing corpus retrieval enhancement on the pest classification result based on a preset corpus includes: Constructing an initial query statement according to the pest classification result, and performing query expansion on the initial query statement to obtain an expanded query statement; Perform document similarity matching between the extended query statement and a preset corpus to obtain relevant documents; Perform sentence similarity matching between the relevant document and the extended query sentence to obtain sentence similarity between each sentence in the relevant document and the extended query sentence; Determine a summary sampling probability according to the sentence similarity, perform summary sampling on the relevant documents according to the summary sampling probability to obtain corresponding compressed summary information, and concatenate the compressed summary information and the initial query sentence to obtain an enhanced query sentence.

7. The method for generating a prevention and control plan according to claim 5, characterized in that: The output of the pest control solution by performing question-answering on the enhanced query statement based on the large language model includes: The enhanced query statement is prompted and adjusted to obtain an optimized query statement, and the optimized query statement is question-answered and output based on a large language model to obtain an output result of a pest control solution.

8. A device for identifying crop pests and diseases, characterized in that: include: A target segmentation unit is used to input the acquired pest and disease image to be identified into a well-trained pest and disease control network, and perform pest and disease target segmentation on the pest and disease image to be identified based on the large segmentation model to obtain a segmentation mask; The type recognition unit is used to perform pest and disease type recognition based on the segmentation mask and the pest and disease image to be recognized to obtain a pest and disease classification result.

9. An intelligent pest identification device, characterized in that: comprising a processor and a memory, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the crop disease and pest identification method described in any one of claims 1 to 4 and / or the steps in the control plan generation method described in any one of claims 5 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the crop disease and pest identification method described in any one of claims 1 to 4 and / or the steps of the control plan generation method described in any one of claims 5 to 7.

Images