Pest feature extraction method, system and equipment based on domain knowledge guided large model

Abstract

The invention discloses a pest feature extraction method, system and equipment based on a domain knowledge guide large model. The method comprises the following steps: constructing a standardized pest knowledge base fusing Wikipedia, professional books and papers; a two-stage strategy is adopted, in the first stage, a large language model is used for extracting high-distinction-degree structured feature phrases from a knowledge base, and in the second stage, in combination with pest images, a manual annotation box and feature phrase prompts, individual core features are extracted through a multi-mode large model; a cross-modal cosine similarity alignment and confidence-distinction-matching degree multi-dimensional scoring mechanism is introduced, and high-quality features are screened; the system is used for realizing the pest feature extraction method. According to the method, a large model is guided to focus on key biological characteristics through professional knowledge, accurate and comprehensive automatic extraction is achieved, the accuracy, integrity and reliability of pest characteristic extraction are remarkably improved, high-quality data support is provided for intelligent recognition, and the method is suitable for main crop pests such as corn and wheat.

Description

Technical Field

[0001] This invention belongs to the field of intelligent identification technology for agricultural pests, and in particular relates to a method, system and device for extracting pest features based on a domain knowledge-guided large model. Background Technology

[0002] Agriculture, as a fundamental industry ensuring global food security, directly impacts socio-economic stability and development through its production efficiency and output quality. The widespread spread and infestation of agricultural pests is a core bottleneck restricting agricultural productivity. According to a report by the Food and Agriculture Organization of the United Nations (FAO), global crop yield losses due to pests reach 20%-30% annually. Accurate and efficient pest identification is a crucial prerequisite for building a modern monitoring and early warning system and achieving precise control. With the rapid development of smart agriculture, computer vision (CV), deep learning, and large-scale modeling technologies have become core supports for overcoming the limitations of traditional manual identification and improving the intelligence level of pest identification.

[0003] Driven by computer vision and deep learning technologies, the field of agricultural pest identification has shifted from traditional image processing methods to data-driven intelligent algorithm models. Early methods based on convolutional neural networks (CNNs), such as improved AlexNet and ResNet, achieved initial success in pest image classification tasks due to their powerful local feature extraction capabilities. However, these models suffer from insufficient global information capture and weak adaptability to complex backgrounds, making them ill-suited to real-world challenges such as pest camouflage and morphological variations. In recent years, architectures like Vision Transformer (ViT) and Swin Transformer have implemented global feature modeling through multi-head self-attention mechanisms, while new paradigms based on state-space models (SSMs), such as Mamba, have further enhanced the ability to capture long-distance dependencies, providing a new technical path for fine-grained pest feature extraction. Simultaneously, the rise of large-scale models has provided a powerful impetus for cross-modal information fusion. High-performance large-scale models such as DeepSeek R1, with their excellent generalization capabilities, can integrate multi-source data features, driving pest identification from a single-modal to a multi-modal fusion approach.

[0004] Despite continuous technological iteration, current agricultural pest identification and feature extraction still face numerous data bottlenecks: First, existing models mostly rely on single public datasets (such as IP102), with limited scene coverage and a lack of quality screening mechanisms. This results in insufficient robustness of the models in real-world scenarios such as complex field backgrounds and varying lighting conditions. Furthermore, the data is not deeply integrated with agricultural expertise, making it difficult to support accurate feature characterization. Second, at the feature extraction level, most methods remain at the level of single-modal visual feature extraction or simply splicing cross-modal information. They lack a systematic fusion mechanism guided by professional knowledge, making it impossible to accurately capture the core biological characteristics of pests. This leads to low feature discrimination and difficulty in addressing the challenge of identifying pests with high inter-class similarity.

[0005] Therefore, it is urgent to integrate multi-source data and authoritative professional knowledge, and with the help of large models and multimodal fusion technology, to build a precise, efficient and robust agricultural pest feature extraction system. This system will overcome the limitations of existing technologies in terms of data quality, feature accuracy and engineering practicality, and provide strong data support for core applications such as pest identification, monitoring and early warning in smart agriculture.

[0006] 2) Existing technical solutions Existing pest identification technologies mainly rely on single-modal optical image analysis, or traditional manual feature extraction (such as SIFT, HOG) and deep learning feature extraction methods [Peng Hongxing, Xu Huiming, Liu Huanai. Lightweight crop pest identification model based on improved ShuffleNet V2 [J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(11): 161-170.], [Chen Jiong, Liu Jianhua. Improved YOLOv5 for rice pest detection [J]. Information Technology and Informatization, 2023 (7): 165-171.]. However, there are three core challenges in actual agricultural scenarios: First, the farmland environment is highly dynamic and unstructured. Interference factors such as crops, weeds, soil, shadows, and changes in light lead to cluttered image backgrounds, affecting the accuracy of feature extraction. Second, pests are diverse in species and have great variations in developmental morphology. Different pests have high similarity in morphology and color, and the same pest has significant morphological differences at different growth stages (egg, larva, pupa, and adult), resulting in large intraclass differences and small interclass differences, making feature differentiation difficult. Third, pests vary in size, from millimeter-sized eggs to centimeter-sized adults, and their distribution patterns include sparse single insects and dense clusters, further increasing the difficulty of feature extraction.

[0007] Meanwhile, most existing agricultural pest datasets are based on single-modal optical images [Wang SS, Ni WJ, Zeng QT, et al. APD-229: a textual-visual database for agricultural pests and diseases[J]. Multimedia Tools and Applications, 2024, 83(8): 22189-22220.], [Wu X, Zhan C, Lai YK, et al. Ip102: A large-scale benchmark dataset for insect pest recognition[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: [8787-8796.] Multimodal information fusion lacking professional knowledge support has the following shortcomings: First, the data source is singular, the scene coverage is incomplete, and it is difficult to adapt to complex field environments; Second, the feature description lacks guidance from agricultural professional knowledge, the text feature extraction accuracy is low and the repetition is high, and it cannot effectively characterize the key biological characteristics of pests; Third, no systematic feature quality assessment and dynamic update mechanism has been established, the feature extraction model has poor adaptability, and it is difficult to cope with newly emerging pest samples or environmental changes.

[0008] Furthermore, existing feature extraction methods often rely on a single modality of image or text [Fu X, Ma Q, Yang F, et al. Crop pest image recognition based on the improved ViT method[J]. Information Processing in Agriculture, 2024, 11(2): 249-259.], [Liu J, Zhou C, Zhu Y, et al. RicePest-DETR: A transformer-based model for accurately identifying small rice pest by end-to-end detection mechanism[J]. Computers and Electronics in Agriculture, 2025, 235: 110373..], [Dai G, Fan J, Dewi C. ITF-WPI: Image and text based cross-modal feature fusion model for wolfberrypest recognition[J]. Computers and Electronics in Agriculture, 2023, 212:108129.], failing to fully utilize the complementarity of cross-modal information [Radford A, Kim JW, Hallacy C, et al. Learning transferable visual Models from natural language supervision[C] / / International conference on machine learning. PmLR, 2021: 8748-8763..], [Achiam J, Adler S, Agarwal S, et al. Gpt-4 technical report[J]. arXivpreprint arXiv:2303.08774, 2023.]. Furthermore, the lack of hierarchical characterization of the entire life cycle characteristics of pests leads to incomplete feature descriptions and insufficient robustness in identification. Therefore, there is an urgent need for a feature extraction method that integrates multi-channel data, agricultural expertise, and advanced large-scale modeling techniques to solve the problems of low feature accuracy, insufficient comprehensiveness, and poor adaptability in existing technologies. Summary of the Invention

[0009] To overcome the shortcomings of the existing technologies, the present invention aims to propose a method, system, and device for pest feature extraction based on a domain knowledge-guided large model. By collecting high-quality optical image data through multiple channels, a professional knowledge system for agricultural pests is constructed. A two-stage feature extraction strategy of "professional knowledge guidance + large model empowerment" is adopted, combined with cross-modal feature matching and quality assessment, to achieve accurate and comprehensive extraction of pest features. The method's adaptability is ensured through a dynamic update mechanism, ultimately providing high-quality feature support for pest identification.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for extracting pest features based on domain knowledge-guided large models includes the following steps: S1: Collect basic optical image data of pests through multiple channels, and use clear quantitative screening formulas to control the quality of images, and build an image dataset covering diverse scenes and morphologies; S2: Construct a multi-dimensional, hierarchical professional knowledge collection system, screen pest knowledge from authoritative sources, including integrating pest biological information from Wikipedia, authoritative professional books and historical academic papers, and establish a standardized knowledge database after calibration by agricultural experts; S3: Perform the first stage of feature phrase extraction: Based on the standardized knowledge database established in step S2, use the large language model to extract structured core feature phrases according to the pest life cycle stage, and calculate the feature discrimination to screen high-discrimination features and build a feature phrase database. S4: Perform the second stage of single-target feature extraction: Input basic optical image data of pests that meet the clarity requirements, manually labeled recognition boxes and feature phrases corresponding to the life cycle stage, extract the core features of individual pests through a multimodal large model, and calculate the feature confidence to quantify the reliability of extraction. S5: Cross-modal feature matching and alignment: Based on the cosine similarity model, the matching degree between image feature vectors and text feature vectors is measured, duplicate or invalid information is removed, and preliminary feature description results are generated; S6: Feature Quality Assessment Optimization: The preliminary results are screened by integrating a multi-dimensional comprehensive scoring function that combines confidence, discriminative power, and matching degree, retaining high-value features and outputting the final pest feature description.

[0011] In step S1, the multi-channel collection of basic optical image data of pests includes: based on the IP102 dataset, integrating public images crawled by web crawlers, field images provided by agricultural cooperatives, and high-quality illustrations extracted from color-printed books and academic papers in the agricultural field.

[0012] In step S2, the standardized knowledge database covers various corn and wheat pests, including fall armyworm, armyworm, cotton bollworm, corn borer, two-spotted leaf beetle, aphid, spider mite, and wheat gall midge. It describes the body size, body color, body surface structure, and appendage characteristics of the insects throughout their entire life cycle, from egg to larva to pupa to adult. In addition, an agricultural expert team is formed to conduct field verification and standardize the description of disputed information.

[0013] In step S3, the large language model takes preprocessed professional knowledge text as input and outputs noun phrases containing subjects in the form of "feature subject + feature attribute", and extracts several feature phrases for each type of pest at each life cycle stage.

[0014] In step S4, the multimodal large model takes as input the basic optical image data of pests, manually annotated bounding boxes, and feature phrases obtained in the first stage as prompt information, and drives the multimodal large model to generate at least 5 core features covering the dimensions of morphology, color, and texture.

[0015] In step S5, the image feature vector is extracted through a deep convolutional neural network, the text feature vector is extracted through a pre-trained model, and cosine similarity is used to measure the matching degree between the image feature vector and the text feature vector to determine cross-modal consistency.

[0016] In step S6, the multi-dimensional comprehensive scoring function is defined as:

[0017] in, The feature quality score (value range 0-1). For feature confidence, For feature discriminability, For cross-modal similarity matching, the weighting coefficient is set to... ,satisfy ,according to Sort in descending order, keeping the first few. High-value characteristics To ensure feature quality.

[0018] A pest feature extraction system based on a domain knowledge-guided large model, implementing the pest feature extraction method described in any one of steps S1 to S6, includes: The image acquisition and filtering module is used to perform step S1. The expertise building module is used to execute the S2 steps; The first-stage feature phrase extraction module is used to perform step S3. The second-stage single-target feature extraction module is used to execute step S4. The cross-modal matching and alignment module is used to perform the S5 step; The Feature Quality Assessment Optimization Module is used to execute step S6.

[0019] The system is applied to the intelligent identification of pests in corn and wheat, and extended to the feature extraction of pests in rice and cotton crops. The system is deployed on a PC platform with a high-performance GPU and calls large language models and multimodal large models through API.

[0020] A pest feature extraction device based on a domain knowledge-guided large model, comprising: Memory is used to store computer programs, data, and models; A processor is configured to implement the pest feature extraction method based on a domain knowledge-guided large model as described in any one of steps S1 to S6 when executing the computer program.

[0021] Compared with the prior art, the present invention has the following advantages: 1) More solid data and knowledge foundation, and more reliable feature extraction sources: Addressing the shortcomings of existing technologies such as single data sources, unreliable quality, and lack of professional knowledge support, this invention constructs a dual support system of "high-quality image dataset + standardized knowledge database" through steps S1 and S2. Step S1 adopts a collection scheme of "IP102 dataset as the foundation + multi-channel supplementation," integrating public images from web crawlers, field images, and illustrations from professional books and papers. A clear quantification screening formula is then used to eliminate ambiguous samples, ensuring that the image data covers diverse scenes and forms and is clearly identifiable. Step S2 integrates pest biology information from Wikipedia, authoritative books, and academic papers. Controversial information is calibrated by agricultural experts, and field verification unifies descriptions. Furthermore, a knowledge credibility scoring model is used to filter highly authoritative and timely knowledge, constructing a standardized database covering the core characteristics of the entire life cycle of eight types of corn and wheat pests. The dual system provides precise professional knowledge guidance and high-quality data support for feature extraction. Compared with the existing single dataset solution, the accuracy of feature characterization is improved by 35%, and the accuracy rate is 75.9% when applied to pest classification tasks and 79.3% mAP.50 when applied to detection tasks.

[0022] 2) More scientific feature extraction logic and more accurate and comprehensive core feature capture: Addressing the problems of single-modal extraction, low discriminative power, and lack of reliability assessment in existing technologies, this invention constructs a two-stage extraction strategy of "professional knowledge guidance + large-scale model empowerment" through steps S3 and S4. Step S3 utilizes a large language model to extract structured feature phrases of "feature subject + feature attribute" from a standardized knowledge database according to the pest's life cycle stage. High-discrimination features are then selected through discriminative power calculation to avoid feature redundancy. Step S4, based on the clear image and manually labeled bounding boxes from step S1, and combined with the feature phrases from step S3 as prompts, drives a multimodal large-scale model to generate at least five core features covering morphology, color, and texture. Simultaneously, the reliability of the extraction is assessed through a confidence quantification model. This two-stage strategy relies on professional knowledge to lock in key directions and leverages a large-scale model to achieve accurate cross-modal extraction, solving the problem of single-modal extraction's difficulty in capturing core biological features. It improves the discriminative power of pests with high inter-class similarity (such as armyworms and armyworm larvae) by more than 35%.

[0023] 3) Improved Feature Optimization Mechanism, Enhanced Model Practicality and Robustness: Addressing the shortcomings of existing technologies such as feature redundancy and lack of optimization mechanisms, this invention establishes a "cross-modal deduplication + multi-dimensional screening" feature optimization system through steps S5 and S6. Step S5 utilizes a cosine similarity model to measure the matching degree between the image feature vector extracted by ResNet50 and the text feature vector extracted by Bert-base, eliminating duplicate or invalid information that does not match across modalities. Step S6 retains high-value features in descending order of scores by fusing a multi-dimensional comprehensive scoring function that integrates confidence, discriminability, and matching degree. This system not only solves the feature redundancy problem, reducing the amount of feature data by 30%, improving model inference efficiency by 20%, and reducing hardware computing costs by 15%, but also adapts to edge device deployment. Furthermore, through cross-modal consistency verification and multi-dimensional quality control, it enhances adaptability to complex scenarios, improving feature extraction accuracy by more than 25% for special scenarios such as complex backgrounds in farmland, changes in lighting, millimeter-level insect eggs, and densely clustered pests, effectively addressing practical challenges such as pest morphological variations and large intra-class differences.

[0024] In summary, this invention collects high-quality optical image data through multiple channels, constructs an authoritative and accurate professional knowledge system for agricultural pests, adopts a two-stage feature extraction strategy of "professional knowledge guidance + large model empowerment", and combines cross-modal feature matching and quality assessment to achieve accurate and comprehensive extraction of pest features. The method's adaptability is ensured through a dynamic update mechanism, ultimately providing high-quality feature support for pest identification. It has the advantages of high accuracy, low deployment difficulty, wide applicability, and high efficiency. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the overall process of feature extraction methods for agricultural pests.

[0026] Figure 2 This is a software illustration of the image quality control algorithm proposed in this invention.

[0027] Figure 3 This is an example diagram of the first-stage feature phrase extraction method proposed in this invention.

[0028] Figure 4 This is an example diagram of the second-stage single-target feature extraction method of the present invention.

[0029] Figure 5 This is a diagram illustrating the structure of the dataset obtained from the simulation experiment of this invention.

[0030] Figure 6 This is an example diagram of data obtained from the simulation experiment of this invention. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] A method for pest feature extraction based on domain knowledge-guided large models includes the following steps, such as... Figure 1 As shown: S1: Basic optical image data of pests is collected through multiple channels, and a clear quantified screening formula is used to control image quality, constructing an image dataset covering diverse scenes and morphologies. The multiple channels include: public images crawled by web crawlers, field images provided by agricultural partners, and high-quality illustrations extracted from color-printed books and academic papers in the agricultural field, all based on the IP102 dataset framework. A clear quantified screening formula is used to control the quality of the collected images.

[0033] in, As an evaluation metric for image sharpness, Image pixel size and Each pixel Set a sharpness threshold based on the gradient values ​​in the horizontal and vertical directions. (Can be adjusted according to actual application scenarios), only retain Images are entered into the dataset, ensuring they are clearly identifiable. For example... Figure 2 The image shown is a software demonstration diagram of the image quality control algorithm proposed in this invention. The value in the lower left corner of the software is the sharpness value calculated by the algorithm.

[0034] S2: Construct a multi-dimensional, hierarchical professional knowledge collection system, screening authoritative sources of pest knowledge, including integrating pest biology information from Wikipedia, authoritative professional books, and recent academic papers. After calibration by agricultural experts, a standardized knowledge database is established. This standardized knowledge database covers various corn and wheat pests, including fall armyworm, bollworm, corn borer, two-spotted leaf beetle, aphid, spider mite, and wheat gall midge, providing information on body size, color, surface structure, and appendage characteristics throughout the entire life cycle of egg, larva, pupa, and adult. For controversial information, an agricultural expert team is formed to conduct field verification and standardize the description of the information. A knowledge credibility scoring model is established.

[0035] in, Assess the credibility of the knowledge (value range 0-1). The authority coefficient of the information source is 0.8 for Wikipedia, 0.9 for authoritative books, and 0.85 for core journal articles. To verify the consensus rate among experts (1.0 for unanimous approval and 0.8 for majority approval), we will use the consensus rate. The time validity coefficient is set as follows (1.0 for knowledge within the last five years, and 0.8 for knowledge older than five years), and the weighting coefficient is set as follows: , , (satisfy ), only retain The knowledge is entered into the database to ensure its accuracy.

[0036] S3: Perform the first stage of feature phrase extraction: Based on the standardized knowledge database established in step S2, use a large language model to extract structured core feature phrases according to the pest's life cycle stage, and calculate the feature discrimination to filter high-discrimination features, constructing a feature phrase database; the large language model is GPT-4, the input is preprocessed professional knowledge text, and the output is a noun phrase containing a subject in the form of "feature subject + feature attribute", and 30-50 feature phrases are extracted for each type of pest for each life cycle stage; Calculate the feature phrase discriminative power to filter for high-discrimination features:

[0037] in, For the first The discriminative power of each feature phrase (value range 0-1). This represents the total number of characteristic phrases for a certain type of pest at a certain stage of its life cycle. Feature phrases calculated based on cosine similarity and semantic similarity, retain We construct a structured feature phrase database to ensure the uniqueness of each feature. For example... Figure 3 The image shown is an example of the first-stage feature phrase extraction method proposed in this invention, taking the corn borer larval stage as an example.

[0038] S4: Perform the second stage of single-target feature extraction: Input basic optical image data of pests that meet the clarity requirements, manually annotated bounding boxes, and feature phrases corresponding to the life cycle stage. Extract the core features of individual pests using a multimodal large model, and calculate the feature confidence to quantify the extraction reliability. The multimodal large model is Gemini or Tongyi Qianwen. Input includes basic optical image data of pests, manually annotated bounding boxes, and feature phrases corresponding to the stage obtained in the first stage as prompt information, driving the model to generate at least 5 core features covering morphology, color, and texture dimensions; simultaneously, calculate the feature confidence to quantify the extraction reliability.

[0039] in, Features The confidence level (value range 0-1). The feature text description completeness is scored (based on professional knowledge assessment, with 1.0 for complete core attributes and 0.5-0.9 for partial missing attributes). The matching degree between features and image visual information (calculated by the cosine similarity between image features and text features). and These are the maximum values ​​of completeness score and matching degree among all features, respectively, and are retained. The features ensure that the features are true and reliable. For example... Figure 4 The figure shown is an example of the second-stage single-target feature extraction method of the present invention, taking the corn borer larvae and pupae as examples.

[0040] S5: Cross-modal feature matching and alignment: Based on the cosine similarity model, the matching degree between image feature vectors and text feature vectors is measured, duplicate or invalid information is removed, and preliminary feature description results are generated; the image feature vectors are extracted using a deep convolutional neural network ResNet50 model, and the text feature vectors are extracted using a Bert-base pre-trained model. Both have a dimension of 2048. Cosine similarity is used to measure the matching degree between image feature vectors and text feature vectors to determine cross-modal consistency.

[0041] Where V is the image feature vector extracted by ResNet50, T is the text feature vector extracted by Bert-base, and d is the feature vector dimension (set to 2048), retaining only the... Feature pairs are used to eliminate invalid information that does not match across modes.

[0042] S6: Feature Quality Assessment Optimization: The preliminary results are screened by integrating a multi-dimensional comprehensive scoring function that combines confidence, discriminative power, and matching degree. High-value features are retained, and the final pest feature description is output. The multi-dimensional comprehensive scoring function is defined as follows:

[0043] in, The feature quality score (value range 0-1). For feature confidence, For feature discriminability, For cross-modal similarity matching, the weight coefficients are set to satisfy the following conditions: ,according to Sort in descending order, keeping the first few. High-value characteristics To ensure feature quality.

[0044] A pest feature extraction system based on a domain knowledge-guided large model, implementing the pest feature extraction method described in any one of steps S1 to S6, includes: The image acquisition and filtering module is used to perform step S1. The expertise building module is used to execute the S2 steps; The first-stage feature phrase extraction module is used to perform step S3. The second-stage single-target feature extraction module is used to execute step S4. The cross-modal matching and alignment module is used to perform the S5 step; The Feature Quality Assessment Optimization Module is used to execute step S6.

[0045] The system employs a two-stage strategy of "professional knowledge guidance + multimodal large model empowerment" to achieve accurate, comprehensive, and reliable extraction of pest characteristics.

[0046] The system is applied to the intelligent identification of pests in corn and wheat, and extended to the feature extraction of pests in rice and cotton crops. The system is deployed on a PC platform with a high-performance GPU, and the operating environment includes the Ubuntu 20.04 operating system, Python 3.9 programming language, PyTorch 2.0 deep learning framework, and calls the large language model GPT-4 and the multimodal large model Gemini Pro through API.

[0047] A pest feature extraction device based on a domain knowledge-guided large model, comprising: Memory is used to store computer programs, data, and models; A processor is configured to implement the pest feature extraction method based on a domain knowledge-guided large model as described in any one of steps S1 to S6 when executing the computer program.

[0048] Example A method for extracting pest features based on domain knowledge-guided large models includes the following steps: Step 1: Image Data Acquisition and Filtering Basic data integration: Based on the IP102 dataset, valid samples of 8 types of pests related to corn and wheat were selected, and a total of 1200 initial images were obtained; Supplementary data collection: 450 pest images were crawled from public online resources such as the official website of the Ministry of Agriculture and Rural Affairs and the pest database of the Chinese Academy of Agricultural Sciences using the Beautiful Soup web crawler; 237 pest images taken in the field were collected by signing data sharing agreements with 3 agricultural partners; and 100 high-quality illustrations were extracted from professional books such as "Atlas of Chinese Crop Pests and Diseases" and "Agricultural Entomology" as well as journal articles such as "Journal of Plant Protection" in the past five years. Sharpness filtering: Sharpness index was calculated for each of the 1987 acquired images. Set threshold Remove The blurry images were ultimately filtered out, and 1987 high-quality images (all acquired images met the sharpness requirements) were retained to construct an image dataset.

[0049] Step 2: Expertise Collection and Calibration Knowledge gathering: Download basic information such as biological classification, life cycle, and morphological characteristics of 8 types of pests from Wikipedia; supplement detailed morphological descriptions by referring to authoritative books such as "Handbook for Identifying Agricultural Pests" and "Atlas of Control of Common Pests in Corn and Wheat"; retrieve 30 relevant core journal papers published between 2019 and 2024 to collect pest characteristic information from the latest research. Expert calibration: A team of five agricultural entomologists was formed to review the collected knowledge. For three disputed pieces of information, including "the range of body color variation of fall armyworm larvae" and "antennae characteristics of adult wheat gall midge", a unified description was formed through field observation and verification. Credibility Score: Calculate a credibility score R for each piece of knowledge. For example, for information about the characteristics of fall armyworm larvae from an authoritative book... , , ,but All rights reserved Based on this knowledge, a standardized knowledge database containing over 2,000 records was constructed.

[0050] Step 3: First-stage feature phrase extraction Knowledge input processing: Preprocess the text information categorized by lifecycle stage in the standardized knowledge database, remove redundant modifiers, and retain the core descriptions; Feature phrase extraction: Call the GPT-4 API, input the preprocessed knowledge text, and set the prompt "Extract the core morphological, color, and body surface structure features of the following pests [life cycle stages], and output them in the form of noun phrases of 'feature subject + feature attribute'", and extract 30-50 feature phrases for each life cycle stage of each type of pest; Discriminative filtering: Calculate the discriminative score for each feature phrase. For example, the semantic similarity between the feature phrase "Y-shaped black spots on the body surface" of the fall armyworm larva and other feature phrases. ,but ,reserve We construct a structured feature phrase database based on the feature phrases.

[0051] Step 4: Second-stage single-target feature extraction Input information preparation: Select fall armyworm larvae images as test samples, input the multimodal large model GeminiPro, and simultaneously input manually annotated bounding box coordinates (precisely selecting the larval region) and fall armyworm larvae feature phrases extracted in the first stage (such as "black spots on the body surface", "brown head", "cylindrical body", etc.). Feature extraction: The model output requirement is set to "generate at least 5 core features, covering the dimensions of morphology, color, and texture". The model output features include "the larva is about 15mm long, with black Y-shaped spots on its body surface, a brown head, a light green cylindrical body, and fine hairs on its body surface". Confidence Calculation: Evaluation (All features are fully described, take 1.0), calculate (The mean cosine similarity between image features and text features is 0.85). , ,but The feature set is retained.

[0052] Step 5: Cross-modal feature matching and alignment Feature vector extraction: The feature vector V (dimension 2048) of the test image is extracted using the ResNet50 model, and the feature vector T (dimension 2048) of the output feature text is extracted using the Bert-base model. Similarity calculation: Calculate ,because The feature pair is retained to generate a preliminary feature description result.

[0053] Step 6: Feature Quality Assessment and Optimization Multi-dimensional scoring: Calculating the features of test samples , , ,but ; Filter output: By All features are sorted in descending order, and the top 5 high-value features are retained. The final feature description is: "The larva of the fall armyworm is about 15mm long, with a light green cylindrical body, a brown head, black Y-shaped spots on the body surface, and fine hairs on the body surface."

[0054] As can be seen from the embodiments, compared with the prior art, the advantages and innovations of the present invention are specifically reflected in the following aspects: 1) A more complete data and knowledge support system significantly improves the reliability of feature extraction sources: Existing technologies mostly rely on a single IP102 dataset, which has limited scene coverage and lacks quality control. The knowledge also lacks authoritative calibration, leading to significant biases in feature characterization. This invention, through steps 1 and 2 in the embodiments, constructs a dual support system of "multi-source high-quality images + authoritative standardized knowledge": Step 1, based on IP102, integrates 1987 images from web crawlers (450 images), field observations (237 images), and illustrations from professional books and papers (100 images). These images undergo clear quantification screening (all samples meet threshold requirements) to ensure that the images cover complex field scenes and are clearly identifiable. Step 2 integrates knowledge from Wikipedia, authoritative books, and 30 core journal papers. Five agricultural experts verify and calibrate three controversial pieces of information in the field. A knowledge credibility scoring model is used to select over 2000 high-credibility records (R≥0.85), covering the core features of eight types of pests throughout their entire life cycle. The dual system provides precise professional guidance and high-quality data support for feature extraction. Compared with the existing single dataset solution, the feature discrimination is improved by 35%, and the accuracy rate is 75.9% when applied to pest classification tasks and 79.3% mAP.50 when applied to detection tasks. It effectively solves the problems of one-sided data and inaccurate knowledge in the existing technology.

[0055] 2) The two-stage feature extraction strategy is more scientific, and the accuracy and comprehensiveness of core feature capture are greatly optimized: Existing technologies mostly use single-modal extraction, which is difficult to cover multi-dimensional features such as morphology, color, and texture, and lacks reliability assessment. Steps 3 and 4 in this embodiment of the invention construct a two-stage strategy of "professional knowledge guidance + large model empowerment": Step 3 calls GPT-4 to extract 30-50 structured phrases of "feature subject + feature attribute" according to the life cycle stage from the standardized knowledge database, and selects high-discrimination features (such as "Y-shaped black spots on the body surface" of the fall armyworm larva) through discrimination calculation (D≥0.7) to avoid feature redundancy; Step 4 uses clear images and manually annotated boxes as a basis, combined with feature phrase prompts, to drive Gemini Pro to generate at least 5 multi-dimensional core features (such as "body length 15mm, light green cylindrical shape, brown head" of the fall armyworm larva), and quantifies the reliability through a confidence model (confidence S=0.85≥0.7 in this embodiment). This strategy relies on professional knowledge to identify key directions and uses a multimodal large model to achieve accurate cross-modal extraction, solving the problem that single-modal extraction is difficult to capture core biological features. For pests with high inter-class similarity, such as fall armyworm and common armyworm, the accuracy of differentiation is improved by more than 40%.

[0056] 3) Improved cross-modal matching and multi-dimensional evaluation mechanisms significantly enhance feature usability and model robustness: Existing technologies lack effective feature deduplication and quality screening mechanisms, leading to feature redundancy, low inference efficiency, and poor adaptability to complex scenarios. Steps 5 and 6 in this embodiment establish an optimized system of "cross-modal deduplication + multi-dimensional screening": Step 5 extracts image feature vectors using ResNet50 and text feature vectors using Bert-base (both 2048-dimensional), calculates cosine similarity (Cos(V,T)=0.88≥0.7 in this embodiment), and removes invalid information that does not match across modalities; Step 6 retains the top 5 high-value features in descending order by fusing a multi-dimensional scoring function (Q=0.86) of confidence (0.85), discriminability (0.82), and matching degree (0.88). This system not only solves the problem of feature redundancy, reducing the amount of feature data by 30%, improving model inference efficiency by 20%, and reducing hardware computing costs by 15%, but also adapts to edge device deployment. Furthermore, through cross-modal consistency verification and multi-dimensional quality control, it enhances the adaptability to complex scenarios. For special scenarios such as millimeter-level insect eggs, densely clustered pests, and complex field backgrounds and light changes, the feature extraction accuracy is improved by more than 25%, effectively addressing the shortcomings of existing technologies in terms of weak anti-interference ability and insufficient robustness.

[0057] Simulation Experiment 1. Hardware Platform: Utilizing a stable PC platform with a high-performance GPU, sufficient computing power, and ample storage space to meet the needs of image processing, large model inference, and data storage; 2. Software Platform: The operating system is Ubuntu 20.04, the programming language is Python 3.9, the deep learning framework is PyTorch 2.0, the image processing library is OpenCV 4.8, large model calls use API interfaces (GPT-4, GeminiPro), and data storage uses a MySQL database; 3. Test data: Based on the MWPest8 multimodal dataset, covering 8 types of corn and wheat pests (earthworm, armyworm, cotton bollworm, corn borer, two-spotted leaf beetle, aphid, spider mite, and wheat gall midge), including 1987 optical images, 3398 bounding box annotations and corresponding bilingual (Chinese and English) professional text descriptions.

[0058] Simulation Experiment Results and Effect Analysis: To verify the effectiveness of this invention, comparative experiments were conducted based on the aforementioned hardware and software platform and the MWPest8 multimodal dataset. The invention was quantitatively compared with existing mainstream technologies (improved ShuffleNet V2, improved YOLOv5, RicePest-DETR) in four core dimensions: classification accuracy, detection precision, feature efficiency, and adaptability to complex scenes. Qualitative analysis was also used to verify the technological advantages. The experimental results are as follows: 1. Quantitative experimental results Evaluation index ShuffleNet V2 YOLOv5 RicePest-DETR The present technology Technical improvement range Detection task mAP.50(%) 64.5 70.1 73.8 79.3 7.4% Model inference speed(ms) 85.6 72.3 68.9 55.1 20.0% Complex background feature extraction accuracy(%) 58.4 63.7 67.9 84.2 24.0% 2. Qualitative experimental results Based on the performance indicators of the aforementioned quantitative experiments, the core advantages of this invention can be qualitatively determined from three dimensions: the effectiveness of the technical mechanism, the completeness of scenario adaptation, and the resistance to environmental interference. 1. The cross-modal feature alignment mechanism is more accurate, and invalid interference is significantly reduced. This invention achieves precise alignment of "image-knowledge" cross-modal information through cosine similarity matching between ResNet50 image feature vectors and Bert-base text feature vectors. This mechanism is demonstrated in quantitative experiments by achieving an 84.2% accuracy rate in extracting complex background features (a 24.0% improvement over the best existing technology), underpinned by a 41.5% removal rate of invalid cross-modal features (significantly higher than the 18.7% achieved by simply concatenating cross-modal information in existing technologies). This indicates that the feature fusion in this invention is not merely a formal accumulation of information, but rather ensures the consistency and effectiveness of features from the source, avoiding redundant information from interfering with subsequent tasks.

[0059] 2. More complete lifecycle characteristics and more comprehensive monitoring scenario coverage. In quantitative experiments, the present invention achieved a 7.4% lead in pest target detection mAP.50 (%), relying on the feature design of "hierarchical extraction from a knowledge database + multimodal large-scale model characterization": its feature coverage of the entire life cycle of pests—eggs, larvae, pupae, and adults—reached 91.2%, especially in the pupal stage where morphological features are relatively ambiguous, the coverage still remained at 85.7%. In contrast, existing technologies, lacking the "knowledge-guided hierarchical feature extraction" step, only achieved 76.5% feature coverage across the entire life cycle, and less than 65% for the pupal stage. This means that the feature system of the present invention can support more complete pest life cycle monitoring needs, rather than being limited to the identification of a single stage.

[0060] 3. It has stronger resistance to interference in actual field scenarios and higher engineering practicality. The quantitative experiment showing a 24.0% improvement in the accuracy of feature extraction from complex backgrounds reveals that the present invention achieves a feature extraction accuracy fluctuation of only 3.2% under complex backgrounds such as weeds and shadows in the field, as well as under varying environmental conditions such as strong light and high humidity. In contrast, existing technologies typically exhibit accuracy fluctuations exceeding 8% under similar conditions. This qualitative difference indicates that the feature system of the present invention is more adaptable to the interference conditions of actual agricultural scenarios, exhibits stronger robustness, and can be truly applied to complex field scenarios.

Claims

1. A method for extracting pest features based on a domain knowledge-guided large model, characterized in that, Includes the following steps: S1: Collect basic optical image data of pests through multiple channels, and use clear quantitative screening formulas to control the quality of images, and build an image dataset covering diverse scenes and morphologies; S2: Construct a multi-dimensional, hierarchical professional knowledge collection system, screen pest knowledge from authoritative sources, including integrating pest biological information from Wikipedia, authoritative professional books and historical academic papers, and establish a standardized knowledge database after calibration by agricultural experts; S3: Perform the first stage of feature phrase extraction: Based on the standardized knowledge database established in step S2, use the large language model to extract structured core feature phrases according to the pest life cycle stage, and calculate the feature discrimination to screen high-discrimination features and build a feature phrase database. S4: Perform the second stage of single-target feature extraction: Input basic optical image data of pests that meet the clarity requirements, manually labeled recognition boxes and feature phrases corresponding to the life cycle stage, extract the core features of individual pests through a multimodal large model, and calculate the feature confidence to quantify the reliability of extraction. S5: Cross-modal feature matching and alignment: Based on the cosine similarity model, the matching degree between image feature vectors and text feature vectors is measured, duplicate or invalid information is removed, and preliminary feature description results are generated; S6: Feature Quality Assessment Optimization: The preliminary results are screened by integrating a multi-dimensional comprehensive scoring function that combines confidence, discriminative power, and matching degree, retaining high-value features and outputting the final pest feature description.

2. The method for extracting pest features according to claim 1, characterized in that, In step S1, the multi-channel collection of basic optical image data of pests includes: based on the IP102 dataset, integrating public images crawled by web crawlers, field images provided by agricultural cooperatives, and high-quality illustrations extracted from color-printed books and academic papers in the agricultural field.

3. The method for extracting pest features according to claim 1, characterized in that, In step S2, the standardized knowledge database covers various corn and wheat pests, including fall armyworm, armyworm, cotton bollworm, corn borer, two-spotted leaf beetle, aphid, spider mite, and wheat gall midge. It describes the body size, body color, body surface structure, and appendage characteristics of the insects throughout their entire life cycle, from egg to larva to pupa to adult. In addition, an agricultural expert team is formed to conduct field verification and standardize the description of disputed information.

4. The method for extracting pest features according to claim 1, characterized in that, In step S3, the large language model takes preprocessed professional knowledge text as input and outputs noun phrases containing subjects in the form of "feature subject + feature attribute", and extracts several feature phrases for each type of pest at each life cycle stage.

5. The method for extracting pest features according to claim 1, characterized in that, In step S4, the multimodal large model takes as input the basic optical image data of pests, manually annotated bounding boxes, and feature phrases obtained in the first stage as prompt information, and drives the multimodal large model to generate at least 5 core features covering the dimensions of morphology, color, and texture.

6. The method for extracting pest features according to claim 1, characterized in that, In step S5, the image feature vector is extracted through a deep convolutional neural network, the text feature vector is extracted through a pre-trained model, and cosine similarity is used to measure the matching degree between the image feature vector and the text feature vector to determine cross-modal consistency.

7. The method for extracting pest features according to claim 1, characterized in that, In step S6, the multi-dimensional comprehensive scoring function is defined as: in, The feature quality score (value range 0-1). For feature confidence, For feature discriminability, For cross-modal similarity matching, the weighting coefficient is set to... (satisfy ),according to Sort in descending order, keeping the first few. High-value characteristics To ensure feature quality.

8. A pest feature extraction system based on a domain knowledge-guided large model, implementing the method described in any one of claims 1 to 7, characterized in that, include: An image acquisition and filtering module is used to perform step S1 in claim 1; A expertise building module for performing step S2 in claim 1; The first-stage feature phrase extraction module is used to perform step S3 in claim 1; The second-stage single-target feature extraction module is used to perform step S4 in claim 1; A cross-modal matching and alignment module for performing step S5 in claim 1; The feature quality evaluation and optimization module is used to perform step S6 in claim 1.

9. The pest feature extraction system according to claim 8, characterized in that, The system is applied to the intelligent identification of pests in corn and wheat, and extended to the feature extraction of pests in rice and cotton crops. The system is deployed on a PC platform with a high-performance GPU and calls large language models and multimodal large models through API.

10. A pest feature extraction device based on a domain knowledge-guided large model, characterized in that, include: Memory is used to store computer programs, data, and models; A processor, configured to implement the method of any one of claims 1 to 7 when executing the computer program, specifically the pest feature extraction method based on a domain knowledge-guided large model as described in any one of claims 1 to 7.

Images