Multi-modal information fused corn disease intelligent grading evaluation and treatment recommendation method

Abstract

The invention relates to a corn disease intelligent grading evaluation and renovation recommendation method fusing multi-modal information, and relates to a corn diversity disease identification and multi-stage evaluation model based on position coding. The method comprises the following steps: introducing a position attention mechanism, and carrying out multi-layer stacking by taking an inverse movement residual module integrated with position coding as an element movement block; the capabilities of complex image feature extraction and spatial information analysis are enhanced, and a scab region is segmented, so that disease type distinguishing and severity evaluation are realized; constructing a model based on a corn disease entity relationship triple library: performing deep context coding on a corn disease text, realizing classification and prediction of complex relationships among disease entities through a multi-layer structure, and analyzing complex links and influence factors among diseases; and a corn disease treatment method recommendation model combined with an implicit relationship: according to a disease identification and grading result and a disease triple, mining an optimal implicit relationship among different disease entities through complex number embedding and a graph neural network, and giving an accurate disease treatment method recommendation.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural disease diagnosis and assessment, and in particular to a method for intelligent graded assessment and control recommendation of corn diseases integrating multimodal information. Background Art

[0002] In the traditional practice of agricultural disease diagnosis and assessment, farmers and agricultural experts mainly rely on intuitive observation of disease symptoms and accumulated experience and knowledge to make judgments. This method is more subjective and uncertain, especially when dealing with complex and diverse corn disease phenomena and their severity assessments. Due to the diversity and confusion of disease symptoms, the diagnosis results are often not accurate enough. In addition, some current disease identification technologies are mostly based on quantitative analysis of single disease characteristics (such as the color and shape of lesions), which serve as the main basis for disease judgment. Although this method simplifies the disease identification process to some extent, its comprehensive assessment and precise policy-making capabilities are stretched when faced with complex situations involving the interweaving of multiple disease characteristics and different stages of disease development.

[0003] There are also studies in the prior art that explore the combination of deep learning and modal data analysis methods to improve the recognition and classification of corn disease characteristics. The existing method is: some corn disease recognition and evaluation models integrate multiple disease features, and conduct comprehensive evaluation and graded identification of corn diseases by constructing a multi-task learning model or an integrated learning framework. Specific steps: First, collect field crop disease pictures, and use models such as convolutional neural networks to extract the characteristics of disease images, such as lesion distribution, color changes, texture characteristics, etc., and then preliminarily identify the types of diseases. Secondly, use intelligent algorithms to compare and match these multi-level and multi-dimensional disease characteristics with preset disease classification standards, and strive to accurately locate the disease level. Finally, based on this refined disease classification evaluation result, the system recommends corresponding prevention and control measures and remediation plans. However, most of these methods only stay at the level of identifying a single disease type, and the grading evaluation of different severity levels of the same disease and the proposal of targeted remediation measures for diseases at all levels are still imperfect.

[0004] In summary, the technical problems of existing agricultural disease diagnosis and assessment technologies for corn diseases are:

[0005] (1) In terms of multi-level identification and evaluation of corn diseases, traditional manual methods are subject to strong subjectivity, resulting in low efficiency; the recognition accuracy of multiple diseases is low when the disease manifestations are subtle and the morphology is similar; for irregular lesions, the edges of the cut lesions are relatively rough.

[0006] (2) In terms of disease entity and relationship extraction, when processing multi-feature texts rich in complex corn-related knowledge, due to the complexity and diversity of entity types and the lack of detailed classification of entity categories in existing methods, it is difficult to accurately identify different entities and their attributes, and thus it is difficult to reveal the implicit relationships in the text, especially when capturing the interactions and potential connections between multi-dimensional information such as diseases, environmental factors and growth cycles.

[0007] (3) In terms of corn disease control recommendations, traditional representation learning methods may lose the ability to capture complex logical relationships in the process of simplified modeling, while graph-based methods face the problems of insufficient modeling of complex relationships and ignoring the hidden characteristics of entity relationships.

[0008] (4) Most existing methods only stay at the level of identifying a single disease type. The grading assessment of the severity of the same disease and the proposal of targeted remediation measures for each level of disease are still imperfect. Summary of the invention

[0009] The present invention aims to solve the technical problems in the prior art and provide a method for intelligent grading assessment and control recommendation of corn diseases by integrating multimodal information.

[0010] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0011] A method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information includes the following steps:

[0012] Step 1: Disease recognition and multi-level assessment model based on position attention mechanism, combined with the position attention mechanism, through convolutional neural network to enhance image feature extraction and spatial analysis, distinguish similar diseases, calculate the area and proportion of lesions to achieve disease grade assessment;

[0013] Step 2: Based on the corn disease entity recognition and relationship extraction model, automatically annotate data, deeply mine the key entities and their complex relationships in the disease text, build a disease knowledge triple library, and deepen the understanding of corn diseases and related factors;

[0014] Step 3: A disease remediation recommendation model based on a graph attention network enhanced with implicit dependencies is developed. The entity relationships in the disease knowledge triple library are learned in the complex space to generate link predictions. The graph attention network is used to strengthen node features to capture implicit relationships. The weighted adjacency information and multi-head attention mechanism are combined to dynamically optimize the deep representation of entities and relationships to achieve disease remediation recommendations.

[0015] In the above technical solution, step 1 includes the following steps:

[0016] Step 1.1, by improving the EMO-Transformer model, the inverted moving residual module with integrated position coding is stacked as a meta-moving block to enhance the ability of complex image feature extraction and spatial information analysis, and to distinguish between disease types with highly similar morphology;

[0017] Step 1.2, passing the features identified in step 1.1 to the improved Unet-SE to map the distribution of the disease on the plant, and optimizing the segmentation results in combination with the conditional random field (CRF) to improve the recognition accuracy of the edge of the lesion;

[0018] Step 1.3, calculate the area and proportion of the diseased spots in the marked area, and classify the disease severity according to the preset standards to evaluate the severity of the disease.

[0019] In the above technical solution, in step 1.1, the steps of performing multi-layer stacking of the elementary moving blocks are:

[0020] ① Input dimension increase: increase the depth of the input feature map through a dimension-upgraded multi-layer perceptron to provide richer feature dimensions. This step is expressed as:

[0021] X′ icm =MLP up (X icm )

[0022] Among them, MLP up is a multi-layer perceptron with dimensionality enhancement function, which inputs the feature map X icm The number of channels increases, X′ icm Represents the new feature map after dimensionality upgrade;

[0023] ② Intermediate operator enhancement: applying a flexibly selected intermediate operator to further enhance feature representation;

[0024] Y efm =Operator(X′ icm )

[0025] Among them, Y efm Represents the enhanced feature map, and Operator represents the Identity mapping;

[0026] ③ Dimensionality reduction and residual connection, through another MLP down Perform dimensionality reduction to reduce the depth of the feature map to control model complexity, while introducing residual connections to promote gradient flow and training stability. This step is formulated as:

[0027] Z mbo =MLP down (Y efm )+X isr

[0028] Among them, Z mbo is the output of the current meta movement block, +X isr represents the inverse shifted residual.

[0029] In the above technical solution, in step 1.1, the inverted moving residual module adopts the extreme separation convolution block (XSep-Conv) through spatial separation convolution, deep convolution and point-by-point operation to efficiently extract local features, reduce the number of parameters, and ensure the accuracy of feature capture; at the same time, the extended window multi-head self-attention (EW-MHSA) is used to decompose the complex self-attention calculation into multiple small-scale operations by block calculation, effectively controlling the calculation complexity.

[0030] In the above technical solution, in step 1.1, the disease types that are highly similar in morphology are distinguished by compressing the feature map through global average pooling, and then mapped to the disease category through the fully connected layer, and the disease probability distribution is output, and the highest probability category is selected as the accurate classification prediction result for leaf diseases.

[0031] In the above technical solution, step 2 includes the following steps:

[0032] Step 2.1: Use the RoBERTa model to perform deep context encoding on large-scale pre-training data to capture the nuances and implicit information in agricultural texts.

[0033] Step 2.2, using the multi-level structure of the PCNN model to realize disease entity recognition and relationship extraction, classify the complex associations between crop disease entities, and reveal the influencing relationship between the symptoms, onset period and other key factors behind the crop name and disease name;

[0034] Step 2.3, form a corn disease triple library containing entities and relationships.

[0035] In the above technical solution, step 3 includes the following steps:

[0036] Step 3.1, at the global level, the ComplEx model is used to learn the triple representation of all entities and their relationships related to corn diseases, embed them into the complex space, and capture the interaction between entities and relationships through inner products;

[0037] Step 3.2, focusing on the local structure, using the learned entity embedding to initialize node features through the GAT model, strengthening implicit dependencies through the dynamic weighted neighbor features of the graph structure, capturing the implicit relationship between nodes, and generating unknown relationship link predictions;

[0038] Step 3.3, dynamically optimize the deep representation of entities and relationships, combine weighted adjacency information and multi-head attention mechanism, use interval sorting loss function to improve the discrimination ability, obtain the optimal implicit relationship between different entities, and realize disease control recommendation.

[0039] In the above technical solution, before step 1, it also includes data set construction and preprocessing steps.

[0040] The beneficial effects of the present invention are:

[0041] The advantages of the corn disease intelligent grading assessment and control recommendation method integrating multimodal information of the present invention are as follows:

[0042] (1) A model for identifying and evaluating diverse maize diseases based on position coding: The position attention mechanism is introduced, and the inverted moving residual module with integrated position coding is stacked as a meta-moving block in multiple layers to enhance the ability of complex image feature extraction and spatial information analysis, so as to distinguish disease types that are highly similar in morphology. The area and proportion of disease spots in the marked area are calculated, and the disease severity is graded according to the preset standards to evaluate the severity of the disease.

[0043] (2) Construction model of triple library based on maize disease entity relationships: Deep context encoding of maize disease text is performed, and the complex relationships between disease entities are classified and predicted through a multi-level structure. The complex connections and influencing factors between diseases are analyzed to construct a disease triple library of entities and relationships, including symptoms, onset period, meteorological conditions of causes, prevention methods and treatment methods of maize diseases.

[0044] (3) Corn disease control method recommendation model based on implicit relationships: Combined with graph attention network and plural embedding, the implicit dependency relationship between corn disease triplets is mined, the intricate relationship network and imperceptible correlation in corn disease control are analyzed, and accurate disease control method recommendations are achieved. According to the results of disease identification and classification, the corresponding effective corn disease control strategy is given based on the triplets constructed by entity relationship identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0046] Figure 1 The present invention is a flowchart of an implementation method of the intelligent grading assessment and control recommendation method for corn diseases integrating multimodal information.

[0047] Figure 2 This is the network structure diagram of the multi-level disease identification and assessment model.

[0048] Figure 3 This is the iRMB module structure diagram.

[0049] Figure 4 This is the structure diagram of the RoBERTa-PCNN model.

[0050] Figure 5 Diagram of the multi-core convolution process in the BERT-PCNN model.

[0051] Figure 6 A flowchart is proposed for the disease management method based on entity relationship optimization of graph attention network. DETAILED DESCRIPTION

[0052] The inventive concept of the present invention is as follows: The present invention is aimed at the technical problems existing in the existing agricultural disease diagnosis and assessment technology of corn diseases: (1) Disease identification and grading assessment: The traditional manual-based method is highly subjective and produces judgment bias due to individual differences; when faced with a variety of diseases with subtle manifestations and similar morphology, it lacks the understanding of complex image features and spatial context; for irregular lesions, there are problems such as blurred lesion edges, loss of details and noise interference that affect the accurate segmentation of lesions. (2) Entity recognition and relationship extraction: The existing technology is difficult to accurately identify and distinguish due to the complexity and diversity of corn disease-related entity types; the relationship between entities is complex, making it difficult to capture the interaction between diseases and environmental factors; the implicit relationship in the disease text is difficult to discover, which aggravates the difficulty of capturing the potential connection between multi-dimensional information. (3) Disease link prediction: The traditional representation learning method cannot capture the complex logical relationship in the disease text. The graph-based method faces the problem of ignoring the hidden characteristics of entity relationships and failing to discover the implicit dependencies between entities. The occurrence and development of diseases are dynamic processes, and their symptoms will change over time, while the existing methods cannot adapt to dynamic changes. In view of the above shortcomings, the present invention proposes an intelligent graded assessment and remediation recommendation method for corn diseases that integrates multimodal information. First, a disease recognition and multi-level assessment model based on a position attention mechanism is combined with a position attention mechanism to enhance image feature extraction and spatial analysis through a convolutional neural network, distinguish similar diseases, and calculate the area and proportion of lesions to achieve disease grade assessment. Secondly, a corn disease entity recognition and relationship extraction model automatically annotates data, deeply mines key entities and their complex relationships in disease texts, constructs a disease knowledge triple library, and deepens the understanding of corn diseases and related factors. Finally, a disease remediation recommendation model based on a graph attention network with implicit dependency enhancement learns entity relationships in disease triplets in complex space to generate link predictions, uses a graph attention network to enhance node features to capture implicit relationships, and combines weighted adjacency information and a multi-head attention mechanism to dynamically optimize the deep representation of entities and relationships to achieve disease remediation recommendations.

[0053] The present invention is described in detail below with reference to the accompanying drawings.

[0054] A specific implementation of the method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information of the present invention first constructs and preprocesses a data set, and then executes steps 1 to 3:

[0055] Step 1: Disease recognition and multi-level assessment model based on position attention mechanism, combined with the position attention mechanism, through convolutional neural network to enhance image feature extraction and spatial analysis, distinguish similar diseases, calculate the area and proportion of lesions to achieve disease grade assessment;

[0056] Step 2: Based on the corn disease entity recognition and relationship extraction model, automatically annotate data, deeply mine the key entities and their complex relationships in the disease text, build a disease knowledge triple library, and deepen the understanding of corn diseases and related factors;

[0057] Step 3: A disease remediation recommendation model based on a graph attention network enhanced with implicit dependencies is developed. The entity relationships in the disease knowledge triple library are learned in the complex space to generate link predictions. The graph attention network is used to strengthen node features to capture implicit relationships. The weighted adjacency information and multi-head attention mechanism are combined to dynamically optimize the deep representation of entities and relationships to achieve disease remediation recommendations.

[0058] Combine the following Figure 1 Each step of the method of the present invention is described in more detail.

[0059] 1. Dataset Construction and Preprocessing

[0060] 1. Corn disease identification and multi-level evaluation dataset

[0061] The data comes from the Plant Village database and the Plant Protection Professional Database, covering common corn diseases such as about 1,000 images of corn leaf spot, about 1,000 images of corn brown spot, about 1,100 images of corn gray spot, and about 1,200 images of corn southern rust, as well as the construction of evaluation standards for different diseases. The above disease data are divided into 70% of the data for training, 20% for adjusting model parameters, and 10% for testing to evaluate the generalization ability of the model. The LabelMe tool is used to manually annotate the images, annotating the lesion areas of various disease types and multi-level diseases to support disease identification and graded evaluation. As shown in Table 1-1, this is the severity graded evaluation standard for various corn leaf diseases.

[0062] Table 1-1 Corn disease severity classification table

[0063]

[0064] 2. Disease entity relationship extraction and remediation reasoning data set

[0065] The dataset of the present invention integrates the information of corn disease literature in CNKI, books such as "Color Illustrations of Crop Disease and Pest Diagnosis and Control", and multiple professional websites. In the text information preprocessing stage, first, the known relationship is used as an anchor point to automatically match and mark the corresponding entity pairs in the unlabeled text. Then, by intelligently comparing the entity mentions in the text with the records in the knowledge base, the implicit relationships in a large amount of text are inferred and marked, thereby automatically generating a large number of labeled instances for model training.

[0066] With reference to professional knowledge, named entities in the field of corn diseases are defined into seven categories, namely disease name, onset period, meteorological conditions of etiology, absolute conditions of etiology, symptoms, prevention methods and treatment methods. As shown in Table 1-2, the named entity types and examples in the field of corn disease detection are given. At the same time, in order to improve the quality of the data, non-text data such as labels and special characters are deleted. Then the sentences are divided and the noisy parts are removed, and finally 125,786 valid sentences are obtained.

[0067] Table 1-2 Named entity types and examples in the field of corn diseases

[0068]

[0069]

[0070] In the dataset based on entity recognition, the maize disease domain relations are defined into seven categories: disease name, disease-symptoms, disease-onset period, disease-cause meteorological conditions, disease-prevention measures, disease-treatment methods, and disease-cause absolute conditions. At the same time, the corresponding text data is divided into 97,486 training sentences, 125,429 test sentences, and 10,942 verification sentences.

[0071] The construction of the data set in Part 2 provides a theoretical basis and practical guidance for the disease identification and classification assessment in Part 1. By integrating detailed information on maize disease entities, relationships, and remediation methods, we can deepen our understanding of disease characteristics and thus improve the accuracy and efficiency of disease identification. In addition, the analysis results of these data can be directly applied to the decision-making process of maize disease management, providing a scientific basis for the formulation of effective preventive measures and treatment plans.

[0072] The following content of the present invention takes corn gray spot as an example to introduce steps 1 to 3 of the method of the present invention.

[0073] First, the following detailed description is given for step 1:

[0074] 2. Corn disease identification and evaluation method based on improved EMO-Transformer and Unet-SE

[0075] In view of the diversity and complexity of diseases faced by corn disease identification and evaluation methods, the present invention proposes a corn disease identification and evaluation method combining improved EMO-Transforme and Unet-SE. First, the inverted moving residual module with integrated position coding is stacked as a meta-moving block through the improved EMO-Transformer model to enhance the ability of complex image feature extraction and spatial information analysis, and to distinguish between morphologically highly similar disease types. Secondly, the identified features are passed to the improved Unet-SE to map the distribution of diseases on the plants, and the segmentation results are optimized in combination with the conditional random field (CRF) to improve the recognition accuracy of the edges of the lesions. Finally, the area and proportion of the lesions in the marked area are calculated, and the degree of the disease is graded according to the preset standards to achieve an assessment of the severity of the disease. Through experimental verification, the present invention performs well in plant disease identification and damage area assessment, and provides an efficient and accurate technical means for agricultural disease prevention and control. The network structure of the multi-level disease identification and evaluation model (improved EMO-Transformer and Unet-SE model) is as follows Figure 2 shown.

[0076] 1. Improved EMO-Transformer corn disease feature extraction based on position encoding

[0077] The improved EMO-Transformer model adopts a multi-layer nested moving block structure design. By stacking diversified feature maps, the depth and breadth of the representation of input data are enhanced, and the hierarchy and comprehensiveness of feature learning are promoted. Secondly, the extreme separation convolution block is integrated with the improved expansion window multi-head self-attention mechanism, and an inverted moving residual module is constructed as a moving block to capture local features and integrate global context information. Finally, the feature map is compressed through global average pooling, mapped to the disease category using a fully connected layer, and the disease probability distribution is output, and the category with the highest probability is selected as the classification prediction result of leaf disease.

[0078] (1) Multi-layer meta-movement block stacking (i.e., the step of performing multi-layer stacking of meta-movement blocks). By dynamically adjusting the selection of intermediate operators, the model is allowed to explore and optimize feature combinations at different feature extraction stages, achieving a more comprehensive and in-depth understanding of image features. The meta-movement block structure consists of the following three parts.

[0079] ① Input dimension increase: increase the depth of the input feature map through a dimension-upgraded multi-layer perceptron to provide richer feature dimensions. This step is expressed as:

[0080] X′ icm =MLP up (X icm )

[0081] Among them, MLP upis a multi-layer perceptron with dimensionality enhancement function, which inputs the feature map X icm The number of channels increases, X′ icm Represents the new feature map after dimensionality increase.

[0082] ② Intermediate operator enhancement: Apply a flexibly selected intermediate operator Operator to further enhance feature representation.

[0083] Y efm =Operator(X′ icm )

[0084] Among them, Y efm Represents the enhanced feature map, and Operator represents the Identity mapping.

[0085] ③ Dimensionality reduction and residual connection, through another MLP down Perform dimensionality reduction to reduce the depth of the feature map to control model complexity, while introducing residual connections to promote gradient flow and training stability. This step is formulated as:

[0086] Z mbo =MLP down (Y efm )+X isr

[0087] Among them, Z mbo is the output of the current meta movement block, +X isr represents the inverse shifted residual described below.

[0088] (2) Inverted Moving Residual Module (iRMB). The XSep-Conv block is used to efficiently extract local features and reduce the number of parameters through spatial separation convolution, deep convolution and point-by-point operation, while ensuring the accuracy of feature capture; at the same time, the extended window multi-head self-attention (EW-MHSA) is used to decompose the complex self-attention calculation into multiple small-scale operations through block calculation, effectively controlling the computational complexity. The iRMB structure is as follows: Figure 3 shown.

[0089] ① Extreme separation convolution block. First, use spatial separation convolution to refine the features in each channel. Secondly, use 2x2 depth convolution combined with improved symmetric filling strategy to compensate for the information loss caused by spatial separation convolution and keep capturing local features. Finally, adjust the number of output channels through point-by-point convolution to reduce computational complexity. The formula is expressed as:

[0090] Y local =XSepConv(X icm )=2×2DW+1×KDW+K×1DW+1×PW

[0091] Among them, DW is improved symmetric padding, 1×KDW and K×1DW represent the components of spatial separation convolution, 1×PW represents point-by-point convolution, which is used to adjust the number of output channels, and Y local It is the processed local feature output.

[0092] ② Improve the multi-head self-attention mechanism (i.e., extended window multi-head self-attention (EW-MHSA)). Introduce block calculation, subdivide the huge attention matrix into multiple small blocks, and perform self-attention operations independently in each block, rather than in the broad range of the entire sequence or image. By controlling the size of the block, the complexity is maintained at a low level. The block strategy formula is:

[0093]

[0094] in, and are the i-th block of the query and key matrices after segmentation, d k is the dimension of the key vector, A segmented It is the attention matrix obtained after the above block calculation.

[0095] ③ Position coding integration: Position coding is embedded in the above EW-MHSA to enhance the perception of image spatial structure.

[0096] First, the query Q is derived from the input features using a linear transformation isr , key K isr Sum value V isr matrices, representing different perspectives on the input data. isr , K isr and V isr is defined as follows:

[0097]

[0098] Among them, X cm is the input feature matrix, W Q , W K and W V is the weight matrix used to generate queries, keys, and values.

[0099] Then, we integrate the position encoding vector P isr , giving each position a unique absolute position code, so that the model can distinguish the features of different positions in the image. When calculating self-attention, the position code vector P isr It is integrated into the key-value pair matching process, and the formula is expressed as:

[0100]

[0101] Among them, dk is the dimension of the key vector, which is used to scale the dot product to avoid too large values ​​in the softmax function and ensure the stability of the learning process.

[0102] By integrating position attention, we learn the intrinsic correlation between input features and also focus on the relative position of features in the image space layout. At the same time, position encoding can assist in calculating area estimation, showing stronger spatial understanding and recognition capabilities in tasks such as disease recognition.

[0103] Finally, after the above three steps, the complete expression of the reverse moving residual module is:

[0104] F isr =(XSep-Conv,Skip)(EW-MHSA(·))

[0105] (3) Disease category prediction. The feature map is compressed by global average pooling and then mapped to the disease category through a fully connected layer. The disease probability distribution is output and the category with the highest probability is selected as the accurate classification prediction result for leaf diseases.

[0106] ① The features are compressed into vectors. Global average pooling (GAP) is used to transform the final feature map Z final Compress it and convert it into a fixed-length vector to reduce information redundancy while retaining key discrimination. This step is expressed as:

[0107] Z compressed =GAP(Z final )

[0108] ② Feature mapping disease category. These refined features are fed into the classification prediction module, and the high-level abstract features are mapped to each predefined category through the fully connected layer to achieve accurate classification prediction of the input samples. The whole process ensures efficient conversion and accuracy from complex features to target category prediction. The formula for this step is expressed as:

[0109] y pred =FC(Z compressed )

[0110] Among them, y pred represents the predicted probability distribution of the model for leaf disease categories, and Z compressed It is the feature vector obtained after the pooling operation.

[0111] The obtained y pred is a probability distribution vector, each element of which represents the probability that the input sample belongs to each disease category. In order to obtain a specific category prediction from these probabilities, the category corresponding to the highest probability is used as the final prediction result.

[0112] 2. Disease area segmentation based on improved Unet-SE

[0113] In the multi-level disease modality grading assessment, a lightweight improved Unet-SE lesion segmentation network is connected in series after the EMO model, and the generated high-level feature map is directly passed to the encoder of the segmentation network. The semantically rich features extracted by the position attention mechanism and the inverted shifted residual module are used to output a pixel-level lesion segmentation mask.

[0114] (1) Improved Unet-SE lesion segmentation network

[0115] ① In the encoder, after each layer of convolution and downsampling operation, the corresponding level feature map provided by the EMO model is fused with the features of the current layer of the model. This feature fusion not only transmits high-level semantic information, but also retains the underlying spatial details, helping the model understand the precise location and range of the lesions.

[0116] Let the first layer of the encoder be denoted as U l , after convolution and pooling, the feature map is obtained, and the formula is expressed as:

[0117] U l =f l enc (U l-1 )

[0118] Among them, f l enc represents the encoder function of the lth layer, including convolution, activation function and possible maximum pooling operation, U l-1 is the output of the previous layer.

[0119] For each encoder layer, the feature map obtained from the EMO model is mapped to E l , these feature maps are adjusted as necessary to ensure that their spatial dimensions match the feature maps of the corresponding layers of Unet-SE. l It carries high-level semantic information extracted by the EMO model to enhance the understanding of lesion characteristics.

[0120] In the lth layer, the Unet-SE feature U l EMO model provides the following features: l Fusion is performed, combining the improved local detail preservation capability of Unet-SE and the global semantic understanding of the EMO model. The weighted summation formula is as follows:

[0121] FusedFeature l =α enc *U l +β enc *E l

[0122] Among them, α enc , β enc is a weight parameter used to balance the importance of features from the improved Unet-SE and EMO models.

[0123] Through feature fusion, the improved Unet-SE model strengthens the learning of lesion features in the encoder stage, while maintaining the ability to accurately recognize the location and shape of lesions.

[0124] ② In the decoder stage, the spatial resolution of the feature map is gradually restored, while integrating the deep semantic information from the encoder and the high-level features of the EMO model to achieve accurate lesion boundary definition. The goal of each layer of the decoder is to enlarge the spatial size of the feature representation through upsampling operations and effectively integrate it with the features from the corresponding encoder layer and the corresponding features of the EMO model.

[0125] First, the feature map of the encoder is upsampled to increase the spatial resolution. The formula is expressed as:

[0126] U′ l =UpSample(U l+1 )

[0127] Among them, U′ l represents the feature map sampled on the l-th layer decoder, and U l+1 It is the feature map of the next level encoder.

[0128] The upsampled feature maps are fused with the features of the corresponding encoder layer to combine local details and global semantic information. Assume that the feature fusion operation of the decoder layer l is D l , then the feature fusion process can be expressed as:

[0129] D l =α dec *U′ l +β dec *U l +γ dec *E l

[0130] Among them, α dec , β dec , γ dec is the weight parameter used to balance the features from the upsampled features, the corresponding encoder layer features, and the EMO model features; E l is the EMO model feature map corresponding to the lth layer of the decoder, appropriately scaled to match the feature space dimension of the decoder.

[0131] Finally, when there are multiple types of lesions that need to be distinguished, the Softmax function is used to ensure that the sum of the probabilities of all class outputs is 1, that is, a class probability vector is assigned to each pixel. For a problem with C classes, the network output for pixel i is a C-dimensional vector in is the logits of pixel i belonging to class j. After applying the Softmax function, we get the probability distribution of each class:

[0132]

[0133] in, Represents the probability that pixel i belongs to the jth class. That is, for each pixel, a probability distribution is given about which class the pixel is most likely to belong to.

[0134] (2) Conditional random field optimization. Since the segmentation of lesions is easily affected by blurred boundaries, loss of details and noise interference, and it is difficult to handle irregular lesions, CRF is used to optimize the lesion boundary positioning and refine the lesion edge using the dependency relationship between pixels and neighborhood information.

[0135] First, for each pixel, define t k is a label that depends on a single pixel, and s l It is based on the relationship between pixels, which reflects the relationship between the category label of the pixel and its context information, such as the characteristic function of the shape, texture, position and color characteristics of the lesion to quantify the properties of the pixel and its neighborhood.

[0136] Assign a weight, λ, to each feature function k The weight corresponding to the unary potential energy measures the importance of a single pixel feature; μ l is the weight of the binary potential energy, which controls the strength of the interaction between pixels.

[0137] Then, the global energy function E(x) is constructed by combining the feature function and the weight. The formula is expressed as:

[0138] E(x lab ) mine =-∑ k λ k t k (x lab )-∑ l μ l s l (x lab )+log(Z(x lab ) nor )

[0139] Among them, x lab represents the label configuration of all pixels, ∑ kλ k t k (x lab ) is the sum of all unary potential energies, ∑ l μ l s l (x lab ) is the sum of all binary potential energies, Z(·) nor is the normalization factor.

[0140] By minimizing the energy function E(·) mine , find the optimal label assignment The formula is:

[0141]

[0142] get Finally, the initial segmentation results based on single pixel features will be corrected by these global optimal labels, solving the problems of blurred boundaries, loss of details and noise interference that may exist in the original segmentation.

[0143] However, due to the problem of similar disease spots in the corn field, in order to enhance the effect of distinguishing similar diseases, the present invention combines Foeal Loss and Dice Loss as the loss function, and its formula is expressed as follows:

[0144] L Z =L FL +L DL

[0145] The Foeal Loss loss function can optimize the network's ability to extract inter-class judgment features, so the Foeal Loss loss is used to achieve a good classification effect on similar diseases. The definition formula is as follows:

[0146]

[0147] Among them, p dis is the probability of the disease category, α dis is the balance factor, is the regulating factor, γ dis ≥0 is an adjustable focus parameter. By adjusting the focus parameter γ dis , reducing the weight of easy-to-classify samples during training. For the problem of imbalance between classes, the balancing factor α is introduced dis ∈[0,1], suppressing the imbalance in the number of positive and negative samples.

[0148] Dice Loss is a regional metric function used to evaluate the similarity of two samples. It not only calculates the target pixel loss, but also the adjacent pixel loss. Its formula is as follows:

[0149]

[0150] Among them, beta and smooth are the hyperparameters of the loss function, TP stands for true positive, that is, the prediction is positive and the actual result is also positive; FN stands for false negative, that is, the prediction is negative and the actual result is positive.

[0151] 3. Multi-level disease classification assessment based on lesion area

[0152] After achieving accurate segmentation of corn lesions, the severity of different lesion areas is carefully graded. The disease classification is often expressed in the image as the ratio of the number of pixels in the diseased area to the number of pixels in the total area of ​​the plant leaf. The calculation formula for the percentage of the lesion area to the leaf area is:

[0153]

[0154] in, is the area of ​​leaf lesions; is the leaf area, is the healthy area of ​​the leaf; p dis is the area of ​​a unit pixel; It is the lesion area; is the leaf area.

[0155] The EMO-Transformer model was used to analyze the selected test images, and the case of corn leaves was successfully identified as being affected by corn gray spot disease, and the diseased area was segmented by the improved Unet-SE model. According to the area calculation results, the area covered by the diseased spots occupies 25% of the entire leaf area. This proportion is based on the evaluation criteria in the corn gray spot disease severity grading table. When the diseased area accounts for 5% to 30%, the corresponding disease severity is classified as level 5, which is a moderate infection.

[0156] Next, the following detailed description is given for step 2:

[0157] 3. Disease entity recognition and relationship extraction method based on RoBERTa-PCNN

[0158] In the study of corn entity recognition and relationship extraction, due to the complexity and diversity of entity types in text data, and the insufficient detailed classification of entity categories by existing methods, it is difficult to reveal the implicit relationships when processing multi-feature texts rich in complex corn-related knowledge. To address the problem, the present invention proposes a RoBERTa-PCNN model for corn disease entity recognition and relationship extraction. First, the RoBERTa model is used to perform deep context encoding on large-scale pre-training data to capture subtle differences and implicit information in agricultural texts. Secondly, the multi-level structure of the PCNN model is used to realize disease entity recognition and relationship extraction, and the complex associations between crop disease entities are classified to reveal the influence of symptoms, onset period and other key factors behind crop names and disease names. Finally, a corn disease triple library containing entities and relationships is formed. Figure 4 Shown is the structural diagram of the RoBERTa-PCNN model.

[0159] 1. Text feature encoding based on RoBERTa-PCNN model

[0160] The RoBERTa encoding layer uses the Transformer's self-attention mechanism to capture the deep features of the text, including vocabulary, syntax, and paragraph logic, through word segmentation and word embedding, and provides fine features for tasks such as text classification. Its fine-tuning capability ensures that the model can quickly adapt to different scenarios and efficiently extract task-related features.

[0161] First, for each token after word segmentation, not only is the token embedded in the matrix (W token ) is converted into a vector to capture the meaning at the lexical level, and two complementary embedding information are also incorporated: sentence embedding (W seg ) and position embedding (W pos ). Sentence embedding provides the ability to handle more complex document structures by treating each token as belonging to the same sentence. Position embedding is to make up for the limitation that the Transformer architecture does not natively contain position information, ensuring that the model can understand the position relationship of words in the text. When encoding, the comprehensive vector expression of the i-th token follows the formula:

[0162] E i sem =W token (t i )+W seg (seg i )+W pos (i)

[0163] Among them, W token[i] , W seg[i] and W pos[i]They represent the token embedding vector of the i-th token obtained by table lookup, the unified sentence embedding vector, and the position embedding vector based on position i. Then the entire input sequence of length n is converted into a matrix E of shape n×d sem , where d is 768, corresponding to the hidden layer size of the Transformer layer in the RoBERTa model.

[0164] Then, the sequence matrix E rich in semantic information is sem It is fed into the core of the RoBERTa model, which consists of a 12-layer Transformer encoder. The pre-trained parameters of the model are used to initialize the network to deepen the understanding and encoding of the text. After this series of Transformer layers are processed layer by layer, the deep semantic features of each token and its context are gradually extracted. The output of the last layer of the Transformer stack is selected as the semantic encoding of the entire sentence. The formula is:

[0165]

[0166] Among them, H RB Indicates that the RoBERTa model is used to process E sem The obtained output condenses the global semantic representation of the sentence and acts on the subsequent PCNN disease entity recognition and relationship extraction.

[0167] 2. Disease entity recognition and relationship extraction based on RoBERTa-PCNN model

[0168] In the task of disease entity recognition and relationship extraction, a four-stage processing flow is used to accurately capture the names of crops, diseases and their complex relationships in response to the specific needs of the agricultural field. Figure 6 As shown in the figure, first, the disease feature extraction layer uses a multi-scale kernel to capture the local features of the crop name, disease name and its symptoms. Secondly, the disease text feature aggregation layer focuses on the disease entity relationship, such as the surrounding information of the disease name, highlighting the closely related etiological conditions and prevention methods. Then, the obtained features are imported into the disease feature fully connected layer, and multivariate nonlinear transformation is implemented to enhance the model's learning ability for complex and nonlinear relationship patterns, capturing the interaction between the degree of pathogen infection, the symptom evolution stage and environmental conditions. Finally, in the disease entity relationship classification prediction layer, the features are converted into probability distributions through the Softmax function, and argmax determines the most likely relationship category. In this way, the classification of complex associations between crop disease entities is achieved, revealing the influencing relationship between the symptoms, onset period and other key factors behind the crop name and the disease name.

[0169] (1) Disease feature extraction layer. Local features closely related to the relationship between crop names and disease names are extracted from the input features generated by RoBERTa. The convolution kernel slides on the input feature vector and performs convolution operations to capture the potential onset period, cause absolute condition patterns and features related to crop diseases in the text. The multi-scale convolution strategy strengthens the model to capture a wide range of features, covering symptoms, prevention and treatment information.

[0170] ① Each convolution kernel is a weight matrix with a specific size, usually represented by W con ∈R kcon ×d ifv , where R kcon represents the width of the convolution kernel (corresponding to the number of words in the text sequence), and d ifv matches the dimension of the input feature vector. These convolution kernels cover different spans in the text with different widths to capture local context features. The convolution kernels are located along H RB Scan one by one with a fixed step size and perform convolution operation on each window. For a given input feature vector sequence The convolution formula is:

[0171]

[0172] in, represents the feature map value obtained by applying the convolution kernel at position j, f naf is a nonlinear activation function, b pot is the bias term, H RB [j:j+k-1] represents the input feature vector subsequence starting from position j and with length k.

[0173] ② In order to fully capture feature information of different ranges, we traverse each position and apply convolution kernels of different scales. The accumulated feature value sequences together constitute a representation of a high-dimensional feature space, that is, multiple feature maps are generated. Each feature map directly reflects the response intensity distribution of a convolution kernel of a specific size on the original input features. This enables the RoBERTa-PCNN model to parse text structure from a multi-dimensional granularity, significantly enhancing the ability to identify complex semantic relationships.

[0174] like Figure 5 As shown in Figure 1, it is the detailed process of multi-kernel convolution. By summarizing the feature maps C generated at all different scales i mc (i=1,2,...,m), constructing a comprehensive feature representation that integrates spatial and semantic information at each unique scale. This multi-scale feature fusion strategy significantly enhances the model's ability to capture complex text structures, subtle visual patterns, and deep semantic associations.

[0175] ③ Add L2 regularization to the convolutional layer, which effectively limits the size of the model parameters by forcing the elements of the weight vector to approach zero, thereby suppressing the tendency of overfitting and improving the generalization performance of the model. The regularization formula is:

[0176]

[0177] Among them, L total Represents the final loss function, L data is the error of the model on the training data, W i con refers to the weight matrix of the lth layer, and λ con is the regularization coefficient, which is used to balance the model complexity and error.

[0178] By flexibly designing convolution kernels, adopting multi-scale kernel strategies and integrating regularization and activation functions, we can deeply explore and extract input features to provide feature description for the RoBERTa-PCNN model.

[0179] (2) Disease text feature aggregation layer. Focus on the key information around the disease entity, such as the text related to the crop name and disease name, selectively aggregate the features of the adjacent entities, extract the local optimal information that is most directly related to the core relationships such as the cause of the disease, prevention measures, etc., and reduce the high dimensionality of the convolution output.

[0180] ① Segment the text region. Given a text sequence S containing two entities ts =[w ts1 ,w ts2 ,...,w tsn ], where the entity pair is e 1 and e 2 , their starting and ending positions in the sequence are start(e 1 ), end(e 1 ), start(e 2 ), end(e 2 ), and assuming start(e 1 )<end(e 1 )<start(e 2 )<end(e 2 ). The entire text sequence is divided into three parts with specific semantic meanings through the PCNN model:

[0181] Entity 1 preamble area: from the beginning of the sequence to the end (e 1 ) before, denoted as C pre .

[0182] Inter-entity area: from end(e 1 ) to start(e2 ), denoted as C mid .

[0183] Entity 2 post-text area: from end(e 2 ) to the end of the sequence, denoted by C post .

[0184] ②Use the RoBERTa pre-training model to transform each word w in the text i Converted into a high-dimensional vector to provide rich semantic information for subsequent processing. Accordingly, each region C can be regarded as a collection of a series of continuous vectors This lays the foundation for further mining of contextual features.

[0185] ③Use multi-scale convolution kernel K mc Convolution operations are performed on each region to capture local feature patterns within different ranges. The formula is:

[0186]

[0187] Among them, f mck is the ReLU nonlinear activation function, w k is the weight of the kth convolution kernel, b k is the bias term, is the value of the feature map obtained by applying the kth convolution kernel at position j.

[0188] ④PCNN implements segmented maximum pooling on the convolutional feature map of each region to filter out the features that are most closely associated with the entity. For each region C, its pooling result P C It is obtained by aggregating the maximum value of all convolution feature responses in the region. The formula is:

[0189]

[0190] The segmented maximum pooling results of the three regions (preface, inter-region, and post-text) are concatenated to form a comprehensive feature vector F smp =(P pre ,P mid ,P post ), which comprehensively reflects all the key contextual information closely related to the two entities in the text.

[0191] The relevance and recognition accuracy of features are enhanced through segmented pooling, and the formed comprehensive feature vector provides a more concentrated and discriminative input for the disease feature fully connected layer, promoting the understanding of the entity relationship of complex corn diseases.

[0192] (3) Disease feature fully connected layer. The comprehensive feature vector F is formed by segment pooling. smp =(Ppre ,P mid ,P post ), a set of learnable weight matrices W is used fc ∈R m×n and the bias vector b fc ∈R m To further refine its features. Where m represents the number of neurons in the hidden layer, and n is the dimension of the input feature vector. And integrate the complex information of the previous layer, introduce the nonlinear activation function σ fc , so that the model can fit nonlinear data relationships. The formula is:

[0193] h fc =σ fc (W fc ·flat(X conv )+b fc )

[0194] Among them, X conv It is the feature matrix after convolution output and pooling, and flat() represents the flattening operation.

[0195] This layer generates a prediction vector obtained after a deep and comprehensive analysis of disease characteristics, which is expressed as the probability distribution of various diseases and is used in the next step of classification and prediction of complex relationships between disease entities.

[0196] (4) Disease entity relationship classification prediction layer. Based on the obtained disease entity relationship prediction vector, the features are mapped into the probability distribution of various relationships through Softmax transformation, and the highest probability category is confirmed using argmax to achieve fine classification of the complex relationship between crop names and disease names, revealing the multi-factor interaction mechanism of crop health. Each probability value intuitively reflects the model's confidence in the category. The formula is:

[0197]

[0198] Among them, x ci is the original value of the i-th category in the fully connected layer output, t rc is the total number of relationship categories.

[0199] Finally, to derive the final predicted category from the probability, the argmax function is used to traverse the entire probability vector, find and return the index position with the highest value, which directly corresponds to the relationship category, that is, the most likely correct answer considered by the model. It is expressed as:

[0200]

[0201] Through the above processing, a large amount of corn disease-related text data is converted into meaningful low-level feature representations, and then multi-level feature learning and integration are carried out, and these low-level features are refined so that they can be mapped to disease entities and their complex relationships, thereby predicting the type, degree and possible transmission relationship of corn diseases at the high-level category prediction level.

[0202] 3. Construction of disease triple library based on entities and relationships

[0203] The present invention successfully realizes the accurate identification of multiple disease entities from corn-related literature, covering key disease types such as corn leaf blight, corn leaf blight, rust, gray leaf spot and black smut, and deeply explores and extracts the complex relationships between these disease entities and between them and the crop growth cycle and environmental conditions, such as the prerequisites for disease occurrence, transmission paths, and interactive relationships with crop growth stages. It also includes effective disease management and prevention strategies, and constructs a disease ternary library for corn for the recommendation of disease control strategies.

[0204] Among them, the disease triplets related to corn gray spot disease include corn gray spot disease, disease-symptoms, gray spots appearing on leaves; corn gray spot disease, disease-onset period, the prone period of leaf disease before heading; corn gray spot disease, disease-cause meteorological conditions, high humidity and heavy rain; corn gray spot disease, disease-prevention measures, reasonable close planting to improve ventilation; corn gray spot disease, disease-treatment methods, spraying mancozeb; corn gray spot disease, disease-cause absolute conditions, zinc deficiency leads to decreased resistance.

[0205] Finally, step 3 of the method of the present invention is described in detail:

[0206] 4. Recommendation of disease control methods based on entity relationship optimization of graph attention network

[0207] In view of the problems that existing methods ignore the complex relationships between nodes, lack effective modeling of multi-relational data, and lack learning of entity hidden relationship representations in dealing with disease control recommendations, the present invention proposes a disease control method recommendation that combines the complex embedding (ComplEx) model and the graph attention network (GAT). First, at the global level, the ComplEx model is used to learn the triple representation of all entities related to corn diseases and their mutual relationships, which are embedded into the complex space, and the interaction between entities and relationships is captured through the inner product. Secondly, focusing on the local structure, the GAT model uses the learned entity embedding to initialize node features, and the implicit dependencies are strengthened through the dynamic weighted neighbor features of the graph structure, the implicit relationships between nodes are captured, and unknown relationship link predictions are generated. Finally, the deep representation of entities and relationships is dynamically optimized, and the weighted adjacency information and multi-head attention mechanism are combined. The interval sorting loss function is used to improve the discrimination power, and the optimal implicit relationship between different entities is obtained. Figure 6As shown, it is a flowchart for recommending disease control methods based on graph attention network entity relationship optimization.

[0208] 1. Disease multi-relation complex embedding based on ComplEx

[0209] Starting from the global structure of the corn disease knowledge base, in order to realize the link prediction of disease control methods, the present invention encodes the entities and relations in the corn disease triples in the complex vector space through the ComplEx model, analyzes the multi-dimensional associations between diseases and crops and prevention and control measures, processes complex multi-relationship data through complex vectors, and combines the attention mechanism to capture the deep semantic associations unique to the agricultural field, such as pathogen transmission, disease evolution related to crop growth stages, etc.

[0210] (1) Entity embedding, let the entity embedding matrix Each entity is mapped to a point in the complex vector field. Each row of the matrix corresponds to the c-dimensional complex vector representation of an entity in the knowledge graph. etotal represents the total number of entities in the knowledge graph, c es is the dimension of the embedding space. This matrix not only preserves the relative distance and directionality between entities, but also encodes the complex characteristics of entity relationships through the real and imaginary parts of complex numbers. The formula is:

[0211] e i =[e i1 ,e i2 ,...,e in ] T ∈C d

[0212] (2) Relational embedding, assuming a complex matrix Map each relationship in the knowledge graph to a complex vector space. Each row of the matrix represents a d-dimensional complex vector of a relationship, q rtotal is the total number of relations, d red It is the relationship embedding dimension, which uses the structural characteristics of complex numbers to describe the directionality, symmetry and even deeper semantic associations of the relationship. The formula is:

[0213] r i =[r j1 ,r j2 ,...,r jd ] T ∈C d

[0214] (3) Fusion of multi-dimensional features and complex relationships of entities. First, entities are integrated. Its neighbors and the relationship between them Embedding information of the entity and The embedding vector and relationship The embedding vectors of the entities are concatenated to generate a comprehensive vector. This joint representation not only takes into account the characteristics of the entities themselves, but also incorporates the complexity of the interaction between the two. By bridging the relationship vectors, it provides the model with a comprehensive perspective for evaluating the closeness of the relationship between the two. The formula is:

[0215]

[0216] Then the joint vector z obtained above is ijk It is fed into a linear transformation layer, which is passed through a set of learnable weight matrices W a ∈R d′×3d and the bias term b a ∈R d′ Map the joint vector to extract the deep correlation pattern between features. The formula is:

[0217] z ij =W a z ijk +b a

[0218] Then, the LeakyReLU activation function is used to increase the nonlinear expression ability of the model, alleviate the gradient vanishing problem, and ensure that the attention mechanism can more sensitively distinguish important and minor adjacency relationships, thereby calculating the absolute attention score of each neighbor entity to the central entity. The formula is:

[0219]

[0220] Among them, W z is an appropriate weight matrix, is the LeakyReLU activation function, b z is the bias term.

[0221] Finally, softmax is used to encourage the model to pay more attention to key information and suppress secondary information, thereby generating a relative attention weight for each neighbor entity. The formula is:

[0222]

[0223] 2. Construction of entity relationship recommendation graph based on graph attention network

[0224] For the local information of a single node, the triple representation learned from ComplEx is used as the initial feature of the node in the graph attention network model. GAT is used to focus on the local structure. Graph structure modeling is used to ensure the accurate matching of identified entities and relationships, generate a complete entity relationship structure, and dynamically assign different weights to each pair of nodes in the graph through the self-attention mechanism to capture the implicit relationship between nodes and generate corresponding relationship link predictions.

[0225] (1) Entity node initialization. Embed the entities learned from ComplEx into i =[e i1 ,e i2 ,...,e in ] T Directly use it as the initial feature of the node in GAT, let is each disease entity node v i The starting eigenvector of is:

[0226]

[0227] (2) Disease remediation relationship graph reasoning. Based on the hierarchical architecture of the graph attention network, at its first layer, for a given triplet of entity nodes v i , whose neighbor node set is N(v i ),new The embedding consists of all neighbor nodes j∈N i The features and edge relationships of i =[r j1 ,r j2 ,...,r jd ] T The weighted sum is obtained, and the weight reflects the strength and implicit dependency of the relationship between nodes. The formula is:

[0228]

[0229] Among them, N i is the set of neighbor nodes of node i, W (l) is the weight matrix of this layer, is the attention coefficient of node i to neighbor node j, and the formula is:

[0230]

[0231] in, is the relative attention weight of the central node for each neighbor entity in the ComplEx model, and || is the concatenation of feature vectors to enhance the ability to identify subtle differences and deep connections between nodes. The whole process dynamically assigns different weights to each pair of nodes in the graph through the self-attention mechanism, so as to better capture the implicit relationship between nodes and generate more accurate relationship link predictions.

[0232] Finally, through the learning of multi-layer GAT, the relationship link prediction between unknown disease entities, that is, the new disease-crop-environmental condition triples, can be achieved. This can not only help understand the behavior of known diseases, but also predict the potential transmission mode of new diseases, providing a scientific basis for disease prevention and control.

[0233] 3. Implicit entity relationship optimization based on embedding update

[0234] The static representation of GAT limits entities and relationships to automatically adapt to new information over time and in context, making it difficult to cover all potential associations and ignoring the depth and diversity of complex cross-hop relationships. The present invention dynamically optimizes the deep representation of entities and relationships, deepens entity embedding with weighted adjacency information from a multi-source perspective, enables relationship embedding to flexibly adapt to context fluctuations, incorporates a multi-head attention mechanism to reveal implicit patterns in the data, and refines the distinguishing ability with an interval sorting loss function, so that the model can obtain the optimal implicit relationship between different entities.

[0235] (1) Entity embedding update. For the entity e i , whose new embedding is obtained by summarizing all the surrounding related triplets, each of which is assigned different degrees of importance according to the attention weights calculated above. These weights reflect the relationship between the neighboring entities and e through different relationships. i The closeness of the association. By weighted summation of neighbors, entity e i The embedding of can absorb the characteristics of the social environment of corn gray spot, reflect the network structure and relationship semantics, and promote a deeper understanding and expression of entities. The formula is:

[0236]

[0237] Among them, T tr Representing entity e i The set of all participating triples, ⊙ represents the Hadamard product of complex vectors, reflecting the directionality and composite nature of the relationship in the ComplEx model.

[0238] In order to enable the model to understand the relationship between corn gray spot and other entities from different perspectives and deeply explore the implicit associations between entities, multiple independent attention heads are run in parallel, each focusing on a different representation subspace of the input information to capture multiple types of contextual features and relationship patterns. The formula is:

[0239]

[0240] Where M represents the number of attention heads, are the weights of each attention head, W O is the projection matrix.

[0241] (2) Dynamic relationship embedding update. In view of the dynamic association between operations such as treatment and prevention and diseases, a parameter generator mechanism is integrated to dynamically evolve the relationship embedding according to the current context information. k The embedding update process is to apply a parameter generator driven by the context to produce a series of linear transformation parameters Then adjust the original relation embedding. The formula is:

[0242]

[0243] in, and Represents the relationship r at time step t and t+1 respectively k The embedding vector of is the relation-specific linear transformation matrix computed by the parameter generator based on the current context, and is the bias term, which works together on the embedding update to ensure that the model can reflect and adapt to changes in relationship features in real time.

[0244] Updated embed It will be used in subsequent reasoning processes, such as predicting the probability of corn gray spot or recommending the best prevention and control measures, to ensure that the model's decisions can promptly reflect the latest impact of humidity changes on the disease.

[0245] (3) Optimize triplet scores. This is used to distinguish between effective and incorrect triplet relationships, ensuring that positive recommendations—using a certain fungicide to get a high score for corn gray spot—are better than negative examples, that is, incorrectly recommending inappropriate control measures. Ensuring that the correct triplet gets a higher score than the incorrect triplet can improve the accuracy and reliability of the control measure recommendations. The formula is:

[0246]

[0247] Among them, S tr is the positive sample set, is a set of negative samples, s(h,r,t) is a score function for a given triple, [·] + represents the positive part, γ tr Is a preset margin interval.

[0248] In summary, according to the test image input disease recognition and classification model, the result of corn gray spot disease is level 5, and combined with the disease triple library constructed by corn disease entity recognition and relationship extraction, this model recommends: first, apply Bacillus subtilis spray to inhibit pathogens in a biological way; secondly, physical prevention and control, use silver reflective film to reduce the damp environment preferred by pathogens, and regularly rotate crops to reduce the accumulation of pathogens in the soil; at the same time, strengthen field management, reasonably densely plant to improve ventilation and light conditions, and manually remove diseased leaves to reduce the spread of diseases.

[0249] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information, characterized in that: The following steps are involved: Step 1: Disease recognition and multi-level assessment model based on position attention mechanism, combined with the position attention mechanism, through convolutional neural network to enhance image feature extraction and spatial analysis, distinguish similar diseases, calculate the area and proportion of lesions to achieve disease grade assessment; Step 2: Based on the corn disease entity recognition and relationship extraction model, automatically annotate data, deeply mine the key entities and their complex relationships in the disease text, build a disease knowledge triple library, and deepen the understanding of corn diseases and related factors; Step 3: A disease remediation recommendation model based on a graph attention network enhanced with implicit dependencies is developed. The entity relationships in the disease knowledge triple library are learned in the complex space to generate link predictions. The graph attention network is used to strengthen node features to capture implicit relationships. The weighted adjacency information and multi-head attention mechanism are combined to dynamically optimize the deep representation of entities and relationships to achieve disease remediation recommendations.

2. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to claim 1 is characterized in that: Step 1 includes the following steps: Step 1.1, by improving the EMO-Transformer model, the inverted moving residual module with integrated position coding is stacked as a meta-moving block to enhance the ability of complex image feature extraction and spatial information analysis, and to distinguish between disease types with highly similar morphology; Step 1.2, passing the features identified in step 1.1 to the improved Unet-SE to map the distribution of the disease on the plant, and optimizing the segmentation results in combination with the conditional random field (CRF) to improve the recognition accuracy of the edge of the lesion; Step 1.3, calculate the area and proportion of the diseased spots in the marked area, and classify the disease severity according to the preset standards to evaluate the severity of the disease.

3. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to claim 2 is characterized in that: In step 1.1, the steps of performing multi-layer stacking of meta-movement blocks are as follows: ① Input dimension increase: increase the depth of the input feature map through a dimension-upgraded multi-layer perceptron to provide richer feature dimensions. This step is expressed as: X i ' cm =MLP up (X icm ) Among them, MLP up is a multi-layer perceptron with dimensionality enhancement function, which inputs the feature map X icm The number of channels increases, X' icm Represents the new feature map after dimensionality upgrade; ② Intermediate operator enhancement: applying a flexibly selected intermediate operator to further enhance feature representation; Y efm =Operator(X’ icm ) Among them, Y efm Represents the enhanced feature map, and Operator represents the Identity mapping; ③ Dimensionality reduction and residual connection, through another MLP down Perform dimensionality reduction to reduce the depth of the feature map to control model complexity, while introducing residual connections to promote gradient flow and training stability. This step is formulated as: Z mbo =MLP down (AND efm )+X isr Among them, Z mbo is the output of the current meta movement block, +X isr represents the inverse shifted residual.

4. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to claim 2 is characterized in that: In step 1.1, the inverted moving residual module uses the extreme separation convolution block (XSep-Conv) to efficiently extract local features through spatial separation convolution, deep convolution and point-by-point operation to reduce the number of parameters and ensure the accuracy of feature capture; at the same time, the extended window multi-head self-attention (EW-MHSA) is used to decompose the complex self-attention calculation into multiple small-scale operations through block calculation, effectively controlling the computational complexity.

5. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to claim 2 is characterized in that: In step 1.1, the disease types that are highly similar in morphology are distinguished by compressing the feature map through global average pooling, and then mapped to the disease category through the fully connected layer, outputting the disease probability distribution, and selecting the highest probability category as the accurate classification prediction result for leaf diseases.

6. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to claim 1 is characterized in that: Step 2 includes the following steps: Step 2.1: Use the RoBERTa model to perform deep context encoding on large-scale pre-training data to capture the nuances and implicit information in agricultural texts; Step 2.2, using the multi-level structure of the PCNN model to realize disease entity recognition and relationship extraction, classify the complex associations between crop disease entities, and reveal the influencing relationship between the symptoms, onset period and other key factors behind the crop name and disease name; Step 2.3, form a corn disease triple library containing entities and relationships.

7. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to claim 1 is characterized in that: Step 3 includes the following steps: Step 3.1, at the global level, the ComplEx model is used to learn the triple representation of all entities and their relationships related to corn diseases, embed them into the complex space, and capture the interaction between entities and relationships through inner products; Step 3.2, focusing on the local structure, using the learned entity embedding to initialize node features through the GAT model, strengthening implicit dependencies through the dynamic weighted neighbor features of the graph structure, capturing the implicit relationship between nodes, and generating unknown relationship link predictions; Step 3.3, dynamically optimize the deep representation of entities and relationships, combine weighted adjacency information and multi-head attention mechanism, use interval sorting loss function to improve the discrimination ability, obtain the optimal implicit relationship between different entities, and realize disease control recommendation.

8. The method for intelligent grading assessment and control recommendation of corn diseases integrating multimodal information according to any one of claims 1 to 7, characterized in that: Before step 1, the steps of data set construction and preprocessing are also included.

Images