An interpretable agricultural intelligent prediction method and system based on neural symbol fusion

Abstract

This invention belongs to the field of artificial intelligence and agricultural technology, proposing an interpretable intelligent agricultural prediction method and system based on neural symbol fusion. The method includes: preprocessing user-uploaded agricultural image data and environmental data acquired by IoT sensors to obtain standardized image data and structured environmental data; constructing a bi-branch deep learning model and training it with symbolic regularization; inputting the standardized image data into the trained bi-branch deep learning model for decision-making, outputting preliminary diagnostic results and confidence levels; using interpretive tools to generate a saliency heatmap highlighting the most important image regions for the decision; performing knowledge graph verification, rule extraction verification, and symbolic reasoning verification on the preliminary diagnostic results, generating verification results for each; and checking the consistency between the preliminary diagnostic results and the verification results to determine whether to output a decision. This invention solves the problem of opaque model decision-making processes in existing technologies.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and agricultural technology, and in particular relates to an interpretable agricultural intelligent prediction method and system based on neural symbol fusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Deep learning models play a crucial role in agricultural applications. First, they can process and analyze agricultural data, including soil quality, weather conditions, and crop growth, providing decision support for agricultural production. Second, through continuous learning and optimization, deep learning models can identify and issue early warnings for crop diseases and pests, improving crop yield and quality. Furthermore, deep learning models can be applied to the navigation and operation control of agricultural robots, enhancing the automation and intelligence of agricultural production, reducing labor costs, and increasing production efficiency.

[0004] However, existing deep learning models for agricultural applications have the following main drawbacks and unresolved technical problems:

[0005] The "black box" nature of deep learning models means their decision-making process is opaque, making it difficult for users to understand the basis of their predictions or recommendations. This leads to insufficient trust in AI systems, especially in critical decision-making areas like agriculture, where inaccurate advice can cause significant losses and environmental impacts.

[0006] Lack of agronomic compliance verification: Existing models struggle to directly verify whether their predictions conform to established agronomic principles and rules. Explanations are often based on statistical derivation rather than human-understandable agronomic logic.

[0007] Trust gap hinders application: Due to a lack of explainability, farmers and agronomists hesitate to adopt AI technology, limiting the full realization of AI's application value and potential benefits in actual agricultural production.

[0008] Difficulty in handling uncertainty and incompleteness: The inherent uncertainties in the agricultural field (such as weather changes, soil heterogeneity, and sensor data noise) and the incompleteness of knowledge pose challenges to the robustness of pure deep learning models. Summary of the Invention

[0009] To overcome the shortcomings of the prior art, this invention provides an interpretable agricultural intelligent prediction method and system based on neural symbol fusion. It proposes an end-to-end architecture of "model prediction, three-way verification, consistency check, and reliable output" to solve the problems of opaque model decision-making process, lack of agronomic compliance verification, difficulty in handling uncertainty and incompleteness, and the existence of trust gap obstacles in the prior art.

[0010] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0011] The first aspect of this invention provides an interpretable intelligent agricultural prediction method based on neural symbol fusion, comprising:

[0012] The agricultural image data uploaded by users and the environmental data obtained by IoT sensors are preprocessed to obtain standardized image data and structured environmental data.

[0013] Construct a dual-branch deep learning model and perform symbolic regularization training on the dual-branch deep learning model;

[0014] Standardized image data is input into a trained dual-branch deep learning model to make decisions, and the model outputs preliminary diagnostic results and confidence levels.

[0015] Interpretive tools are used to generate saliency heatmaps that highlight the image regions most important for decision-making;

[0016] The preliminary diagnostic results are verified using knowledge graphs, rule extraction, and symbolic reasoning, and verification results are generated for each.

[0017] The initial diagnostic results are checked for consistency with the validation results, and a decision is made on whether to output a decision.

[0018] Furthermore, the dual-branch deep learning model includes a feature extraction module, a feature transformation module, a concept encoder, and a concept inferencer. Specifically, the standardized image data is input into the feature extraction module, which outputs a high-dimensional feature map; the high-dimensional feature map is input into the feature transformation module, which outputs a global visual feature vector; the global visual feature vector is input into the concept encoder, which outputs a concept vector; and the concept vector is input into the concept inferencer, which outputs a preliminary diagnostic result and confidence level.

[0019] Furthermore, the concept encoder maps high-dimensional global visual feature vectors to low-dimensional concept vectors; specifically, the concept encoder consists of two fully connected layers. The first layer uses the ReLU activation function to achieve non-linearity, and the second layer uses the Sigmoid function to compress each dimension to [0,1]. Each concept vector corresponds to a predefined agricultural semantic concept.

[0020] The concept inference engine processes the concept vectors. Specifically, the concept inference engine includes a two-layer MLP classifier. The first layer maps the low-dimensional concept vectors to a space and activates them using the ReLU activation function. The second layer outputs multiple unnormalized disease classifications, which are then converted into probability distributions using the Softmax function. The maximum probability is taken as the preliminary diagnosis result, and its confidence level is recorded.

[0021] Furthermore, the knowledge graph verification matches the image regions of interest to the dual-branch deep learning model with symptoms and environmental conditions in the agricultural knowledge graph; specifically, the knowledge graph verification includes:

[0022] Image thresholding is performed on the saliency heatmap to preserve high-activation regions;

[0023] Local image patches are extracted using connected component analysis;

[0024] Use a lightweight CNN classifier to identify typical symptom features;

[0025] The graph database is used to verify the consistency between symptoms, environmental conditions, and the agricultural knowledge graph. When both symptom matching and environmental support are met, the knowledge graph verification result is determined as "consistent" or "inconsistent".

[0026] Furthermore, the rule extraction and verification includes using the LIME algorithm to extract decision-related local rules from the two-branch deep learning model, and semantically aligning these rules with the agricultural knowledge graph to achieve transparency in the model's behavior; specifically, the implementation of the LIME algorithm includes:

[0027] The preprocessed standardized image data is divided into superpixels to generate perturbation samples;

[0028] Calculate the weights of the perturbed samples and fit a local linear model;

[0029] Extract the agronomic concepts corresponding to the top three superpixels with the highest weights and generate IF-THEN rules;

[0030] The concept numbers in the rules are mapped to knowledge graph entities, and the consistency with medical consensus is verified. The rule extraction verification determines whether the verification result is "correct" or "incorrect".

[0031] Furthermore, the symbolic reasoning verification includes probabilistic symbolic reasoning and fuzzy logic; specifically, the probabilistic symbolic reasoning includes:

[0032] The concept vectors and structured environment data generated by the dual-branch deep learning model are converted into first-order logical predicates;

[0033] Load the Markov logic network rule base, which contains weighted first-order logic formulas;

[0034] The posterior probability of various diseases is calculated using a probabilistic inference algorithm, and the verification results are output.

[0035] Furthermore, the consistency check includes:

[0036] If the verification results of knowledge graph verification are "consistent", rule extraction verification are "correct", and symbolic reasoning verification are the preliminary diagnostic results output by the model, then it is determined to be consistent, enters the credible decision channel, and generates a final diagnostic report with multi-source evidence support.

[0037] Otherwise, it is determined to be a "conflict", and the arbitration mechanism is initiated to calculate the weighted score and select the result with the highest score as the compromise solution; or a diagnostic conflict report is directly generated and a manual review is requested.

[0038] A second aspect of the present invention provides an interpretable agricultural intelligent prediction system based on neural symbol fusion, comprising:

[0039] The data preprocessing module is configured to preprocess the agricultural image data uploaded by the user and the environmental data acquired by the IoT sensor to obtain standardized image data and structured environmental data.

[0040] The model training module is configured to: construct a two-branch deep learning model and perform symbolic regularization training on the two-branch deep learning model;

[0041] The model decision module is configured to input standardized image data into the trained dual-branch deep learning model to make decisions and output preliminary diagnostic results and confidence levels.

[0042] The decision visualization module is configured to use interpretive tools to generate a saliency heatmap that highlights the image regions most important to the decision.

[0043] The multi-path verification module is configured to perform knowledge graph verification, rule extraction verification, and symbolic reasoning verification on the preliminary diagnosis results, and generate verification results respectively.

[0044] The consistency check module is configured to perform a consistency check between the preliminary diagnostic results and the verification results and decide whether to output a decision.

[0045] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of an interpretable intelligent agricultural prediction method based on neural symbol fusion as described in the first aspect of the present invention.

[0046] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of an interpretable agricultural intelligent prediction method based on neural symbol fusion as described in the first aspect of the present invention.

[0047] The above one or more technical solutions have the following beneficial effects:

[0048] This invention integrates agronomic knowledge directly into model training and architecture design through symbolic regularization and concept-based models. This allows the model to learn patterns that conform to agronomic rules from the outset and reason using human-understandable concepts, achieving endogenous interpretability and agronomic compliance in prediction results, rather than simple ex post facto attribution. By providing agronomically logical and human-understandable explanations, it directly addresses the "trust gap" problem.

[0049] This invention, through the integration of knowledge graphs, can elevate low-level feature attribution to high-level, semantically rich agronomic explanations, providing a complete logical chain and background knowledge behind the predictions, making the explanations more practical.

[0050] This invention combines probabilistic symbolic reasoning and fuzzy logic to quantify uncertainty in prediction and handle fuzzy agronomic knowledge, providing more detailed, confidence-based recommendations, thereby offering more reliable decision support in complex and ever-changing agricultural environments.

[0051] The end-to-end integrated neural-symbolic hybrid architecture proposed in this invention combines data-driven learning with symbolic logical reasoning, achieving seamless integration of learning and reasoning. From raw data input to final agronomic decision output, the symbolic reasoning engine performs logical reasoning on these concepts to form the final decision recommendations, ensuring the transparency and traceability of the entire decision-making process. This is crucial for high-risk agricultural decision-making and is particularly suitable for applications in high-risk fields such as agriculture.

[0052] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0053] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0054] Figure 1 This is a flowchart of the interpretable intelligent agricultural forecasting method in the first embodiment;

[0055] Figure 2 This is a flowchart of the knowledge graph verification process for the first embodiment;

[0056] Figure 3 Here is a flowchart of the rule extraction and verification process for the first embodiment;

[0057] Figure 4 This is a flowchart of the symbolic reasoning verification process for the first embodiment. Detailed Implementation

[0058] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0059] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0060] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0061] Example 1

[0062] like Figure 1 As shown, this embodiment discloses an interpretable agricultural intelligent prediction method based on neural symbol fusion, including:

[0063] Step S1: Preprocess the agricultural image data uploaded by the user and the environmental data obtained by the IoT sensor to obtain standardized image data and structured environmental data.

[0064] Specifically,

[0065] S1-1: Upon receiving an agricultural image uploaded by a user, the image is first decoded into an RGB three-channel tensor, and its size is uniformly adjusted to 224×224 pixels to meet the input requirements of the subsequent deep neural network. Bilinear interpolation is used during the adjustment process to preserve image details. Normalization is performed on each pixel value.

[0066] S1-2: Analyze environmental data from IoT sensors, including key fields such as temperature, relative humidity, light intensity, and soil moisture. Standardize these values ​​to a unified dimension, for example, convert humidity H to H / 100∈[0,1], and encode crop type and growth stage as category indexes for subsequent knowledge verification.

[0067] Step S2: Construct a two-branch deep learning model and perform symbolic regularization training on the two-branch deep learning model.

[0068] Specifically, the constructed dual-branch deep learning model includes a feature extraction module, a feature transformation module, a concept encoder, and a concept inferencer.

[0069] In step S2, symbolic regularization training integrates explicit agronomic rules (e.g., logical rules extracted from agronomic ontology) as regularization terms into the training loss function of the neural network model, ensuring that the model follows agronomic rules from the training stage and has interpretability and agronomic compliance.

[0070] In this embodiment, a "logistic loss" function is introduced to guide the model's learning process, penalizing any predictions that contradict agronomic rules. The specific process is as follows:

[0071] In the standard loss function Introducing logical loss terms based on This forms the final hybrid loss function. :

[0072]

[0073] Here, λ is a hyperparameter used to balance data fitting and logical consistency.

[0074] The design is based on first-order logic rules. For example, an agronomic rule can be expressed as "If the crop exhibits symptom A (…)", which means "If the crop exhibits symptom A (…)", then the agronomic rule is correct. If it is not infected with disease B, then it is impossible for it to have disease B. ), its logical form is " "For the model's predicted probability P( |x) and P( |x), given an input image x, construct a loss using the t-norm. The degree of violation of this rule can be quantified as:

[0075]

[0076] During the model's prediction process, if a crop is predicted with a high probability... And predicted a higher probability In situations where this contradicts agronomic rules, a large loss value will be obtained according to the loss calculation method. However, under the backpropagation mechanism in neural network training, this large loss value will prompt the model to adjust its parameters. By continuously optimizing the parameters, the model's subsequent predictions can follow the established agronomic rule of "if the crop exhibits symptom A, then it cannot have disease B," thereby improving the logical rationality and accuracy of the model's predictions. Losses are all related to agronomic rules. The sum:

[0077]

[0078] In addition, a regularizer based on the Jacobian matrix is ​​introduced. This encourages neural networks to learn functions that are smoother and easier to extract into human-readable notation. The regularizer calculates the norm of the gradient of the model output f with respect to the input x:

[0079]

[0080] Represents the Jacobian matrix. Let represent the squared norm of the Jacobian matrix; This represents the i-th component of the model output, such as the predicted probability of the i-th disease. This represents the j-th dimension of the input feature, such as the intensity of a symptom, temperature, humidity, etc. Indicates output For input The partial derivatives reflect How to influence .

[0081] Adding it to the total loss function can effectively improve the fidelity of subsequent rule extraction algorithms:

[0082]

[0083] Indicates mission loss. This represents logical rule constraints. Indicates smoothness regularization, , is a coefficient.

[0084] In this embodiment, knowledge guidance based on symbolic regularization not only enables the model to intrinsically adhere to specific logical rules during data fitting, but also improves the interpretability and credibility of its prediction results. For example, if there is an agronomic rule: "If the leaves show ring-shaped spots with yellow edges, it is highly likely to be anthracnose," this rule will be incorporated into the training process as a loss term, thereby guiding the model to learn knowledge representations that conform to this logic.

[0085] Step S3: Input standardized image data into the trained dual-branch deep learning model to make a decision and output preliminary diagnostic results and confidence levels.

[0086] S3-1: Feature extraction module: Preprocessed standardized image data x∈ The image is fed into the backbone feature extraction network ResNet-50. This network consists of multiple stacked residual blocks, which extract local and global features of the image layer by layer through convolution, batch normalization, and ReLU activation functions. During forward propagation, the input image passes through an initial 7×7 convolutional layer and a max-pooling layer, then enters a four-stage residual structure, ultimately outputting a high-dimensional feature map A∈ with a spatial dimension of 7×7 and 2048 channels. .

[0087] S3-2: Feature Transformation Module: To transform spatial features into a global representation, a global average pooling operation is performed on the high-dimensional feature map A, that is, the average value is calculated for each channel in the spatial dimension, resulting in a 2048-dimensional global visual feature vector. ∈ The vector Considered a "global visual fingerprint" of an image, it contains deep semantic information about the disease and forms the basis for all subsequent interpretability analyses.

[0088] The model's decision-making process consists of two steps: first, mapping the original input data to a high-level "concept space," and then making a final prediction based on these meaningful concepts. That is, the model f(x) can be represented as a composite function of two sub-networks:

[0089]

[0090] S3-3: Introducing the Concept Encoder Abstract global visual feature vectors Mapped to a concept vector The concept encoder It consists of two fully connected layers: the first layer maps 2048-dimensional features to 512 dimensions, using the ReLU activation function to achieve non-linearity. The second layer further compresses the 512-dimensional vector to 50 dimensions, and uses the Sigmoid function to compress each dimension to the interval [0,1], resulting in the final concept vector c∈[0,1]. Each concept vector corresponds to a predefined agricultural concept, for example... Indicates the prominence of "angular lesions". This design, which indicates the degree of "leaf yellowing," makes the model's internal representation readable. The formula is as follows:

[0091]

[0092]

[0093] in, For preprocessed standardized image data, This represents the confidence level for the existence of the i-th agricultural concept, where i∈[1,k]. This represents the global visual feature vector, where σ is the Sigmoid activation function. This represents the transpose of the weight vector (column vector) corresponding to the i-th agricultural concept. This represents the bias term corresponding to the i-th concept.

[0094] S3-4: After obtaining the semantic concept vector c, introduce a concept inference engine. Perform the final disease classification. (In the conceptual reasoning engine) In this conceptual inference engine, the concept vector c is taken as input, and the preliminary diagnostic result y (such as the specific disease) is output. A two-layer MLP classifier is used: the first layer maps 50-dimensional concept vectors to a 128-dimensional space and uses ReLU activation; the second layer outputs 10 unnormalized disease classifications. Then, the class is transformed into a probability distribution p using the Softmax function, and the class with the highest probability is taken as the preliminary diagnosis result y, with its confidence level conf = max(p) recorded. The formula is as follows:

[0095]

[0096] in, This is the first layer weight matrix of the concept inference engine. This is the bias vector for the first layer of the concept inference engine.

[0097] The input to the concept inferencer g is a well-defined concept vector c aligned with human cognition, and its internal decision-making logic can be directly interpreted. For example, if y corresponds to "downy mildew," and the weights... The highest weight values ​​were associated with the agricultural concepts of "white mold on the underside of leaves" and "angular lesions," which provided a clear and agronomically logical explanation for the model's predictions.

[0098] In this embodiment, the dual-branch deep learning model, consisting of a concept encoder h(x) and a concept inferencer g(c), enables the model to align its internal feature representations with easily understood agricultural concepts such as "yellowing," "wilt," and "leaf spots." In this way, the model's decision path is no longer a black box operation, but rather based on reasoning according to specific agricultural concepts, enhancing the intuitiveness and transparency of the model's output. For example, when the model diagnoses a crop with "downy mildew," it can indicate that the conclusion is based on the features of "white mold on the underside of the leaves" and "angular lesions," which is entirely consistent with the understanding of agronomic experts.

[0099] Step S4: Use interpretive tools to generate a saliency heatmap that highlights the image regions most important for decision-making.

[0100] To support subsequent interpretability verification, a saliency heatmap is generated simultaneously to visualize the image regions upon which the model's decisions depend. In this embodiment, the saliency heatmap is generated using the Grad-CAM algorithm.

[0101] S4-1: Let ∈ This is the feature map of the k-th feature channel of the last convolutional layer in the ResNet-50 backbone feature extraction network. If the target category is the output of the model, then this channel is... Sensitivity is determined by gradient express. This represents the activation value of the k-th feature channel at position (i,j) in the last convolutional layer of ResNet-50, where i represents the row index in the vertical direction of the feature map and j represents the column index in the horizontal direction.

[0102] S4-2: Average the gradient of each channel in the spatial dimension to obtain the global average gradient. :

[0103]

[0104] Global average gradient This reflects the importance of the k-th feature channel to the final decision.

[0105] S4-3: Extract the feature map of each channel Its corresponding global average gradient Multiplying and summing along the channel dimensions yields a coarse-grained heatmap:

[0106]

[0107] The heatmap only retains a 7×7 spatial resolution and needs to be further upsampled to the original image size.

[0108] S4-4: Using bilinear interpolation to... The spatial resolution was enlarged to 224×224, and negative values ​​were filtered out using the ReLU function to retain the regions that contribute to positive decisions.

[0109]

[0110] This represents the image region on which the model's decisions depend, i.e., the final saliency heatmap.

[0111] The final saliency heatmap exists as a grayscale image; higher brightness indicates that the region is more important to the model's decision. This image will serve as a key input to the knowledge graph validation module. For example, after analyzing an image of a cucumber leaf, the model predicts a 95% probability of "downy mildew." Simultaneously, the heatmap generated by the interpretive tool highlights the yellow angular spots on the upper surface of the leaf.

[0112] Step S5: Perform knowledge graph verification, rule extraction verification, and symbolic reasoning verification on the preliminary diagnosis results, and generate verification results respectively. The three verification paths are parallel and independent.

[0113] S5-1: Knowledge Graph Validation K – Matching the image regions of interest to the dual-branch deep learning model with symptoms and environmental conditions in the agricultural knowledge graph.

[0114] After the model makes a decision, its output (such as category labels or saliency heatmaps) is associated with the Agriculture Knowledge Graph (AKG). The AKG is a structured semantic network that contains entities such as crops, diseases, symptoms, environment, and control measures, as well as the complex relationships between them.

[0115] The specific process is as follows:

[0116] S5-1-1: First, read the significance heatmap. Image processing is performed: a threshold τ = 0.5 × max( ), set pixels with values ​​below this value to zero, and preserve highly active areas;

[0117] S5-1-2: Perform connected component analysis to identify multiple salient activation regions. For each region, extract its bounding box and crop it from the original image to form a local image patch. ;

[0118] S5-1-3: Use a lightweight CNN classifier to perform texture recognition on each image patch to determine whether it contains typical symptoms. For example, if a region is identified as "angular macular degeneration," it is converted into a node in the agricultural knowledge graph using a predefined semantic mapping table;

[0119] S5-1-4: Connect to the Neo4j graph database and execute a Cypher query to obtain a list of all symptoms corresponding to "downy mildew". If the node from S5-1-3 appears in the returned results, it is determined that there is a causal relationship between the symptom and the disease. Simultaneously, query the environmental preference relationship; if "high humidity environment" is returned and the current humidity H > 80%, it is determined that the environmental conditions support the occurrence of the disease. Only when both symptom matching and environmental support are met is the knowledge graph verification result determined. "Consistent" is selected; otherwise, "Inconsistent" is selected. The graph database includes an agricultural knowledge graph.

[0120] For example, after analyzing an image of a cucumber leaf, the model predicts a 95% probability of "downy mildew." Simultaneously, the generated heatmap highlights the yellow angular spots on the upper surface of the leaf. This information is then fed into the Agricultural Knowledge Graph (AKG) for querying and inference. AKG returns related information, ultimately interpreting the diagnosis as: the model diagnoses "downy mildew" based on the detection of "yellow angular spots" on the leaf. According to the agricultural knowledge graph, yellow angular spots are a typical symptom of downy mildew. This disease is caused by *Pseudomonas columbia* and is prone to occur in high humidity environments. Fungicide spraying is recommended for control.

[0121] In this embodiment, a structured agricultural knowledge graph is used as an external semantic context to transform the low-level feature attribution or prediction labels of the deep learning model into high-level, agronomically meaningful explanations. When the model predicts a crop disease, the agricultural knowledge graph can provide detailed information such as typical symptoms, predisposing environmental conditions, and recommended control measures, thus providing users with comprehensive background knowledge support. Furthermore, by querying entities and relationships, the model's output can be mapped to a broader agronomic knowledge system, further enhancing the depth and practicality of the explanation.

[0122] S5-2: Rule Extraction and Validation R – From a trained neural network, rules in a human-understandable form are extracted using algorithms. Instructional algorithms approximate the behavior of the model by making local perturbations around it.

[0123] In this embodiment, the LIME (Local Interpretable Model-agnostic Explanations) algorithm is used to provide local interpretations of the model decisions.

[0124] The specific process is as follows:

[0125] S5-2-1: Divide the preprocessed standardized image data into several superpixels, and perform "masking" perturbation on each superpixel with a 50% probability to generate 1000 perturbation samples x′;

[0126] S5-2-2: For each perturbation sample, record the probability distribution p′ of the model output and calculate its similarity to the original prediction p, using it as the weight π(x′) for that sample. The core of this step is to quantify the impact of each perturbation sample on the model's original prediction. Specifically, by comparing the similarity between the model's predicted probability distribution p on the original image and the predicted probability distribution p′ on each perturbation sample x′, we can assess which superpixel occlusions have a key impact on the model's decision. Similarity calculation can employ various metrics, such as cosine similarity and KL divergence, with the aim of identifying those perturbation samples that significantly change the model's prediction, thereby revealing the key features or regions behind the model's decision.

[0127] Subsequently, a local linear model is fitted to calculate the importance weight π(x′) for each perturbed sample:

[0128]

[0129] Where z′∈{0,1}, m is a binary vector of a superpixel (1 indicates existence, 0 indicates occlusion). ∈ is the weight vector to be learned.

[0130] S5-2-3: By minimizing the weighted error:

[0131]

[0132] Solving for optimal weights Take the agronomic concepts corresponding to the top three superpixels with the highest weights and generate IF-THEN rules.

[0133] S5-2-4: Map the concept numbers in the rules to knowledge graph entities, verify consistency with medical consensus, extract and verify rules, and determine the verification results. For "correct" or "incorrect".

[0134] In this embodiment, the instructional algorithm LIME is used to extract local decision rules from the trained model and map these rules to semantic concepts and relationships in the agricultural knowledge graph, thereby achieving transparency in the model's behavior. For example, when the model determines that an image is "powdery mildew," a locally linear interpretation model can be obtained through the LIME algorithm. This model reveals that the most important decision criterion is the presence of a superpixel with a "white powdery coating" in the image, which is transformed into a specific decision rule:

[0135] IF 'Texture characteristics of region A' = 'White powder' AND 'Color of region A' = 'White'

[0136] THEN Prediction = 'Powdery mildew'

[0137] The extracted rule can be further semantically aligned with AKG, mapping the "white powdery" texture feature to the concept of nodes in S5-1-3, thereby verifying the agronomic rationality of the model's decision and revealing the key decision basis - "the texture feature of region A is white powdery". This transforms the originally difficult-to-understand "black box" model into a clear agronomic logic expression.

[0138] S5-3: Symbolic Reasoning Verification S – This includes two consecutive sub-steps: First, based on the confidence level of each agricultural concept in the concept vector, candidate diagnostic rules are generated through probabilistic symbolic reasoning; then, the candidate rules are input into a pre-constructed agricultural knowledge graph, and fuzzy logic is used to verify their semantic consistency. Only when the rule does not conflict with the knowledge of domain experts is it considered a valid explanation.

[0139] (1) Probabilistic symbolic reasoning—converting concepts and data into logical predicates and reasoning through probabilistic logic networks.

[0140] The specific process is as follows:

[0141] (1.1): Convert the concept vector c and structured environment data into first-order logical predicates;

[0142] For example, the concept of a "white powdery coating" identified in an image, along with related structured environmental data such as region location, color, and texture, can be transformed into a first-order logical predicate expression: "There exists a region A, the texture feature of region A is white powdery, and the color of region A is white." In this way, probabilistic logic networks can be used to reason about these predicates, further verifying the rationality and accuracy of the model's decisions.

[0143] (1.2): Load the Markov logic network rule base, where each rule has the form ( , ), It is a first-order logic formula. Assign a weight to this rule based on its importance throughout the reasoning process;

[0144] The probability distribution of a specific "world" state S is defined as follows:

[0145]

[0146] in, This represents the number of instances of the i-th rule that is true in state s, i.e., the number of specific instances that satisfy the first-order logic formula. Then, based on the weight of each rule, these base formula counts are weighted and summed, and the probability distribution of world s is finally obtained through an exponential function and a normalization factor Z. This allows the system to handle rules with uncertainty. For example, when the rule "the presence of severe symptoms indicates a high risk to the crop" has a high weight, the system will infer with high probability that the crop is in a high-risk state after observing severe symptoms.

[0147] (1.3): The posterior probability of various diseases is calculated using a probabilistic inference algorithm, and the symbolic inference results are finally output. It also includes the reasoning path and confidence level.

[0148] In this embodiment, probabilistic symbolic reasoning effectively addresses the inherent uncertainties in the agricultural field, such as weather changes and sensor data noise.

[0149] (2) Fuzzy logic – provides a mathematical framework for handling imprecise agronomic concepts, such as high temperature or yellowing leaves.

[0150] Specifically:

[0151] (2.1): Define fuzzy sets and membership functions for imprecise agronomic concepts, mapping precise sensor input values ​​to the [0,1] interval to represent the degree to which they belong to a certain fuzzy set. For example, define three fuzzy sets for "soil moisture": "dry," "moist," and "overly moist," and use trapezoidal or Gaussian membership functions to calculate the degree to which precise moisture values ​​belong to each set. This can be defined as follows:

[0152]

[0153] The relative humidity h of the soil is mapped to a value indicating a degree of "humidity".

[0154] (2.2): Reasoning is based on defined fuzzy rules, with the rule form: IF [premise] THEN [conclusion]; where the premise is a combination of multiple fuzzy propositions, and the conclusion is a fuzzy set. For example,

[0155] IF soil moisture IS moist AND temperature IS high

[0156] THEN'Root Rot Risk' IS'Medium'

[0157] (2.3): The output fuzzy set obtained by fuzzy inference is converted into a clear risk index or decision value by defuzzification method.

[0158] In this embodiment, fuzzy logic is used to effectively address the incompleteness of knowledge in the agricultural field.

[0159] By combining probabilistic symbolic reasoning and fuzzy logic, it is possible to quantify the uncertainty in forecasting and handle fuzzy agronomic knowledge, providing more detailed, confidence-based recommendations, thereby offering more reliable decision support in complex and ever-changing agricultural environments.

[0160] In this embodiment, this end-to-end integrated neural-symbolic hybrid architecture ensures transparency and traceability throughout the decision-making process, making it particularly suitable for applications in high-risk fields such as agriculture. The front end uses a neural network to extract sub-symbolic features from raw data and transforms them into advanced agronomic concepts; the back end utilizes a symbolic reasoning engine to perform logical reasoning on these concepts, forming the final decision recommendations. The neural network is a pre-trained deep convolutional neural network, ResNet-50, whose backbone architecture includes an initial convolutional layer, four residual block groups (containing 3, 4, 6, and 3 Bottleneck residual units respectively), and outputs a 2048-dimensional feature vector before a global average pooling layer. This network has been pre-trained on the ImageNet dataset and further fine-tuned using a labeled image dataset containing 12 types of crop diseases to adapt to specific agricultural diagnostic tasks.

[0161] Step S6: Perform a consistency check between the preliminary diagnostic results and the verification results and decide whether to output a decision.

[0162] Consistency checks include:

[0163] If the verification results of knowledge graph verification are "consistent", rule extraction verification are "correct", and symbolic reasoning verification are the preliminary diagnostic results output by the model, then it is determined to be consistent, enters the credible decision channel, and generates a final diagnostic report with multi-source evidence support.

[0164] Otherwise, it is determined to be a "conflict," and the arbitration mechanism is initiated to calculate the weighted score:

[0165]

[0166] This indicates whether the verification results of the knowledge graph validation are consistent with the preliminary diagnostic results. This indicates whether the IF-THEN rule extracted from the model is judged as "correct". This indicates whether the verification results of symbolic reasoning verification are consistent with the preliminary diagnostic results of the model.

[0167] Choose the result with the highest score as a compromise; or directly generate a diagnostic conflict report and push it to the agricultural technology expert service platform to request manual review.

[0168] Example 2

[0169] This embodiment discloses an interpretable agricultural intelligent prediction system based on neural symbol fusion, comprising:

[0170] The data preprocessing module is configured to preprocess the agricultural image data uploaded by the user and the environmental data acquired by the IoT sensor to obtain standardized image data and structured environmental data.

[0171] The model training module is configured to: construct a two-branch deep learning model and perform symbolic regularization training on the two-branch deep learning model;

[0172] The model decision module is configured to input standardized image data into the trained dual-branch deep learning model to make decisions and output preliminary diagnostic results and confidence levels.

[0173] The decision visualization module is configured to use interpretive tools to generate a saliency heatmap that highlights the image regions most important to the decision.

[0174] The multi-path verification module is configured to perform knowledge graph verification, rule extraction verification, and symbolic reasoning verification on the preliminary diagnosis results, and generate verification results respectively.

[0175] The consistency check module is configured to perform a consistency check between the preliminary diagnostic results and the verification results and decide whether to output a decision.

[0176] Example 3

[0177] The purpose of this embodiment is to provide a computer-readable storage medium.

[0178] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an interpretable agricultural intelligent prediction method based on neural symbol fusion as described in Embodiment 1 of this disclosure.

[0179] Example 4

[0180] The purpose of this embodiment is to provide an electronic device.

[0181] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in an interpretable agricultural intelligent prediction method based on neural symbol fusion as described in Embodiment 1 of this disclosure.

[0182] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0183] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0184] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An interpretable intelligent agricultural prediction method based on neural symbol fusion, characterized in that, include: The agricultural image data uploaded by users and the environmental data obtained by IoT sensors are preprocessed to obtain standardized image data and structured environmental data. Construct a dual-branch deep learning model and perform symbolic regularization training on the dual-branch deep learning model; Standardized image data is input into a trained dual-branch deep learning model to make decisions, and the model outputs preliminary diagnostic results and confidence levels. Interpretive tools are used to generate saliency heatmaps that highlight the image regions most important for decision-making; The preliminary diagnostic results are verified using knowledge graphs, rule extraction, and symbolic reasoning, and verification results are generated for each. The initial diagnostic results are checked for consistency with the verification results, and a decision is made on whether to output a decision. The dual-branch deep learning model includes a feature extraction module, a feature transformation module, a concept encoder, and a concept inferencer. Specifically, the standardized image data is input into the feature extraction module, which outputs a high-dimensional feature map; the high-dimensional feature map is input into the feature transformation module, which outputs a global visual feature vector; the global visual feature vector is input into the concept encoder, which outputs a concept vector; and the concept vector is input into the concept inferencer, which outputs a preliminary diagnostic result and confidence level. The concept encoder maps high-dimensional global visual feature vectors to low-dimensional concept vectors. Specifically, the concept encoder consists of two fully connected layers. The first layer uses the ReLU activation function to achieve non-linearity, and the second layer uses the Sigmoid function to compress each dimension to [0,1]. Each concept vector corresponds to a predefined agricultural semantic concept. The concept inference engine processes the concept vectors. Specifically, the concept inference engine includes a two-layer MLP classifier. The first layer maps the low-dimensional concept vectors to a space and activates them using the ReLU activation function. The second layer outputs multiple unnormalized disease classifications, which are then converted into probability distributions using the Softmax function. The maximum probability is taken as the preliminary diagnosis result, and its confidence level is recorded.

2. The interpretable agricultural intelligent prediction method based on neural symbol fusion as described in claim 1, characterized in that, The knowledge graph verification matches the image regions that the dual-branch deep learning model focuses on with symptoms and environmental conditions in the agricultural knowledge graph; specifically, the knowledge graph verification includes: Image thresholding is performed on the saliency heatmap to preserve high-activation regions; Local image patches are extracted using connected component analysis; Use a lightweight CNN classifier to identify typical symptom features; The graph database is used to verify the consistency between symptoms, environmental conditions, and agricultural knowledge graphs. When both symptom matching and environmental support are met, the knowledge graph verification result is determined as "consistent" or "inconsistent".

3. The interpretable agricultural intelligent prediction method based on neural symbol fusion as described in claim 1, characterized in that, The rule extraction and verification process includes using the LIME algorithm to extract decision-related local rules from the two-branch deep learning model, and semantically aligning these rules with the agricultural knowledge graph to achieve transparency in the model's behavior. Specifically, the implementation of the LIME algorithm includes: The preprocessed standardized image data is divided into superpixels to generate perturbation samples; Calculate the weights of the perturbed samples and fit a local linear model; Extract the agronomic concepts corresponding to the top three superpixels with the highest weights and generate IF-THEN rules; The concept numbers in the rules are mapped to knowledge graph entities, and the consistency with medical consensus is verified. The rule extraction verification determines whether the verification result is "correct" or "incorrect".

4. The interpretable agricultural intelligent prediction method based on neural symbol fusion as described in claim 1, characterized in that, The symbolic reasoning verification includes probabilistic symbolic reasoning and fuzzy logic; specifically, the probabilistic symbolic reasoning includes: The concept vectors and structured environment data generated by the dual-branch deep learning model are converted into first-order logical predicates; Load the Markov logic network rule base, which contains weighted first-order logic formulas; The posterior probability of various diseases is calculated using a probabilistic inference algorithm, and the verification results are output.

5. The interpretable agricultural intelligent prediction method based on neural symbol fusion as described in claim 1, characterized in that, The consistency check includes: If the verification results of knowledge graph verification are "consistent", rule extraction verification are "correct" and symbolic reasoning verification are the preliminary diagnostic results output by the model, then it is determined to be consistent, enters the credible decision channel, and generates a final diagnostic report with multi-source evidence support. Otherwise, it is determined to be a "conflict", and the arbitration mechanism is initiated to calculate the weighted score and select the result with the highest score as the compromise solution; or a diagnostic conflict report is directly generated and a manual review is requested.

6. An interpretable agricultural intelligent prediction system based on neural symbol fusion, employing the interpretable agricultural intelligent prediction method based on neural symbol fusion as described in claim 1, characterized in that, include: The data preprocessing module is configured to preprocess the agricultural image data uploaded by the user and the environmental data acquired by the IoT sensor to obtain standardized image data and structured environmental data. The model training module is configured to: construct a dual-branch deep learning model and perform symbolic regularization training on the dual-branch deep learning model; The model decision module is configured to input standardized image data into the trained dual-branch deep learning model to make decisions and output preliminary diagnostic results and confidence levels. The decision visualization module is configured to use interpretive tools to generate a saliency heatmap that highlights the image regions most important to the decision. The multi-path verification module is configured to perform knowledge graph verification, rule extraction verification, and symbolic reasoning verification on the preliminary diagnosis results, and generate verification results respectively. The consistency check module is configured to perform a consistency check between the preliminary diagnostic results and the verification results and decide whether to output a decision.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps of an interpretable agricultural intelligent prediction method based on neural symbol fusion as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the interpretable agricultural intelligent prediction method based on neural symbol fusion as described in any one of claims 1-5.

Images