Multi-modal intelligent irrigation decision algorithm based on causal inference

Abstract

The invention provides a multi-modal intelligent irrigation decision algorithm based on causal inference, and the algorithm comprises the steps: carrying out the knowledge distillation of a pre-trained image feature extractor based on causal inference, and fully extracting the features of crop time sequence information and image information; according to the method, multi-modal data is considered, and heterogeneous data is deeply fused. Through knowledge distillation of an image feature extractor, the reasoning speed of the model is higher, the prediction accuracy is higher, end-to-end water shortage state prediction can be carried out based on greenhouse environment data, root system environment data and crop images, and a prediction result can be used for guiding crop irrigation and correcting the irrigation frequency and the irrigation amount of crops. And a more intelligent and comprehensive irrigation decision is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural greenhouse crop irrigation prediction and decision-making, and in particular to a multi-modal intelligent irrigation decision-making algorithm for agricultural greenhouse matrix-cultivated crops. Background Art

[0002] Smart irrigation is a technology that integrates information, communication and control technologies in the irrigation process, accurately predicts and controls the amount of water required for crop growth and the time according to crop needs, so as to obtain the best use of water resources and minimize the impact on the environment. Smart irrigation has the potential to save water, energy, improve yield, improve quality and increase efficiency, and has become a major development trend of modern agricultural irrigation technology. As a frontier field of "smart agriculture", modern large greenhouses have the ability to accurately classify and predict the water shortage status of crops and manage water conditions by integrating equipment such as microclimate detection, soil moisture detection and high-definition cameras. Greenhouse substrate-grown crops are very sensitive to water demand. Excessive irrigation may lead to root hypoxia, root diseases and nutrient loss in the root system. Insufficient irrigation may cause water stress in crops, reduce crop production quality and slow down their growth rate. Correctly judging the water shortage of crops can ensure the high quality and high yield of substrate-grown crops, improve the utilization efficiency of irrigation water, and achieve energy saving and increased production.

[0003] In practice, traditional irrigation methods are usually based on some agronomic experience to determine the water shortage of crops. Some use lysimeters to calculate evapotranspiration and then multiply it by a ratio to determine the amount of irrigation. Some are based on the intensity threshold of outdoor solar radiation or greenhouse light transmittance radiation to trigger quantitative irrigation. They do not take into account the comprehensive impact of microclimate changes in the greenhouse on crop water consumption, nor do they consider the water stress phenomenon shown by crop morphological information. This makes the planning of irrigation water for greenhouse crops lack a reasonable basis. Summary of the invention

[0004] The purpose of the present invention is to provide a multimodal intelligent irrigation decision-making algorithm based on causal inference that combines the crop growth environment with the characteristics expressed by images and can classify and predict the water shortage status of crops more comprehensively and accurately.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A multimodal intelligent irrigation decision-making algorithm based on causal inference is used for classification prediction of crop water shortage status, including the following steps:

[0007] Step S1: collecting crop environment and crop image data respectively;

[0008] Step S2: performing image enhancement and data preprocessing on the collected data;

[0009] Step S3: Perform knowledge distillation on the image feature extractor based on causal inference, obtain a lightweight model of ViT through training and use it for image feature extraction;

[0010] Step S4: extracting image data features and time series data features through an image encoder and a time series encoder;

[0011] Step S5: training the multimodal fusion model to obtain multimodal fusion features, inputting the multimodal fusion features into the classification head to classify the water shortage state, and comparing them with the true labels; calculating the cross entropy loss, and updating the model parameters through back propagation until the classification model converges;

[0012] Step S6: Apply the trained multimodal fusion model to classify and predict the water shortage status of crops, thereby achieving classification prediction of the water shortage status of crops.

[0013] Preferably, in step S1, the crop environment and crop image data are collected separately, including the following steps: taking a crop image vertically from above the crop every hour through a high-definition camera, and collecting crop environmental information every 5 minutes through a microclimate sensor and a soil moisture sensor, wherein the crop environmental information includes air temperature, air humidity, water vapor pressure difference, light intensity / photosynthetically active radiation, CO2 concentration, soil / substrate temperature, soil / substrate humidity, soil / substrate EC value and soil / substrate pH value.

[0014] Preferably, in step 2, the data preprocessing includes data normalization and data filling for missing data, including the following steps: inputting the crop image data into ViT after camera calibration, image enhancement and image compression; converting the timestamp in the time series data into time features, performing data filling for a small amount of missing data, and after data normalization, dividing the time series data into time series of equal length, aligning them with the time of image acquisition, and inputting them into Informer.

[0015] Preferably, the image feature extractor in step 3 adopts a pre-trained ViT model, and calibrates and lightweights the model based on a knowledge distillation algorithm based on causal inference. By training three loss functions, namely, alignment loss of comprehensive sample representation, alignment loss of class representation, and intervention distillation loss, a lightweight model of ViT is obtained and used for image feature extraction.

[0016] Preferably, the pre-trained image feature extractor ViT is distilled using a knowledge distillation algorithm of causal inference, and the specific process is as follows:

[0017] Step S31: define the “teacher model” and “student model”: use the ViT model and parameters pre-trained on the ImageNet dataset as the teacher model T; use n-layer (n≤6) ViT and initialized parameters as the student model S;

[0018] Step S32: Calculate the alignment loss of sample representation: For a specific input image sample

[0019] The teacher model T outputs d t The length of the sample representation logits, the student model S output d s The sample representation logits of length is expressed as follows:

[0020] h t =T(X)

[0021] h s =S(X)

[0022] in, represents the transposed representation of the samples of the student model, represents the sample representation of the teacher model, so the alignment loss of the sample representation It can be calculated using the following formula:

[0023]

[0024] in, It is a projection matrix used to map the representation of the student model to the same space as the representation of the teacher model. It is a weight coefficient used to control the proportion of loss in the total loss function. Represents the normalized mean square error, which is expressed as follows:

[0025]

[0026] Among them, V a and V b They refer to two vectors of equal length, respectively, and ||·|| represents the L2 norm;

[0027] Step S33: Calculate the alignment loss of class representation: Minimize the difference between the class centers of the student model and the teacher model through the alignment loss of class representation, which is specifically expressed as follows:

[0028]

[0029] Among them, m is the total number of classes, β is the weight coefficient that controls the proportion of class representation alignment loss in the total loss function, and C i is a class representation, which can be expressed by the following formula:

[0030]

[0031] in, is the representation vector of sample j in class i; the class representation is calculated by the feature vectors of all samples in the class and can be regarded as the center of the class;

[0032] Step S34: Calculate context representation: For the sample representation h of the teacher model t and the sample representation h of the student model s , we can extract the context coefficients for different classifications

[0033] The calculation formula is as follows:

[0034]

[0035] Among them, σ represents the Softmax function, τ is the temperature coefficient, which is used to soften the sample representation of the model; then the context representation J is generated based on the class center, which is specifically expressed as follows:

[0036]

[0037] Step S35: Calculate intervention distillation loss: Calculate the error of the student model in classifying the label under the context intervention of the teacher model, and eliminate the influence of these contextual confusion factors on the prediction results, intervention distillation loss It can be expressed by the following formula:

[0038]

[0039] Among them, P(c i ) is the proportion of the i-th class in the sample, y is the one-hat encoding of the true classification label, is the cross entropy loss, is the KL divergence, which can be expressed by the following formulas:

[0040]

[0041] in, and Generally refers to a probability distribution of two equal dimensions;

[0042] Step S36: Back propagate the loss function: Use the Adam parameter optimizer to update the model parameters until the student model converges; the comprehensive loss function can be expressed as follows:

[0043]

[0044] Preferably, the pre-trained 12-layer ViT model is distilled into a shallower model (n layers, n≤6) using a causal inference algorithm, and the confusion factors are removed by the backdoor adjustment method in causal inference to ensure that the student model learns the truly useful information in the teacher model.

[0045] Preferably, in step 4, image data features and time series data features are extracted by an image encoder and a time series encoder. The time series data feature extractor selects some high-contribution features for feedforward calculation based on probabilistic sparse self-attention. ProbSparseAttention can be calculated using the following formula:

[0046]

[0047] Among them, σ represents the Softmax function, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. represents the query matrix after L2 norm, d f Represents the feature dimension.

[0048] Preferably, the image encoder composed of n layers of ViT in step 4 receives the preprocessed image data and extracts high-dimensional semantic features from the image through a multi-head self-attention layer and a feed-forward layer;

[0049] The self-attention layer generates query, key and value based on the convolution embedding of the image, and the complex features and patterns in the image can be effectively captured by calculating the dot product attention. At the same time, the time series data encoder is composed of m layers of Informer, which receives the preprocessed time series data, generates query, key and value based on the data embedding of the time series data, and selects some queries that contribute highly to the results for feedforward calculation by calculating the probability sparse self-attention, so as to extract high-dimensional time series features from the time series data. The architecture design of the Informer is particularly suitable for processing long sequence data. Through the sparse self-attention mechanism and convolution operation, the trend and periodicity features in the time series data can be efficiently extracted.

[0050] Preferably, the multimodal fusion model in step 5 realizes deep fusion of image features and temporal features based on cross attention, and the multimodal fusion model is fused and trained through cross attention, and the specific process is as follows:

[0051] Perform multimodal interaction between image and time series features: f Image features Transformed into a query matrix, the length of which is d f The timing characteristics of

[0052] Converted into key matrix and value matrix, expressed as follows:

[0053] Q=W Q IK=W K TV=W V T

[0054] Among them, W Q , W K , W V Represent the transformation matrices of the query matrix, key matrix, and value matrix respectively; the formula for calculating the cross attention score using the query matrix, key matrix, and value matrix is ​​as follows:

[0055]

[0056] Step S52: Classifying the crop water shortage status using multimodal features: Connect a classification head, which consists of two linear layers and a ReLU layer. The linear layer can be expressed by the following formula:

[0057] FC(M)=W FC ·M+b FC

[0058] Among them, W FC and b FC Represent the weights and biases of the linear layer respectively. The classification head is applied to obtain the sample representation logits, which is specifically expressed as:

[0059] logits = FC(ReLU(FC(M)))

[0060] Step S53: Model training: Use the output of the model to calculate the cross entropy loss for the label, use the loss function for back propagation, use the Adam parameter optimizer to update the model parameters, and train the model to achieve the classification prediction accuracy of the crop water shortage status.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] (1) Based on causal inference, the present invention performs knowledge distillation on the pre-trained image feature extractor and fully extracts the features of crop temporal information and image information.

[0063] (2) The present invention takes multimodal data into consideration and deeply integrates heterogeneous data. After knowledge distillation of the image feature extractor, the model has a faster reasoning speed and higher prediction accuracy. It can perform end-to-end water shortage prediction based on greenhouse environment data, root environment data and crop images. The prediction results can be used to guide crop irrigation, correct crop irrigation frequency and irrigation amount, and achieve smarter and more comprehensive irrigation decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a flow chart of a multimodal intelligent irrigation decision algorithm based on causal inference provided by an embodiment of the present invention;

[0065] Figure 2 A flowchart of multimodal fusion model training in a multimodal intelligent irrigation decision algorithm based on causal inference provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of a model of a multimodal intelligent irrigation decision-making algorithm based on causal inference provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0068] like Figures 1 to 3 As shown, a multimodal intelligent irrigation decision algorithm based on causal inference disclosed by the present invention is used for classification prediction of crop water shortage status. The specific steps are as follows:

[0069] Step S1: Collect crop environment and crop image data respectively. Use a high-definition camera to take a crop image vertically from above the crop every hour, and use a microclimate sensor and a soil moisture sensor to collect crop environment information every 5 minutes. The crop environment information includes air temperature, air humidity, water vapor pressure difference, light intensity / photosynthetically active radiation, CO2 concentration, soil / substrate temperature, soil / substrate humidity, soil / substrate EC value and soil / substrate pH value data.

[0070] Step S2: Perform image enhancement and data preprocessing on the collected data. After camera calibration, image enhancement and image compression, the crop image data is compressed to 224*224 pixels for easy input into ViT. The timestamps in the time series data are converted into time features, a small amount of missing data is filled, and after data normalization, the time series data is divided into time series of equal length, aligned with the time of image acquisition, and input into Informer.

[0071] Step S3: Based on causal inference, the image feature extractor is subjected to knowledge distillation. The image feature extractor uses a pre-trained 12-layer ViT model to distill into a shallower model (n layers, n≤6). In this embodiment, the pre-trained 12-layer ViT model is distilled into a 6-layer ViT, and the model is calibrated and lightweight based on the knowledge distillation algorithm of causal inference. The confusion factor is removed by the backdoor adjustment method in causal inference to ensure that the "student model" learns the truly useful information in the "teacher model". By integrating the three loss functions of alignment loss of sample representation, alignment loss of class representation, and intervention distillation loss, a lightweight model of ViT is trained and used for image feature extraction.

[0072] The knowledge distillation algorithm of causal inference is used to distill the pre-trained image feature extractor ViT. The specific process is as follows:

[0073] Step S31: define the "teacher model" and "student model": use the ViT model and parameters pre-trained on the ImageNet dataset as the teacher model T; use the 6-layer ViT and initialized parameters as the student model S;

[0074] Step S32: Calculate the alignment loss of sample representation: For a specific input image sample The teacher model T outputs d t The length of the sample representation logits, the student model S output d s The sample representation logits of length is expressed as follows:

[0075] h t =T(X)

[0076] h s =S(X)

[0077] in, represents the transposed representation of the samples of the student model, represents the sample representation of the teacher model, so the alignment loss of the sample representation It can be calculated using the following formula:

[0078]

[0079] in, is a projection matrix used to map the representation of the student model to the same space as the representation of the teacher model. α is a weight coefficient used to control the proportion of loss in the total loss function. Represents the normalized mean square error, which is expressed as follows:

[0080]

[0081] Among them, Va and V b They refer to two vectors of equal length, respectively, and ||·|| represents the L2 norm;

[0082] Step S33: Calculate the alignment loss of class representation: Minimize the difference between the class centers of the student model and the teacher model through the alignment loss of class representation, which is specifically expressed as follows:

[0083]

[0084] Among them, m is the total number of classes, β is the weight coefficient that controls the proportion of class representation alignment loss in the total loss function, and C i is a class representation, which can be expressed by the following formula:

[0085]

[0086] in, is the representation vector of sample j in class i; the class representation is calculated by the feature vectors of all samples in the class and can be regarded as the center of the class;

[0087] Step S34: Calculate context representation: For the sample representation h of the teacher model t and the sample representation h of the student model s , we can extract the context coefficients for different classifications The calculation formula is as follows:

[0088]

[0089]

[0090] Among them, σ represents the Softmax function, τ is the temperature coefficient, which is used to soften the sample representation of the model; then the context representation J is generated based on the class center, which is specifically expressed as follows:

[0091]

[0092] Step S35: Calculate intervention distillation loss: Calculate the error of the student model in classifying the label under the context intervention of the teacher model, and eliminate the influence of these contextual confusion factors on the prediction results, intervention distillation loss It can be expressed by the following formula:

[0093]

[0094] Among them, P(c i ) is the proportion of the i-th class in the sample, y is the one-hat encoding of the true classification label, is the cross entropy loss, is the KL divergence, which can be expressed by the following formulas:

[0095]

[0096] in, and Generally refers to a probability distribution of two equal dimensions;

[0097] Step S36: Back propagate the loss function: Use the Adam parameter optimizer to update the model parameters until the student model converges; the comprehensive loss function can be expressed as follows:

[0098]

[0099] Step S4: Extract image data features and time series data features through the image encoder and time series encoder. The image encoder composed of 6 layers of ViT receives the preprocessed image data and extracts a 768-dimensional feature vector from the image through the multi-head self-attention layer and the feedforward layer.

[0100] In the self-attention layer, the convolutional embedding (patch embedding) based on the image generates query, key and value. By calculating the dot product attention, the complex features and patterns in the image can be effectively captured. At the same time, the time series data encoder consists of 3 layers of Informer, which receives the preprocessed time series data, generates query, key and value based on the data embedding (data embedding) of the time series data, and selects the top 20 queries with the attention score for feedforward calculation by calculating the ProbSparse Self-attention, and extracts 768-dimensional time series features from the time series data. The architecture design of Informer is particularly suitable for processing long sequence data. Through the sparse self-attention mechanism and convolution operation, the trend and periodicity features in the time series data can be efficiently extracted.

[0101] Image data features and time series data features are extracted through the image encoder and the time series encoder. The time series data feature extractor selects some high-contribution features for feedforward calculation based on probabilistic sparse self-attention. The probabilistic sparse self-attention ProbSparseAttentiom can be calculated using the following formula:

[0102]

[0103] Among them, σ represents the Softmax function, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. represents the query matrix after L2 norm, d f Represents the feature dimension.

[0104] Step S5: training the multimodal fusion model. The multimodal fusion model realizes the deep fusion of image features and temporal features based on cross attention, the multimodal fusion model is fused and trained through cross attention, and the high-dimensional features extracted by ViT and Informer are input into the cross attention layer.

[0105] The cross-attention mechanism can effectively fuse information from different modalities, allowing image features and temporal features to complement and reinforce each other in the multimodal space, thereby generating a richer and more expressive multimodal fusion feature representation.

[0106] The multimodal fusion feature representation output by the cross-attention mechanism is input into the classification head to classify the water shortage status. The classifier generates logits and compares them with the true labels to predict the water shortage status of the crop.

[0107] Calculate the cross entropy loss between the predicted result and the true label, and use the loss function to back propagate and update the model parameters until the classification model converges. The cross entropy loss function is as follows:

[0108]

[0109] in, and It generally refers to the probability distribution of two equal dimensions. By optimizing the objective function, the prediction error is gradually reduced and the prediction accuracy of the model is improved.

[0110] The method for training the multimodal fusion model includes the following sub-steps:

[0111] Step S51: Perform multimodal interaction between image and time series features: f Image features Transformed into a query matrix, the length of which is d f The timing characteristics of Converted into key matrix and value matrix, expressed as follows:

[0112] Q=W Q IK=W K TV=W V T

[0113] Among them, W Q , W K , W V Represent the transformation matrices of the query matrix, key matrix, and value matrix respectively; the formula for calculating the cross attention score using the query matrix, key matrix, and value matrix is ​​as follows:

[0114]

[0115] Step S52: Classifying the crop water shortage status using multimodal features: Connect a classification head, which consists of two linear layers and a ReLU layer. The linear layer can be expressed by the following formula:

[0116] FC(M)=W FC ·M+b FC

[0117] Among them, W FC and b FC Represent the weights and biases of the linear layer respectively. The classification head is applied to obtain the sample representation logits, which is specifically expressed as:

[0118] logits = FC(ReLU(FC(M))

[0119] Step S53: Model training: Use the output of the model to calculate the cross entropy loss for the label, use the loss function for back propagation, use the Adam parameter optimizer to update the model parameters, and train the model to achieve the classification prediction accuracy of the crop water shortage status.

[0120] Step S6: Apply the trained multimodal fusion model to classify and predict the water shortage status of crops. Use the trained model to classify and predict the water shortage status of crops based on crop environment and image data collected in real time.

[0121] The present invention utilizes a microclimate sensor, a soil moisture sensor and a high-definition camera to collect crop environment and crop image data respectively; performs data enhancement and data preprocessing on the collected time series and image data respectively; performs knowledge distillation on an image feature extractor based on causal inference, reduces the model complexity of the image feature extractor, and improves the applicability and prediction accuracy of the model for the task; extracts image data features and time series data features respectively through an image encoder and a time series encoder; utilizes a cross-attention mechanism to fuse the extracted high-dimensional features of different modes, and trains a multimodal fusion model; applies the trained model to classify and predict the water shortage status of crops, and the predicted water shortage level of the crops can be used for irrigation guidance, and the irrigation frequency and amount of the crops can be corrected.

[0122] The present invention performs knowledge distillation on the pre-trained image feature extractor based on causal inference, fully extracts the features of crop time series information and image information; takes into account multimodal data, and deeply integrates heterogeneous data. After knowledge distillation of the image feature extractor, the model has faster reasoning speed and higher prediction accuracy. It can perform end-to-end water shortage state prediction based on greenhouse environment data, root environment data and crop images. The prediction results can be used to guide crop irrigation, correct crop irrigation frequency and irrigation amount, and achieve smarter and more comprehensive irrigation decisions.

[0123] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0124] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multimodal intelligent irrigation decision-making algorithm based on causal inference for classification prediction of crop water shortage status, characterized in that: The steps include: Step S1: collecting crop environment and crop image data respectively; Step S2: performing image enhancement and data preprocessing on the collected data; Step S3: Perform knowledge distillation on the image feature extractor based on causal inference, obtain a lightweight model of ViT through training and use it for image feature extraction; Step S4: extracting image data features and time series data features through an image encoder and a time series encoder; Step S5: training the multimodal fusion model to obtain multimodal fusion features, inputting the multimodal fusion features into the classification head to classify the water shortage state, and comparing them with the true labels; calculating the cross entropy loss, and updating the model parameters through back propagation until the classification model converges; Step S6: Apply the trained multimodal fusion model to classify and predict the water shortage status of crops, thereby achieving classification prediction of the water shortage status of crops.

2. The multimodal intelligent irrigation decision-making algorithm based on causal inference according to claim 1 is characterized in that: In the step S1, the crop environment and crop image data are collected separately, including the following steps: a crop image is taken vertically from above the crop every hour by a high-definition camera, and crop environment information is collected every 5 minutes by a microclimate sensor and a soil moisture sensor, wherein the crop environment information includes air temperature, air humidity, water vapor pressure difference, light intensity / photosynthetically active radiation, CO2 concentration, soil / substrate temperature, soil / substrate humidity, soil / substrate EC value and soil / substrate pH value.

3. The multimodal intelligent irrigation decision-making algorithm based on causal inference according to claim 1 is characterized in that: In step 2, the data preprocessing includes data normalization and data filling for missing data, including the following steps: inputting the crop image data into ViT after camera calibration, image enhancement and image compression; converting the timestamp in the time series data into time features, performing data filling for a small amount of missing data, and after data normalization, dividing the time series data into time series of equal length, aligning with the time of image acquisition, and inputting into Informer.

4. The multimodal intelligent irrigation decision-making algorithm based on causal inference according to claim 1 is characterized in that: The image feature extractor in step 3 adopts the pre-trained ViT model, and calibrates and lightweights the model based on the knowledge distillation algorithm of causal inference. By training three loss functions including the alignment loss of comprehensive sample representation, the alignment loss of class representation and the intervention distillation loss, a lightweight model of ViT is obtained and used for image feature extraction.

5. The multimodal intelligent irrigation decision-making algorithm based on causal inference according to claim 4 is characterized in that: The knowledge distillation algorithm of causal inference is used to distill the pre-trained image feature extractor ViT. The specific process is as follows: Step S31: define the "teacher model" and "student model": use the ViT model and parameters pre-trained on the ImageNet dataset as the teacher model T; use n-layer (n≤6) ViT and initialized parameters as the student model S; Step S32: Calculate the alignment loss of sample representation: For a specific input image sample The teacher model T outputs d t The length of the sample representation logits, the student model S output d s The sample representation logits of length is expressed as follows: h t =T(X) h s =S(X) in, represents the transposed representation of the sample of the student model, represents the sample representation of the teacher model, so the alignment loss of the sample representation It can be calculated using the following formula: in, is a projection matrix used to map the representation of the student model to the same space as the representation of the teacher model. α is a weight coefficient used to control the proportion of loss in the total loss function. Represents the normalized mean square error, which is expressed as follows: Among them, V a and V b They refer to two vectors of equal length, respectively, and ||·|| represents the L2 norm; Step S33: Calculate the alignment loss of class representation: Minimize the difference between the class centers of the student model and the teacher model through the alignment loss of class representation, which is specifically expressed as follows: Among them, m is the total number of classes, β is the weight coefficient that controls the proportion of class representation alignment loss in the total loss function, and C i is a class representation, which can be expressed by the following formula: in, is the representation vector of sample j in class i; the class representation is calculated by the feature vectors of all samples in the class and can be regarded as the center of the class; Step S34: Calculate context representation: For the sample representation h of the teacher model t and the sample representation h of the student model s , we can extract the context coefficients for different classifications The calculation formula is as follows: Among them, σ represents the Softmax function, τ is the temperature coefficient, which is used to soften the sample representation of the model; then the context representation J is generated based on the class center, which is specifically expressed as follows: Step S35: Calculate intervention distillation loss: Calculate the error of the student model in classifying the label under the context intervention of the teacher model, and eliminate the influence of these contextual confusion factors on the prediction results, intervention distillation loss It can be expressed by the following formula: Among them, P(c i ) is the proportion of the i-th class in the sample, y is the one-hat encoding of the true classification label, is the cross entropy loss, is the KL divergence, which can be expressed by the following formulas: in, and Generally refers to a probability distribution of two equal dimensions; Step S36: Back propagate the loss function: Use the Adam parameter optimizer to update the model parameters until the student model converges; the comprehensive loss function can be expressed as follows:

6. The multimodal intelligent irrigation decision algorithm based on causal inference according to claim 5 is characterized in that: The pre-trained 12-layer ViT model is distilled into a shallower model (n layers, n≤6) using a causal inference algorithm, and the confusion factors are removed by the backdoor adjustment method in causal inference to ensure that the student model learns the truly useful information in the teacher model.

7. The multimodal intelligent irrigation decision-making algorithm based on causal inference according to claim 6 is characterized in that: In step 4, image data features and time series data features are extracted by an image encoder and a time series encoder. The time series data feature extractor selects some high-contribution features for feedforward calculation based on probabilistic sparse self-attention. ProbSparseAttention can be calculated using the following formula: Among them, σ represents the Softmax function, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. represents the query matrix after L2 norm, d f Represents the feature dimension.

Images