Tea garden state monitoring method fusing visual time sequence text pre-training model

By integrating visual temporal text pre-training model and space-time heterogeneous graph neural network, the problem of difficult to explore the relationship between physiological changes and environmental factors in tea garden monitoring is solved, and panoramic perception and accurate prediction of tea garden growth status is realized, and tea garden management is improved level of intelligence and automation.

CN119046673AInactive Publication Date: 2024-11-29NATIONAL AGRO-TECH EXTENSION & SERVICE CENTER
View PDF 0 Cites 28 Cited by

Patent Information

Application Number
CN202411557904.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing tea garden monitoring technology is difficult to accurately reflect the internal physiological changes of tea trees, and it lacks in-depth exploration of the internal relationship between environmental factors and tea tree growth. Agricultural management lacks quantitative basis and intelligent assistance, making it difficult to adapt to the needs of modern refined and digital management.

Method used

The tea garden state monitoring method using a fusion visual timing text pre-trained model is used to realize the deep fusion and synergistic gain of tea garden environmental parameters, tea tree images and agricultural log data through multimodal feature extraction, space-time modeling and expert knowledge-guided semi-supervised learning. , build a spatiotemporal heterogeneous graph neural network and multi-task prediction framework to improve the intelligent perception and prediction capabilities of tea tree growth state.

Benefits of technology

It realizes panoramic perception and accurate prediction of the growth status of tea gardens, improves the intelligence and automation level of tea garden management, provides more accurate, comprehensive and reliable data support, reduces the intensity of tea farmers' work, and improves the efficiency of tea garden management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046673A_ABST
    Figure CN119046673A_ABST
Patent Text Reader

Abstract

The invention discloses a tea garden state monitoring method fusing a visual time sequence text pre-training model, and the method comprises the following steps: 1, data collection and preprocessing; step 2, multi-modal feature extraction; step 3, feature fusion and space-time modeling; step 4, semi-supervised learning guided by expert knowledge; and 5, multi-task tea tree growth state prediction. According to the tea garden state monitoring method fusing the visual time sequence text pre-training model, the accuracy and comprehensiveness of tea garden monitoring are improved, and the tea garden state monitoring accuracy is improved by fusing tea garden environment parameters, tea tree images, farming logs and other multi-source heterogeneous data and deeply mining the space-time dependency relationship among the data. According to the method, the dynamic change of the tea garden growth state can be sensed more accurately and comprehensively, the intelligence and automation level of tea garden management is improved, and based on space-time heterogeneous graph representation learning and a multi-task prediction framework, integrated intelligent analysis of tea tree growth state recognition, anomaly detection and risk early warning can be realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of tea tree growth status monitoring, and in particular to a tea garden status monitoring method integrating a visual temporal text pre-training model. Background Art

[0002] As one of the three major beverages in the world, tea has a long history of cultivation and profound cultural heritage in my country. In recent years, with the rapid development of social economy and the continuous improvement of consumer demand, the tea market has shown a trend of product diversification and high-end quality. In order to adapt to the market competition under the new situation, ensure the quality and safety of tea, improve the efficiency of resource utilization, and realize the transformation and upgrading of the tea industry and high-quality development, it is urgent to introduce advanced information technology means to carry out real-time dynamic monitoring of the growth status of tea gardens around the clock, in all directions, and throughout the process.

[0003] The traditional tea garden monitoring model mainly relies on manual regular inspections and farmers' experience judgment, which often has many limitations such as low efficiency, strong subjectivity, and difficulty in quantification. In recent years, some studies have tried to use IoT sensors to collect environmental parameter data and apply machine vision technology to analyze tea garden images, which has achieved automation and digitization of the monitoring process to a certain extent.

[0004] However, as a complex ecosystem, the growth state of the tea garden is not only closely related to its own physiological characteristics, but also affected by multiple factors such as environmental conditions and management measures. It is difficult to accurately describe the dynamic changes of tea tree growth and development by relying solely on structured environmental parameter data or unstructured image data. In addition, the large amount of text logs accumulated by tea farmers in daily management contain rich prior knowledge, but there is still a lack of effective mining and utilization methods. Therefore, how to integrate multi-source heterogeneous data such as tea garden environmental parameters, tea tree images, and farming logs, and use advanced artificial intelligence analysis and mining technology to achieve accurate perception, scientific management and intelligent early warning of the growth state of tea trees has become a key scientific issue that needs to be broken through in the current construction of smart tea gardens.

[0005] In recent years, artificial intelligence technology has developed rapidly, providing a new solution for the intelligent monitoring of tea garden growth. In the field of computer vision, deep learning models represented by convolutional neural networks have made significant progress in tasks such as image recognition and target detection. From the earliest AlexNet and VGGNet, to deeper and wider network structures such as ResNet and Inception, to new paradigms that introduce attention mechanisms such as VisionTransformer and SwinTransformer, the feature extraction and semantic understanding capabilities of visual models have been continuously enhanced. It is worth mentioning that the ConvNeXt model inherits the advantages of CNN and Transformer, and achieves a good balance between accuracy and computational efficiency, bringing new opportunities for intelligent analysis of tea tree images.

[0006] In terms of time series prediction, recurrent neural network models such as long short-term memory networks and gated recurrent units have been gradually replaced by the Transformer paradigm. Work represented by Informer introduced the ProbSparse attention mechanism and generative decoding strategy, making breakthrough progress in long-sequence time series prediction and providing new ideas for modeling dynamic changes in tea garden environments. Autoformer and Pyraformer further improved the performance of Transformer in time series prediction tasks by introducing autoregressive mechanisms and pyramid structures.

[0007] In the field of natural language processing, pre-trained language models represented by BERT and GPT have set off a research boom. Through self-supervised pre-training on large-scale text corpora, the above models can learn the general semantic representation of language and can be easily transferred to downstream tasks, greatly improving the effect of text intelligent processing. The knowledge-enhanced pre-training model represented by ERNIE3.0 further integrates external knowledge graphs. By combining knowledge representation with language representation, it has achieved performance far exceeding the existing optimal models in machine reading comprehension, question answering, dialogue and other tasks, showing broad application prospects.

[0008] However, there are still many challenges in directly applying the above-mentioned cutting-edge artificial intelligence models to the intelligent monitoring of tea garden growth. On the one hand, tea garden environmental parameters, tea tree images, and agricultural logs obey different data distributions. How to learn consistent representations of different modal data under a unified framework and achieve efficient fusion is a key issue; on the other hand, there are intricate spatiotemporal correlations between different data modalities. How to model the dynamic interaction and co-evolution of cross-modal data is another core challenge. In addition, the construction of the tea garden intelligent monitoring model cannot be separated from the guidance of expert knowledge. It is necessary to explore how to embed expert experience into deep learning model training to improve the generalization performance and robustness of the model.

[0009] Existing tea garden growth monitoring technologies are mainly concentrated in the fields of environmental parameter collection, image analysis and agricultural management. ① In terms of environmental monitoring, traditional methods mainly rely on IoT sensors to collect temperature, humidity, light and other parameter data in real time, but lack in-depth exploration of the intrinsic relationship between environmental factors and tea tree growth. ② In terms of image analysis, machine vision-based methods attempt to extract tea tree growth status characteristics from tea garden images, but are limited to surface morphological analysis and are difficult to accurately reflect the internal physiological changes of tea trees. ③ In terms of agricultural management, farmers rely on experience to conduct manual inspections and management decisions on tea gardens, but lack quantitative basis and intelligent assistance, making it difficult to adapt to the needs of modern refined and digital management.

[0010] Therefore, we propose a tea garden status monitoring method that integrates a visual temporal text pre-training model to solve the above problems. Summary of the invention

[0011] The purpose of the present invention is to provide a tea garden status monitoring method integrating a visual temporal text pre-training model, so as to solve the problems raised by the above-mentioned background technology. In terms of environmental monitoring in the current market, traditional methods mainly rely on Internet of Things sensors to collect parameter data such as temperature, humidity, and light in real time, but lack in-depth exploration of the intrinsic relationship between environmental factors and tea tree growth. In terms of image analysis, machine vision-based methods attempt to extract tea tree growth status characteristics from tea garden images, but are limited to surface morphological analysis and are difficult to accurately reflect the internal physiological changes of tea trees. In terms of agricultural management, farmers rely on experience to conduct manual inspections and management decisions on tea gardens, but lack quantitative basis and intelligent assistance, making it difficult to adapt to the needs of modern refined and digital management.

[0012] To achieve the above object, the present invention provides the following technical solutions: A tea garden status monitoring method integrating a visual temporal text pre-training model, comprising the following methods;

[0013] Step 1: Data collection and preprocessing;

[0014] Step 2: Multimodal feature extraction;

[0015] Step 3: Feature fusion and spatiotemporal modeling;

[0016] Step 4: Semi-supervised learning guided by expert knowledge;

[0017] Step 5: Multi-task tea tree growth status prediction.

[0018] Preferably, in step 1, the data collection and preprocessing step aims to collect multi-source heterogeneous data such as tea garden environmental parameters, tea tree images and text logs with high quality and efficiency, and perform intelligent cleaning and anomaly correction to provide reliable data support for subsequent tea tree growth status detection. An AIoT sensor with an adaptive sampling rate and a high-definition camera are used to collect tea garden environmental parameters, tea tree images and text log data, an attention mechanism is introduced to optimize data quality, and GPT is used to achieve intelligent data cleaning and anomaly correction;

[0019] In terms of environmental parameter collection, the present invention uses IoT sensors, including sensors for temperature, humidity, light intensity, and soil pH, and introduces an adaptive sampling strategy to dynamically adjust the sampling frequency.

[0020] Preferably, in step 1, in terms of tea tree image acquisition, the present invention uses a high-definition camera installed above the tea garden, and the inclination angle is generally set to 30°~60° to obtain the overall image of the tea tree canopy. The shooting resolution is determined according to the tea garden area and the tea tree planting density, generally not less than 1080p, and the frame rate is not less than 30fps to ensure the clarity and continuity of the image. In order to highlight the key features of the growth status of the tea tree, the present invention introduces an attention mechanism in image acquisition, calculates the significance weight of the image area in real time through a convolutional neural network, dynamically adjusts the shooting parameters, adaptively increases the shooting frequency and resolution of the key growth area of ​​the tea tree, and reduces the shooting frequency of the relatively stable area, thereby improving data quality while reducing transmission and storage overhead;

[0021] Text log data is collected through the tea farmers' mobile phone APP, and the recorded content includes tea garden management operations, pest and disease observation, fertilization and medication, etc. It adopts a structured form, and the fields include time, location, operation type, operation parameters, text description and picture attachments, etc. The collection frequency is not less than once a day;

[0022] To ensure the quality of collected data, the present invention uses a large language model GPT for intelligent data cleaning and anomaly detection. GPT learns data cleaning rules from a small number of manually labeled samples through Few-shot Learning, and generates self-correction prompts to optimize data.

[0023] GPT minimizes the loss function through optimization algorithms such as gradient descent, quickly adapts to new data cleaning tasks, and identifies abnormal data through statistical tests and cross-validation of data distribution, and fills or deletes them according to data types and business rules;

[0024] The cleaned data is stored in the time series database InfluxDB, which supports efficient data writing, query and aggregation analysis. InfluxDB uses a SQL-like query language and has built-in multiple aggregation functions, which is suitable for processing the time series data of tea garden sensors. At the same time, its horizontally scalable architecture can support the storage and query of TB-level data, providing convenience for subsequent large-scale data analysis.

[0025] Preferably, in step 2, ConvNeXtV2 is used to extract local and global visual features of tea garden images, Informer is used to capture long-distance dependencies of environmental time series data, and ERNIE3.0 is used to extract semantic features of tea garden text logs to achieve comprehensive feature representation of tea garden multimodal data.

[0026] In terms of tea garden image feature extraction, the present invention adopts the advanced visual transformer model ConvNeXtV2. Compared with the traditional convolutional neural network, ConvNeXtV2 introduces the self-attention mechanism and dynamic deep convolution, which can more effectively capture the global dependencies and local details of the image.

[0027] The self-attention mechanism captures global dependencies by calculating the correlation between image blocks; the dynamic deep convolution extracts multi-scale local features by adaptively adjusting the depth and width of the convolution kernel. Finally, ConvNeXtV2 obtains the global feature vector of the image as a learnable linear transformation matrix through global average pooling and linear transformation.

[0028] In terms of feature extraction of environmental time series data, the present invention uses the Informer model, which is a time series prediction model based on the self-attention mechanism. By introducing probabilistic self-attention and adaptive distillation, it can effectively capture the long-distance dependency of time series data and significantly reduce the computational complexity.

[0029] Compared with the traditional self-attention mechanism, probabilistic self-attention introduces random sampling and distillation operations, which reduces the computational complexity while maintaining the feature expression capability. Finally, Informer obtains the long-distance dependency features of the environmental time series data through adaptive distillation and linear transformation.

[0030] Preferably, in step 2, in terms of text log feature extraction, the present invention adopts the knowledge-enhanced pre-trained language model ERNIE3.0. Compared with the traditional pre-trained language model, ERNIE3.0 can better understand the semantic information of the text by integrating knowledge graph and multi-granularity language unit modeling.

[0031] Preferably, in step 3, a Performer-based spatiotemporal heterogeneous graph neural network TSHG-PF is used to construct a spatiotemporal heterogeneous graph containing visual, temporal and textual semantic nodes, and DisentangledAttention is introduced to achieve efficient fusion of different modal features, so as to model the spatiotemporal dynamic correlation between the tea garden environment and the growth status of tea trees;

[0032] In order to achieve efficient and explainable multimodal feature fusion, this paper designs a Performer-based spatiotemporal heterogeneous graph neural network TSHG-PF. TSHG-PF models tea garden data in the form of a graph, represents the features of different modalities as nodes in the graph, and describes the spatiotemporal dependency between nodes by introducing edge connections in the time and space dimensions.

[0033] In the aggregation update phase of the graph neural network, TSHG-PF introduces a Performer-based attention mechanism to reduce computational complexity and improve interpretability;

[0034] To further improve the fusion effect of different modal features, TSHG-PF introduces the DisentangledAttention mechanism based on Performer. DisentangledAttention decouples the content and position encoding of node features, allowing features of different modalities to interact in the two dimensions of content and position, thereby enhancing the transmission and fusion of cross-modal information.

[0035] Preferably, in step 4, an expert-guided contrastive learning framework EGCL-IGPT based on InstructGPT is used to construct a small number of high-quality contrast samples using expert knowledge, and the expert annotation cost is reduced through contrastive learning. At the same time, an active learning strategy is used to select the most representative samples for expert annotation, and expert knowledge is effectively integrated to guide model training;

[0036] By constructing a comparative learning task guided by expert knowledge and using a small amount of high-quality labeled samples, EGCL-IGPT learns a discriminative and semantically consistent representation in the feature space of tea garden growth status. At the same time, EGCL-IGPT adopts an active learning strategy to select samples with the most value for model discrimination for expert labeling, further reducing the labeling cost and improving the efficiency of expert knowledge utilization.

[0037] By constructing a comparative learning task guided by expert knowledge and using a small amount of high-quality labeled samples, a discriminative and semantically consistent representation is learned in the feature space of tea garden growth status. At the same time, EGCL-IGPT adopts an active learning strategy to select samples with the most valuable value for model discrimination for expert labeling, further reducing the labeling cost and improving the utilization efficiency of expert knowledge.

[0038] Preferably, in step 4, in terms of active learning strategy, EGCL-IGPT adopts a committee-based query method, selects the nodes with the most value and the most information for the current model to be labeled by experts, generates corresponding queries using InstructGPT, asks experts to label them, and adds the labeled results to the training set, and updates the positive and negative samples for comparative learning;

[0039] By iteratively executing contrastive learning and active learning, EGCL-IGPT gradually optimizes the node embedding representation, while making targeted use of expert knowledge to improve the accuracy and interpretability of tea garden growth status prediction. In the inference stage, for the newly collected tea garden data, EGCL-IGPT first generates node representation through TSHG-PF, then maps it to the embedding space using the trained mapping function, and finally predicts the risk level of the node based on the similarity between the embedded vector and the labeled node.

[0040] Preferably, in step 5, the tea tree growth status monitoring multi-task prediction framework TGM-MTL simultaneously optimizes the tea tree growth status recognition, anomaly detection and risk warning tasks, adaptively adjusts the loss weight of each task, and uses FLAN to perform instruction fine-tuning to improve the model robustness and generalization performance;

[0041] In order to realize multi-task prediction of tea tree growth status, the present invention proposes a multi-task prediction framework TGM-MTL for tea tree growth status monitoring. TGM-MTL integrates task-specific prediction layers into a unified backbone network in an end-to-end manner, dynamically balances the learning progress of different tasks by adaptively adjusting the loss weights of each task, and uses FLAN to fine-tune instructions to enhance the model's ability to understand and execute natural language instructions.

[0042] Preferably, in step 5, in terms of instruction fine-tuning, TGM-MTL uses FLAN to fine-tune the model so that it can understand and execute instructions in natural language. FLAN learns to map instructions to corresponding operations by pre-training a large-scale language model and fine-tuning it on a large number of instruction-response pairs. In TGM-MTL, by concatenating task descriptions and node features into an instruction sequence and taking the task output as the target response, FLAN can learn to adaptively perform different prediction tasks according to instructions;

[0043] In the reasoning stage, given a new spatiotemporal heterogeneous graph of a tea garden, TGM-MTL first generates representations and embeddings of nodes through TSHG-PF and EGCL-IGPT, which are then input into the shared backbone network and task-specific prediction heads. Combined with the instruction fine-tuning capability of FLAN, it outputs the prediction results of all tasks at the same time, thus achieving comprehensive monitoring and early warning of the growth status of tea trees.

[0044] Compared with the prior art, the beneficial effects of the present invention are that, in order to optimize and improve the shortcomings of traditional tea garden monitoring methods, such as data islands, shallow feature expression, and reliance on expert experience, the present invention comprehensively utilizes technologies such as visual-temporal-text multimodal pre-training, spatiotemporal heterogeneous graph representation learning, and expert-guided comparative learning, thus overcoming the shortcomings of the prior art solutions and improving the accuracy, comprehensiveness, and automation of intelligent perception and prediction of tea garden growth status, as follows:

[0045] (1) In the field of intelligent monitoring of tea gardens, we realize the deep fusion and synergistic gain of multimodal data such as vision, time series, and text. By introducing leading deep learning models such as ConvNeXt, Informer, and ERNIE, we conduct deep feature mining on tea garden images, environmental parameters, and agricultural logs, and use spatiotemporal heterogeneous graph neural networks to achieve efficient interaction and aggregation of cross-modal information, maximize the value of data, and reveal the inherent laws between multi-dimensional elements of tea gardens, thereby achieving a panoramic perception of the status of tea gardens. Compared with traditional single data source analysis, this innovative method can provide a more accurate, comprehensive, and three-dimensional portrait of tea garden growth, providing a solid foundation for precise management and control;

[0046] (2) The expert-guided comparative learning paradigm was proposed in a pioneering way, breaking through the bottleneck of small sample learning through knowledge-driven learning. The small number of samples in the tea garden scene and the high labeling cost seriously restrict the training quality of the data-driven model. To address this problem, the present invention cleverly uses expert knowledge to construct high-quality comparative samples, and couples active learning strategies to optimize the model step by step. This paradigm allows experts to flexibly adjust learning goals and optimization paths based on prior experience, minimize manpower input and ensure results, so that the model can continue to learn and quickly adapt to the complex and changing tea garden environment. Compared with traditional data volume-dependent methods, this innovative paradigm can maximize the expert wisdom dividend and significantly improve the generalization ability and robustness of the model;

[0047] (3) Improve the accuracy and comprehensiveness of tea garden monitoring. By integrating multi-source heterogeneous data such as tea garden environmental parameters, tea tree images, and farming logs, and deeply mining the spatiotemporal dependencies between them, the present invention can more accurately and comprehensively perceive the dynamic changes in the growth status of tea gardens, provide tea farmers with detailed and reliable data support, and guide scientific management decisions;

[0048] (4) Improving the intelligence and automation level of tea garden management. Based on spatiotemporal heterogeneous graph representation learning and multi-task prediction framework, the present invention can realize integrated intelligent analysis of tea tree growth status recognition, anomaly detection and risk warning, so that traditional tea garden manual monitoring and experience-based judgment and other dependent tasks can be automated, reducing the workload of tea farmers and improving tea garden management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1It is a flow chart of the detection method of the present invention;

[0050] Figure 2 It is the environmental parameter data diagram of the present invention;

[0051] Figure 3 It is the tea tree image data graph of the present invention;

[0052] Figure 4 It is the tea tree image feature map of the present invention;

[0053] Figure 5 It is the time series characteristic diagram of the environmental parameters of the present invention;

[0054] Figure 6 This is the log text semantic feature graph of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The present invention provides the following technical solutions:

[0057] The following methods are included;

[0058] Step 1: Data collection and preprocessing;

[0059] Step 2: Multimodal feature extraction;

[0060] Step 3: Feature fusion and spatiotemporal modeling;

[0061] Step 4: Semi-supervised learning guided by expert knowledge;

[0062] Step 5: Multi-task tea tree growth status prediction.

[0063] In step 1, the data collection and preprocessing step aims to collect multi-source heterogeneous data such as tea garden environmental parameters, tea tree images and text logs with high quality and efficiency, and perform intelligent cleaning and anomaly correction to provide reliable data support for subsequent tea tree growth status detection. Adaptive sampling rate AIoT sensors and high-definition cameras are used to collect tea garden environmental parameters, tea tree images and text log data, introduce attention mechanisms to optimize data quality, and use GPT to achieve intelligent data cleaning and anomaly correction;

[0064] In terms of environmental parameter collection, the present invention uses IoT sensors, including sensors for temperature, humidity, light intensity, and soil pH, and introduces an adaptive sampling strategy to dynamically adjust the sampling frequency.

[0065] In step 1, in terms of tea tree image acquisition, the present invention uses a high-definition camera installed above the tea garden, and the tilt angle is generally set to 30°~60° to obtain the overall image of the tea tree canopy. The shooting resolution is determined according to the tea garden area and the tea tree planting density, generally not less than 1080p, and the frame rate is not less than 30fps to ensure the clarity and continuity of the image. In order to highlight the key features of the growth status of the tea tree, the present invention introduces an attention mechanism in image acquisition, calculates the significance weight of the image area in real time through a convolutional neural network, dynamically adjusts the shooting parameters, and adaptively increases the shooting frequency and resolution of the key growth area of ​​the tea tree, reduces the shooting frequency of the relatively stable area, and reduces the transmission and storage overhead while improving data quality;

[0066] Text log data is collected through the tea farmers' mobile phone APP, and the recorded content includes tea garden management operations, pest and disease observation, fertilization and medication, etc. It adopts a structured form, and the fields include time, location, operation type, operation parameters, text description and picture attachments, etc. The collection frequency is not less than once a day;

[0067] To ensure the quality of collected data, the present invention uses a large language model GPT for intelligent data cleaning and anomaly detection. GPT learns data cleaning rules from a small number of manually labeled samples through Few-shot Learning, and generates self-correction prompts to optimize data.

[0068] GPT minimizes the loss function through optimization algorithms such as gradient descent, quickly adapts to new data cleaning tasks, and identifies abnormal data through statistical tests and cross-validation of data distribution, and fills or deletes them according to data types and business rules;

[0069] The cleaned data is stored in the time series database InfluxDB, which supports efficient data writing, query and aggregation analysis. InfluxDB uses a SQL-like query language and has built-in multiple aggregation functions, which is suitable for processing the time series data of tea garden sensors. At the same time, its horizontally scalable architecture can support the storage and query of TB-level data, providing convenience for subsequent large-scale data analysis.

[0070] In step 2, ConvNeXtV2 is used to extract local and global visual features of tea garden images, Informer is used to capture the long-distance dependencies of environmental time series data, and ERNIE3.0 is used to extract semantic features of tea garden text logs to achieve comprehensive feature representation of tea garden multimodal data.

[0071] In terms of tea garden image feature extraction, the present invention adopts the advanced visual transformer model ConvNeXtV2. Compared with the traditional convolutional neural network, ConvNeXtV2 introduces the self-attention mechanism and dynamic deep convolution, which can more effectively capture the global dependencies and local details of the image.

[0072] The self-attention mechanism captures global dependencies by calculating the correlation between image blocks; the dynamic deep convolution extracts multi-scale local features by adaptively adjusting the depth and width of the convolution kernel. Finally, ConvNeXtV2 obtains the global feature vector of the image as a learnable linear transformation matrix through global average pooling and linear transformation.

[0073] In terms of feature extraction of environmental time series data, the present invention uses the Informer model, which is a time series prediction model based on the self-attention mechanism. By introducing probabilistic self-attention and adaptive distillation, it can effectively capture the long-distance dependency of time series data and significantly reduce the computational complexity.

[0074] Compared with the traditional self-attention mechanism, probabilistic self-attention introduces random sampling and distillation operations, which reduces the computational complexity while maintaining the feature expression capability. Finally, Informer obtains the long-distance dependency features of the environmental time series data through adaptive distillation and linear transformation.

[0075] In step 2, in terms of text log feature extraction, the present invention adopts the knowledge-enhanced pre-trained language model ERNIE3.0. Compared with the traditional pre-trained language model, ERNIE3.0 can better understand the semantic information of the text by integrating knowledge graph and multi-granularity language unit modeling.

[0076] In step 3, based on Performer’s spatiotemporal heterogeneous graph neural network TSHG-PF, a spatiotemporal heterogeneous graph containing visual, temporal, and textual semantic nodes is constructed. DisentangledAttention is introduced to achieve efficient fusion of different modal features and model the spatiotemporal dynamic relationship between the tea garden environment and the growth status of tea trees.

[0077] In order to achieve efficient and explainable multimodal feature fusion, this paper designs a Performer-based spatiotemporal heterogeneous graph neural network TSHG-PF. TSHG-PF models tea garden data in the form of a graph, represents the features of different modalities as nodes in the graph, and describes the spatiotemporal dependency between nodes by introducing edge connections in the time and space dimensions.

[0078] In the aggregation update phase of the graph neural network, TSHG-PF introduces a Performer-based attention mechanism to reduce computational complexity and improve interpretability;

[0079] To further improve the fusion effect of different modal features, TSHG-PF introduces the DisentangledAttention mechanism based on Performer. DisentangledAttention decouples the content and position encoding of node features, allowing features of different modalities to interact in the two dimensions of content and position, thereby enhancing the transmission and fusion of cross-modal information.

[0080] In step 4, the expert-guided contrastive learning framework EGCL-IGPT based on InstructGPT uses expert knowledge to construct a small number of high-quality contrast samples, reduces the expert annotation cost through contrastive learning, and adopts active learning strategy to select the most representative samples for expert annotation, effectively integrating expert knowledge to guide model training;

[0081] By constructing a comparative learning task guided by expert knowledge and using a small amount of high-quality labeled samples, EGCL-IGPT learns a discriminative and semantically consistent representation in the feature space of tea garden growth status. At the same time, EGCL-IGPT adopts an active learning strategy to select samples with the most value for model discrimination for expert labeling, further reducing the labeling cost and improving the efficiency of expert knowledge utilization.

[0082] By constructing a comparative learning task guided by expert knowledge and using a small amount of high-quality labeled samples, a discriminative and semantically consistent representation is learned in the feature space of tea garden growth status. At the same time, EGCL-IGPT adopts an active learning strategy to select samples with the most valuable value for model discrimination for expert labeling, further reducing the labeling cost and improving the utilization efficiency of expert knowledge.

[0083] In step 4, in terms of active learning strategy, EGCL-IGPT adopts a committee-based query method to select the nodes that are most valuable and informative for the current model to be labeled by experts, and uses InstructGPT to generate corresponding queries, let experts label them, and add the labeled results to the training set to update the positive and negative samples for comparative learning;

[0084] By iteratively executing contrastive learning and active learning, EGCL-IGPT gradually optimizes the node embedding representation, while making targeted use of expert knowledge to improve the accuracy and interpretability of tea garden growth status prediction. In the inference stage, for the newly collected tea garden data, EGCL-IGPT first generates node representation through TSHG-PF, then maps it to the embedding space using the trained mapping function, and finally predicts the risk level of the node based on the similarity between the embedded vector and the labeled node.

[0085] In step 5, the tea tree growth status monitoring multi-task prediction framework TGM-MTL simultaneously optimizes the tea tree growth status recognition, anomaly detection and risk warning tasks, adaptively adjusts the loss weight of each task, and uses FLAN to fine-tune the instructions to improve the model robustness and generalization performance;

[0086] In order to realize multi-task prediction of tea tree growth status, the present invention proposes a multi-task prediction framework TGM-MTL for tea tree growth status monitoring. TGM-MTL integrates task-specific prediction layers into a unified backbone network in an end-to-end manner, dynamically balances the learning progress of different tasks by adaptively adjusting the loss weights of each task, and uses FLAN to fine-tune instructions to enhance the model's ability to understand and execute natural language instructions.

[0087] In step 5, in terms of instruction fine-tuning, TGM-MTL uses FLAN to fine-tune the model so that it can understand and execute instructions in natural language. FLAN learns to map instructions to corresponding operations by pre-training a large-scale language model and fine-tuning it on a large number of instruction-response pairs. In TGM-MTL, by concatenating task descriptions and node features into an instruction sequence and using the task output as the target response, FLAN can learn to adaptively perform different prediction tasks based on instructions.

[0088] In the reasoning stage, given a new spatiotemporal heterogeneous graph of a tea garden, TGM-MTL first generates node representations and embeddings through TSHG-PF and EGCL-IGPT, then inputs them into the shared backbone network and task-specific prediction heads, and combines the instruction fine-tuning capabilities of FLAN to output the prediction results of all tasks at the same time, thus achieving comprehensive monitoring and early warning of the growth status of tea trees.

[0089] Example:

[0090] 1. Data acquisition

[0091] The TG-ConIE method mainly collects three types of data: tea garden environmental parameters, tea tree images and text logs.

[0092] ① Environmental parameter data: such as Figure 2As shown in the figure, the AIoT wireless sensor network is used to collect tea garden microclimate and soil parameters in real time. Each monitoring point can represent at least 20 mu of tea garden. The typical deployment scheme of sensor nodes is as follows:

[0093] The sensors are networked via ZigBee / LoRa, and upload data to the tea garden edge gateway every 30 minutes, which is then transmitted back to the Alibaba Cloud IoT platform via 4G / NBIoT.

[0094] ②Tea tree image data: Figure 3 As shown, a high-definition network camera is used to take panoramic and local photos of the tea garden 4 times a day. The image specifications are as follows:

[0095] The video stream is transmitted to the tea garden server via the RTSP protocol, and after frame cutting, encoding and encryption, it is uploaded to Alibaba Cloud OSS. To protect privacy, the image metadata is encrypted using the national SM4 block encryption.

[0096] ③Text log data: Through the self-developed tea farmer APP, the following information is recorded at least once a day:

[0097] Tea tree growth: number of new shoots, bud and leaf shape, young leaf color, growth volume, etc.;

[0098] Pest and disease situation: pest type, occurrence site, degree of damage, prevention and control measures, etc.;

[0099] Tea garden management: content and time of farming operations such as fertilizing, pruning, weeding, and picking;

[0100] The text data is encapsulated in JSON format and encrypted using the SM2 elliptic curve public key, with limited access rights open to scientific research institutions. Visitors must pass the OAuth2.0 protocol authentication, and the data transmission channel uses TLS1.3 encryption. The original data is automatically cold-backed to OSS deep archive storage after being stored for 3 months.

[0101] 2. Typical Python libraries for data processing

[0102] ① Tea garden environmental parameters:

[0103] PandasDataFrame caches sensor time series data, fillna interpolates to fill missing values, rolling_mean smoothes short-term disturbances, and remove_outliers removes physical outliers.

[0104] DaskDataFrame accelerates parallel processing of tens of millions of data, and dash.dataframe.from_pandas batch conversion.

[0105] The aggregated environment parameters are written into the time series database using IoTDB. create_timeseries creates a timeline, insert_tablet inserts in batches, and delete_storage_group deletes expired data periodically.

[0106] StreamSetsDataCollector builds a real-time data collection channel and deploys KafkaConnectIoTDBSource / SinkConnector to synchronize the tea garden environmental parameters to the Kafka cluster in real time.

[0107] ②Tea tree image:

[0108] The PillowImage module reads the original image file, converts it to map the color space, resizes it to adjust the image size, and transposes it to correct the direction. OpenCV performs preprocessing on the tea tree image, such as smoothing, sharpening, and contrast enhancement. FastNlMeansDenoising reduces noise, GaussianBlur removes Gaussian blur, and equalizeHist performs histogram equalization. Use the ImageAIYOLOv4 / YOLOv5 model to detect and segment tea trees, crop out the tea tree area of ​​interest, and filter out the background noise area. Use Baidu PaddleDetection to deploy the PP-YOLOv2 tea tree pest and disease detection model, with an mAP of more than 95% for target detection, and supports the identification of 20 common pests and diseases. Use the PaddleClas image classification SDK to classify the morphology of tea tree buds and leaves and the tenderness of new shoots. Use the PP-ShiTu backbone network to achieve 10 types of morphology and 5 levels of tenderness evaluation, with an overall accuracy of more than 90%. The generated tea tree target detection and attribute classification annotation information, together with the original Figure 1 And save it to the MongoDBGridFS system, and use the pymongo library create_indexes to create a spatiotemporal composite index.

[0109] ③Tea Garden Log Text:

[0110] Pandas read_json loads the original JSON log, json_normalize flattens the nested structure, and uses ETL rules to automatically extract fields such as time, location, and event. Use jieba.cut and NLTK wordpunct_tokenize to segment Chinese and English texts, english.stopwords.words removes English stop words, and zhon.hanzi.punctuation removes Chinese punctuation. Use BaiduLAC lexical analysis and named entity recognition to extract tea garden management behaviors and managed objects, and use the ERNIE3.0 sklearn interface to retrain fine-grained NER in the tea field. Use the ERNIE3.0 sentiment analysis model to discriminate the positive / negative sentiment of tea farmers' logs, automatically classify according to sentiment bias, and weightedly count the frequency, spatiotemporal distribution, and personnel distribution of positive and negative events. Use PaddleNLPUIE to deploy tea garden domain knowledge extraction, build a causal graph of agricultural operations-tea tree growth response, and use ERNIE-GNN to realize agricultural knowledge reasoning. Redisredisearch is used to build reverse indexes for text semantic vectors and graph entities, and vector retrieval, knowledge question answering, and aggregate analysis are implemented through the ft.create, ft.search, and ft.aggregate interfaces of the redis-py library. The structured extraction results are written to MongoDB, the original text is stored in Elasticsearch, and the SparkSQL external table mapping is implemented using ES-HadoopConnector.

[0111] 3. Core model construction

[0112] The core model of TG-ConIE covers multimodal feature extraction, spatiotemporal heterogeneous graph neural network and multi-task prediction.

[0113] ①Multimodal feature extraction

[0114] Tea tree image features: Figure 4 As shown, the ConvNeXtv2 image classification model is used, pre-trained on the tea fine-grained classification dataset TeaTree-197, and then fine-tuned on the actual tea garden monitoring images:

[0115] ConvNeXt-B is used as the backbone for tea tree image feature extraction, with output channels of [128, 256, 512, 1024], a spatial downsampling rate of 32, and a corresponding image feature dimension of 1024. Main parameter settings:

[0116] ConvNeXt(depths=[3, 3, 27, 3], dims=[128, 256, 512, 1024], drop_path_rate=0.4, layer_scale_init_value=1.0, num_classes=num_classes)

[0117] Environmental parameter time series characteristics: such as Figure 5 As shown in the figure, the Informer encoder is used to extract the long-range time series representation of the tea garden environmental factors to fill the sensor sampling blind area:

[0118] The dimension of the environmental parameter encoding is 512. During training, samples are constructed with a sliding window of 6 hours (1 point every 10 minutes, lookback=36) to predict the next 24 hours (horizon=144):

[0119] Informer(enc_in=11, dec_in=11, c_out=11, out_len=144, factor=5, d_model=512, dropout=0.05, attn='prob', embed='fixed', freq='h', activation='gelu', num_workers=4)

[0120] Log text semantic features: such as Figure 6 As shown in the figure, Baidu ERNIE3.0 is used to extract the semantic representation of tea farmers' logs and capture the implicit causal knowledge contained in agricultural operations:

[0121] Use paddlenlp.transformers.ErnieModel to load the pre-trained model, using the bert-base-chinese word segmenter, and the text semantic feature dimension is 768:

[0122] tokenizer=ppnlp.transformers.ErnieTokenizer.from_pretrained('ernie-3.0-base-zh')model=ppnlp.transformers.ErnieModel.from_pretrained(pretrained_model_name_or_path='ernie-3.0-base-zh', num_hidden_layers=12)

[0123] ②Spatiotemporal Heterogeneous Graph Neural Network

[0124] The PGL framework is used to construct the tea garden spatiotemporal heterogeneous graph TSHG-PF, with tea trees as the central nodes, environmental factors and logs as edge nodes, to construct a multimodal spatiotemporal heterogeneous graph HIN=(V, E), integrating node attributes and edge relationships:

[0125] Tea tree node vt∈V: The attribute is the 1024-dimensional image feature extracted by ConvNeXt, and the edge set E is the heterogeneous edge set between the node vt at the current moment and its K nearest neighbor nodes and its own node in the past T time steps.

[0126] Environmental node ve∈V: The attributes are the 512-dimensional environmental parameter time series features extracted by Informer, and the edge set E is the heterogeneous association edges between node ve and tea tree node vt in the time and space dimensions.

[0127] Log node vl∈V: The attributes are the 768-dimensional agricultural log semantic features extracted by ERNIE3.0, and the edge set E is the implicit causal relationship edge between node vl and the environmental node ve and tea tree node vt.

[0128] Use PGL's GNNModel class to encapsulate spatiotemporal heterogeneous graph neural networks, and use DisentangledAttention and pgl.nn.GATConv to implement node information propagation.

[0129] TSHG-PF key parameter settings:

[0130] Number of graph convolution layers num_layers=6;

[0131] The number of multi-head attention heads num_heads=8;

[0132] The node dimension hidden_size=512 aligns the output features of the temporal encoder Informer;

[0133] Attention dropout=0.1, feature dropout=0.2;

[0134] The classifier output dimension output_size=3, corresponding to the tea tree growth status category {healthy, sub-healthy, unhealthy}

[0135] ③Multi-task prediction framework

[0136] Based on the high-dimensional spatiotemporal features of TSHG-PF, a multi-task prediction framework TGM-MTL was built to combine the three major tasks of tea tree growth status recognition, environmental anomaly detection, and drought level warning to improve the generalization performance of the model.

[0137] Task 1: Tea tree growth state y1∈{0, 1, 2}, the loss function uses cross entropy CrossEntropyLoss;

[0138] Task 2: Environment anomaly y2∈{0, 1}, the loss function uses focal loss FocalLoss, focusing on sample imbalance;

[0139] Task 3: Drought level y3∈[0, 4], the loss function uses mean square error MSELoss;

[0140] The overall multi-task loss function adopts a weighted summation strategy, and the weights are adaptively adjusted according to the difficulty of the task:

[0141]

[0142] Use BayesianWeightUncertainty to achieve adaptive update of task weights and alleviate multi-task gradient interference.

[0143] Finally, the multi-task prediction model TGM-MTL is fine-tuned using the FLAN paradigm on large-scale instruction data, greatly improving the generalization and question-answering capabilities of the model.

[0144] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A tea garden status monitoring method integrating a visual temporal text pre-training model, comprising the following methods; Step 1: Data collection and preprocessing; Step 2: Multimodal feature extraction; Step 3: Feature fusion and spatiotemporal modeling; Step 4: Semi-supervised learning guided by expert knowledge; Step 5: Multi-task tea tree growth status prediction.

2. The tea garden status monitoring method integrating the visual temporal text pre-training model according to claim 1 is characterized in that: In step 1, the data collection and preprocessing step aims to collect tea garden environmental parameters, tea tree images and text log multi-source heterogeneous data with high quality and efficiency, and perform intelligent cleaning and anomaly correction to provide reliable data support for subsequent tea tree growth status detection. Adaptive sampling rate AIoT sensors and high-definition cameras are used to collect tea garden environmental parameters, tea tree images and text log data, introduce attention mechanism to optimize data quality, and use GPT to achieve intelligent data cleaning and anomaly correction; In terms of environmental parameter collection, IoT sensors are used, including temperature, humidity, light intensity and soil pH sensors, and an adaptive sampling strategy is introduced to dynamically adjust the sampling frequency.

3. The tea garden status monitoring method integrating the visual temporal text pre-training model according to claim 1 is characterized in that: In step 1, in terms of tea tree image acquisition, a high-definition camera is used, which is installed above the tea garden, and the tilt angle is generally set to 30°~60° to obtain the overall image of the tea tree canopy. The shooting resolution is determined according to the tea garden area and the tea tree planting density, generally not less than 1080p, and the frame rate is not less than 30fps to ensure the clarity and continuity of the image. In order to highlight the key features of the tea tree growth state, an attention mechanism is introduced in image acquisition. The convolutional neural network is used to calculate the significance weight of the image area in real time, and the shooting parameters are dynamically adjusted to adaptively increase the shooting frequency and resolution of the key growth area of ​​the tea tree, and reduce the shooting frequency of the relatively stable area, thereby improving data quality while reducing transmission and storage overhead; Text log data is collected through the tea farmers' mobile phone APP. The recorded content includes tea garden management operations, pest and disease observation, fertilization and medication. It adopts a structured form with fields including time, location, operation type, operation parameters, text description and picture attachments. The collection frequency is no less than once a day. To ensure the quality of collected data, the large language model GPT is used for intelligent data cleaning and anomaly detection. GPT learns data cleaning rules from a small number of manually labeled samples through Few-shot Learning, and generates self-correction prompts to optimize data. GPT minimizes the loss function through the gradient descent optimization algorithm and quickly adapts to new data cleaning tasks. Abnormal data is identified through statistical tests and cross-validation of data distribution, and filled or deleted according to data types and business rules. The cleaned data is stored in the time series database InfluxDB, which supports efficient data writing, query and aggregation analysis. InfluxDB uses a SQL-like query language and has built-in multiple aggregation functions, which is suitable for processing the time series data of tea garden sensors. At the same time, its horizontally scalable architecture can support the storage and query of TB-level data, providing convenience for subsequent large-scale data analysis.

4. The tea garden status monitoring method integrating the visual temporal text pre-training model according to claim 1 is characterized in that: In step 2, ConvNeXtV2 is used to extract local and global visual features of tea garden images, Informer is used to capture the long-distance dependencies of environmental time series data, and ERNIE3.0 is used to extract semantic features of tea garden text logs to achieve comprehensive feature representation of tea garden multimodal data. In terms of tea garden image feature extraction, the advanced visual transformer model ConvNeXtV2 is used. Compared with the traditional convolutional neural network, ConvNeXtV2 introduces the self-attention mechanism and dynamic deep convolution, which can more effectively capture the global dependencies and local details of the image. The self-attention mechanism captures global dependencies by calculating the correlation between image blocks; the dynamic deep convolution extracts multi-scale local features by adaptively adjusting the depth and width of the convolution kernel. Finally, ConvNeXtV2 obtains the global feature vector of the image as a learnable linear transformation matrix through global average pooling and linear transformation. In terms of feature extraction of environmental time series data, the Informer model is used. Informer is a time series prediction model based on the self-attention mechanism. By introducing probabilistic self-attention and adaptive distillation, it can effectively capture the long-distance dependencies of time series data and significantly reduce the computational complexity. Compared with the traditional self-attention mechanism, probabilistic self-attention introduces random sampling and distillation operations, which reduces the computational complexity while maintaining the feature expression capability. Finally, Informer obtains the long-distance dependency features of the environmental time series data through adaptive distillation and linear transformation.

5. The tea garden status monitoring method integrating visual temporal text pre-training model according to claim 1 is characterized in that: In step 2, in terms of text log feature extraction, the knowledge-enhanced pre-trained language model ERNIE3.0 was used. Compared with traditional pre-trained language models, ERNIE3.0 can better understand the semantic information of the text by integrating knowledge graphs and multi-granularity language unit modeling.

6. The tea garden status monitoring method integrating visual temporal text pre-training model according to claim 1 is characterized in that: In step 3, based on Performer’s spatiotemporal heterogeneous graph neural network TSHG-PF, a spatiotemporal heterogeneous graph containing visual, temporal, and textual semantic nodes is constructed. DisentangledAttention is introduced to achieve efficient fusion of different modal features and model the spatiotemporal dynamic relationship between the tea garden environment and the growth status of tea trees. In order to achieve efficient and explainable multimodal feature fusion, a Performer-based spatiotemporal heterogeneous graph neural network TSHG-PF is designed. TSHG-PF models tea garden data in the form of a graph, represents the features of different modalities as nodes in the graph, and describes the spatiotemporal dependency between nodes by introducing edge connections in the time and space dimensions. In the aggregation update phase of the graph neural network, TSHG-PF introduces a Performer-based attention mechanism to reduce computational complexity and improve interpretability; To further improve the fusion effect of different modal features, TSHG-PF introduces the DisentangledAttention mechanism based on Performer. DisentangledAttention decouples the content and position encoding of node features, allowing features of different modalities to interact in the two dimensions of content and position, thereby enhancing the transmission and fusion of cross-modal information.

7. The tea garden status monitoring method integrating visual temporal text pre-training model according to claim 1 is characterized in that: In step 4, the expert-guided contrastive learning framework EGCL-IGPT based on InstructGPT uses expert knowledge to construct a small number of high-quality contrast samples, reduces the expert annotation cost through contrastive learning, and adopts active learning strategy to select the most representative samples for expert annotation, effectively integrating expert knowledge to guide model training; By constructing a comparative learning task guided by expert knowledge and using a small amount of high-quality labeled samples, a discriminative and semantically consistent representation is learned in the feature space of tea garden growth status. At the same time, EGCL-IGPT adopts an active learning strategy to select samples with the most valuable value for model discrimination for expert labeling, further reducing the labeling cost and improving the utilization efficiency of expert knowledge.

8. The tea garden status monitoring method integrating visual temporal text pre-training model according to claim 1 is characterized in that: In step 4, in terms of active learning strategy, EGCL-IGPT adopts a committee-based query method to select the nodes that are most valuable and informative for the current model to be labeled by experts, and uses InstructGPT to generate corresponding queries, let experts label them, and add the labeled results to the training set to update the positive and negative samples for comparative learning; By iteratively executing contrastive learning and active learning, EGCL-IGPT gradually optimizes the node embedding representation, while making targeted use of expert knowledge to improve the accuracy and interpretability of tea garden growth status prediction. In the inference stage, for the newly collected tea garden data, EGCL-IGPT first generates node representation through TSHG-PF, then maps it to the embedding space using the trained mapping function, and finally predicts the risk level of the node based on the similarity between the embedded vector and the labeled node.

9. The tea garden status monitoring method integrating visual temporal text pre-training model according to claim 1 is characterized in that: In step 5, the tea tree growth status monitoring multi-task prediction framework TGM-MTL simultaneously optimizes the tea tree growth status recognition, anomaly detection and risk warning tasks, adaptively adjusts the loss weight of each task, and uses FLAN to fine-tune the instructions to improve the model robustness and generalization performance; In order to realize multi-task prediction of tea tree growth status, a multi-task prediction framework TGM-MTL for tea tree growth status monitoring was proposed. TGM-MTL integrates task-specific prediction layers into a unified backbone network in an end-to-end manner, dynamically balances the learning progress of different tasks by adaptively adjusting the loss weights of each task, and uses FLAN to fine-tune instructions to enhance the model's ability to understand and execute natural language instructions.

10. The tea garden status monitoring method integrating visual temporal text pre-training model according to claim 1 is characterized in that: In step 5, in terms of instruction fine-tuning, TGM-MTL uses FLAN to fine-tune the model so that it can understand and execute instructions in natural language. FLAN learns to map instructions to corresponding operations by pre-training a large-scale language model and fine-tuning it on a large number of instruction-response pairs. In TGM-MTL, by concatenating task descriptions and node features into an instruction sequence and using the task output as the target response, FLAN can learn to adaptively perform different prediction tasks based on instructions. In the reasoning stage, given a new spatiotemporal heterogeneous graph of a tea garden, TGM-MTL first generates representations and embeddings of nodes through TSHG-PF and EGCL-IGPT, which are then input into the shared backbone network and task-specific prediction heads. Combined with the instruction fine-tuning capability of FLAN, it outputs the prediction results of all tasks at the same time, thus achieving comprehensive monitoring and early warning of the growth status of tea trees.

Citation Information

Cited By

  • Assessment method and system of tea growth model

    CN119761637A

  • Campus monitoring behavior identification method and system based on reasoning enhancement

    CN119785296A

  • Optical storage and charging station multi-strategy operation method, device and equipment based on deep learning

    CN119904121A

  • Action generation method and device based on frequency domain and text semantic information joint optimization

    CN119920005A

  • Day lily field management optimization method and system based on big data

    CN119991333A