Cloud edge efficient synchronization mechanism and distributed management scheduling system based on crop water shortage data

Abstract

The invention discloses a cloud edge efficient synchronization mechanism and distributed management scheduling system based on crop water shortage data, and relates to the technical field of intelligent agriculture. The system comprises an edge end, an edge computing node module, a cloud edge communication module and a cloud storage and management module, wherein the edge end is used for collecting multi-source original data related to crop water shortage; the edge computing node module is used for preprocessing the multi-source original data and extracting a feature vector comprehensively reflecting a crop water shortage state; the cloud side communication module is used for constructing a two-way communication link and adjusting the data transmission rate according to an adaptive window adjustment strategy; and the cloud storage and management module is used for carrying out encrypted storage on the data by adopting a distributed hash table technology and carrying out dynamic scheduling and distribution on the stored data by utilizing a distributed management scheduling algorithm. According to the invention, efficient collection, accurate transmission, intelligent scheduling and safe storage of crop water shortage data can be realized, and agricultural data management precision and decision-making efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent agriculture, and particularly to an efficient cloud-edge synchronization mechanism and a distributed management and scheduling system based on crop water shortage data. Background Art

[0002] With the development of intelligent agriculture, data-driven agricultural production management has gradually become one of the key technologies for improving agricultural production efficiency and coping with climate change. During the growth process of crops, water is one of the important factors affecting crop growth. Crop water shortage directly affects the growth, yield and quality of crops. In order to ensure the water supply of crops, the agricultural field has begun to widely apply Internet of Things (IoT) technology, combined with various data collection devices such as soil moisture sensors, weather stations, and crop physiological monitoring sensors, to monitor the water demand of crops in real time. However, with the increase in monitoring data, how to efficiently collect, store, transmit, process and schedule data has become an important problem faced in modern agricultural management.

[0003] Traditional agricultural water shortage monitoring technologies mainly rely on a single sensor for data collection, usually using fixed-frequency soil moisture measurement and weather data collection. These data are usually uploaded to a central server through a wireless transmission protocol and are processed centrally for decision support. In this traditional method, the agricultural irrigation decision-making system usually uses a static collection strategy, and the sensors collect data at a fixed frequency without dynamically adjusting the collection frequency according to the growth stage of the crops, soil properties or weather conditions. This method leads to an increase in energy consumption and data redundancy, and it is difficult to accurately reflect the real-time water demand of crops, and it cannot provide timely and accurate irrigation decision support. In addition, traditional data storage methods mostly use centralized databases, and with the continuous increase in the amount of data, the retrieval efficiency and storage pressure have gradually become bottlenecks.

[0004] Although traditional technologies can meet the basic needs of agricultural production to a certain extent, there are many problems in their practical applications. Especially in large-scale agricultural production and diverse application scenarios, traditional methods often cannot efficiently meet the requirements of real-time, accuracy, and large-scale management of crop water shortage data. First, the static data collection frequency is difficult to cope with the dynamic changes in water requirements during the crop growth process, and it cannot improve the timeliness of data collection at critical moments. Especially in the case of drought warnings or rainfall predictions, it cannot automatically adjust the collection frequency, resulting in energy waste and data lag. Second, traditional centralized data storage and management methods greatly reduce the retrieval efficiency and data access speed in the case of a large amount of data, causing inconvenience to users. Especially in multi-user and multi-application scenarios, it is difficult to meet the requirements of fast and accurate data acquisition. In addition, traditional network transmission protocols have not been effectively optimized for network congestion and bandwidth fluctuations, resulting in high data transmission delays. Especially in scenarios with high real-time requirements such as agricultural disaster warnings, key data may be lost or the transmission may be delayed, thus affecting the decision-making effect. Summary of the Invention

[0005] The purpose of the present invention is to provide a cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data, which can improve the accuracy of agricultural data management and decision-making efficiency.

[0006] To achieve the above purpose, the present invention provides the following solutions:

[0007] A cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data, comprising:

[0008] An edge side, used for collecting multi-source raw data related to crop water shortage;

[0009] An edge computing node module, used for preprocessing the multi-source raw data, extracting a feature vector comprehensively reflecting the crop water shortage state, and encapsulating the feature vector in a preset format;

[0010] A cloud-edge communication module, used for building a two-way communication link and adjusting the data transmission rate according to an adaptive window adjustment strategy; the adaptive window adjustment strategy includes an exponential backoff algorithm and a slow start algorithm;

[0011] A cloud storage and management module, used for receiving the multi-source raw data and the corresponding feature vector according to the adjusted data transmission rate, encrypting and storing them using a distributed hash table technology, and dynamically scheduling and allocating the stored data using a distributed management and scheduling algorithm.

[0012] Optionally, the edge side is provided with a soil humidity sensor, a weather station sensor, and a crop physiological index monitoring sensor.

[0013] Optionally, a data acquisition module is further provided at the edge side;

[0014] The data acquisition module is used to dynamically adjust the acquisition frequency according to the crop growth stage, environmental changes and historical data, and perform data acquisition according to the adjusted frequency to obtain multi-source raw data;

[0015] Among them, the relationship between the crop growth stage and frequency adjustment is expressed as:

[0016]

[0017] In the formula, w i is the acquisition weight of the i-th layer of soil sensors, f i is the acquisition frequency of the i-th layer of sensors, C i is the crop growth stage adjustment coefficient of the i-th layer of sensors, and n is the total number of sensor layers;

[0018] The function expression of the environmental change is specifically:

[0019] S t = α·S t-1 + β·P t + γ·T t

[0020] In the formula, S t is the soil humidity at the current moment, S t-1 is the soil humidity at the previous moment, P t is the precipitation at the current moment, T t is the temperature at the current moment, and α, β, γ are set coefficients determined according to the historical data.

[0021] Optionally, the edge computing node module specifically includes:

[0022] A model construction unit for constructing a multi-modal data fusion model based on deep learning;

[0023] A data fusion unit for removing noise and outliers from the multi-source raw data, and using the multi-modal data fusion model to extract features from the processed data to obtain a feature vector comprehensively reflecting the water shortage state of the crop, and encapsulating the feature vector in a preset format.

[0024] Optionally, the cloud-edge communication module specifically includes:

[0025] A communication link construction unit for constructing a two-way communication link;

[0026] A transmission rate adjustment unit for adjusting the data transmission rate based on the two-way communication link according to the exponential backoff algorithm and the slow start algorithm;

[0027] Among them, the exponential backoff algorithm is expressed as:

[0028] W n+1 = W n ·γ

[0029] In the formula, W n is the current window size, W n+1 is the adjusted window size, and γ is the attenuation factor;

[0030] The slow start algorithm is expressed as:

[0031] W n+1 = W n + ΔW

[0032] In the formula, ΔW = 512 bytes, indicating that the window size is increased by a fixed 512 bytes each time.

[0033] Optionally, the cloud storage and management module is deployed on a cloud server.

[0034] Optionally, the cloud storage and management module specifically includes:

[0035] An encryption unit for performing a hash operation on the received data to determine the corresponding hash value; the hash value is used to describe the specific node location where the data is stored in the distributed storage cluster; a local index structure based on an inverted index is also constructed on each of the nodes;

[0036] A scheduling unit for determining user requirements and dynamically scheduling and allocating the stored data according to the reward function and the distributed management scheduling algorithm.

[0037] Optionally, the cloud storage and management module further includes:

[0038] A data backup and recovery module for performing backup storage using redundant array technology and performing data recovery using a recovery algorithm based on differential backup; among them, the recovery algorithm is specifically:

[0039]

[0040] In the formula, D restore represents the recovered data, D full is the latest full backup data, ΔD i represents the incremental data between the i-th backups, and N is the number of differential backups since the last full backup.

[0041] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0042] The present invention discloses an efficient cloud-edge synchronization mechanism and a distributed management and scheduling system based on crop water shortage data. The system includes an edge side, an edge computing node module, a cloud-edge communication module, and a cloud storage and management module. Among them, the edge side is used to collect multi-source raw data related to crop water shortage. The edge computing node module is used to preprocess the multi-source raw data, extract a feature vector that comprehensively reflects the crop water shortage status, and encapsulate the feature vector in a preset format. The cloud-edge communication module is used to build a two-way communication link and adjust the data transmission rate according to an adaptive window adjustment strategy. The adaptive window adjustment strategy includes an exponential backoff algorithm and a slow start algorithm. The cloud storage and management module is used to receive the multi-source raw data and the corresponding feature vector according to the adjusted data transmission rate, perform encrypted storage using a distributed hash table technology, and perform dynamic scheduling and allocation on the stored data using a distributed management and scheduling algorithm. The present invention can achieve efficient collection, accurate transmission, intelligent scheduling, and secure storage of crop water shortage data, provide reliable data support for agricultural irrigation decision-making, disaster warning, and precise resource management, and improve the intelligent and sustainable development level of agricultural production. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 It is the architecture diagram of the data acquisition module in this embodiment;

[0045] Figure 2 It is the data processing flow chart of the edge computing node in this embodiment;

[0046] Figure 3 It is the transmission mechanism diagram of the cloud-edge communication module in this embodiment;

[0047] Figure 4 It is the principle diagram of data storage and retrieval of the cloud storage module in this embodiment;

[0048] Figure 5 It is the interaction diagram of the scheduling algorithm based on reinforcement learning in the cloud in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0050] The object of the present invention is to provide a cloud-edge efficient synchronization mechanism and a distributed management and scheduling system based on crop water shortage data, which can improve the accuracy of agricultural data management and the decision-making efficiency.

[0051] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] As Figures 1 - 5 shown, the present invention provides a cloud-edge efficient synchronization mechanism and a distributed management and scheduling system based on crop water shortage data, including:

[0053] An edge side, which is used to collect multi-source raw data related to crop water shortage; an edge computing node module, which is used to preprocess the multi-source raw data, extract a feature vector comprehensively reflecting the crop water shortage state, and encapsulate the feature vector in a preset format; a cloud-edge communication module, which is used to build a bidirectional communication link and adjust the data transmission rate according to an adaptive window adjustment strategy; the adaptive window adjustment strategy includes an exponential backoff algorithm and a slow start algorithm; a cloud storage and management module, which is used to receive the multi-source raw data and the corresponding feature vector according to the adjusted data transmission rate, perform encrypted storage using a distributed hash table technology, and perform dynamic scheduling and allocation on the stored data using a distributed management and scheduling algorithm.

[0054] A more specific processing method is as follows:

[0055] The edge side is provided with a data acquisition module, which is used to collect multi-source raw data related to crop water shortage in real time from a variety of different types of sensors. The sensors include a soil humidity sensor, a weather station sensor, and a crop physiological index monitoring sensor, and the acquisition frequency can be dynamically adjusted according to a preset strategy; as Figure 1 shown, it is an architecture diagram of the data acquisition module.

[0056] An edge computing node, which is connected to the data acquisition module, receives and preprocesses the multi-source raw data, removes noise and outliers, and encapsulates the preprocessed data in a preset format; as Figure 2 shown, it is a data processing flow chart of the edge computing node.

[0057] The cloud-edge communication module establishes a two-way communication link between the edge computing node and the cloud server, and ensures the efficiency and stability of data transmission based on an improved transmission protocol to achieve real-time upload of encapsulated data to the cloud server and rapid distribution of cloud control instructions to the edge side; as Figure 3 shown, it is a diagram of the transmission mechanism of the cloud-edge communication module.

[0058] The cloud storage and management module is deployed on the cloud server, used to store a large amount of crop water shortage data synchronized from the edge side, and constructs a distributed database for classified storage according to data characteristics and timestamps. At the same time, based on the distributed management and scheduling algorithm, the stored data is dynamically scheduled and allocated according to different regions, different crop types and real-time data requirements to meet the accurate and rapid acquisition requirements of crop water shortage data in multi-user and multi-application scenarios.

[0059] As a specific implementation method, the processing processes of the above modules are as follows:

[0060] In the data acquisition module, the multi-layer structure design adopted by the soil humidity sensor can not only accurately capture the subtle changes in soil humidity at different depths, but also dynamically adjust the acquisition weights and frequencies of each layer of sensors according to different crop growth stages, soil types and environmental changes to achieve efficient and accurate data acquisition. Specifically, the acquisition weights and frequencies of each layer of sensors are flexibly configured according to crop growth stages and soil conditions. The changes in water requirements of crops at different growth stages determine the focus of attention for collecting soil humidity data at each layer. In the seedling stage of crops, the change in surface soil humidity directly affects the growth of crops. Therefore, the acquisition weight of the surface sensing layer is set to 0.6 at this time, while the weights of the middle and deep layers are 0.3 and 0.1 respectively. When entering the vigorous growth stage of crops, the role of the middle layer of soil in water supply is more significant. Therefore, the acquisition weight of the middle layer sensor is increased to 0.5, the surface layer is reduced to 0.3, and the deep layer is 0.2 to ensure better capture of the water changes in the middle layer of soil.

[0061] To further improve data acquisition efficiency and energy conservation, the acquisition frequency is dynamically adjusted according to meteorological prediction information and historical data. When rainfall or irrigation events are predicted, the acquisition frequency of the deep soil detection layer is optimized through algorithms to avoid unnecessary data acquisition and improve energy use efficiency. For example, in the case of rainfall prediction, the water change in the deep soil is small, and the acquisition frequency of the deep layer can be reduced from once every 30 minutes to once every 2 hours; during the drought warning period, to ensure timely monitoring of soil moisture, the system will increase the acquisition frequency of all sensor layers to once every 15 minutes. This dynamic adjustment mechanism can not only effectively reduce energy consumption, but also ensure the provision of accurate soil humidity data to support irrigation decisions under different climate conditions.

[0062] The implementation of this dynamic adjustment mechanism can be described by the following formula to represent the relationship between data acquisition weight and frequency adjustment:

[0063]

[0064] where w i is the acquisition weight of the i-th layer of soil sensors, f i is the acquisition frequency of the i-th layer of sensors, C i is the crop growth stage adjustment coefficient of the i-th layer of sensors, and n is the total number of sensor layers. The purpose of this formula is to dynamically adjust the acquisition weight and frequency of each layer according to the requirements of the crop growth stage. By adjusting the weight and frequency, it is ensured that under different crop growth stages and different environmental conditions, the soil moisture data can more accurately reflect the changes in crop water requirements while reducing energy consumption. This method enables the system to flexibly respond to different agricultural scenarios and improve the overall intelligent level of agricultural irrigation management.

[0065] In addition, the combination of historical soil moisture data and meteorological data can also predict the future soil moisture trend and adjust the acquisition strategy through the following comprehensive judgment formula:

[0066] S t = α·S t-1 + β·P t + γ·T t

[0067] where S t is the soil moisture at the current moment, S t-1 is the soil moisture at the previous moment, P t is the precipitation at the current moment, T t is the temperature at the current moment, and α, β, γ are undetermined coefficients obtained through training with historical data. This formula predicts the future soil moisture by comprehensively considering factors such as the historical changes in soil moisture, precipitation, and temperature, so as to adjust the acquisition strategy in advance during drought or rainfall to maximize the data acquisition efficiency and energy-saving effect. This way of combining historical data with real-time meteorological data enhances the adaptability of the system in a dynamic environment and ensures that optimal water management can always be obtained during the crop growth process.

[0068] In the preprocessing process of edge computing nodes, the data fusion link is crucial for the accuracy and real-time performance of the crop water shortage monitoring system. Through the multi-modal data fusion model, the system can effectively integrate data from different sensors, including temperature, humidity, and light intensity data from weather station sensors, as well as leaf water content and stomatal conductance data from crop physiological index monitoring sensors. The output data of each sensor contains different environmental information and crop growth status. How to effectively fuse this information to obtain a comprehensive feature that accurately reflects the water status of the crop is the key to improving the accuracy of the system.

[0069] First, the model extracts features from information from different data sources. In terms of meteorological data, temperature trend features can be extracted from temperature data, representing the impact of rising or falling temperature on crop water evaporation; humidity data can extract water vapor saturation features, reflecting the adequacy of water in the air and affecting the evaporation rate; light intensity data can extract features of light duration and intensity fluctuations, affecting the photosynthesis process and water demand of plants. In terms of crop physiological data, leaf water content data can reflect the water stress of the crop, while stomatal conductance can reveal the fluidity of water in crop leaves, further helping to evaluate the degree of water deficit. The extraction of these features is completed through a multi-modal feature extraction network based on deep learning. The features of each data source are extracted through methods such as convolutional neural networks (CNNs), and then these features are concatenated into a unified feature vector for further fusion.

[0070] Next, the fused feature vector is input into a neural network for training. To optimize the model, the cross-entropy loss function is used as the optimization objective. The formula for the cross-entropy loss function is:

[0071]

[0072] where y i is the true label, is the model prediction value, and N is the number of samples. In crop water shortage monitoring, the label y i indicates whether the crop is water short (such as water short or not), and the prediction value of the model is the water shortage risk value output by the neural network after the fused feature vector. By minimizing the cross-entropy loss function, the neural network can automatically adjust the weights and learn the internal correlation between the data of each sensor, thereby improving the ability to identify the water shortage status of the crop.

[0073] The core objective of this multi-modal fusion model is to output a feature vector that comprehensively reflects the water shortage status of crops by integrating information from different data sources. For example, when the leaf water content decreases and the stomatal conductance decreases, while the light intensity increases, the temperature rises, and the humidity drops, the feature vector output by the model will indicate that the crop is in a high water shortage risk state, thereby triggering the warning mechanism at the edge. At this time, the edge computing node makes real-time decisions based on the fused features and pushes alarm information to farmers or agricultural managers to remind them to take countermeasures such as irrigation and shading to avoid serious damage to the crops due to insufficient water. Through continuous optimization and learning, the model can cope with complex agricultural environmental changes, improve the accuracy and timeliness of crop water management, and thus achieve the goal of precision agriculture management.

[0074] This deep learning-based multi-modal data fusion method can automatically identify and associate the key features in different sensor data, not only improving the accuracy of crop water shortage warning, but also enabling the system to flexibly respond under different environments and crop growth conditions, thereby enhancing the intelligent level and resource utilization efficiency of agricultural production.

[0075] In the design of the cloud-edge communication module, the improved transmission protocol solves the network congestion problem through an adaptive window adjustment strategy, ensuring efficient and stable data upload under different network conditions. This strategy is particularly important because the crop water shortage monitoring system needs to upload key data to the cloud in real time and accurately, and network congestion may lead to data loss or transmission delay, thus affecting the timeliness of crop water management and disaster warning.

[0076] During the process of transmitting data over the network, the edge computing node first determines the network status by real-time monitoring of the network bandwidth utilization rate and round-trip delay. When the bandwidth utilization rate exceeds 80% and the round-trip delay increases by more than 20%, it is determined as network congestion. At this time, the system gradually reduces the size of the sending window through an exponential backoff algorithm. Starting from the initial 1024 bytes, the attenuation ratio each time is 0.5, that is, the window size becomes 50% of the original. This backoff mechanism can effectively slow down the data sending rate and avoid packet loss caused by overload. In addition, to ensure the upload priority of key data, the system will suspend the transmission of low-priority data and give priority to ensuring the upload of key data related to crop water shortage (such as severe drought warnings, physiological indicators of acute crop water shortage, etc.) to ensure timely acquisition of these data with high timeliness requirements.

[0077] The formula of the exponential backoff algorithm can be expressed as:

[0078] W n+1 =W n ·γ

[0079] Where, W n is the current window size, Wn+1 is the adjusted window size, and γ is the attenuation factor, usually set to 0.5. The core purpose of this algorithm is to alleviate network congestion by gradually reducing the sending window, reducing the competition for excessive data, and thus improving the stability of the network. When the network condition recovers, the bandwidth utilization rate is lower than 60% and the round-trip delay is stable, the system will gradually increase the sending window according to the slow start algorithm to restore the normal rate of data transmission.

[0080] The formula for the slow start algorithm is:

[0081] W n+1 = W n + ΔW

[0082] where ΔW = 512 bytes is the window increment, meaning that the window size is increased by a fixed 512 bytes each time. Through this process, the system can gradually restore the data transmission rate and re-include the transmission of low-priority data after the network returns to stability. The purpose of the slow start algorithm is to prevent the network from becoming congested again due to a sudden increase in the window size and ensure a smooth transition of the transmission.

[0083] Through the combination of these two algorithms, the system can flexibly adjust the data transmission rate in the case of network bandwidth and delay fluctuations, ensuring that the transmission of high-priority crop water shortage data is not affected while guaranteeing the overall transmission efficiency of the system. This adaptive adjustment strategy not only effectively avoids the impact of network congestion on the crop water shortage monitoring system but also ensures that the cloud can obtain key real-time data in a timely manner, providing strong support for precise irrigation and agricultural disaster warning.

[0084] The cloud storage and management module can efficiently manage and store a large amount of crop water shortage data by adopting the distributed hash table (DHT) technology. As Figure 4 shown, it is the schematic diagram of data storage and retrieval of the cloud storage module. Under this storage architecture, the crop water shortage data is subjected to a hash operation according to its key attributes such as region, crop type, and collection time to generate a unique hash value. These hash values determine the specific node positions in the distributed storage cluster where the data is stored, thus realizing the decentralized storage of data and avoiding the single-point failure and performance bottlenecks that may be brought about by centralized storage.

[0085] Specifically, taking "North China - Wheat - 20230601" as an example, the system performs a hash operation on these identification information to generate the corresponding hash value. For example, assume the hash function is H(X), where X is the string "North China - Wheat - 20230601", and the hash value H(X) is used to locate the specific node where the data is stored. The role of the hash function is to disperse the data storage in the storage node cluster, ensure the load balance of the data, and also enable the data to be efficiently located in the cluster.

[0086] Assume that the size of the data storage node cluster is N, and the output range of the hash function H(X) is [0, N-1]. Then the location of the data in the storage cluster nodes is determined by the following formula:

[0087] Node = H(X) mod N

[0088] Where H(X) is the hash value of the input data, N is the total number of nodes, and Node is the node number storing the data. In this way, data of different regions and different crops are automatically dispersed to multiple nodes, thus improving the scalability and fault tolerance of the system.

[0089] To improve the data retrieval efficiency, the system also constructs a local index structure based on the inverted index on each storage node. The inverted index takes data features such as soil moisture range and crop water shortage level as index keywords, and records the data storage addresses corresponding to each keyword. In this way, the system can perform efficient retrieval in the indexes of local storage nodes according to the query conditions, reducing unnecessary global search time.

[0090] For example, when the user queries the data under the specific condition of "North China region, wheat crop, mild water shortage", the system first queries the relevant index information in the global index server and quickly locates the node range storing the target data. Then, in the local inverted indexes of each node, the data is accurately located according to the specific query conditions, thus greatly shortening the retrieval time and improving the response speed of the system. This mechanism can efficiently process large-scale data requests and meet the high requirements for data real-time in multi-scenario applications such as agricultural scientific research and production management.

[0091] The construction and query of the inverted index can be described by the following formula:

[0092] I(k) = {D 1 , D 2 ,..., D m}

[0093] Where I(k) represents the inverted index of keyword k, D 1 , D 2 ,..., D m are all data storage addresses related to keyword k, and m is the number of documents containing this keyword. Through the inverted index, the system can quickly locate the data containing specific keywords, thus greatly improving the data retrieval efficiency.

[0094] The combination of this distributed storage and retrieval strategy not only improves the storage and query efficiency of crop water shortage data, but also maintains high performance and scalability in the face of massive data. Through the combination of hash distribution and inverted index, the system can quickly and accurately provide the required data, supporting the real-time needs of precision irrigation and crop growth monitoring in agricultural production, thus providing strong support for agricultural management.

[0095] The cloud storage and management module can dynamically adjust the data scheduling strategy according to the needs of different users and application scenarios by introducing a distributed management scheduling algorithm based on reinforcement learning, ensuring the efficient acquisition and application of crop water shortage data. As Figure 5 shown, it is the interaction diagram of the scheduling algorithm based on reinforcement learning in the cloud. Through environmental interaction and reward mechanism, the reinforcement learning algorithm can self-optimize in the changing demands and data distributions, thus achieving optimal data scheduling. Specifically, the state space of the reinforcement learning model consists of user demands and data distribution situations, and the action space includes selecting the storage nodes for scheduling and the transmission priorities of data.

[0096] In this scheduling strategy, the algorithm first identifies the needs of different users. The agricultural irrigation decision-making system may need to monitor the water shortage trend of crops in a certain area within the next 24 hours in real time, while research institutions need to study the water shortage patterns during the growth cycle of a certain variety of crops based on long-term time series data. Each demand corresponds to a different environmental state. For example, if the agricultural irrigation system needs the crop water shortage data of a certain area within the next 24 hours, this demand will be regarded as a state, and in this state, the algorithm will decide which storage node to retrieve data from and how to set the transmission priority of this data.

[0097] The core of reinforcement learning is to drive the optimization of the scheduling strategy by constructing a reward function. During the scheduling process, when the system's decision can accurately meet the user's needs, a positive reward will be given. For example, if the scheduling strategy successfully provides an accurate water shortage warning for irrigation decision-making, resulting in an irrigation water saving rate increase of more than 10%, the system will give a reward. On the contrary, if the scheduling strategy fails to effectively support the demand, causing the user to fail to obtain the required data in a timely manner, the system will give a negative reward. This process enables the reinforcement learning model to gradually learn the optimal scheduling strategy through the balance of exploration and exploitation in continuous interaction with the environment.

[0098] To make the system schedule more efficiently, the reward function can be expressed by the following formula:

[0099] R t = λ 1 ·ΔS irrigation + λ 2 ·ΔT research - λ3 ·ΔT delay

[0100] where R t is the total reward at time step t, ΔS irrigation is the change in the water-saving rate of the irrigation decision, ΔT research is the change in the lead time of the research result, ΔT delay is the change in the data scheduling delay, λ 1 , λ 2 , λ 3 are the weight coefficients of the corresponding objectives. This formula provides positive or negative rewards for the system scheduling strategy by comprehensively considering the satisfaction of different user requirements, such as water-saving rate, research progress, and data delay. These reward values will in turn affect the policy update in the learning process, enabling the algorithm to automatically optimize the scheduling scheme according to different scenarios.

[0101] As time goes by and the model continuously interacts with the environment, the reinforcement learning algorithm can adaptively adjust the scheduling strategy, enabling the system to continuously learn and optimize in the face of changes in data requirements, user behavior, and data distribution. For example, during peak demand periods, the system can automatically increase resource allocation to high-priority users; while during low data load periods, the system can adjust the data scheduling strategy to ensure the efficient transmission of low-priority data and reduce resource waste.

[0102] Through this distributed management scheduling algorithm based on reinforcement learning, the system can dynamically adjust the scheduling scheme in real time according to actual needs, data distribution, and network conditions, thereby ensuring that different types of users can efficiently obtain the required crop water shortage data. The introduction of this algorithm not only improves the operation efficiency of the agricultural data management system but also can flexibly adapt to changes according to the requirements of different scenarios, enhancing the accuracy and intelligence level of crop water management.

[0103] The cloud storage and management module has a data backup and recovery function, further enhancing the reliability and data security of the system. In the face of data loss, damage, or other failures, it can quickly recover and ensure the continuity and accuracy of crop water shortage data, thus providing stable support for agricultural production. The data backup strategy is customized according to the importance and update frequency of crop water shortage data. For data that changes frequently in real time and has a significant impact on agricultural production, such as soil moisture and crop acute water shortage physiological index data, the backup period is set to once a day; for relatively stable historical meteorological statistical data, the backup period is set to once a week. This backup strategy can effectively balance storage efficiency and the timeliness of data recovery.

[0104] Backup data is stored in off-site redundant storage servers, and redundant array technology (RAID) is used to improve data security. Redundant array technology disperses data storage across multiple hard drives. Even if some hard drives fail, the data can still be recovered, thus avoiding data loss problems caused by single-point failures. The core idea of redundant array technology is to divide and store data in a certain redundant manner so that the original data can be restored through redundant data in case of hardware failures. Suppose the data is divided into n parts and stored with m parts of redundant data. When m hard drives are damaged, the data can still be restored through the remaining n - m hard drives.

[0105] When data loss or damage occurs, the system can quickly detect the anomaly and initiate the data recovery process. To improve the recovery efficiency, the system adopts a recovery algorithm based on differential backup. The core of differential backup is to record the data changes between each backup and only save the data that has been modified or newly added since the last backup. This can significantly reduce the storage space requirements and speed up the recovery process. During data recovery, the system first restores from the latest full backup data to ensure the integrity of the basic data. Then, it traces the incremental data since the last full backup through the backup log and quickly supplements these newly added or modified data.

[0106] The specific recovery process can be described by the following formula:

[0107]

[0108] In the formula, D restore represents the recovered data, D full is the latest full backup data, ΔD i represents the incremental data between the i-th backups, and N is the number of differential backups since the last full backup. In this way, the recovery process can first restore the general data structure and then gradually restore the detailed data according to the differential backup files, ensuring that the recovery process is both efficient and complete.

[0109] This differential backup and recovery algorithm not only ensures data integrity and efficient recovery but also reduces the consumption of storage resources and improves system performance. When emergencies such as hardware failures and network attacks occur, it can minimize the risk of data loss and recovery time, ensuring the availability of crop water shortage data. Through this reliable data backup and recovery mechanism, the agricultural production management system can maintain continuous and stable monitoring and decision support, ensuring that the system can be quickly restored to normal operation under any circumstances and providing uninterrupted precise irrigation and crop management services for farmers.

[0110] Example:

[0111] Taking an agricultural greenhouse planted with wheat as the set scenario, the water shortage situation of crops is monitored in real time and precisely regulated through this solution. The specific process is as follows:

[0112] 1. Multi-source data collection and preprocessing

[0113] 1.1 Data collection module

[0114] In the agricultural greenhouse, multiple sensors are used to collect data related to crop water shortage in real time. These sensors include:

[0115] Soil moisture sensors: Arranged at different soil layers in the greenhouse, including surface soil, middle soil, and deep soil. Sensors at each layer collect soil moisture data at the corresponding depth according to different sensitivities and detection depths. The system dynamically adjusts the collection weights of sensors at each layer according to the crop growth stage. For example, the weight of surface soil moisture is relatively large during the seedling stage of the crop, while the weight of middle soil moisture is relatively large during the vigorous growth stage.

[0116] Meteorological station sensors: Include temperature and humidity sensors, light intensity sensors, and are used to collect meteorological data such as environmental temperature and humidity, light intensity, etc.

[0117] Crop physiological monitoring sensors: Collect physiological data such as leaf water content and stomatal conductance of crops, reflecting the water stress state of crops.

[0118] The data collection frequency of the sensors is adaptively adjusted according to dynamic changes such as weather forecasts and soil moisture. When a drought warning occurs, the system will increase the collection frequency to once every 15 minutes. When rainfall is predicted, the system will reduce the collection frequency of deep soil sensors to save energy consumption.

[0119] 1.2 Data preprocessing

[0120] The original data collected by these sensors is preprocessed through edge computing nodes. The preprocessing steps include:

[0121] Denoising and outlier detection: By setting thresholds and data filtering algorithms, noise data and outliers of the sensors are removed.

[0122] Data standardization and encapsulation: Convert the data of different sensors into a unified format, and after standardization processing, encapsulate it into data packets for preparation to be transmitted to the cloud.

[0123] After data processing, the system uploads the preprocessed data to the cloud for storage and further analysis through edge computing nodes via a wireless network.

[0124] 2. Cloud-edge communication module and data upload

[0125] 2.1 Improved transmission protocol

[0126] The edge-cloud communication module is responsible for ensuring the real-time upload of data from edge computing nodes to the cloud. When the network bandwidth exceeds 80% and the round-trip delay increases by more than 20%, the system determines that there is network congestion, activates the exponential backoff algorithm, gradually reduces the sending window size from the initial 1024 bytes, and suspends the transmission of low-priority data to prioritize the upload of critical data. For example, when the soil moisture sensor reports an extreme drought situation, the system preferentially uploads this critical data to ensure that agricultural managers receive important data in a timely manner.

[0127] When the network bandwidth returns to normal, the system gradually increases the sending window according to the slow start algorithm, resumes the upload of low-priority data, and ensures the stability and efficiency of the entire data transmission process.

[0128] 2.2 Data Transmission and Priority Management

[0129] According to the requirements of agricultural scenarios, the system can set priorities for different types of data to ensure that the physiological index data of acute crop water shortage is uploaded first, and meteorological data and historical statistical data can be uploaded with appropriate delays. For example, when a crop water shortage warning occurs, the system will immediately send soil drought data and meteorological prediction information to farmers so that they can take irrigation measures in a timely manner.

[0130] 3. Data Storage and Distributed Management

[0131] 3.1 Data Storage Module

[0132] Cloud storage uses the Distributed Hash Table (DHT) technology to store crop water shortage data in a decentralized manner. Each piece of data is first hashed through a hashing algorithm (e.g., MD5 or SHA-256) to generate a unique hash value, and the storage location of the data is determined based on this hash value. For example, for the identification data such as "North China - Wheat - 20230601", the system calculates the hash value and stores it in a certain node in the cloud storage cluster. To improve the storage and retrieval efficiency, the system uses the inverted index technology on each storage node to index features such as soil moisture and crop water shortage level for quick location and retrieval.

[0133] 3.2 Data Scheduling and Query

[0134] When querying data, the user requests data for a specific region, crop, and time period. The system first searches for relevant index information in the global index server to quickly locate the range of storage nodes. Then, it uses the inverted index to accurately search for the target data in each storage node. Due to the use of distributed storage, the data query time is greatly reduced, ensuring that users can obtain the required data efficiently and accurately.

[0135] 4. Data Backup and Recovery Mechanism

[0136] 4.1 Data Backup

[0137] To prevent data loss or damage, the system regularly backs up the crop water shortage data. According to the nature of the data, the system sets daily backups for the soil moisture data and crop physiological data that change frequently in real time, while for the relatively stable meteorological data, a backup is performed once a week. The backup data is stored in a remote redundant storage server, and redundant array technology is used to ensure the security of the data.

[0138] 4.2 Data Recovery

[0139] When a data loss or damage event occurs, the system performs data recovery through a recovery algorithm based on differential backup. The system first restores the most recent full backup data, and then uses the backup log file to supplement the data that has been newly added or modified since the last backup. The recovery process can be automated to ensure that data recovery can be quickly completed in the event of disasters such as hardware failures and cyberattacks, maximizing the integrity and availability of the crop water shortage data.

[0140] 5. Reinforcement Learning Scheduling Algorithm

[0141] 5.1 Data Scheduling Strategy

[0142] The system performs dynamic scheduling through a reinforcement learning model. In the scenarios of agricultural irrigation and scientific research data requirements, the system first evaluates the environmental state (such as user requirements, the distribution of data storage nodes, etc.), and then takes appropriate actions according to the state (such as selecting storage nodes, adjusting transmission priorities, etc.). Through interaction with the environment, the reinforcement learning model can continuously optimize the scheduling strategy to ensure that the data transmission requirements in different user and application scenarios are met.

[0143] 5.2 Reward Mechanism and Learning Process

[0144] In reinforcement learning, the system sets a reward function. When the scheduling strategy can improve the irrigation water saving rate or accelerate the output of scientific research results, a positive reward is given; otherwise, a negative reward is given. Over time, the reinforcement learning model can continuously optimize its scheduling strategy, automatically adapt to user requirements and data changes, and enhance the adaptive ability and overall efficiency of the system.

[0145] Through the above implementation manners, the present invention can effectively achieve the efficient acquisition, real-time transmission, intelligent scheduling and reliable storage of agricultural water shortage data, not only improving the efficiency of agricultural production management, but also optimizing the energy use and data access efficiency, and promoting the practical application of agricultural intelligent management and precise irrigation.

[0146] According to the above technical solutions, the beneficial effects of this solution include:

[0147] (1) Improve the timeliness and accuracy of crop water shortage data: Through multi-source data collection and real-time data preprocessing by edge computing nodes, the solution can dynamically adjust the data collection frequency and weight to ensure the accurate collection of key data such as soil moisture and crop physiological indicators, and then provide efficient real-time data support to help the agricultural production management system reflect the crop water shortage situation in real time.

[0148] (2) Reduce energy consumption and improve system efficiency: Under the strategy of dynamically adjusting the data collection frequency, the solution can reasonably adjust the collection frequency of each layer of sensors according to the crop growth stage and meteorological prediction. Especially in the case of drought warning and rainfall prediction, by reducing the collection frequency of non-critical layers, it can save energy consumption while maintaining the accuracy of data collection.

[0149] (3) Enhance the stability and efficiency of data transmission: Through the improved cloud-edge communication module transmission protocol, adopting an adaptive window adjustment strategy and congestion control mechanism, it ensures that key data is given priority for uploading during network congestion, avoiding the loss or delay of important data due to network instability, and improving the stability and efficiency of data transmission to meet the real-time requirements of agricultural irrigation decision-making and disaster warning.

[0150] (4) Optimize data storage and retrieval efficiency: By using the distributed hash table (DHT) technology to distribute the storage of crop water shortage data and combining the inverted index technology to improve data retrieval efficiency, the solution can quickly locate and accurately obtain data under specific regions, crop types, and water shortage states, greatly improving the efficiency and accuracy of data retrieval and supporting the multi-scenario needs of agricultural scientific research and production management.

[0151] (5) Ensure data security and integrity: Through regular backup, redundant storage, and differential backup recovery algorithms, the solution effectively prevents the risk of data loss or damage. When a failure occurs in cloud storage, it can quickly recover the data to ensure the integrity and availability of crop water shortage data, providing reliable data guarantee for the continuous and stable monitoring and decision-making of agricultural production.

[0152] (6) Automatically optimize the data scheduling strategy and improve the system's adaptability: The distributed management scheduling algorithm based on reinforcement learning can automatically optimize the scheduling strategy according to different user requirements and data distribution situations to ensure that in a dynamically changing environment, it can efficiently and intelligently meet the data needs of application scenarios such as agricultural irrigation and scientific research, improving the system's adaptability and data management efficiency.

[0153] (7) Improve the accuracy and intelligence of agricultural production decision-making: By combining multi-source data analysis and deep learning models, the solution can accurately reflect the water shortage status and change trends of crops, helping agricultural managers make more scientific irrigation decisions, disaster warnings, and resource allocations, and promoting the development of agricultural production towards a more efficient and intelligent direction.

[0154] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0155] Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The descriptions of the above embodiments are only used to help understand the core idea of the present invention; at the same time, for those of ordinary skill in the art, based on the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data, characterized in that: include: The edge is used to collect multi-source raw data related to crop water shortage; An edge computing node module, used for preprocessing the multi-source raw data, extracting a feature vector that comprehensively reflects the water shortage state of the crop, and packaging the feature vector in a preset format; A cloud-edge communication module is used to establish a two-way communication link and adjust the data transmission rate according to an adaptive window adjustment strategy; the adaptive window adjustment strategy includes an exponential backoff algorithm and a slow start algorithm; The cloud storage and management module is used to receive the multi-source original data and the corresponding feature vectors according to the adjusted data transmission rate, and use the distributed hash table technology for encrypted storage, and use the distributed management scheduling algorithm to dynamically schedule and allocate the stored data.

2. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 1 is characterized in that: The edge end is provided with a soil moisture sensor, a weather station sensor and a crop physiological index monitoring sensor.

3. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 1 is characterized in that: The edge end is also provided with a data acquisition module; The data acquisition module is used to dynamically adjust the acquisition frequency according to the crop growth stage, environmental changes and historical data, and to acquire data according to the adjusted frequency to obtain multi-source original data; The relationship between the crop growth stage and the frequency adjustment is expressed as: In the formula, w i is the collection weight of the soil sensor at the i-th layer, f i is the acquisition frequency of the sensor at the i-th layer, C i is the crop growth stage adjustment coefficient of the i-th sensor layer, and n is the total number of sensor layers; The functional expression of the environmental change is specifically: S t =α·S t-1 +β·P t +γ·T t In the formula, S t is the soil moisture at the current moment, S t-1 is the soil moisture at the previous moment, P t is the current precipitation, T t is the current temperature, and α, β, and γ are setting coefficients determined based on the historical data.

4. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 1 is characterized in that: The edge computing node module specifically includes: A model building unit, used to build a multimodal data fusion model based on deep learning; The data fusion unit is used to remove noise and outliers from the multi-source raw data, and use the multimodal data fusion model to extract features from the processed data to obtain a feature vector that comprehensively reflects the water shortage state of the crop, and encapsulate the feature vector in a preset format.

5. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 1 is characterized in that: The cloud-edge communication module specifically includes: A communication link building unit, used for building a bidirectional communication link; A transmission rate adjustment unit, configured to adjust the data transmission rate based on the bidirectional communication link according to an exponential backoff algorithm and a slow start algorithm; Wherein, the exponential backoff algorithm is expressed as: W n+1 =W n ·γ Where W n is the current window size, W n+1 is the adjusted window size, γ is the attenuation factor; The slow start algorithm is expressed as: IN n+1 =In n +ΔW Wherein, ΔW=512 bytes, indicating that the window size is increased by a fixed 512 bytes each time.

6. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 1 is characterized in that: The cloud storage and management module is deployed on a cloud server.

7. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 1 is characterized in that: The cloud storage and management module specifically includes: The encryption unit is used to perform a hash operation on the received data to determine a corresponding hash value; the hash value is used to describe the specific node location where the data is stored in the distributed storage cluster; each of the nodes is also constructed with a local index structure based on an inverted index; The scheduling unit is used to determine user needs and dynamically schedule and allocate stored data according to the reward function and the distributed management scheduling algorithm.

8. The cloud-edge efficient synchronization mechanism and distributed management and scheduling system based on crop water shortage data according to claim 7 is characterized in that: The cloud storage and management module further includes: The data backup and recovery module is used to use redundant array technology for backup storage and to use a recovery algorithm based on differential backup for data recovery; wherein the recovery algorithm is specifically: Where D restore Indicates the restored data, D full It is the latest full backup data, ΔD i Indicates the incremental data between the i-th backups, and N is the number of differential backups since the last full backup.

Images