Data optimization storage method and system for smart agriculture

Abstract

The invention discloses a data optimization storage method and system for smart agriculture, and relates to the technical field of smart agriculture, and the method comprises the steps: collecting agricultural field data, remote sensing data and management data, and carrying out the preprocessing of the data, and constructing a multi-source heterogeneous data matrix; the method comprises the steps of calculating fuzzy similarity of a multi-source heterogeneous data matrix, constructing a time sequence distance matrix, calculating weight vectors of different data types, constructing a weighted distance matrix according to distribution weights of different data types, calculating a weighted fuzzy similarity matrix, constructing a Boolean similarity matrix, distinguishing redundant nodes and representative nodes, and determining difference vectors. According to the method, redundant data are accurately identified by using the weighted fuzzy similarity matrix, only the difference vector is stored, the storage space occupation is greatly reduced, the consumption of cloud or chain end storage resources is effectively reduced, the actually transmitted data volume is obviously reduced through storage compression and differential storage, and the data transmission efficiency is improved. And the data transmission burden of the edge device or the cloud node is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of smart agriculture, and particularly to a data optimization storage method and system for smart agriculture. Background Art

[0002] Currently, smart agriculture widely relies on field sensors, satellite remote sensing technology, and agricultural management information systems to obtain a large amount of multi-source heterogeneous data, realizing real-time perception and precise decision-making of agricultural production status. These data usually exhibit characteristics such as high dimensionality, multiple sources, and strong temporality, posing extremely high requirements for the capacity, efficiency, and security of data storage;

[0003] Traditional smart agriculture data storage methods mainly adopt the methods of complete backup of original data or simple redundancy removal, with a large amount of data redundancy and high consumption of storage resources; moreover, most existing technologies do not fully consider the importance differences of different data types, often using a unified fixed weight for processing, which is difficult to adapt to the dynamic characteristics of data, affecting the accuracy of data redundancy detection, and thus reducing the data compression and storage efficiency. In addition, existing agricultural data storage methods usually lack an effective data priority prediction and intelligent optimization scheduling mechanism, and cannot accurately distinguish and differentiate the value and timeliness of stored data, resulting in unreasonable allocation of storage resources, making it difficult to save data with high value density in a timely manner, and even possibly risking the loss of high-value data. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a data optimization storage method for smart agriculture to solve the problems that existing agricultural data storage methods usually lack an effective data priority prediction and intelligent optimization scheduling mechanism, cannot accurately distinguish and differentiate the value and timeliness of stored data, resulting in unreasonable allocation of storage resources, making it difficult to save data with high value density in a timely manner, and even possibly risking the loss of high-value data.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a data optimization storage method for smart agriculture, which includes:

[0008] Collect agricultural field data, remote sensing data, and management data, and perform preprocessing to construct a multi-source heterogeneous data matrix;

[0009] Calculate the fuzzy similarity of the multi-source heterogeneous data matrix, construct a time series distance matrix to calculate the weight vectors of different data types, construct a weighted distance matrix according to the weights assigned to different data types, calculate the weighted fuzzy similarity matrix, construct a Boolean similarity matrix, distinguish redundant nodes and representative nodes, and determine the difference vector;

[0010] Form a feature matrix based on the representative node data and the difference vector data, calculate the feature weights by the entropy weight method, construct a linear logistic regression model to predict the agricultural yield results of different nodes, obtain a sorted sequence, and calculate the fitness value of each value in the sequence. Use the differential evolution algorithm for mutation and crossover, and update the population based on the genetic algorithm to optimize the sorted sequence;

[0011] Generate a hash digest of the sorted sequence data for secure storage, perform data consistency verification, and generate a log record for cloud storage.

[0012] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the calculation of the weighted fuzzy similarity matrix and the construction of the Boolean similarity matrix include,

[0013] Calculate the fuzzy similarity of the multi-source heterogeneous data matrix, where calculate the sequence distance between different nodes, and calculate the time series distance matrix for each data type k;

[0014] Adopt the feature variance normalization method to calculate the weight vectors of different data types, construct a weighted distance matrix according to the weights assigned to different data types, and calculate the weighted fuzzy similarity matrix;

[0015] Adopt a dynamic similarity threshold mechanism, set the thresholds corresponding to different data types according to the sum of the similarity mean and twice the standard deviation of different data types, and calculate the global threshold weighted by the weight vectors of different data types. Based on the weighted fuzzy similarity matrix, filter out redundant data point pairs through the dynamic threshold to construct the final Boolean similarity matrix;

[0016] Extract the node groups with similarity greater than the global threshold from the Boolean similarity matrix, select the node with the highest similarity as the representative node, and mark other nodes as redundant nodes;

[0017] Record the mapping relationship between the redundant nodes and the representative nodes to form a mapping table. For each redundant node, calculate the difference vector from the representative node, and perform storage compression by only storing the representative node data and the difference vector of the redundant node, and construct a sleep queue according to the index of the redundant node and its difference vector information.

[0018] As a preferred solution of the data optimization storage method for smart agriculture according to the present invention, wherein: constructing a linear logistic regression model to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence, including,

[0019] Read the representative node data and difference vector data after storage compression, extract the multi-source heterogeneous data of each node, and splice them to form a feature matrix, where the feature matrix contains f data type dimensions;

[0020] Calculate the information entropy of the features by the entropy weight method, update and calculate the feature weights based on the information entropy, and perform pre-weighting calculation on the feature matrix according to the feature weights;

[0021] Construct a linear logistic regression model to predict the agricultural yield results of different nodes, and convert the linear prediction results into prediction probabilities through the Sigmoid function;

[0022] Use the training set to train the linear logistic regression model, calculate the error between the prediction probability and the true label of the training set through the cross-entropy loss function, and perform iterative optimization through batch gradient descent until the calculation error no longer decreases significantly and then stop the iteration;

[0023] Use the trained linear logistic regression model for probability prediction;

[0024] Based on the probability of each data reaching the target, sort it as the storage priority value to obtain a sorting sequence, and calculate the fitness value of each value in the sequence;

[0025] Use the differential evolution algorithm DE for mutation operation, randomly select different individuals for mutation to generate new mutant individuals;

[0026] Perform crossover operation on the new mutant individuals and the original parent individuals to generate new candidate individuals, and compare the fitness of the new candidate individuals and the parent individuals. If the fitness of the new candidate individuals is greater than that of the parent individuals, perform greedy selection;

[0027] Perform tournament selection and population update based on the genetic algorithm GA, and calculate the best fitness value after each round of population update until the difference in the calculated fitness value no longer decreases significantly and then stop the update;

[0028] Select the chromosome individual with the highest fitness value in the final population, and generate an optimized sorting sequence as the final priority sorting result for data storage.

[0029] As a preferred solution of the data optimization storage method for smart agriculture according to the present invention, wherein: generating a hash digest of the sorted sequence data for secure storage, including,

[0030] Use a smart contract to write data according to the final priority sorting result, and generate a data hash digest for the written data using the SHA-256 algorithm;

[0031] After writing the data to IPFS, return a clear storage path address to the smart contract to form a binding record between the chain and IPFS.

[0032] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the data consistency verification includes,

[0033] Only store the data hash digest and the IPFS path index through the blockchain, perform data consistency verification on the hash digest of the data stored in IPFS, and the data access request accesses through the REST API gateway to perform user access permission verification.

[0034] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the collection of agricultural field data, remote sensing data, and management data includes,

[0035] Collect multi-source heterogeneous data through field sensors, remote sensing data, and agricultural management data, perform data preprocessing, construct a multi-source heterogeneous data matrix, and use the number of data collection nodes and the collection time period of different data types as matrix dimensions;

[0036] Based on the multi-source heterogeneous data matrix, construct a star-shaped data warehouse structure.

[0037] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the generation of log records for cloud storage includes,

[0038] Automatically generate log records during each data write, access, modification, or deletion operation, perform secure storage on the generated log records using the SHA-256 algorithm, and perform cloud storage through wireless transmission technology.

[0039] In a second aspect, the present invention provides a system for a data optimization storage method for smart agriculture, including,

[0040] A data collection and processing module, responsible for collecting field sensor data, uniformly standardizing and normalizing the collected data, and generating a multi-source heterogeneous data matrix;

[0041] A data redundancy identification module, calculates the fuzzy similarity according to the multi-source heterogeneous data matrix, generates a time series distance matrix, calculates the weight vector of different data types, constructs a weighted distance matrix and a fuzzy similarity matrix, and clearly distinguishes representative nodes and redundant nodes according to the Boolean similarity matrix;

[0042] The data sorting and optimization module reads the representative node data and difference vectors to form a feature matrix, determines the feature weights through the entropy weight method, uses a linear logistic regression model to predict agricultural yields, calculates the prediction probabilities of each node, and optimizes the priority sorting sequence of data storage using differential evolution and genetic algorithms;

[0043] The data security storage module generates a data hash digest according to the sorting sequence for secure storage;

[0044] The log storage module automatically generates operation logs and uploads them to cloud storage.

[0045] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the data optimization storage method for smart agriculture described in the first aspect of the present invention is implemented.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the data optimization storage method for smart agriculture described in the first aspect of the present invention is implemented.

[0047] The beneficial effects of the present invention are as follows: By using a weighted fuzzy similarity matrix to accurately identify redundant data and only storing difference vectors, the storage space occupancy is significantly reduced, effectively reducing the consumption of storage resources at the cloud or chain end. Through storage compression and differential storage, the actual amount of data transmitted is significantly reduced, reducing the data transmission burden of edge devices or cloud nodes. By the entropy weight method, the actual contribution degree of various data types in yield prediction is clarified, making the prediction results closer to the true characteristics of agricultural production, thereby directly improving the accuracy of yield prediction and the quality of storage priority decision-making. Based on the prediction probability, the data storage priority is determined, automatically realizing the priority storage of high-value and high-information data. By combining the priority sorting mechanism optimized by differential evolution, high-priority data can be quickly identified and sorted, reducing decision-making lag and enhancing the real-time response ability of overall data scheduling and storage management. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a schematic flowchart of the data optimization storage method for smart agriculture in Embodiment 1.

[0050] Figure 2 It is a schematic structural diagram of the data optimization storage system for smart agriculture in Embodiment 1. Detailed implementation manners

[0051] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings of the specification.

[0052] In the following description, many specific details are set forth to facilitate a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0053] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or alternative embodiments that exclude each other from other embodiments.

[0054] Embodiment 1, referring to From Figure 1 to Figure 2 , which is the first embodiment of the present invention. This embodiment provides a data optimization storage method for smart agriculture, including the following steps:

[0055] S1. Collect agricultural field data, remote sensing data, and management data, and perform preprocessing to construct a multi-source heterogeneous data matrix;

[0056] Preferably, collecting agricultural field data, remote sensing data, and management data includes,

[0057] Collect multi-source heterogeneous data through field sensors, including temperature, humidity, soil pH, soil nutrients [NPK], pest and disease conditions, remote sensing data, including plot area monitoring, vegetation index, soil moisture change, and agricultural management data, including fertilization amount, sowing date, and field management actions, and perform data preprocessing to construct a multi-source heterogeneous data matrix, using the number of data collection nodes and collection time period of different data types as matrix dimensions;

[0058] Based on the multi-source heterogeneous data matrix, construct a star-shaped data warehouse structure.

[0059] By constructing a multi-source heterogeneous data matrix, data from different sources are represented in a unified structure, simplifying the complexity of data analysis and retrieval, improving the efficiency of subsequent data analysis and the construction of prediction models. With a star data warehouse structure centered around crops or plots, different types of agricultural data are clearly associated, greatly enhancing the ability to integrate and analyze cross-source data, facilitating the rapid discovery of the internal relationships and patterns among data. The standardized star data warehouse defines the standard structure for data storage and management, reducing redundant data storage and management overhead, and effectively lowering the overall cost of data maintenance.

[0060] S2. Calculate the fuzzy similarity of the multi-source heterogeneous data matrix, construct a time series distance matrix, calculate the weight vectors of different data types, construct a weighted distance matrix based on the weights assigned to different data types, calculate the weighted fuzzy similarity matrix, construct a Boolean similarity matrix, distinguish redundant nodes and representative nodes, and determine the difference vector;

[0061] Preferably, calculating the weighted fuzzy similarity matrix and constructing the Boolean similarity matrix includes

[0062] Calculating the fuzzy similarity of the multi-source heterogeneous data matrix, where the sequence distance between different nodes is calculated as:

[0063] ;

[0064] where represents the time series distance between nodes a and b, t represents the time window, and respectively represent the data values of nodes a and b at the i-th time point;

[0065] And calculate the time series distance matrix for each data type k, expressed as:

[0066] ;

[0067] where n represents the number of data nodes, represents the time series distance matrix of the k-th data type, represents the time series distance between nodes a and b of the k-th data type;

[0068] Use the feature variance normalization method to calculate the weight vectors of different data types, expressed as:

[0069] ;

[0070] where represents the assigned weight of the k-th data type, construct a weight vector from the assigned weights of different data types, represents the standard deviation of the k-th data type, Denotes the standard deviation of the j-th data type except the k-th one, Denotes the number of data types;

[0071] Calculate the allocation weight through the variance of the data type, which is used for the weighted distance calculation of the fuzzy similarity matrix. The larger the variance, the greater the volatility of the data type in the data distribution, and the greater the contribution to the fuzzy similarity matrix. By calculating In the redundancy detection and data compression stage, improve the accuracy of data similarity detection, ensure that the fuzzy similarity matrix can more accurately reflect the changes of data types, and improve the quality of the compressed data retention;

[0072] And construct a weighted distance matrix according to the allocation weights of different data types, and calculate the weighted fuzzy similarity matrix, which is expressed as:

[0073] ;

[0074] ;

[0075] ;

[0076] Where Denotes the final comprehensive distance after weighting the distances of different data types, Denotes the weighted fuzzy similarity between nodes a and b, and S denotes the weighted fuzzy similarity matrix;

[0077] Adopt a dynamic similarity threshold mechanism, set the thresholds corresponding to different data types according to the sum of the similarity mean and twice the standard deviation of different data types, and calculate the global threshold by weighting according to the weight vectors of different data types. Based on the weighted fuzzy similarity matrix, filter out redundant data point pairs through the dynamic threshold, and construct the final Boolean similarity matrix, which is expressed as:

[0078] ;

[0079] Where Denotes an element in the Boolean similarity matrix, which is used to judge whether the weighted fuzzy similarity between nodes ab Exceeds the global threshold;

[0080] Extract the node groups with similarity greater than the global threshold from the Boolean similarity matrix, select the node with the highest similarity as the representative node, and mark other nodes as redundant nodes;

[0081] Record the mapping relationship between the redundant nodes and the representative nodes to form a mapping table. For each redundant node, calculate the difference vector from the representative node, which is expressed as:

[0082] ;

[0083] wherein represents the difference vector of redundant nodes, represents the redundant nodes of the original data, represents the original data of the representative node ;

[0084] And perform storage compression that only stores the data of the representative node and the difference vector of the redundant node, and construct a sleep queue according to the index of the redundant node and its difference vector information.

[0085] By using the weighted fuzzy similarity matrix to accurately identify redundant data, not only storing the difference vector, significantly reducing the storage space occupancy, effectively reducing the consumption of storage resources at the cloud or chain end, based on the data type weights determined by feature variance normalization, more reasonably reflecting the actual contribution of different data types to the fuzzy similarity, avoiding misjudgment or missed judgment phenomena caused by too large differences in data types, making the redundant detection results more accurate and stable, the dynamic similarity threshold mechanism automatically adapts to data fluctuations, avoiding the blind area of a single fixed threshold, and being able to automatically adapt to the fluctuation conditions and trend changes of different data types. The way of storing the difference vector clearly records the details of data changes between redundant nodes and representative nodes, which is conducive to quickly and accurately restoring the data of redundant nodes, avoiding the time and space costs brought by the complete backup of the original data, and significantly improving the data backtracking efficiency;

[0086] Through storage compression and differential storage, the actual amount of data transmitted is significantly reduced, reducing the data transmission burden of edge devices or cloud nodes, and thus indirectly reducing the network bandwidth cost and system energy consumption. After the data volume is compressed, the system's search and analysis operation speed for high-dimensional data is significantly accelerated, enhancing the operation efficiency of subsequent intelligent analysis or prediction models, ensuring the real-time nature of subsequent prediction or decision-making tasks. The reduction of storage space directly reduces the storage overhead of distributed systems such as blockchains or IPFS, thereby reducing the total cost of ownership. The data compression and differential storage mechanism indirectly increases the update frequency of on-chain records, improving the real-time nature and reliability of on-chain data verification and traceability.

[0087] S3. Form a feature matrix according to the representative node data and the difference vector data, calculate the feature weights by the entropy weight method, construct a linear logistic regression model for predicting the agricultural yield results of different nodes, obtain a sorting sequence, and calculate the fitness value of each value in the sequence. Use the differential evolution algorithm for mutation and crossover, and update the population based on the genetic algorithm to optimize the sorting sequence;

[0088] Preferably, a linear logistic regression model is constructed to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence, including,

[0089] Read the stored and compressed representative node data and difference vector data, extract the multi-source heterogeneous data of each node, and splice them to form a feature matrix, where the feature matrix contains f data type dimensions;

[0090] Calculate the information entropy of the features by the entropy weight method, quantify the sensitivity of different features to data changes, and use it as the basis for the initial assignment of feature weights, expressed as:

[0091] ;

[0092] where represents the information entropy of data type j, represents the normalized probability value of data type j in sample i;

[0093] Update and calculate the feature weights based on the information entropy, expressed as:

[0094] ;

[0095] where represents the type weight of the jth feature weight;

[0096] Calculate the data type weights by the information entropy for feature weighting in the training of the logistic regression model. The information entropy weight reflects the amount of information of the data type, so a higher weight is given in the prediction of data storage priority. Calculate the data type weights by the information entropy and perform feature weighting according to the amount of information of the data type during the training stage of the logistic regression model to ensure that in the logistic regression model, data types with greater information content account for a larger proportion in the prediction of data storage priority;

[0097] And perform pre-weighting calculation on the feature matrix according to the feature weights;

[0098] Construct a linear logistic regression model to predict the agricultural yield results of different nodes, expressed as:

[0099] ;

[0100] where represents the linear combination prediction value of the ith data, used to predict the crop price or yield, represents the bias term, represents the model weight of the jth data type, It represents the i-th data type value weighted by information entropy weight for the j-th data type;

[0101] Convert the linear prediction result to a prediction probability through the Sigmoid function, which is expressed as:

[0102] ;

[0103] where represents the probability that the i-th data agricultural output result reaches the target;

[0104] Use the training set to train the linear logistic regression model, calculate the error between the prediction probability and the true label of the training set through the cross-entropy loss function, and perform iterative optimization through batch gradient descent until the calculation error no longer decreases significantly, then stop the iteration;

[0105] Through the gradient descent method, adaptively update the type during the training process of the logistic regression model. As the training data changes continuously, adjust the model weights to minimize the prediction error. By continuously adjusting the model weights, adapt to different data distribution situations and improve the generalization ability of the model;

[0106] Use the trained linear logistic regression model for probability prediction;

[0107] When constructing the data storage priority prediction model (logistic regression), more reasonably reflect the contribution degree of each data type to the prediction target. After weighting, the data types with large amounts of information have a greater impact on the logistic regression model, which is more in line with the actual needs, and clearly improve the accuracy and rationality of the logistic regression model for predicting data storage priorities;

[0108] Based on the probability of each data reaching the target, sort it as the storage priority value to obtain a sorted sequence, and calculate the fitness value of each value in the sequence, which is expressed as:

[0109] ;

[0110] where represents the fitness value of the value of the j-th sorted sequence C, v represents the total number of sequence values, represents the i-th node data storage value, in kb;

[0111] Use the differential evolution algorithm DE to perform mutation operations, randomly select different individuals for mutation to generate new mutant individuals, which is expressed as:

[0112] ;

[0113] where represents the new individual generated by mutating the i-th individual in the (s + 1)-th generation, 、 and respectively represent the first, second, and third individual values randomly selected from the s-th generation of the parent generation, and G represents the differential scaling factor;

[0114] Perform a crossover operation on the newly mutated individual and the original parent individual to generate a new candidate individual, and compare the fitness of the new candidate individual and the parent individual. If the fitness of the new candidate individual is greater than that of the parent individual, perform greedy selection;

[0115] Perform tournament selection and population update based on the genetic algorithm GA, and calculate the best fitness value after each round of population update until the difference in the calculated fitness value no longer decreases significantly, then stop the update;

[0116] Select the chromosome individual with the highest fitness value in the final population and generate an optimized sorting sequence as the final priority sorting result for data storage.

[0117] The actual contribution degree of various data types in yield prediction is clarified through the entropy weight method, making the prediction result closer to the actual agricultural production characteristics, thereby directly improving the accuracy of yield prediction, improving the quality of storage priority decision-making, determining the data storage priority based on the prediction probability, automatically realizing the priority storage of high-value and high-information data, effectively avoiding the occupation of precious storage resources by low-information redundant data, reducing the storage cost, and introducing the mutation and crossover mechanism of the differential evolution algorithm significantly improves the global search ability of the traditional genetic algorithm, reduces the risk of the algorithm falling into local optimum, and speeds up the convergence rate of sorting sequence optimization. Through the priority sorting mechanism optimized by combining differential evolution, high-priority data can be quickly identified and sorted, reducing decision-making lag and enhancing the real-time response ability of the overall data scheduling and storage management;

[0118] Due to the precise improvement of data storage priority, the quality and efficiency of data query and analysis are significantly improved, further improving the performance of the intelligent agricultural decision support system. The efficient priority sorting optimization and storage scheme significantly reduce data storage and computing costs, indirectly enhancing the economic and environmental benefits of the smart agriculture management platform, reducing the uncertainty caused by inconsistent data quality in the data mining and analysis process, and further improving the reliability of subsequent agricultural decision-making and intelligent operations.

[0119] S4. Generate a hash digest of the sorting sequence data for secure storage, perform data consistency verification, and generate a log record for cloud storage;

[0120] Preferably, generating a hash digest of the sorting sequence data for secure storage includes

[0121] Using a smart contract to write data according to the final priority sorting result, and generating a data hash digest for the written data using the SHA-256 algorithm;

[0122] After writing the data to IPFS, a clear storage path address is returned to the smart contract to form a binding record between the chain and IPFS.

[0123] Through the sorting sequence data, a hash digest is generated by the SHA-256 algorithm to achieve data anti-tampering verification, ensuring that the data has not been modified without authorization, guaranteeing the credibility of the data. By storing the IPFS data path address and the hash digest on the chain, it is convenient for users to quickly and accurately audit and trace the data, improving the transparency of data management. By actually storing the data in the IPFS off-chain system and only saving the hash digest and path on the chain, the load and cost of on-chain storage are significantly reduced, and the storage efficiency is improved.

[0124] Furthermore, data consistency verification is carried out, including,

[0125] Only the data hash digest and the IPFS path index are stored on the blockchain, without storing the original data content, reducing the on-chain storage burden. The hash digest of the data stored in IPFS is used for data consistency verification. The data access request accesses through the RESTAPI gateway, and user access permission verification is executed.

[0126] By comparing the hash digest on the blockchain with the hash of the actual data in IPFS, data consistency verification is quickly realized, effectively avoiding the risk of data tampering, ensuring the true and valid nature of the data. Only storing the hash digest and index path of the data reduces the data storage volume of the blockchain nodes, effectively reducing the blockchain maintenance cost and management burden.

[0127] Preferably, log records are generated for cloud storage, including,

[0128] During each data write, access, modification, or deletion operation, log records are automatically generated, and the generated log records are securely stored using the SHA-256 algorithm and stored in the cloud through wireless transmission technology.

[0129] By automatically generating logs and protecting them with the SHA-256 algorithm, it effectively prevents log tampering, ensuring that the entire process of data operation is transparent and traceable. Using the SHA-256 digest to securely store the logs ensures the integrity of the logs, prevents illegal tampering, and guarantees the credibility of the audit data. By uploading the log records to the cloud in real time through wireless transmission, the log collection and maintenance process are simplified, effectively improving the log management efficiency and real-time performance. The log records centrally stored and managed in the cloud are convenient for quick retrieval and analysis, significantly improving the efficiency of system fault diagnosis and security audit.

[0130] This embodiment also provides a system for the data optimization storage method for smart agriculture, including,

[0131] The data acquisition and processing module is responsible for collecting field sensor data, uniformly standardizing and normalizing the collected data, and generating a multi-source heterogeneous data matrix;

[0132] The data redundancy identification module calculates the fuzzy similarity based on the multi-source heterogeneous data matrix, generates a time series distance matrix, calculates the weight vectors of different data types, constructs a weighted distance matrix and a fuzzy similarity matrix, and clearly distinguishes representative nodes and redundant nodes according to the Boolean similarity matrix;

[0133] The data sorting and optimization module reads the representative node data and difference vectors to form a feature matrix, determines the feature weights by the entropy weight method, uses a linear logistic regression model to predict agricultural yields, calculates the prediction probabilities of each node, and optimizes the priority sorting sequence of data storage using differential evolution and genetic algorithms;

[0134] The data secure storage module generates a data hash digest for secure storage according to the sorting sequence;

[0135] The log storage module automatically generates operation logs and uploads them to cloud storage.

[0136] This embodiment also provides a computer device applicable to the data optimization storage method for smart agriculture, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the data optimization storage method for smart agriculture proposed in the above embodiment.

[0137] This computer device can be a terminal. This computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of this computer device is used to provide computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of this computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of this computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of this computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0138] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the data optimization storage method for smart agriculture proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disc.

[0139] In summary, the present invention accurately identifies redundant data by using a weighted fuzzy similarity matrix, stores only the difference vectors, significantly reduces the storage space occupancy, effectively reduces the consumption of storage resources at the cloud or chain end. Through storage compression and differential storage, the actual amount of data transmitted is significantly reduced, reducing the data transmission burden of edge devices or cloud nodes. By using the entropy weight method, the actual contribution degree of various data types in yield prediction is clarified, making the prediction result closer to the actual agricultural production characteristics, thereby directly improving the accuracy of yield prediction and the quality of storage priority decision-making. Based on the prediction probability, the data storage priority is determined, and the priority storage of high-value and high-information data is automatically realized. By combining the priority sorting mechanism optimized by differential evolution, high-priority data is quickly identified and sorted, reducing decision-making lag and improving the real-time response ability of overall data scheduling and storage management.

[0140] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A data optimization storage method for smart agriculture, characterized in that Including: Collecting agricultural field data, remote sensing data, and management data, and performing preprocessing to construct a multi-source heterogeneous data matrix; Calculating the fuzzy similarity of the multi-source heterogeneous data matrix, constructing a time series distance matrix to calculate the weight vectors of different data types, constructing a weighted distance matrix according to the weights assigned to different data types, calculating a weighted fuzzy similarity matrix, constructing a Boolean similarity matrix, distinguishing redundant nodes and representative nodes, and determining the difference vector; Forming a feature matrix based on the representative node data and the difference vector data, calculating the feature weights through the entropy weight method, constructing a linear logistic regression model to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence; Generating a hash digest of the sorting sequence data for secure storage, performing data consistency verification, and generating log records for cloud storage.

2. The data optimization storage method for smart agriculture according to claim 1, wherein: The calculating of the weighted fuzzy similarity matrix and the constructing of the Boolean similarity matrix include: Calculating the fuzzy similarity of the multi-source heterogeneous data matrix, where calculating the sequence distance between different nodes and calculating the time series distance matrix for each data type k; Using the feature variance normalization method to calculate the weight vectors of different data types, constructing a weighted distance matrix according to the weights assigned to different data types, and calculating the weighted fuzzy similarity matrix; Adopting a dynamic similarity threshold mechanism, setting the thresholds corresponding to different data types according to the sum of the similarity mean and twice the standard deviation of different data types, calculating the global threshold weighted by the weight vectors of different data types, and screening out redundant data point pairs based on the weighted fuzzy similarity matrix through the dynamic threshold to construct the final Boolean similarity matrix; Extracting the node groups with similarity greater than the global threshold from the Boolean similarity matrix, selecting the node with the highest similarity as the representative node, and marking other nodes as redundant nodes; Recording the mapping relationship between the redundant nodes and the representative nodes to form a mapping table, for each redundant node, calculating the difference vector with the representative node, and performing storage compression by only storing the representative node data and the difference vectors of the redundant nodes, and constructing a sleep queue according to the index of the redundant nodes and their difference vector information.

3. The data optimization storage method for smart agriculture according to claim 2, wherein: The constructing of the linear logistic regression model to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence includes: Reading the representative node data and the difference vector data after storage compression, extracting the multi-source heterogeneous data of each node, and splicing them to form a feature matrix, where the feature matrix contains f data type dimensions; Calculating the information entropy of the features through the entropy weight method, updating and calculating the feature weights based on the information entropy, and performing pre-weighting calculation on the feature matrix according to the feature weights; Constructing a linear logistic regression model to predict the agricultural yield results of different nodes, and converting the linear prediction result into a prediction probability through the Sigmoid function; Train a linear logistic regression model using the training set, calculate the error between the predicted probability and the true labels of the training set through the cross-entropy loss function, and perform iterative optimization through batch gradient descent until the calculated error no longer decreases significantly, then stop the iteration; Use the trained linear logistic regression model to make probability predictions; Based on the probability of each data reaching the target, sort it as the storage priority value to obtain a sorted sequence, and calculate the fitness value of each value in the sequence; Use the differential evolution algorithm DE to perform mutation operations, randomly select different individuals for mutation to generate new mutant individuals; Perform crossover operations on the new mutant individuals and the original parent individuals to generate new candidate individuals, and compare the fitness of the new candidate individuals and the parent individuals. If the fitness of the new candidate individuals is greater than that of the parent individuals, perform greedy selection; Perform tournament selection and population update based on the genetic algorithm GA, and calculate the best fitness value after each round of population update until the difference in the calculated fitness value no longer decreases significantly, then stop the update; Select the chromosome individual with the highest fitness value in the final population and generate an optimized sorted sequence as the final priority sorting result for data storage.

4. The data optimization storage method for smart agriculture according to claim 3, characterized in that: Securely store the hash digest of the generated sorted sequence data, including, Use a smart contract to write data according to the final priority sorting result, and generate a data hash digest for the written data using the SHA-256 algorithm; After writing the data to IPFS, return a clear storage path address to the smart contract to form a binding record between the chain and IPFS.

5. The data optimization storage method for smart agriculture according to claim 4, characterized in that: The data consistency verification includes, Only store the data hash digest and the IPFS path index through the blockchain, verify the data consistency of the hash digest of the data stored in IPFS, and the data access request is accessed through the REST API gateway to perform user access permission verification.

6. The data optimization storage method for smart agriculture according to claim 5, wherein: The collection of agricultural field data, remote sensing data, and management data includes, Collect multi-source heterogeneous data through field sensors, remote sensing data, and agricultural management data, and perform data preprocessing to construct a multi-source heterogeneous data matrix. Use the number of data collection nodes and the collection time period of different data types as the matrix dimensions; Based on the multi-source heterogeneous data matrix, construct a star-shaped data warehouse structure.

7. The data optimization storage method for smart agriculture according to claim 6, characterized in that: The generation of log records for cloud storage includes, Automatically generate log records during each data write, access, modification, or deletion operation, securely store the generated log records using the SHA-256 algorithm, and perform cloud storage through wireless transmission technology.

8. A system for a data optimization storage method in smart agriculture, based on the data optimization storage method for smart agriculture according to any one of claims 1 to 7, characterized in that: including, The data collection and processing module is responsible for collecting field sensor data, uniformly standardizing and normalizing the collected data, and generating a multi-source heterogeneous data matrix; The data redundancy identification module calculates the fuzzy similarity based on the multi-source heterogeneous data matrix, generates a time series distance matrix, calculates the weight vectors of different data types, constructs a weighted distance matrix and a fuzzy similarity matrix, and clearly distinguishes representative nodes and redundant nodes according to the Boolean similarity matrix; The data sorting optimization module reads the representative node data and the difference vector to form a feature matrix, determines the feature weights through the entropy weight method, uses a linear logistic regression model to predict agricultural yields, calculates the prediction probabilities of each node, and optimizes the priority sorting sequence of data storage using the differential evolution and genetic algorithms; The data secure storage module generates a data hash digest according to the sorting sequence for secure storage; The log storage module automatically generates operation logs and uploads them to cloud storage.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the data optimization storage method for smart agriculture according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the data optimization storage method for smart agriculture according to any one of claims 1 to 7.

Images