A data optimization storage method and system for smart agriculture

Abstract

The present invention discloses a data optimization storage method and system for smart agriculture, which relates to the technical field of smart agriculture. The method includes collecting agricultural field data, remote sensing data, and management data, and performing preprocessing to construct a multi-source heterogeneous data matrix; calculating the fuzzy similarity of the multi-source heterogeneous data matrix, constructing a time series distance matrix, calculating the weight vectors of different data types, constructing a weighted distance matrix according to the weights assigned to different data types, calculating a weighted fuzzy similarity matrix, constructing a Boolean similarity matrix, distinguishing redundant nodes and representative nodes, and determining the difference vector. The method of the present invention accurately identifies redundant data by using the weighted fuzzy similarity matrix, only stores the difference vector, greatly reduces the storage space occupancy, effectively reduces the consumption of storage resources at the cloud end or the chain end, and significantly reduces the amount of data actually transmitted through storage compression and differential storage, reducing the data transmission burden of edge devices or cloud nodes.

Description

Technical Field

[0001] The present invention relates to the technical field of smart agriculture, and in particular to a data optimization storage method and system for smart agriculture. Background Art

[0002] Currently, smart agriculture widely relies on field sensors, satellite remote sensing technology, and agricultural management information systems to obtain a large amount of multi-source heterogeneous data, and realizes real-time perception and precise decision-making of the agricultural production status. These data usually exhibit characteristics such as high-dimensionality, multi-source, and strong temporality, posing extremely high requirements for the capacity, efficiency, and security of data storage;

[0003] Traditional smart agriculture data storage methods mainly adopt the methods of complete backup of original data or simple redundancy removal. The data redundancy is large, and the storage resource consumption is relatively high; moreover, most of the existing technologies do not fully consider the importance differences of different data types, and often use a unified fixed weight for processing, which is difficult to adapt to the dynamic characteristics of data, affects the accuracy of data redundancy detection, and further reduces the data compression and storage efficiency. In addition, the existing agricultural data storage methods usually lack an effective data priority prediction and intelligent optimization scheduling mechanism, and cannot accurately distinguish and differentiate the value and timeliness of the stored data, resulting in unreasonable allocation of storage resources, difficult timely preservation of data with high value density, and even the risk of loss of high-value data. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a data optimization storage method for smart agriculture to solve the problems that the existing agricultural data storage methods usually lack an effective data priority prediction and intelligent optimization scheduling mechanism, cannot accurately distinguish and differentiate the value and timeliness of the stored data, resulting in unreasonable allocation of storage resources, difficult timely preservation of data with high value density, and even the risk of loss of high-value data.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a data optimization storage method for smart agriculture, which includes:

[0008] Collect agricultural field data, remote sensing data, and management data, and perform preprocessing to construct a multi-source heterogeneous data matrix;

[0009] Calculate the fuzzy similarity of the multi-source heterogeneous data matrix, construct the time series distance matrix to calculate the weight vectors of different data types, construct the weighted distance matrix according to the weights assigned to different data types, calculate the weighted fuzzy similarity matrix, construct the Boolean similarity matrix, distinguish redundant nodes and representative nodes, and determine the difference vector;

[0010] Form the feature matrix based on the representative node data and the difference vector data, calculate the feature weights by the entropy weight method, construct a linear logistic regression model to predict the agricultural yield results of different nodes, obtain the sorting sequence, and calculate the fitness value of each value in the sequence. Use the differential evolution algorithm for mutation and crossover, and update the population based on the genetic algorithm to optimize the sorting sequence;

[0011] Generate the hash digest of the sorting sequence data for secure storage, perform data consistency verification, and generate log records for cloud storage.

[0012] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the calculation of the weighted fuzzy similarity matrix and the construction of the Boolean similarity matrix include,

[0013] Calculate the fuzzy similarity of the multi-source heterogeneous data matrix, where calculate the sequence distance between different nodes, and calculate the time series distance matrix for each data type k;

[0014] Adopt the feature variance normalization method to calculate the weight vectors of different data types, construct the weighted distance matrix according to the weights assigned to different data types, and calculate the weighted fuzzy similarity matrix;

[0015] Adopt the dynamic similarity threshold mechanism, set the thresholds corresponding to different data types according to the sum of the similarity mean and twice the standard deviation of different data types, and calculate the global threshold weighted by the weight vectors of different data types. Based on the weighted fuzzy similarity matrix, screen out redundant data point pairs through the dynamic threshold to construct the final Boolean similarity matrix;

[0016] Extract the node groups with similarity greater than the global threshold from the Boolean similarity matrix, select the node with the highest similarity as the representative node, and mark other nodes as redundant nodes;

[0017] Record the mapping relationship between the redundant nodes and the representative nodes to form a mapping table. For each redundant node, calculate the difference vector from the representative node, and perform storage compression by only storing the representative node data and the difference vector of the redundant node, and construct a dormant queue according to the index of the redundant node and its difference vector information.

[0018] As a preferred solution of the data optimization storage method for smart agriculture according to the present invention, wherein: constructing a linear logistic regression model to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence, including,

[0019] Read the representative node data and difference vector data after storage compression, extract the multi-source heterogeneous data of each node, and splice them to form a feature matrix, where the feature matrix contains f data type dimensions;

[0020] Calculate the information entropy of the features by the entropy weight method, update and calculate the feature weights based on the information entropy, and perform pre-weighting calculation on the feature matrix according to the feature weights;

[0021] Construct a linear logistic regression model to predict the agricultural yield results of different nodes, and convert the linear prediction results into prediction probabilities through the Sigmoid function;

[0022] Use the training set to train the linear logistic regression model, calculate the error between the prediction probability and the true label of the training set through the cross-entropy loss function, and perform iterative optimization through batch gradient descent until the calculation error no longer decreases significantly and then stop the iteration;

[0023] Use the trained linear logistic regression model to perform probability prediction;

[0024] Based on the probability of each data reaching the target, sort as the storage priority value to obtain a sorting sequence, and calculate the fitness value of each value in the sequence;

[0025] Use the differential evolution algorithm DE to perform mutation operations, randomly select different individuals for mutation to generate new mutant individuals;

[0026] Perform crossover operations on the new mutant individuals and the original parent individuals to generate new candidate individuals, and compare the fitness of the new candidate individuals and the parent individuals. If the fitness of the new candidate individuals is greater than that of the parent individuals, perform greedy selection;

[0027] Perform tournament selection and population update based on the genetic algorithm GA, and calculate the best fitness value after each round of population update until the difference in the calculated fitness value no longer decreases significantly and then stop the update;

[0028] Select the chromosome individual with the highest fitness value in the final population, and generate an optimized sorting sequence as the final priority sorting result for data storage.

[0029] As a preferred solution of the data optimization storage method for smart agriculture according to the present invention, wherein: generating a hash digest of the sorting sequence data for secure storage, including,

[0030] Use a smart contract to write data according to the final priority sorting result, and generate a data hash digest for the written data using the SHA-256 algorithm;

[0031] After writing the data to IPFS, return a clear storage path address to the smart contract to form a binding record between the chain and IPFS.

[0032] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the data consistency verification includes,

[0033] Only store the data hash digest and the IPFS path index through the blockchain, verify the data consistency of the hash digest of the data stored in IPFS, and the data access request accesses through the REST API gateway to perform user access permission verification.

[0034] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the collection of agricultural field data, remote sensing data, and management data includes,

[0035] Collect multi-source heterogeneous data through field sensors, remote sensing data, and agricultural management data, and perform data preprocessing to construct a multi-source heterogeneous data matrix, using the number of data acquisition nodes and the acquisition time period of different data types as matrix dimensions;

[0036] Based on the multi-source heterogeneous data matrix, construct a star-shaped data warehouse structure.

[0037] As a preferred solution of the data optimization storage method for smart agriculture described in the present invention, wherein: the generation of log records for cloud storage includes,

[0038] Automatically generate log records during each data write, access, modification, or deletion operation, perform secure storage on the generated log records using the SHA-256 algorithm, and perform cloud storage through wireless transmission technology.

[0039] In a second aspect, the present invention provides a system for a data optimization storage method for smart agriculture, including,

[0040] A data acquisition and processing module, responsible for collecting field sensor data, uniformly standardizing and normalizing the collected data to generate a multi-source heterogeneous data matrix;

[0041] A data redundancy identification module, which calculates the fuzzy similarity according to the multi-source heterogeneous data matrix, generates a time series distance matrix, calculates the weight vectors of different data types, constructs a weighted distance matrix and a fuzzy similarity matrix, and clearly distinguishes representative nodes and redundant nodes according to the Boolean similarity matrix;

[0042] The data sorting optimization module reads the representative node data and difference vectors to form a feature matrix, determines the feature weights through the entropy weight method, uses a linear logistic regression model to predict agricultural yields, calculates the prediction probabilities of each node, and optimizes the priority sorting sequence of data storage using differential evolution and genetic algorithms;

[0043] The data security storage module generates a data hash digest according to the sorting sequence for secure storage;

[0044] The log storage module automatically generates operation logs and uploads them to cloud storage.

[0045] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the data optimization storage method for smart agriculture described in the first aspect of the present invention is implemented.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the data optimization storage method for smart agriculture described in the first aspect of the present invention is implemented.

[0047] The beneficial effects of the present invention are as follows: By using a weighted fuzzy similarity matrix to accurately identify redundant data and only storing difference vectors, the storage space occupancy is significantly reduced, effectively reducing the consumption of storage resources at the cloud or chain end. Through storage compression and differential storage, the actual amount of data transmitted is significantly reduced, reducing the data transmission burden of edge devices or cloud nodes. By using the entropy weight method, the actual contribution degree of various data types in yield prediction is clarified, making the prediction results closer to the actual agricultural production characteristics, thereby directly improving the accuracy of yield prediction and the quality of storage priority decision-making. Based on the prediction probability, the data storage priority is determined, automatically realizing the priority storage of high-value and high-information data. By combining the priority sorting mechanism optimized by differential evolution, high-priority data can be quickly identified and sorted, reducing decision-making lag and improving the real-time response ability of the overall data scheduling and storage management. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a flowchart of the data optimization storage method for smart agriculture in Embodiment 1.

[0050] Figure 2 It is a schematic structural diagram of the data optimization storage system for smart agriculture in Embodiment 1. Detailed implementation manners

[0051] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific implementation manners of the present invention will be given in conjunction with the accompanying drawings of the specification.

[0052] In the following description, many specific details are set forth to facilitate a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0053] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that excludes other embodiments.

[0054] Embodiment 1, referring to From Figure 1 to Figure 2 , which is the first embodiment of the present invention. This embodiment provides a data optimization storage method for smart agriculture, including the following steps:

[0055] S1. Collect agricultural field data, remote sensing data, and management data, and perform preprocessing to construct a multi-source heterogeneous data matrix;

[0056] Preferably, collecting agricultural field data, remote sensing data, and management data includes,

[0057] Collect multi-source heterogeneous data through field sensors, including temperature, humidity, soil pH, soil nutrients [NPK], pest and disease conditions, remote sensing data, including plot area monitoring, vegetation index, soil moisture change, and agricultural management data, including fertilizer application amount, sowing date, and field management actions, and perform data preprocessing to construct a multi-source heterogeneous data matrix, using the number of data collection nodes and collection time period of different data types as matrix dimensions;

[0058] Based on the multi-source heterogeneous data matrix, construct a star-shaped data warehouse structure.

[0059] By constructing a multi-source heterogeneous data matrix, data from different sources are represented in a unified structure, simplifying the complexity of data analysis and retrieval, improving the construction efficiency of subsequent data analysis and prediction models. With a star-shaped data warehouse structure centered around crops or plots, different types of agricultural data are clearly associated, greatly enhancing the cross-source data integration and analysis ability, facilitating the rapid mining of the internal relationships and rules between data. The standardized star-shaped data warehouse defines the standard structure for data storage and management, reducing redundant data storage and management overhead, and effectively lowering the overall cost of data maintenance.

[0060] S2. Calculate the fuzzy similarity of the multi-source heterogeneous data matrix, construct a time series distance matrix to calculate the weight vectors of different data types, construct a weighted distance matrix according to the assigned weights of different data categories, calculate the weighted fuzzy similarity matrix, construct a Boolean similarity matrix, distinguish redundant nodes and representative nodes, and determine the difference vector;

[0061] Preferably, calculating the weighted fuzzy similarity matrix and constructing the Boolean similarity matrix includes

[0062] Calculating the fuzzy similarity of the multi-source heterogeneous data matrix, where calculating the sequence distance between different nodes is expressed as:

[0063] ;

[0064] where represents the time series distance between nodes a and b, t represents the time window, and respectively represent the data values of nodes a and b at the i-th time point;

[0065] And calculate the time series distance matrix for each data type k, expressed as:

[0066] [[ID=D28]] ;

[0067] where n represents the number of data nodes, represents the time series distance matrix of the k-th data type, represents the time series distance between nodes a and b of the k-th data type;

[0068] Adopt the feature variance normalization method to calculate the weight vectors of different data types, expressed as:

[0069] ;

[0070] where represents the assigned weight of the k-th data type, construct a weight vector from the assigned weights of different data types, [[ID=4D9]] represents the standard deviation of the k-th data type, represents the standard deviation of the j-th data type except the k-th type, represents the number of data types;

[0071] Calculate the allocation weight through the variance of the data type, which is used for the weighted distance calculation of the fuzzy similarity matrix. The larger the variance, the greater the volatility of the data type in the data distribution and the greater the contribution to the fuzzy similarity matrix. By calculating In the redundancy detection and data compression stage, improve the accuracy of data similarity detection, ensure that the fuzzy similarity matrix can more accurately reflect the changes of data types, and improve the quality of the retained compressed data;

[0072] And construct a weighted distance matrix according to the allocation weights of different data types, and calculate the weighted fuzzy similarity matrix, which is expressed as:

[0073] ;

[0074] ;

[0075] ;

[0076] where represents the final comprehensive distance after weighting the distances of different data types, represents the weighted fuzzy similarity between nodes a and b, and S represents the weighted fuzzy similarity matrix;

[0077] Adopt a dynamic similarity threshold mechanism, set the thresholds corresponding to different data types according to the sum of the similarity mean and twice the standard deviation of different data types, and calculate the global threshold by weighting according to the weight vectors of different data types. Based on the weighted fuzzy similarity matrix, filter out redundant data point pairs through the dynamic threshold, and construct the final Boolean similarity matrix, which is expressed as:

[0078] ;

[0079] where represents an element in the Boolean similarity matrix, which is used to judge whether the weighted fuzzy similarity between nodes ab exceeds the global threshold;

[0080] Extract the node groups with similarity greater than the global threshold from the Boolean similarity matrix, select the node with the highest similarity as the representative node, and mark other nodes as redundant nodes;

[0081] Record the mapping relationship between the redundant nodes and the representative nodes to form a mapping table. For each redundant node, calculate the difference vector from the representative node, which is expressed as:

[0082] ;

[0083] wherein represents the difference vector of redundant nodes, represents the redundant nodes of the original data, represents the original data of the representative node ;

[0084] And perform storage compression that only stores the data of the representative node and the difference vector of the redundant node, and construct a sleep queue according to the index of the redundant node and its difference vector information.

[0085] By using a weighted fuzzy similarity matrix to accurately identify redundant data, not only store the difference vector, significantly reduce the storage space occupancy, effectively reduce the consumption of storage resources at the cloud or chain end, and based on the data type weights determined by feature variance normalization, more reasonably reflect the actual contribution of different data types to the fuzzy similarity, avoid misjudgment or missed judgment caused by too large differences in data types, make the redundant detection results more accurate and stable, the dynamic similarity threshold mechanism automatically adapts to data fluctuations, avoids the blind area of a single fixed threshold, and can automatically adapt to the fluctuation conditions and trend changes of different data types. The method of storing the difference vector clearly records the details of data changes between redundant nodes and representative nodes, which is beneficial to quickly and accurately restore the data of redundant nodes, avoid the time and space costs brought by the complete backup of the original data, and significantly improve the data traceability efficiency;

[0086] Through storage compression and differential storage, the actual amount of data transmitted is significantly reduced, reducing the data transmission burden of edge devices or cloud nodes, and indirectly reducing the network bandwidth cost and system energy consumption. After the data volume is compressed, the system's search and analysis operation speed for high-dimensional data is significantly accelerated, enhancing the operation efficiency of subsequent intelligent analysis or prediction models, ensuring the real-time nature of subsequent prediction or decision-making tasks. The reduction of storage space directly reduces the storage overhead of distributed systems such as blockchains or IPFS, thereby reducing the total cost of ownership. The data compression and differential storage mechanism indirectly increases the update frequency of on-chain records, improving the real-time nature and reliability of on-chain data verification and traceability.

[0087] S3. According to the representative node data and the difference vector data, form a feature matrix, calculate the feature weights by the entropy weight method, construct a linear logistic regression model for predicting the agricultural yield results of different nodes, obtain a sorting sequence, and calculate the fitness value of each value in the sequence. Use the differential evolution algorithm for mutation and crossover, and update the population based on the genetic algorithm to optimize the sorting sequence;

[0088] Preferably, a linear logistic regression model is constructed to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence, including,

[0089] Read the stored and compressed representative node data and difference vector data, extract the multi-source heterogeneous data of each node, and splice them to form a feature matrix, where the feature matrix contains f data type dimensions;

[0090] Calculate the information entropy of the features by the entropy weight method to quantify the sensitivity of different features to data changes, as the initial assignment basis of the feature weights, expressed as:

[0091] ;

[0092] where represents the information entropy of data type j, represents the normalized probability value of data type j in sample i;

[0093] Update and calculate the feature weights based on the information entropy, expressed as:

[0094] ;

[0095] where represents the type weight of the j-th feature weight;

[0096] Calculate the data type weights by the information entropy for feature weighting in the training of the logistic regression model. The information entropy weight reflects the amount of information of the data type, so a higher weight is given in the prediction of data storage priority. Calculate the data type weights by the information entropy and perform feature weighting according to the amount of information of the data type during the training stage of the logistic regression model to ensure that in the logistic regression model, data types with greater information content account for a larger proportion in the prediction of data storage priority;

[0097] And perform pre-weighting calculation on the feature matrix according to the feature weights;

[0098] Construct a linear logistic regression model to predict the agricultural yield results of different nodes, expressed as:

[0099] ;

[0100] where represents the linear combination prediction value of the i-th data, used to predict the crop price or yield, represents the bias term, represents the model weight of the j-th data type, Represents the value of the i-th data type of the j-th data type weighted by the information entropy weight;

[0101] The linear prediction result is converted into prediction probability through the Sigmoid function, which is expressed as:

[0102] ;

[0103] in represents the probability that the agricultural output result of the i-th data reaches the target;

[0104] Use the training set to train a linear logistic regression model, calculate the error between the predicted probability and the true label of the training set through the cross entropy loss function, and iterate and optimize through batch gradient descent until the calculated error no longer decreases significantly.

[0105] Through the gradient descent method, the type of adaptive update is performed during the training process of the logistic regression model. As the training data changes, the model weights are adjusted to minimize the prediction error. By continuously adjusting the model weights, it adapts to different data distributions and improves the generalization ability of the model.

[0106] Use the trained linear logistic regression model to make probability predictions;

[0107] When building a data storage priority prediction model (logistic regression), the contribution of each data type to the prediction target is more reasonably reflected. After weighting, data types with large amounts of information have a greater impact on the logistic regression model, which is more in line with actual needs and clearly improves the accuracy and rationality of the logistic regression model in predicting data storage priorities.

[0108] Based on the probability of reaching the target for each data, the sorting sequence is obtained as the storage priority value, and the fitness value of each value in the sequence is calculated, which is expressed as:

[0109] ;

[0110] in represents the fitness value of the jth sorted sequence C, v represents the total number of sequence values, Indicates the data storage value of the i-th node, in kb;

[0111] Use the differential evolution algorithm DE to perform mutation operations, randomly extract different individuals to mutate and generate new mutant individuals, which can be expressed as:

[0112] ;

[0113] in represents the new individual generated by the mutation of the i-th individual in the s+1th generation, 、 and respectively represent the first, second, and third individual values randomly selected from the s-th generation of the parent generation, and G represents the differential scaling factor;

[0114] Perform a crossover operation on the newly mutated individual and the original parent individual to generate a new candidate individual, and compare the fitness of the new candidate individual and the parent individual. If the fitness of the new candidate individual is greater than that of the parent individual, perform greedy selection;

[0115] Perform tournament selection and population update based on the genetic algorithm GA, and calculate the best fitness value after each round of population update until the difference in the calculated fitness value no longer decreases significantly, then stop the update;

[0116] Select the chromosome individual with the highest fitness value in the final population and generate an optimized sorting sequence as the final priority sorting result for data storage.

[0117] The actual contribution degree of various data types in yield prediction is clarified through the entropy weight method, making the prediction result closer to the actual agricultural production characteristics, thereby directly improving the accuracy of yield prediction, improving the quality of storage priority decision-making, determining the data storage priority based on the prediction probability, automatically realizing the priority storage of high-value and high-information data, effectively avoiding the occupation of precious storage resources by low-information redundant data, reducing the storage cost. Introducing the mutation and crossover mechanism of the differential evolution algorithm significantly improves the global search ability of the traditional genetic algorithm, reduces the risk of the algorithm falling into local optimum, and speeds up the convergence rate of sorting sequence optimization. Through the priority sorting mechanism optimized by combining differential evolution, high-priority data can be quickly identified and sorted, reducing decision-making lag, and improving the real-time response ability of the overall data scheduling and storage management;

[0118] Due to the precise improvement of data storage priority, the quality and efficiency of data query and analysis are significantly improved, further improving the performance of the intelligent agricultural decision support system. The efficient priority sorting optimization and storage scheme significantly reduce data storage and computing costs, indirectly improving the economic and environmental benefits of the smart agriculture management platform, reducing the uncertainty brought by inconsistent data quality in the data mining and analysis process, and further improving the reliability of subsequent agricultural decision-making and intelligent operations.

[0119] S4. Generate a hash digest of the sorting sequence data for secure storage, perform data consistency verification, and generate a log record for cloud storage;

[0120] Preferably, generating a hash digest of the sorting sequence data for secure storage includes

[0121] Using a smart contract to write data according to the final priority sorting result, and generating a data hash digest for the written data using the SHA-256 algorithm;

[0122] After writing the data to IPFS, a clear storage path address is returned to the smart contract to form a binding record between the chain and IPFS.

[0123] Through the sorting sequence data, a hash digest is generated by the SHA-256 algorithm to achieve data anti-tampering verification, ensure that the data is not modified without authorization, and guarantee the credibility of the data. By storing the IPFS data path address and hash digest on the chain, it is convenient for users to quickly and accurately audit and trace the data, improving the transparency of data management. By actually storing the data in the IPFS off-chain system and only saving the hash digest and path on the chain, the load and cost of on-chain storage are significantly reduced, and the storage efficiency is improved.

[0124] Furthermore, data consistency verification is performed, including,

[0125] Only the data hash digest and IPFS path index are stored on the blockchain, without storing the original data content, reducing the on-chain storage burden. The hash digest of the data stored in IPFS is used for data consistency verification, and the data access request is connected through the RESTAPI gateway to perform user access permission verification.

[0126] By comparing the hash digest on the blockchain with the hash of the actual data in IPFS, data consistency verification is quickly achieved, effectively avoiding the risk of data tampering, ensuring the true and effective nature of the data. Only storing the hash digest and index path of the data reduces the data storage volume of blockchain nodes, effectively reducing the blockchain maintenance cost and management burden.

[0127] Preferably, log records are generated for cloud storage, including,

[0128] Log records are automatically generated during each data write, access, modification, or deletion operation, and the generated log records are securely stored using the SHA-256 algorithm and stored in the cloud through wireless transmission technology.

[0129] By automatically generating logs and protecting them with the SHA-256 algorithm, it effectively prevents log tampering, ensures the transparency and traceability of the entire data operation process. Using the SHA-256 digest to securely store the logs ensures the integrity of the logs, prevents illegal tampering, and guarantees the credibility of the audit data. By uploading the log records to the cloud in real time through wireless transmission, it simplifies the log collection and maintenance process, effectively improving the log management efficiency and real-time performance. The log records stored and managed centrally in the cloud are convenient for quick retrieval and analysis, significantly improving the efficiency of system fault diagnosis and security audit.

[0130] This embodiment also provides a system for the data optimized storage method for smart agriculture, including,

[0131] The data acquisition and processing module is responsible for collecting field sensor data, uniformly standardizing and normalizing the collected data, and generating a multi-source heterogeneous data matrix;

[0132] The data redundancy identification module calculates the fuzzy similarity based on the multi-source heterogeneous data matrix, generates a time series distance matrix, calculates the weight vectors of different data types, constructs a weighted distance matrix and a fuzzy similarity matrix, and clearly distinguishes representative nodes and redundant nodes according to the Boolean similarity matrix;

[0133] The data sorting and optimization module reads the representative node data and the difference vectors to form a feature matrix, determines the feature weights by the entropy weight method, uses a linear logistic regression model to predict agricultural yields, calculates the prediction probabilities of each node, and optimizes the priority sorting sequence of data storage using differential evolution and genetic algorithms;

[0134] The data security storage module generates a data hash digest for secure storage according to the sorting sequence;

[0135] The log storage module automatically generates operation logs and uploads them to cloud storage.

[0136] This embodiment also provides a computer device applicable to the data optimization storage method for smart agriculture, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the data optimization storage method for smart agriculture proposed in the above embodiment.

[0137] This computer device can be a terminal. This computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of this computer device is used to provide computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of this computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of this computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of this computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0138] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the data optimization storage method for smart agriculture proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, magnetic disk or optical disc.

[0139] In summary, the present invention accurately identifies redundant data by using a weighted fuzzy similarity matrix, only stores the difference vectors, greatly reduces the storage space occupation, effectively reduces the consumption of storage resources at the cloud or chain end, and significantly reduces the amount of data actually transmitted through storage compression and differential storage, reducing the data transmission burden of edge devices or cloud nodes. By using the entropy weight method, the actual contribution degree of various data types in yield prediction is clarified, making the prediction result closer to the actual agricultural production characteristics, thereby directly improving the accuracy of yield prediction and the quality of storage priority decision-making. Based on the prediction probability, the data storage priority is determined, and the priority storage of high-value and high-information data is automatically realized. By combining the priority sorting mechanism optimized by differential evolution, high-priority data is quickly identified and sorted, reducing decision-making lag and improving the real-time response ability of overall data scheduling and storage management.

[0140] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A data optimization storage method for smart agriculture, characterized in that, Including: Collecting agricultural field data, remote sensing data, and management data, and preprocessing them to construct a multi-source heterogeneous data matrix; Calculating the fuzzy similarity of the multi-source heterogeneous data matrix, constructing a time series distance matrix to calculate the weight vectors of different data types, constructing a weighted distance matrix according to the weights assigned to different data types, calculating a weighted fuzzy similarity matrix, constructing a Boolean similarity matrix, distinguishing redundant nodes and representative nodes, and determining the difference vector; Forming a feature matrix based on the representative node data and the difference vector data, calculating the feature weights through the entropy weight method, constructing a linear logistic regression model to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence; Generating a hash digest of the sorting sequence data for secure storage, performing data consistency verification, and generating log records for cloud storage; The calculating the weighted fuzzy similarity matrix and constructing the Boolean similarity matrix includes: Calculating the fuzzy similarity of the multi-source heterogeneous data matrix, where the sequence distance between different nodes is calculated, and the time series distance matrix of each data type k is calculated; Calculating the weight vectors of different data types using the feature variance normalization method, constructing a weighted distance matrix according to the weights assigned to different data types, and calculating the weighted fuzzy similarity matrix; Adopting a dynamic similarity threshold mechanism, setting the thresholds corresponding to different data types according to the sum of the similarity mean and twice the standard deviation of different data types, calculating the global threshold weighted by the weight vectors of different data types, and screening out redundant data point pairs based on the weighted fuzzy similarity matrix to construct the final Boolean similarity matrix; Extracting the node groups with similarity greater than the global threshold from the Boolean similarity matrix, selecting the node with the highest similarity as the representative node, and marking other nodes as redundant nodes; Recording the mapping relationship between the redundant nodes and the representative nodes to form a mapping table, calculating the difference vector between each redundant node and the representative node, performing storage compression by only storing the representative node data and the difference vectors of the redundant nodes, and constructing a sleep queue according to the index of the redundant nodes and their difference vector information.

2. The data optimization storage method for smart agriculture according to claim 1, wherein: The constructing the linear logistic regression model to predict the agricultural yield results of different nodes, obtaining a sorting sequence, calculating the fitness value of each value in the sequence, using the differential evolution algorithm for mutation and crossover, and updating the population based on the genetic algorithm to optimize the sorting sequence includes: Reading the representative node data and the difference vector data after storage compression, extracting the multi-source heterogeneous data of each node, and splicing them to form a feature matrix, where the feature matrix contains f data type dimensions; Calculating the information entropy of the features through the entropy weight method, updating and calculating the feature weights based on the information entropy, and performing pre-weighting calculation on the feature matrix according to the feature weights; Constructing a linear logistic regression model to predict the agricultural yield results of different nodes, and converting the linear prediction result into a prediction probability through the Sigmoid function; Train a linear logistic regression model using the training set, calculate the error between the predicted probability and the true labels of the training set through the cross-entropy loss function, and perform iterative optimization through batch gradient descent until the calculated error no longer decreases significantly, then stop the iteration; Use the trained linear logistic regression model to make probability predictions; Based on the probability of each data reaching the target, sort it as the storage priority value to obtain a sorted sequence, and calculate the fitness value of each value in the sequence; Use the differential evolution algorithm DE to perform mutation operations, randomly select different individuals for mutation to generate new mutant individuals; Perform crossover operations on the new mutant individuals and the original parent individuals to generate new candidate individuals, and compare the fitness of the new candidate individuals and the parent individuals. If the fitness of the new candidate individuals is greater than that of the parent individuals, perform greedy selection; Perform tournament selection and population update based on the genetic algorithm GA, and calculate the best fitness value after each round of population update until the difference in the calculated fitness value no longer decreases significantly, then stop the update; Select the chromosome individual with the highest fitness value in the final population, and generate an optimized sorted sequence as the final priority sorting result for data storage.

3. The data optimization storage method for smart agriculture according to claim 2, wherein: Securely store the hash digest of the generated sorted sequence data, including, Use a smart contract to write data according to the final priority sorting result, and generate a data hash digest for the written data using the SHA-256 algorithm; After writing the data to IPFS, return a clear storage path address to the smart contract to form a binding record between the chain and IPFS.

4. The data optimization storage method for smart agriculture according to claim 3, wherein: The data consistency verification includes, Only store the data hash digest and the IPFS path index through the blockchain, verify the data consistency of the hash digest of the data stored in IPFS, and the data access request is connected through the REST API gateway to perform user access permission verification.

5. The data optimization storage method for smart agriculture according to claim 4, characterized in that: The collection of agricultural field data, remote sensing data, and management data includes, Collect multi-source heterogeneous data through field sensors, remote sensing data, and agricultural management data, and perform data preprocessing to construct a multi-source heterogeneous data matrix. Use the number of data collection nodes and the collection time period of different data types as the matrix dimensions; Based on the multi-source heterogeneous data matrix, construct a star-shaped data warehouse structure.

6. The data optimization storage method for smart agriculture according to claim 5, characterized in that: The generation of log records for cloud storage includes, Automatically generate log records during each data write, access, modification, or deletion operation, securely store the generated log records using the SHA-256 algorithm, and perform cloud storage through wireless transmission technology.

7. A system for a data optimization storage method in intelligent agriculture, based on the data optimization storage method for intelligent agriculture according to any one of claims 1 to 6, characterized in that: including, The data acquisition and processing module is responsible for collecting field sensor data, uniformly standardizing and normalizing the collected data to generate a multi-source heterogeneous data matrix; The data redundancy identification module calculates the fuzzy similarity based on the multi-source heterogeneous data matrix, generates a time series distance matrix, calculates the weight vectors of different data types, constructs a weighted distance matrix and a fuzzy similarity matrix, and clearly distinguishes representative nodes and redundant nodes according to the Boolean similarity matrix; The data sorting optimization module reads the representative node data and difference vectors to form a feature matrix, determines the feature weights through the entropy weight method, uses a linear logistic regression model to predict agricultural yields, calculates the prediction probabilities of each node, and optimizes the priority sorting sequence of data storage using the differential evolution and genetic algorithms; The data security storage module generates a data hash digest according to the sorting sequence for secure storage; The log storage module automatically generates operation logs and uploads them to cloud storage.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the data optimization storage method for smart agriculture according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the data optimization storage method for smart agriculture according to any one of claims 1 to 6.

Images