An agricultural data storage method and system for smart villages
By introducing multidimensional minimum distance and operation log matching methods to obtain temporal mutation weight factors, and combining them with the K-means clustering algorithm, the problem of ignoring instantaneous sudden event information in existing technologies is solved, and efficient storage and decision support for key agricultural events are achieved.
Patent Information
- Application Number
- CN202511661987.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-13
- Publication Date
- 2026-05-15
- Estimated Expiration
- 2045-11-13
AI Technical Summary
In existing technologies, the K-means clustering method based on standard Euclidean distance cannot effectively retain key instantaneous sudden event information in the context of agricultural environmental time series data compression. As a result, this information is ignored during the data compression process, which fails to meet the data storage requirements of smart agriculture scenarios.
By introducing multidimensional minimum distance calculation, mutation dimension distribution and operation log matching methods, we obtain the temporal mutation weight factor. Combined with the K-means clustering algorithm, we identify and quantify the abnormality and credibility of instantaneous mutation events, and obtain representative data centroids for data compression and storage.
It achieves high-fidelity retention of key agricultural events, improves the balance between storage efficiency and decision support value of agricultural environmental data, and ensures the effective identification and quantification of transient and sudden events.
Smart Images
Figure CN121479037B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data storage, and more particularly to an agricultural data storage method and system for smart villages. Background Technology
[0002] With the deepening of smart agriculture and smart village construction, various IoT sensor devices deployed in agricultural production environments such as farmland and greenhouses are becoming increasingly widespread. These sensors can collect multi-dimensional environmental parameters at high frequency, including but not limited to ambient temperature, relative humidity, and carbon dioxide concentration, thus forming a continuous and dense time-series data stream of agricultural environment. To address the long-term storage and feature extraction problems of the aforementioned large-scale time-series data, existing technologies widely employ the K-means clustering compression method based on time windows. This method typically divides a day's collected data into preset time windows (e.g., 24 hours) and performs cluster analysis on the multi-dimensional data points within each time period, storing only the centroid of each cluster as the representative feature of the data within that time window. This compression strategy significantly reduces storage costs and facilitates subsequent medium- and long-term trend analysis and modeling based on cluster centroids. In the K-means algorithm, each clustering iteration mainly relies on calculating the Euclidean distance between each data point and each candidate centroid to determine its cluster affiliation and update the cluster center position. This algorithm has been widely used in various industrial scenarios and is also suitable to some extent for the compression needs of conventional agricultural data.
[0003] However, existing K-means clustering methods based on standard Euclidean distance have significant limitations in the compression of agricultural environmental time-series data, failing to effectively preserve information on critical, transient events in agricultural production. Taking a greenhouse with an automated control system as an example, the multidimensional data points collected by agricultural sensors typically include information on ambient temperature, relative humidity, and carbon dioxide concentration. In practical applications, multiple events with critical decision-making value can cause discontinuous jumps in these parameters within a short period. For example, automated carbon dioxide injection can cause a sudden increase in carbon dioxide concentration within seconds, followed by a slow decline; fogging operations can trigger a rapid increase in relative humidity followed by a short-term decrease, forming a typical pulse-like change pattern; ventilation system malfunctions can cause multidimensional coordinated abrupt changes such as temperature rise, humidity accumulation, and carbon dioxide concentration decrease. Although these events have significant physical characteristics, they constitute only a small proportion of the total daily data. During iterative optimization, the K-means algorithm, in pursuit of the overall minimum squared distance, often forcibly merges these key points into clusters composed of a large amount of routine state data. This leads to the "dilution" of critical event data during centroid updates, resulting in centroids that fail to reflect any characteristics of the critical events. Consequently, the final stored representative data loses its ability to record events. More importantly, the distance metric of the standard K-means algorithm does not incorporate any knowledge features related to mutation strength, event credibility, or log matching, lacking a modeling and quantification mechanism for the common characteristic of instantaneous events in agriculture breaking through historical norms. Therefore, under the existing technical architecture, diverse agricultural behaviors and environmental events in smart agriculture scenarios are systematically ignored during data compression and archiving, resulting in a typical problem of information loss due to collection but lack of retention. Summary of the Invention
[0004] In view of this, embodiments of the present invention provide an agricultural data storage method for smart villages to solve the problem that key agricultural events are easily missed during time-series data compression.
[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:
[0006] In a first aspect, this application provides an agricultural data storage method for smart villages, the method comprising the following steps:
[0007] Step S1: Obtain standardized agricultural environmental data and agricultural operation log data by preprocessing and windowing the raw agricultural environmental data;
[0008] Step S2: Obtain the initial mutation intensity factor of the data points by performing multidimensional minimum distance calculation on the local historical baseline;
[0009] Step S3: By comprehensively analyzing the distribution of mutation dimensions of agricultural environmental data points and matching them with operation logs, the semantic consistency factor of events is obtained;
[0010] Step S4: Obtain the temporal mutation weight factor by fusing and normalizing the initial mutation intensity factor and event semantic consistency factor of agricultural environmental data points;
[0011] Step S5: By introducing weighted K-means clustering with time-series mutation weight factors, representative data centroids are obtained and the data is compressed and stored using the representative data centroids.
[0012] Preferably, the step of obtaining the initial mutation intensity factor of the data point by performing multidimensional minimum distance calculation on the local historical baseline includes:
[0013] Set a preceding time window; subtract any target dimension in the agricultural environment data at any target time from the target dimension at any other time in the preceding time window and square the result as the first mutation assessment of the target dimension at the target time; add the first mutation assessments of all dimensions at the target time and square root the result as the comprehensive mutation assessment of the target time; take the minimum comprehensive mutation assessment in the preceding time window of the target time as the initial mutation intensity factor of the target time.
[0014] Preferably, the step of obtaining the event semantic consistency factor by comprehensively analyzing the mutation dimension distribution of agricultural environmental data points and matching them with operation logs includes:
[0015] By calculating the mutation amount of agricultural environmental data points in different dimensions, the degree of mutation contribution in each dimension can be obtained.
[0016] By conducting a balance analysis on the distribution of relative mutation contributions, a dimensional balance index is obtained;
[0017] By matching data points with agricultural operation logs over time, event type matching scores are obtained and an event semantic consistency factor is constructed.
[0018] Preferably, the step of calculating the mutation amount of agricultural environmental data points by dimension to obtain the mutation contribution degree of each dimension includes:
[0019] For any target time in all agricultural environmental data, the square of the difference between the data of any target dimension at the target time and the data of the target dimension at the previous time is taken as the first difference of the target dimension at the target time; the square of the difference between the data of any target dimension at the target time and the data of the target dimension at the next time is taken as the second difference of the target dimension at the target time; the sum of the first and second differences of the target dimension at the target time is taken as the absolute mutation assessment of the target dimension at the target time; the sum of the absolute mutation assessments of all target dimensions at the target time is taken as the total mutation assessment of the target time; and the absolute mutation assessment of the target dimension at the target time divided by the total mutation assessment of the target time is taken as the mutation contribution degree of the target dimension at the target time.
[0020] Preferably, the step of obtaining the dimensional balance index by performing a balance analysis on the distribution of relative mutation contributions includes:
[0021] The square of the result of subtracting the inverse of the number of dimensions from the mutation contribution of the target dimension at the target time is taken as the relative contribution of the target dimension at the target time.
[0022] The result of multiplying the mutation contribution of the target dimension at the target time by the relative contribution of the target dimension at the target time is used as the first dimension balance evaluation of the target dimension at the target time.
[0023] The first-dimensional balance evaluations of all dimensions at the target time are summed and the square root is calculated. The result of subtracting the constant 1 from the corresponding calculation result is taken as the dimensional balance index at the target time.
[0024] Preferably, the step of obtaining an event type matching score and constructing an event semantic consistency factor by performing time matching between data points and agricultural operation logs includes:
[0025] Set a time tolerance window; perform operation log matching on the agricultural environment data in the time tolerance window at the target time. If operation logs exist in the time tolerance window at the target time, set the log matching evaluation to 1; if operation logs do not exist in the time tolerance window at the target time, set the log matching evaluation to 0.
[0026] The result of adding constant 1 to the log matching evaluation is used as the type amplification factor; the larger value between constant 0 and the dimensionality balance index at the target time is used as the first weight factor evaluation at the target time; the result of multiplying the type amplification factor at the target time with the first weight factor evaluation at the target time is used as the event semantic consistency factor at the target time.
[0027] Preferably, the step of obtaining the temporal mutation weight factor by fusing and normalizing the initial mutation intensity factor and event semantic consistency factor of agricultural environmental data points includes:
[0028] The comprehensive credible mutation intensity is obtained by multiplying and fusing the initial mutation intensity factor and the event semantic consistency factor, and the normalized mutation anomaly index is obtained by normalizing the comprehensive credible mutation intensity by the median within the period.
[0029] The final temporal mutation weighting factor is obtained by performing nonlinear activation mapping on the normalized mutation anomaly index.
[0030] Preferably, the step of obtaining a comprehensive credible mutation strength by multiplying and fusing the initial mutation strength factor and the event semantic consistency factor, and obtaining a normalized mutation anomaly index by normalizing the comprehensive credible mutation strength by the median over a period, includes:
[0031] The result of multiplying the initial mutation intensity factor at the target time by the event semantic consistency factor is taken as the comprehensive credible mutation intensity at the target time.
[0032] The analysis period and the gain coefficient used to adjust the steepness of the activation function are set. The median of the comprehensive credible mutation intensity of all agricultural environmental data points in the analysis period of the target time is used as the denominator, and the comprehensive credible mutation intensity of the target time is used as the numerator. The result of subtracting the corresponding fraction from the constant 1 and multiplying it by the gain coefficient is used as the normalized mutation anomaly index of the target time.
[0033] Preferably, the step of obtaining representative data centroids through weighted K-means clustering by introducing a time-series mutation weight factor and then compressing and storing the data using these representative data centroids includes:
[0034] Set the number of clusters for clustering; obtain the Euclidean distance between the agricultural environment data at the target time and any target centroid during the clustering process; multiply the temporal mutation weight factor at the target time by the Euclidean distance as the optimized distance between the agricultural environment data at the target time and the target centroid during the clustering process; complete the K-means clustering process using the optimized distance, and use the obtained cluster centroids as representative data centroids; store the representative data centroids as compressed feature representatives for the corresponding analysis period.
[0035] Secondly, this application provides an agricultural data storage system for smart villages, comprising: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an agricultural data storage method for smart villages is implemented.
[0036] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:
[0037] In practical applications of smart agriculture, the multi-dimensional environmental time-series data continuously collected by sensors often contains key agricultural operation events and sudden abnormal states. Although these events are short in duration and few in number, they are highly valuable for agricultural decision-making. This invention constructs a mutation intensity factor based on historical trajectory deviation and introduces an information perception factor that integrates multi-dimensional synergy and operation log matching. This effectively identifies and quantifies the anomaly and credibility of these instantaneous mutation events, providing accurate mutation perception capabilities for subsequent data compression and clustering processing. At the data processing and compression modeling level, this invention integrates mutation intensity and semantic consistency factors into a weighted index and introduces it into the distance calculation process of the K-means clustering algorithm. This breaks through the dependence of traditional methods on conventional steady-state data and achieves dynamic amplification of the clustering influence of key event points. This design ensures both effective coverage of the overall trend by the representative clustering data and high-fidelity preservation of key agricultural events, significantly improving the balance between storage efficiency and decision support value of agricultural environmental data. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 This is a flowchart of an agricultural data storage method for smart villages provided in Embodiment 1 of the present invention. Detailed Implementation
[0040] Embodiments of this disclosure are described in detail below, with examples of these embodiments illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting it.
[0041] To illustrate the technical solution of the present invention, specific embodiments are described below.
[0042] See Figure 1 This is a flowchart of an agricultural data storage method for smart villages provided in Embodiment 1 of the present invention, as shown below. Figure 1 As shown, the method may include:
[0043] Step S1 involves preprocessing and windowing the raw agricultural environment data to obtain a standardized matching structure between agricultural environment data points and agricultural operation logs.
[0044] In this embodiment of the invention, step S1 aims to construct a unified input data structure upon which subsequent mutation identification and clustering analysis depend. To this end, this step starts from two data sources: the original collected data and the external control log, and sequentially completes the continuous collection of multiple agricultural environmental data, standardized segmentation processing, and time correlation with agricultural operation logs, thereby generating a standardized data point set for key event identification and clustering.
[0045] Specifically, this step first involves deploying environmental sensors for temperature, relative humidity, and carbon dioxide concentration in the target agricultural environment area. The sensor system collects environmental data using a preset sampling strategy. In this embodiment, the sensor system's data collection frequency is set to once per second; however, this frequency can be adjusted based on the actual scenario and is not required. After collecting the raw data, it is uploaded to the data platform of the smart agriculture platform, thus forming raw time-series data indexed primarily by time. To ensure comparability and structural consistency of the raw data during clustering, this embodiment uses a fixed 24-hour time window to segment the continuous raw time-series data. Each time window corresponds to an analysis period. Through this operation, the daily raw time-series data forms an independent set of data points with clear boundaries and time labels, facilitating subsequent periodic processing and archiving. Meanwhile, this step extracts the daily agricultural operation logs from the smart agriculture control system. The agricultural operation logs record all operation instructions triggered by the system or manually within the analysis period, such as carbon dioxide increase instructions, wind turbine start-stop control instructions, etc. Each instruction log includes at least the operation type, triggering method, execution timestamp, and target control object.
[0046] This completes the process of preprocessing and windowing raw agricultural environmental data to obtain a standardized agricultural environmental data point and agricultural operation log matching structure.
[0047] Step S2: Obtain the initial mutation intensity factor of the data points by performing multidimensional minimum distance calculation on the local historical baseline.
[0048] In this embodiment, in order to identify candidate points representing instantaneous key events in agricultural environmental time series data, the system adopts a quantization method based on local historical trajectory deviation to extract data points with abrupt change characteristics.
[0049] Specifically, the essential characteristic of critical events in terms of data form is that the state vector of their corresponding data points will break through their historical conventional change trajectory in the time series. For agricultural environmental data in a stable state or with a slow changing trend, there is usually a high degree of numerical continuity between the current data point and the preceding data point. Conversely, when a transient critical event occurs, this continuity will be significantly disrupted. Therefore, potential critical mutation behaviors can be identified by assessing the degree of deviation between the current data point and its preceding state. Specifically, a preceding time window is set. In this embodiment of the invention, the length of the preceding time window is set to 10, which can be adjusted according to the actual scenario and is not required. The result of subtracting any target dimension in the agricultural environmental data at any target time from the target dimension at any other time in the preceding time window of the target time and squaring it is used as the first mutation assessment of the target dimension at the target time. The result of adding the first mutation assessments of all dimensions at the target time and performing square root processing is used as the comprehensive mutation assessment of the target time. The minimum comprehensive mutation assessment in the preceding time window of the target time is used as the initial mutation intensity factor of the target time.
[0050] In one implementation, assume the first At the moment, the _ Agricultural environmental data in 10 dimensions The number of dimensions in agricultural environmental data is: The preceding time window length is Then the first The expression for calculating the initial mutation intensity factor at time point is:
[0051]
[0052] in, Indicates the first The initial mutation intensity factor at each time point; Indicates the first At the moment, the _ Agricultural environmental data in multiple dimensions; Indicates the first At the moment, the _ Agricultural environmental data in multiple dimensions; This indicates the number of dimensions in the agricultural environmental data; Indicates the length of the preceding time window.
[0053] It should be noted that this embodiment proposes to use the current data point Prelude Each data point constitutes a local historical baseline set, representing the system state before the event occurred. This local historical baseline can more comprehensively depict the stable operating state before the current point, thereby improving the stability and accuracy of mutation identification. After constructing the local historical baseline, the system further analyzes the current data points... Multidimensional Euclidean distances are calculated with all data points in the historical baseline, and the minimum value is extracted as the initial mutation intensity factor for that point. This minimum distance measures the minimum deviation of the current data point from its predecessor state. If the current point is in a normal continuous state, then its distance from a predecessor point in the historical baseline (e.g., a direct predecessor) is considered. The distance in multidimensional space will be very small. If the current point has deviated from its historical trajectory, its distance from all points on the historical baseline will increase significantly, leading to a significant increase in the minimum distance value. In the formula... This section calculates the multidimensional Euclidean distance between the current data point and a point in the preceding historical window. In agricultural data scenarios, the state of a data point is defined by multiple dimensions such as temperature, humidity, and carbon dioxide concentration. Therefore, using multidimensional distance instead of single-dimensional difference comprehensively considers changes in all dimensions, ensuring comprehensive event capture. Secondly, and this is the key to the design of this formula, it employs... The calculation precisely corresponds to the physical phenomenon of breaking through historical norms. A data point in a normal state, whether in an absolute steady state (such as constant temperature and humidity at night) or in a slow, continuous trend (such as a smooth rise in daytime temperature), will inevitably have a highly similar current state to its nearest neighbor's past states. Therefore, its distance from each point in the preceding historical window must have a very small value; the result of the calculation, taking the minimum value, will be close to zero. The value is used to correctly identify the point as a non-mutation point. Conversely, when a transient critical event occurs, such as the starting point of carbon dioxide injection, the value is used to correctly identify the point as a non-mutation point. The carbon dioxide concentration value of agricultural environmental data at a given moment will be much higher than the concentration values of all points within its preceding historical window, leading to... The distances from the agricultural environmental data at any given moment to all points within this window become enormous. In this situation, even taking the minimum value yields... It must also be a large value. Through the aforementioned minimum distance extraction method, a mutation intensity metric that is highly sensitive to changes in state continuity can be constructed, effectively distinguishing mutation points from regular points. It is worth noting that this identification method does not rely on prior knowledge of the direction or form of event mutations (e.g., ascending, descending, or symmetrical), thus possessing strong adaptability and versatility.
[0054] This completes the process of obtaining the initial mutation intensity factor of data points by performing multidimensional minimum distance calculation on local historical baselines.
[0055] Step S3: By comprehensively analyzing the mutation dimension distribution of agricultural environmental data points and matching it with operation logs, the semantic consistency factor of the event is obtained.
[0056] After performing step S2, a series of high-intensity candidate points exhibiting significant mutations in data morphology have been identified. However, the practical significance of these identified mutation points in agricultural production management is not entirely the same. A mutation may originate from an agricultural operation with important record-keeping value (such as mist spraying), an anomaly requiring close monitoring (such as equipment failure), or even random sensor noise with no practical significance. Assigning equal weight to all these high-intensity mutation points would fail to differentiate between events of different natures. Therefore, the purpose of this step is to further screen and characterize the candidate mutation points selected in step S2, assigning them an event semantic consistency factor that can represent their event semantic consistency.
[0057] To achieve this goal, the design philosophy of this step is to introduce two dimensions of information for a comprehensive judgment of mutation points. First, an internal consistency assessment is performed. This assessment aims to distinguish whether a mutation originates from a real, multi-dimensional, synergistic physical process or from isolated noise occurring only in a single or a few dimensions. The basis for this is that a real physical event, such as misting or ventilation malfunction, is essentially a systemic change in the physical environment. This change will inevitably leave traces on multiple relevant sensor dimensions simultaneously; that is, the energy of the mutation will be dispersed across multiple dimensions. Conversely, a typical sensor electronic noise typically has a highly localized impact, with its energy highly concentrated in its own data dimension. This step uses a unique method to quantify the evenness of the energy distribution of a mutation event across all data dimensions to assess its credibility as a real physical event. A more evenly distributed mutation has higher multi-dimensional synergy and thus higher credibility. Next, an external reference assessment is performed. This assessment aims to further distinguish whether a real physical event is a planned operation or an unknown anomaly. For this purpose, this step introduces agricultural operation logs from the smart agriculture system as external reference information. This log database records all control commands triggered by manual or automated rules. By comparing the time of data mutations with the operation logs, if a matching log entry can be found in terms of time, the mutation can be classified with high confidence as a planned agricultural operation. Conversely, a mutation that is not recorded in the logs but has high internal consistency is very likely an unknown event that requires special attention.
[0058] In summary, this step first calculates the mutation amount of agricultural environmental data points by dimension to obtain the degree of mutation contribution in each dimension; then, it performs a balance analysis on the distribution of relative mutation contribution to obtain the dimensional balance index; finally, it performs time matching between data points and agricultural operation logs to obtain event type matching scores and construct an event semantic consistency factor.
[0059] First, by calculating the mutation amount of agricultural environmental data points in different dimensions, the mutation contribution degree of each dimension is obtained. Specifically, for agricultural environmental data at any target time in all agricultural environmental data, the square of the result of subtracting the target dimension data of any target dimension at the target time from the target dimension data of the previous time is taken as the first difference of the target dimension at the target time; the square of the result of subtracting the target dimension data of any target dimension at the target time from the target dimension data of the next time is taken as the second difference of the target dimension at the target time; the sum of the first difference and the second difference of the target dimension at the target time is taken as the absolute mutation assessment of the target dimension at the target time; the sum of the absolute mutation assessments of all target dimensions at the target time is taken as the total mutation assessment of the target time; and the result of dividing the absolute mutation assessment of the target dimension at the target time by the total mutation assessment of the target time is taken as the mutation contribution degree of the target dimension at the target time.
[0060] In one embodiment, the first At the moment, the _ The formula for calculating the contribution of mutations in each dimension is as follows:
[0061]
[0062] in, Indicates the first At the moment, the _ The degree of contribution of mutations in each dimension; Indicates the first At the moment, the _ Agricultural environmental data in multiple dimensions; Indicates the first The first time before the first time Agricultural environmental data in multiple dimensions; Indicates the first In the next moment after the first moment Agricultural environmental data in multiple dimensions; Indicates the first At the moment, the _ Agricultural environmental data in multiple dimensions; Indicates the first The first time before the first time Agricultural environmental data in multiple dimensions; Indicates the first In the next moment after the first moment Agricultural environmental data in multiple dimensions; This indicates the number of dimensions in agricultural environmental data.
[0063] Subsequently, a dimensional balance index was obtained by performing a balance analysis on the distribution of relative mutation contributions. Specifically, the square of the result of subtracting the inverse of the number of dimensions from the mutation contribution of the target dimension at the target time was taken as the relative contribution of the target dimension at the target time; the result of multiplying the mutation contribution of the target dimension at the target time by the relative contribution of the target dimension at the target time was taken as the first dimensional balance assessment of the target dimension at the target time; the first dimensional balance assessments of all dimensions at the target time were added together and the square root was calculated, and the result of subtracting the constant 1 from the corresponding calculation result was taken as the dimensional balance index at the target time.
[0064] In one embodiment, the first The expression for calculating the dimensional equilibrium index at time t is:
[0065]
[0066] in, Indicates the first The dimensional equilibrium index at each moment; Indicates the first At the moment, the _ The degree of contribution of mutations in each dimension; This indicates the number of dimensions in agricultural environmental data.
[0067] Finally, by performing time matching between data points and agricultural operation logs, event type matching scores are obtained, and an event semantic consistency factor is constructed. Specifically, a time tolerance window is set; for agricultural environmental data within the time tolerance window at the target time, operation log matching is performed. If operation logs exist within the time tolerance window at the target time, the log matching evaluation is set to 1; if no operation logs exist within the time tolerance window at the target time, the log matching evaluation is set to 0. The result of adding the constant 1 to the log matching evaluation is used as the type amplification factor; the larger value between the constant 0 and the dimensionality balance index at the target time is used as the first weight factor evaluation at the target time; the result of multiplying the type amplification factor at the target time by the first weight factor evaluation at the target time is used as the event semantic consistency factor at the target time.
[0068] In one implementation, assume the first The log matching evaluation at each time point is Then the first The expression for calculating the event semantic consistency factor at time point is:
[0069]
[0070] in, Indicates the first The semantic consistency factor of events at each moment; Indicates the first Log matching evaluation at each time point; Indicates the first The dimensional equilibrium index at each moment; This represents the maximum value function.
[0071] It should be noted that, firstly The calculation normalizes the absolute mutation amount in each dimension, making it a uniform value. The contribution distribution vector between these points and sums to 1. This step is the foundation for all subsequent calculations. Secondly, and this is the key innovation of this invention, This item calculates the first... The relative contribution of each dimension is measured by its deviation from the expected value under a perfectly balanced distribution. This deviation is then squared to ensure it is positive, amplifying the impact of larger deviations. Finally, the squared deviation is calculated using its own relative contribution. Weighted grading means that the greater the contribution of a single dimension, the greater its influence on the overall imbalance. This makes the measurement not only consider the magnitude of the bias but also which dominant dimension the bias occurs in. For random noise that jumps only in a single dimension, its... The distribution will be extremely concentrated, causing the value within the square root to become very large. The calculation result will approach 0, indicating extremely low synergy. Conversely, for a real physical event that causes a joint response across multiple dimensions, its... The distribution will be relatively dispersed, and the values within the square root will be very small. The calculation result will approach 1, indicating that its synergy is very high. Finally, The operation ensured the non-negativity of the score. Finally, Through type amplifier Planned events that match the agricultural operation logs were given the highest weight, while unknown events that could be identified through... Highly coherent mutations (which may include equipment failures) were given higher weights and clearly distinguished from low-coherence random noise.
[0072] Thus, the semantic consistency factor of events was obtained by comprehensively analyzing the distribution of mutation dimensions of agricultural environmental data points and matching them with operation logs.
[0073] Step S4: Obtain the temporal mutation weight factor by fusing and normalizing the initial mutation intensity factor and the event semantic consistency factor of the agricultural environment data points.
[0074] After obtaining the initial mutation intensity factor and the event semantic consistency factor, these two intermediate quantities are further transformed into a temporal mutation weight factor that can be directly used for weighted distance calculation. Specifically, the comprehensive credible mutation intensity is obtained by multiplying and fusing the initial mutation intensity factor and the event semantic consistency factor, and the normalized mutation anomaly index is obtained by normalizing the comprehensive credible mutation intensity by the median within the period. Specifically, the calculation result of multiplying the initial mutation intensity factor and the event semantic consistency factor at the target time is used as the comprehensive credible mutation intensity at the target time. The analysis period and the gain coefficient used to adjust the steepness of the activation function are set. In this embodiment, a fixed time window of 24 hours is set to segment the continuous raw time series data. Each time window corresponds to an analysis period. In this embodiment, the gain coefficient is set to 10, which can be adjusted according to the actual scenario and is not required. The median of the comprehensive credible mutation intensity of all agricultural environment data points in the analysis period at the target time is used as the denominator, and the comprehensive credible mutation intensity at the target time is used as the numerator. The result of subtracting the constant 1 from the resulting fraction and multiplying it by the gain coefficient is used as the normalized mutation anomaly index at the target time.
[0075] In one embodiment, it is assumed that the gain coefficient is ;No. The set of comprehensive credible mutation intensities for all data points within the calculation period at each time point is: Then the first The formula for calculating the normalized mutation anomaly index at time point is:
[0076]
[0077] in, Indicates the first The period in which the _th moment occurs is the _th Normalized mutation anomaly index at each time point; Indicates the gain coefficient; Indicates the first The initial mutation intensity factor at each time point; Indicates the first The semantic consistency factor of events at each moment; Indicates the first The set of comprehensive credible mutation intensities of all data points within the calculation period at a given moment; This indicates that the median of the set is calculated.
[0078] The final temporal mutation weighting factor is obtained by performing nonlinear activation mapping on the normalized mutation anomaly index.
[0079] In one embodiment, the normalized mutation anomaly index is mapped to the weighted gain region by performing a linear transformation using the arctangent function as a nonlinear activation function. The expression for calculating the temporal mutation weighting factor at each time step is:
[0080]
[0081] in, Indicates the first The temporal mutation weighting factor at each time step; Represents pi; Represents the arctangent function; Indicates the first The period in which the _th moment occurs is the _th The normalized mutation anomaly index at each time point.
[0082] It should be noted that, firstly, The initial mutation strength factor from step S2 and the event semantic consistency factor from step S3 are fused. This design ensures that only event points with high intensity and high credibility can obtain a significant initial evaluation value. Any point with shortcomings in intensity or event semantic consistency will have its evaluation value suppressed accordingly. This is the first layer of protection for the weight assignment logic. Secondly, in the denominator... This is the key to the design; it establishes a reference benchmark of a normal level for the data throughout the computation cycle. The median is used instead of the mean because the median is insensitive to a few extremely large evaluation values in the dataset, thus it can very robustly represent the reliable mutation strength level of the vast majority of normal points in the dataset. By using each point... By comparing the result to this robust, conventional benchmark, an absolute evaluation value is transformed into a normalized anomaly score. Finally, the arctangent function is used. As a non-linear activation function, through This linear transformation maps outliers to a weight gain interval. The sigmoid saturation property of the arctan function ensures that for points with outliers far exceeding the norm, the weight gain will saturate to a maximum value, rather than increasing indefinitely. Finally, by adding to a constant 1, the weight... The range of values for is strictly constrained to within The bounded design ensures that key event points receive a sufficiently large weight (approximately twice the weight of regular points) to prevent them from being overwhelmed by brute-force attacks in K-means distance calculations; at the same time, it avoids the problem of algorithm non-convergence caused by excessively large weights, thus ensuring the stability of the entire clustering process.
[0083] Thus, the temporal mutation weight factor was obtained by fusing and normalizing the initial mutation intensity factor and the event semantic consistency factor of agricultural environmental data points.
[0084] Step S5: By introducing weighted K-means clustering with time-series mutation weight factors, representative data centroids are obtained and the data is compressed and stored using the representative data centroids.
[0085] This step utilizes the temporal mutation weight factor calculated in step S4 to complete the clustering of the data point set and achieve high-fidelity storage of key event information. Specifically, the number of clusters is set, and in this embodiment, the number of clusters is set using the elbow method. The Euclidean distance between the agricultural environment data at the target time and any target centroid during the clustering process is obtained. The result of multiplying the temporal mutation weight factor at the target time by the Euclidean distance is used as the optimized distance between the agricultural environment data at the target time and the target centroid during the clustering process. The K-means clustering process is completed using the optimized distance, and the corresponding obtained cluster centroids are used as representative data centroids. The representative data centroids are used as compressed feature representatives of the corresponding analysis period for data storage, thereby completing the high-fidelity, low-storage-occupancy archiving of time-series data containing key agricultural operations and equipment status information.
[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for storing agricultural data in smart villages, characterized in that, The aforementioned agricultural data storage method for smart villages includes: Step S1: Obtain standardized agricultural environmental data and agricultural operation log data by preprocessing and windowing the raw agricultural environmental data; Step S2: Obtain the initial mutation intensity factor of the data points by performing multidimensional minimum distance calculation on the local historical baseline; Step S3: By comprehensively analyzing the distribution of mutation dimensions of agricultural environmental data points and matching them with operation logs, the semantic consistency factor of events is obtained; Step S4: Obtain the temporal mutation weight factor by fusing and normalizing the initial mutation intensity factor and event semantic consistency factor of agricultural environmental data points; Step S5: By introducing weighted K-means clustering with time-series mutation weight factors, representative data centroids are obtained and the data is compressed and stored using the representative data centroids; The method of obtaining the initial mutation intensity factor of data points by performing multidimensional minimum distance calculation on local historical baselines includes: setting a preceding time window; subtracting any target dimension in the agricultural environment data at any target time from the target dimension at any other time in the preceding time window of the target time and squaring the result as the first mutation assessment of the target dimension at the target time; adding the first mutation assessments of all dimensions at the target time and performing square root calculation as the comprehensive mutation assessment of the target time; and using the minimum comprehensive mutation assessment in the preceding time window of the target time as the initial mutation intensity factor of the target time. The process of obtaining an event semantic consistency factor by comprehensively analyzing the mutation dimension distribution of agricultural environmental data points and matching them with operation logs includes: calculating the mutation amount of agricultural environmental data points by dimension to obtain the mutation contribution degree of each dimension; conducting a balance analysis of the relative mutation contribution distribution to obtain a dimension balance index; and matching data points with agricultural operation logs by time to obtain an event type matching score and construct an event semantic consistency factor. The process of obtaining a temporal mutation weight factor by fusing and normalizing the initial mutation intensity factor and the event semantic consistency factor of agricultural environmental data points includes: obtaining a comprehensive credible mutation intensity by multiplying and fusing the initial mutation intensity factor and the event semantic consistency factor; obtaining a normalized mutation anomaly index by normalizing the comprehensive credible mutation intensity to the median within a period; and obtaining the final temporal mutation weight factor by performing a nonlinear activation mapping on the normalized mutation anomaly index.
2. The agricultural data storage method for smart villages according to claim 1, characterized in that, The method involves calculating the mutation amount of agricultural environmental data points across multiple dimensions to obtain the degree of mutation contribution in each dimension, including: For any target time in all agricultural environmental data, the square of the difference between the data of any target dimension at the target time and the data of the target dimension at the previous time is taken as the first difference of the target dimension at the target time; the square of the difference between the data of any target dimension at the target time and the data of the target dimension at the next time is taken as the second difference of the target dimension at the target time; the sum of the first and second differences of the target dimension at the target time is taken as the absolute mutation assessment of the target dimension at the target time; the sum of the absolute mutation assessments of all target dimensions at the target time is taken as the total mutation assessment of the target time; and the absolute mutation assessment of the target dimension at the target time divided by the total mutation assessment of the target time is taken as the mutation contribution degree of the target dimension at the target time.
3. The agricultural data storage method for smart villages according to claim 1, characterized in that, The step involves performing a balance analysis on the distribution of relative mutation contributions to obtain a dimensional balance index, including: The square of the result of subtracting the inverse of the number of dimensions from the mutation contribution of the target dimension at the target time is taken as the relative contribution of the target dimension at the target time; the result of multiplying the mutation contribution of the target dimension at the target time by the relative contribution of the target dimension at the target time is taken as the first dimension balance assessment of the target dimension at the target time; the first dimension balance assessments of all dimensions at the target time are added together and the square root is calculated, and the result of subtracting the constant 1 from the corresponding calculation result is taken as the dimension balance index at the target time.
4. The agricultural data storage method for smart villages according to claim 1, characterized in that, The process of matching data points with agricultural operation logs over time to obtain event type matching scores and constructing an event semantic consistency factor includes: Set a time tolerance window; perform operation log matching on the agricultural environment data in the time tolerance window at the target time. If operation logs exist in the time tolerance window at the target time, set the log matching evaluation to 1; if operation logs do not exist in the time tolerance window at the target time, set the log matching evaluation to 0. The result of adding constant 1 to the log matching evaluation is used as the type amplification factor; the larger value between constant 0 and the dimensionality balance index at the target time is used as the first weight factor evaluation at the target time; the result of multiplying the type amplification factor at the target time with the first weight factor evaluation at the target time is used as the event semantic consistency factor at the target time.
5. The agricultural data storage method for smart villages according to claim 1, characterized in that, The process involves multiplying and fusing the initial mutation intensity factor and the event semantic consistency factor to obtain a comprehensive credible mutation intensity, and then normalizing the comprehensive credible mutation intensity by the median over a period to obtain a normalized mutation anomaly index, including: The result of multiplying the initial mutation intensity factor at the target time by the event semantic consistency factor is taken as the comprehensive credible mutation intensity at the target time. The analysis period and the gain coefficient used to adjust the steepness of the activation function are set. The median of the comprehensive credible mutation intensity of all agricultural environmental data points in the analysis period of the target time is used as the denominator, and the comprehensive credible mutation intensity of the target time is used as the numerator. The result of subtracting the corresponding fraction from the constant 1 and multiplying it by the gain coefficient is used as the normalized mutation anomaly index of the target time.
6. The agricultural data storage method for smart villages according to claim 1, characterized in that, The step of obtaining representative data centroids through weighted K-means clustering by introducing a time-series mutation weight factor and then compressing and storing the data using these representative data centroids includes: Set the number of clusters for clustering; obtain the Euclidean distance between the agricultural environment data at the target time and any target centroid during the clustering process; multiply the temporal mutation weight factor at the target time by the Euclidean distance as the optimized distance between the agricultural environment data at the target time and the target centroid during the clustering process; complete the K-means clustering process using the optimized distance, and use the obtained cluster centroids as representative data centroids; store the representative data centroids as compressed feature representatives for the corresponding analysis period.
7. An agricultural data storage system for smart villages, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement an agricultural data storage method for smart villages according to any one of claims 1-6.