Metadata-based lims platform data cleaning method and system

Abstract

This invention relates to the field of data processing technology, specifically to a data cleaning method and system for a LIMS platform based on metadata, solving the technical problem of low accuracy in anomaly identification during data cleaning in existing technologies. The method includes: acquiring the metadata set for the current period and the metadata set for historical periods from the LIMS platform; for each ecological data indicator in the sample point detection data corresponding to each sampling point, performing spatiotemporal deviation analysis based on the metadata set for the current period and the metadata set for historical periods to determine the local deviation index of the ecological data indicator in the sample point detection data corresponding to the sampling point; performing trend consistency analysis based on the metadata set for historical periods to determine the local change index of the ecological data indicator in the sample point detection data corresponding to the sampling point; and cleaning the metadata set for the current period based on the local deviation index and the local change index using a local outlier detection algorithm.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a data cleaning method and system for a metadata-based LIMS platform. Background Technology

[0002] Metadata, as core data describing data attributes, characteristics, and context, plays a crucial role in the Laboratory Information Management System (LIMS) platform. It encompasses diverse information such as geographical location, sample origin, ecological data type, and specific ecological data. The LIMS platform relies on metadata to achieve core operations such as data visualization, interaction, and modification. The accuracy of metadata directly determines the reliability of data applications on the platform, thus affecting subsequent data analysis, management decisions, and a series of other processes. Therefore, efficient data cleaning of the metadata in the LIMS platform, removing abnormal data and ensuring data quality, is a vital prerequisite for the stable operation of the LIMS platform.

[0003] In existing technologies, data cleaning methods typically include missing data imputation and anomaly identification. Their core objective is to identify and process null values ​​and anomalous deviations, ensuring the cleaned data conforms to a normal distribution pattern. For anomaly identification in LIMS platform ecological environment metadata, the traditional Local Outlier Factor (LOF) algorithm primarily relies on the density difference of nearest neighbor distances to measure data features. However, ecological environment metadata is characterized by multi-source heterogeneity, complex coupling correlations among various ecological data indicators, and significant influence from temporal and spatial variations. Traditional LOF algorithms struggle to effectively identify these spatiotemporal variation patterns, leading to the inaccurate capture of some genuinely anomalous data and even misclassification of normal data as anomalous, failing to meet the high-precision metadata cleaning requirements of the LIMS platform. Summary of the Invention

[0004] To address the problem of low accuracy in anomaly identification during data cleaning in existing technologies, the present invention aims to provide a data cleaning method and system for a metadata-based LIMS platform. The specific technical solution adopted is as follows:

[0005] This application provides a metadata-based data cleaning method for a LIMS platform, including:

[0006] Obtain the metadata set for the current period and the metadata set for the historical period from the LIMS platform; the metadata set includes the sampling point detection data corresponding to multiple sampling points, and the sampling point detection data includes the spatiotemporal information of the corresponding sampling point and various ecological data indicators;

[0007] For each ecological data indicator in the sampling point detection data corresponding to each sampling point, a spatiotemporal deviation analysis is performed based on the metadata set of the current period and the metadata set of the historical period to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point. The local deviation index is used to characterize the degree of numerical deviation of the ecological data indicator in the sampling point detection data corresponding to the sampling point in the spatiotemporal neighborhood.

[0008] For each ecological data indicator in the sampling point detection data corresponding to each sampling point, a trend consistency analysis is performed based on the metadata set of historical periods to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; the local change index is used to characterize the consistency of the change trend of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0009] Based on the local deviation index and the local change index, the metadata set of the current period is cleaned by the local outlier detection algorithm.

[0010] This application provides a metadata-based LIMS platform data cleaning system, including:

[0011] The data acquisition module is used to acquire the metadata set of the current period and the metadata set of the historical period in the LIMS platform; the metadata set includes the sampling point detection data corresponding to multiple sampling points, and the sampling point detection data includes the spatiotemporal information of the corresponding sampling point and a variety of ecological data indicators;

[0012] The spatiotemporal deviation analysis module is used to perform spatiotemporal deviation analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the metadata set of the current period and the metadata set of the historical period, to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; the local deviation index is used to characterize the degree of numerical deviation of the ecological data indicator in the sampling point detection data corresponding to the sampling point in the spatiotemporal neighborhood.

[0013] The trend consistency analysis module is used to perform trend consistency analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the metadata set of historical periods, to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; the local change index is used to characterize the consistency of the change trend of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0014] The data cleaning module is used to clean the metadata set of the current period based on the local deviation index and the local change index, using a local outlier detection algorithm.

[0015] The present invention has the following beneficial effects:

[0016] In view of the technical problem of low accuracy in anomaly identification during data cleaning in existing technologies, this application provides a data cleaning method and system for LIMS platforms based on metadata. By acquiring the metadata set of the current period and the metadata set of the historical period in the LIMS platform, and performing spatiotemporal deviation analysis and trend consistency analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, local deviation index and local change index are obtained respectively. Based on the above local deviation index and local change index, the anomaly identification logic of the local outlier detection algorithm is optimized. It takes into account both the spatiotemporal correlation of ecological data and the coupling correlation between data indicators, effectively solving the problem that the traditional LOF algorithm is difficult to identify spatiotemporal change patterns and coupled correlation data anomalies, significantly improving the accuracy of metadata cleaning of LIMS platforms, and providing reliable data support for subsequent data display, management and analysis. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A system architecture diagram of a metadata-based LIMS platform data cleaning system provided in one embodiment of the present invention;

[0019] Figure 2 This is a flowchart illustrating a metadata-based data cleaning method for a LIMS platform, as provided in one embodiment of the present invention. Detailed Implementation

[0020] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the metadata-based LIMS platform data cleaning method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] In all division and logarithmic operations covered in this application, a smoothing mechanism is employed to prevent computer program crashes or invalid values ​​from being generated due to a zero denominator or a zero input. Specifically, a positive correction factor is superimposed on the denominator term of the division operation or the argument term of the logarithmic function. For example, the value is This ensures the robustness and feasibility of the algorithm under extreme conditions.

[0023] The normalization function mentioned in this application Unless otherwise specified, all values ​​are normalized using maximum and minimum values. The maximum and minimum values ​​are preset empirical extreme values ​​derived from a large amount of historical experimental data. If the calculated result exceeds the [0,1] interval, it is restricted to the [0,1] range by a truncation function (i.e., if the result is less than 0, it is taken as 0, and if it is greater than 1, it is taken as 1) to eliminate the influence of outliers on the evaluation index.

[0024] In view of the technical problem of low accuracy in anomaly identification during data cleaning in existing technologies, this application provides a data cleaning method and system for LIMS platforms based on metadata. By acquiring the metadata set of the current period and the metadata set of the historical period in the LIMS platform, and performing spatiotemporal deviation analysis and trend consistency analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, local deviation index and local change index are obtained respectively. Based on the above local deviation index and local change index, the anomaly identification logic of the local outlier detection algorithm is optimized. It takes into account both the spatiotemporal correlation of ecological data and the coupling correlation between data indicators, effectively solving the problem that the traditional LOF algorithm is difficult to identify spatiotemporal change patterns and coupled correlation data anomalies, significantly improving the accuracy of metadata cleaning of LIMS platforms, and providing reliable data support for subsequent data display, management and analysis.

[0025] The specific solution of the metadata-based LIMS platform data cleaning method and system provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Please see Figure 1 This diagram illustrates the system architecture of a metadata-based LIMS platform data cleaning system according to an embodiment of the present invention. The metadata-based LIMS platform data cleaning system 10 includes: a data acquisition module 11, a spatiotemporal deviation analysis module 12, a trend consistency analysis module 13, and a data cleaning module 14. The modules communicate bidirectionally via communication links to ensure real-time interaction of data and analysis results. The communication links can employ wired or wireless transmission methods to meet the needs of different deployment scenarios.

[0027] The data acquisition module 11 is used to acquire the current period's metadata set and the historical period's metadata set in the LIMS platform.

[0028] The metadata set includes the sampling point detection data corresponding to multiple sampling points. The sampling point detection data includes the spatiotemporal information of the corresponding sampling points and various ecological data indicators.

[0029] For example, the data acquisition module 11 can connect to the metadata storage database through the database interface of the LIMS platform to periodically collect raw metadata for the current period and cleaned metadata for historical periods. The collection cycle can be set according to monitoring needs; for example, it can be set to collect once per hour in real-time monitoring scenarios and once per day in offline analysis scenarios.

[0030] In some embodiments, the data acquisition module 11 can also normalize the collected metadata to eliminate the impact of dimensional differences on subsequent analysis. Simultaneously, it performs format and integrity checks on the collected metadata, discarding data with format errors or missing key information (such as missing spatiotemporal information) to ensure the quality of data input to subsequent modules.

[0031] The spatiotemporal deviation analysis module 12 is used to perform spatiotemporal deviation analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the metadata set of the current period and the metadata set of the historical period, to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0032] Among them, the local deviation index is used to characterize the degree of deviation of the ecological data indicators in the sampling point detection data corresponding to the sampling point in the spatiotemporal neighborhood.

[0033] For example, the spatiotemporal deviation analysis module 12 can extract the spatial neighborhood set and the temporal neighborhood set from the metadata set based on the spatiotemporal information of the sampling points, and then calculate the spatial deviation component and the temporal deviation component respectively, and fuse them to obtain the local deviation index.

[0034] The spatiotemporal deviation analysis module 12 can be implemented using a hardware acceleration chip to efficiently perform time-consuming operations such as neighborhood search and weighted calculation, ensuring high processing speed even in scenarios with large amounts of metadata. Simultaneously, the spatiotemporal deviation analysis module 12 also supports parameter configuration, allowing implementers to adjust relevant configuration parameters through the configuration interface to adapt to different monitoring scenarios.

[0035] The trend consistency analysis module 13 is used to perform trend consistency analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the metadata set of historical periods, to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0036] Among them, the local change index is used to characterize the consistency of the changing trends of ecological data indicators in the sampling point detection data corresponding to the sampling point.

[0037] The trend consistency analysis module 13 can perform correlation analysis on various ecological data indicators in historical periods to determine the consistency of the changing trends of the various ecological data indicators, thereby determining the local change index. For example, the trend consistency analysis module 13 supports batch processing, which can simultaneously perform trend consistency analysis on multiple ecological data indicators, improving processing efficiency.

[0038] The data cleaning module 14 is used to clean the metadata set of the current period based on the local deviation index and the local change index, using the local outlier detection algorithm.

[0039] The data cleaning module 14 can calculate the initial local outlier factor and the outlier factor correction coefficient, and obtain the corrected local outlier factor. Based on the corrected local outlier factor, anomaly detection is performed, and the outlier data is interpolated and corrected. For example, the data cleaning module 14 has result storage and feedback functions. It can store the cleaned metadata in the target database of the LIMS platform and generate a data cleaning report. The data cleaning report includes information such as the quantity, location, and values ​​before and after correction of the outlier data, facilitating traceability and analysis by the implementer.

[0040] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.

[0041] Please see Figure 2 The diagram illustrates a flowchart of a metadata-based data cleaning method for a LIMS platform according to an embodiment of the present invention. The method includes the following steps:

[0042] Step 201: Obtain the current metadata set and the historical metadata set in the LIMS platform.

[0043] The metadata set includes the sampling point detection data corresponding to multiple sampling points. The sampling point detection data includes the spatiotemporal information of the corresponding sampling points and various ecological data indicators.

[0044] The aforementioned spatiotemporal information includes the spatial coordinates of the sampling point (such as latitude and longitude) and the sampling time (e.g., the unit can be hours). Ecological data indicators can be determined according to the monitoring scenario. For example, if the monitoring scenario is water environment monitoring, the ecological data indicators include parameters that reflect the ecological state of the water environment, such as water temperature, water level, dissolved oxygen concentration, chlorophyll content, and pH value. If the monitoring scenario is soil environment monitoring, indicators such as soil moisture, soil pH value, and heavy metal content may be included. That is, the ecological data indicators in this application embodiment are specifically numerical ecological data indicators.

[0045] In some embodiments, the metadata set for the current period is the raw monitoring data collected by each sampling point within the current sampling period (e.g., today), and the metadata set for the historical period is the data that has been cleaned within a preset number of days in the past (to ensure the reliability of historical data). The range of the preset number of days can be adjusted by the implementer according to the ecological stability of the monitoring scenario. For example, a value of 10 days can be used for scenarios with rapid changes such as the water environment, and a value of 30 days can be used for stable scenarios such as the soil environment.

[0046] To avoid the impact of differences in the dimensions of different ecological data indicators on subsequent calculations, this application can normalize the acquired metadata, such as normalizing the maximum and minimum values.

[0047] Step 202: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, perform spatiotemporal deviation analysis based on the metadata set of the current period and the metadata set of the historical period to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0048] Among them, the local deviation index is used to characterize the degree of deviation of the ecological data indicators in the sampling point detection data corresponding to the sampling point in the spatiotemporal neighborhood.

[0049] It should be noted that the ecological and environmental data in the LIMS platform contains a wide variety of data types, and some ecological data indicators have significant spatiotemporal correlations. For example, water quality indicators at adjacent sampling points within the same river basin can influence each other due to the hydrological cycle, and ecological indicators at the same sampling point can show continuous changes with the seasons and time periods. These changes are not isolated abrupt changes; spatially, these influences result from similar changes in nearby river sections due to the hydrological cycle; temporally, these changes are continuous and slow, evolving over time, and there are no sudden abrupt changes in data indicators.

[0050] However, traditional LOF algorithms are prone to misclassifying normal data as anomalous due to fluctuations in these data, or they may fail to identify data anomalies caused by instrument drift. Therefore, this application uses spatiotemporal deviation analysis to quantify the differences between each data point and its neighboring data, generating a local deviation index to preliminarily determine the likelihood of data anomalies.

[0051] Step 203: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, perform trend consistency analysis based on the metadata set of historical periods to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0052] Among them, the local change index is used to characterize the consistency of the changing trends of ecological data indicators in the sampling point detection data corresponding to the sampling point.

[0053] It should be noted that, in the ecological and environmental data of the LIMS platform, in addition to significant spatiotemporal correlations, some ecological data indicators also exhibit complex coupling correlations. For example, dissolved oxygen concentration usually decreases when water temperature rises, while dissolved oxygen concentration may increase when chlorophyll content increases.

[0054] Therefore, this application can further verify the rationality of the data by comparing whether the changes of relevant ecological data indicators remain correlated over a period of time and quantifying the trend of change to obtain a local change index, thereby avoiding misjudgment caused by single-dimensional analysis.

[0055] Step 204: Based on the local deviation index and the local change index, perform data cleaning on the metadata set of the current period using the local outlier detection algorithm.

[0056] In this application, the local deviation index and local change index obtained through the above steps can comprehensively characterize the abnormal features of the data from two dimensions: spatiotemporal neighborhood differences and consistency of change trends. These features are then integrated into the local outlier detection algorithm, thereby correcting the limitation of the traditional local outlier detection algorithm that only relies on density differences and further improving the accuracy of abnormal data identification.

[0057] For example, this application can optimize the traditional local outlier detection algorithm based on the local deviation index and the local change index, thereby detecting whether there is abnormal data in the metadata set of the current period. If there is abnormal data, the abnormal data is processed by interpolation and other methods to finally obtain the cleaned metadata set of the current period.

[0058] Based on the above technical solution, this application obtains the metadata set of the current period and the metadata set of the historical period in the LIMS platform, and performs spatiotemporal deviation analysis and trend consistency analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, respectively obtaining the local deviation index and the local change index. Based on the above local deviation index and local change index, the anomaly identification logic of the local outlier detection algorithm is optimized. It takes into account both the spatiotemporal correlation of ecological data and the coupling correlation between data indicators, effectively solving the problem that traditional local outlier detection algorithms are difficult to identify spatiotemporal change patterns and coupling correlation data anomalies. It significantly improves the accuracy of metadata cleaning of the LIMS platform and provides reliable data support for subsequent data display, management and analysis.

[0059] As a possible embodiment of this application, step 202 above can be implemented through the following steps:

[0060] Step 301: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the spatiotemporal information of the sampling point, extract the spatial neighborhood set and temporal neighborhood set of the sampling point from the metadata set of the current period and the metadata set of the historical period, respectively.

[0061] Among them, spatial neighborhood set Extraction is based on the spatial coordinates of the sampling points, defined as the extraction of the target sampling points. Centered on the spatial coordinates, with a radius of The set of all other sampling points within a certain range of meters, that is, for a given sampling point ,satisfy Then the sampling point Add target sampling points Spatial neighborhood set Among them For target sampling points spatial coordinates, Sampling points spatial coordinates, The Haversine formula is used to convert the latitude and longitude coordinates of two points into a straight-line distance in space, with radius... The value range can be adjusted according to the distribution density of the sampling points. For example, a value of 150 meters can be used in densely populated areas and a value of 500 meters can be used in sparsely populated areas.

[0062] Temporal Neighborhood Set The extraction is based on the sampling time of the sampling point, defined as the target sampling point. In the metadata set of historical periods, all metadata whose time interval between the current sampling time and the sampling time meets preset conditions. For example, past data can be selected. Historical data within the day ensures the timeliness and comparability of data in the time neighborhood, avoiding the misjudgment of normal fluctuations caused by time period differences as abnormalities.

[0063] Step 302: Calculate the spatial deviation component of the ecological data indicators in the sampling point detection data corresponding to the sampling point based on the spatial neighborhood set.

[0064] Among them, the spatial deviation component is used to quantify the degree of difference between the ecological data indicators of the corresponding sampling point and the ecological data indicators of other sampling points in the spatial neighborhood set.

[0065] In one possible implementation, this application can calculate the spatial deviation component of the ecological data indicator in the sampling point detection data corresponding to the sampling point based on the spatial distance between each neighboring sampling point in the spatial neighborhood set and the difference between each neighboring sampling point in the spatial neighborhood set and the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0066] In some embodiments, considering that the closer other sampling points in the spatial neighborhood set are to the sampling point in space, the stronger the correlation between their ecological data indicators and the sampling point, this application can use a weighted average method to calculate the difference value, wherein the spatial attenuation weight in the weighted calculation can be determined based on the spatial distance between the two sampling points.

[0067] For example, the spatial decay weight satisfies the following formula:

[0068]

[0069] in, Sampling points Sampling points within the corresponding spatial neighborhood set Spatial decay weights between them It is a dimensionless attenuation intensity coefficient used to control the influence of the spatial distance between two sampling points on the weight. Its value can be set by the implementer according to the implementation scenario, for example, the value range can be between 1 and 10. Sampling points Sampling points within the corresponding spatial neighborhood set Spatial distance between them The radius of the spatial neighborhood set. It is an exponential function used to quantify spatial distance as a spatial decay weight.

[0070] For example, the spatial deviation component satisfies the following formula:

[0071]

[0072] in, Sampling points The first of the corresponding sampling point detection data Spatial deviation components of various ecological data indicators Sampling points The corresponding spatial neighborhood set, Sampling points Sampling points within the corresponding spatial neighborhood set Spatial decay weights between them Sampling points The first of the corresponding sampling point detection data The sampled values ​​of various ecological data indicators (the sampled values ​​of ecological data indicators have been normalized in step 201 above). Sampling points The first of the corresponding sampling point detection data Sampling values ​​of various ecological data indicators. It is a safety parameter used to correct denominators of 0 to avoid denominators of 0. Its value can be set by the implementer according to the implementation scenario, such as 0.001.

[0073] Characterize the original differences between the target indicator and neighboring indicators. The difference is weighted according to spatial distance; the closer the spatial distance, the greater the weight, and the higher the contribution of the difference to the spatial deviation component. The weighted total difference value is calculated by dividing by the sum of the weights corresponding to the sampling points in the spatial neighborhood set. Then, the weighted average difference value is obtained, which is the spatial deviation component. The spatial deviation component is positively correlated with the degree of spatial anomaly of the target indicator; the larger the value, the more significant the difference between the target indicator and the spatial neighborhood data.

[0074] Step 303: Calculate the time deviation component of the ecological data indicators in the sampling point detection data corresponding to the sampling point based on the time neighborhood set.

[0075] Among them, the time deviation component is used to quantify the degree of difference between the ecological data indicators of the corresponding sampling point and the ecological data indicators of the historical data in the time neighborhood set.

[0076] In one possible implementation, this application can calculate the time deviation component of the ecological data indicators in the sampling point detection data corresponding to the sampling point based on the time interval between each historical sampling time in the time neighborhood set and the current sampling time of the sampling point, as well as the difference between the ecological data indicators in the sampling point detection data corresponding to each historical sampling time in the time neighborhood set and the current sampling time of the sampling point.

[0077] In some embodiments, the closer the historical sampling time is to the current time, the higher the reference value of the data. Therefore, this application can also use a weighted average method to calculate the difference value. The time decay weight in the weighted calculation can be determined based on the time interval between the historical sampling time and the current time of the sampling point.

[0078] For example, the time decay weight satisfies the following formula:

[0079]

[0080] in, Sampling points The corresponding spatial neighborhood set of the first Historical sampling time and sampling point The time decay weight between the current sampling times, It is a dimensionless attenuation intensity coefficient used to control the impact of the time interval between the historical sampling time and the current sampling time on the weight. Its value can be set by the implementer according to the implementation scenario, for example, the value range can be between 1 and 10. Sampling points The current sampling time, Sampling points The corresponding spatial neighborhood set of the first Each historical sampling time, The time range of the spatial neighborhood set is defined as follows: the sampling time is in hours, and the time range of the spatial neighborhood set is in days. Therefore, through... The time interval can be normalized. It is an exponential function used to quantize time intervals as time decay weights.

[0081] For example, the time deviation component satisfies the following formula:

[0082]

[0083] in, Sampling points The first of the corresponding sampling point detection data The time deviation component of various ecological data indicators Sampling points The corresponding temporal neighborhood set, Sampling points The corresponding spatial neighborhood set of the first Historical sampling time and sampling point The time decay weight between the current sampling times, Sampling points The first sampling point in the detection data corresponding to the current sampling time Sampling values ​​of various ecological data indicators, Sampling points The corresponding time neighborhood set The first of the sampling point detection data corresponding to the historical sampling time. Sampling values ​​of various ecological data indicators.

[0084] Characterize the original difference between the target indicator and historical indicators. The difference is weighted according to the time interval; the shorter the time interval, the greater the weight, and the greater the contribution of the difference to the time deviation component. The total difference value after weighting is calculated by dividing by the sum of the weights corresponding to the historical sampling times. Then, the weighted average difference value is obtained, which is the time deviation component. The time deviation component is positively correlated with the degree of time anomaly of the target indicator; the larger the value, the more significant the difference between the target indicator and historical data.

[0085] Step 304: Based on the spatial deviation component and the temporal deviation component, determine the local deviation index of the ecological data indicators in the sampling point detection data corresponding to the sampling point.

[0086] For example, the local deviation index satisfies the following formula:

[0087]

[0088] in, Sampling points The first of the corresponding sampling point detection data Local deviation index of various ecological data indicators. Sampling points The first of the corresponding sampling point detection data Spatial deviation components of various ecological data indicators Sampling points The first of the corresponding sampling point detection data This application uses an arithmetic mean to fuse two components of ecological data indicators to obtain a local deviation index. This local deviation index comprehensively reflects the degree of deviation of the target data in both spatial and temporal dimensions; the larger the value, the more likely the data is to be abnormal.

[0089] Based on the above technical solution, this application uses the spatiotemporal information of sampling points as a benchmark to accurately extract the spatial neighborhood set and the temporal neighborhood set, thereby calculating the spatial deviation component and the temporal neighborhood component respectively, and then fusing them to obtain the local deviation index. This enables deviation analysis to specifically characterize the differences in the spatiotemporal dimensions, further improving the reliability of the local deviation index and providing more accurate basic data for subsequent anomaly identification.

[0090] As a possible embodiment of this application, step 203 above can be implemented through the following steps:

[0091] Step 401: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, perform correlation analysis on the ecological data indicators in the sampling point detection data corresponding to the sampling point in the metadata set of the historical period with other ecological data indicators, and determine the set of positively correlated indicators and the set of negatively correlated indicators in the sampling point detection data corresponding to the sampling point.

[0092] For example, this application can perform correlation analysis using the Pearson correlation coefficient. First, for each sampling point, the correlation coefficient between the sampling point and the historical data set is extracted. All ecological data indicators within the day ( The value range is 1 to Adjustments are made based on ecological environment stability (e.g., water environment data is taken for 2 days, soil environment data for 10 days), and the difference sequence of each ecological data indicator at the sampling point is calculated (based on the corresponding ecological data indicator in continuous...). The difference between sampled values ​​at adjacent historical times within a day constitutes the trend of change in the difference between sampled values ​​at adjacent historical times.

[0093] Then, for each ecological data indicator, the Pearson correlation coefficient between the difference sequence of the ecological data indicator and the difference sequences of each other ecological data indicator is calculated, and a set of positively correlated indicators and a set of negatively correlated indicators are constructed based on the Pearson correlation coefficient.

[0094] For example, this application can make the Pearson correlation coefficient greater than a preset correlation threshold. Ecological data indicators are added to the positively correlated dataset of this ecological data indicator, and Pearson correlation coefficients less than a preset correlation threshold are selected. The ecological data indicators are added to the negative correlation data set of the ecological data indicators.

[0095] Preset relevant thresholds The threshold value can be set by the implementer based on the data type of the ecological indicators in the implementation scenario. If the changes between data in the ecological environment have a significant impact, the threshold can be appropriately increased. If there are changes between data in the ecological environment but the impact is not obvious, the threshold can be appropriately decreased. The value range can be between 0 and 0.5, for example, 0.3.

[0096] Step 402: Calculate the positive correlation change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point based on the set of positive correlation indicators.

[0097] Among them, the positive correlation change component is used to quantify the consistency between the change trend of the ecological data indicator and the change trend of the positively correlated ecological data indicators in the corresponding positively correlated indicator set. If the change trend of the ecological data indicator is consistent with that of each positively correlated ecological data indicator (e.g., rising or falling at the same time), the value of the positive correlation change component is large, indicating that the data change is reasonable; otherwise, the value is small, suggesting that the data may be abnormal.

[0098] In one possible implementation, this application can determine the first numerical sequence of ecological data indicators in the sampling point detection data corresponding to the sampling point at multiple consecutive historical sampling times based on the metadata set of historical periods.

[0099] For example, this application can determine the difference sequence of the ecological data indicator based on the metadata set of historical periods, and perform rounding on the difference sequence of the ecological data indicator. In order to eliminate the influence of random fluctuations such as sensor white noise on trend analysis, a change threshold can be set. If the difference is greater than If the difference is less than 1, then set the corresponding element to 1; if the difference is less than 1, then set the corresponding element to 1. If the difference is greater than or equal to 1, then the corresponding element is set to -1; if the difference is greater than or equal to 1, then the corresponding element is set to -1. and less than or equal to If the corresponding element is set to 0, then the first numerical sequence is obtained. This first numerical sequence is used to characterize the direction of change of ecological data indicators over multiple consecutive historical sampling times. The change threshold is... The settings can be configured based on the historical fluctuation characteristics of specific ecological data indicators and the accuracy of sensor measurements. For example, it can be set to the median of the absolute value of the historical difference sequence, a certain percentage of the mean (such as 10%), or an empirical fixed value based on measurement error.

[0100] Furthermore, for each positively correlated ecological data indicator in the set of positively correlated indicators, a second numerical sequence of the positively correlated ecological data indicator at multiple consecutive historical sampling times is determined based on the metadata set of historical periods.

[0101] The calculation method of the second numerical sequence is the same as that of the first numerical sequence. This application can determine the difference sequence of each positively correlated ecological data indicator in the set of positively correlated indicators based on the metadata set of historical periods, and perform rounding on the difference sequence of the ecological data indicator (refer to the above method) to obtain the second numerical sequence. The second numerical sequence is used to characterize the change direction of the positively correlated ecological data indicator in multiple consecutive historical sampling times.

[0102] In this embodiment of the application, before constructing the numerical sequence, the multi-source data can be resampled or interpolated and aligned based on the same time granularity (e.g., by hour) to ensure that the length of the first numerical sequence is consistent with that of the second numerical sequence.

[0103] Thus, this application can calculate the positive correlation change components of ecological data indicators in the sampling point detection data corresponding to the sampling point based on the first numerical sequence and the second numerical sequence.

[0104] For example, the positively correlated change component satisfies the following formula:

[0105]

[0106] in, Sampling points The first of the corresponding sampling point detection data The positive correlation components of the changes in various ecological data indicators. Sampling points The first of the corresponding sampling point detection data A set of positively correlated indicators for various ecological data indicators. Sampling points The first of the corresponding sampling point detection data The number of positively correlated ecological data indicators in the set of positively correlated ecological data indicators. Sampling points The first of the corresponding sampling point detection data The first numerical sequence of various ecological data indicators across multiple consecutive historical sampling periods. For the first numerical sequence, the first... One element, Sampling points The first of the corresponding sampling point detection data The set of positively correlated indicators of various ecological data indicators The second numerical sequence of a positively correlated ecological data indicator over multiple consecutive historical sampling periods. For the second numerical sequence, the first One element, The number of elements in the first (or second) numerical sequence.

[0107] Characterizing the first Ecological data indicators and their corresponding first The degree of consistency in the numerical changes of positively correlated ecological data indicators over multiple consecutive historical sampling periods. Characterizing the first The degree of consistency in the numerical changes of various positively correlated ecological data indicators within the set of ecological data indicators and their corresponding positively correlated indicators. The larger the value, the stronger the consistency.

[0108] Step 403: Calculate the negative correlation change components of ecological data indicators in the sampling point detection data corresponding to the sampling point based on the negative correlation index set.

[0109] Among them, the negative correlation change component is used to quantify the consistency between the change trend of ecological data indicators and the change trend of negatively correlated ecological data indicators in the negative correlation indicator set. If the change trends of ecological data indicators and negatively correlated ecological data indicators are opposite (for example, the target indicator rises and the negatively correlated indicator falls), the absolute value of the negative correlation change component is large, indicating that the data change is reasonable; otherwise, the absolute value of the value is small, suggesting that the data may be abnormal.

[0110] In one possible implementation, this application can determine the first numerical sequence of ecological data indicators in the sampling point detection data corresponding to the sampling point at multiple consecutive historical sampling times based on the metadata set of historical periods. At the same time, for each negatively correlated ecological data indicator in the negatively correlated indicator set, a third numerical sequence of the negatively correlated ecological data indicator at multiple consecutive historical sampling times is determined based on the metadata set of historical periods.

[0111] The first numerical sequence is used to characterize the direction of change of ecological data indicators over multiple consecutive historical sampling times, and the third numerical sequence is used to characterize the direction of change of negatively correlated ecological data indicators over multiple consecutive historical sampling times. The calculation method can also refer to the above description, and will not be repeated here.

[0112] Thus, this application can calculate the negative correlation change component of the ecological data indicators in the sampling point detection data corresponding to the sampling point based on the first numerical sequence and the third numerical sequence.

[0113] For example, the negatively correlated change component satisfies the following formula:

[0114]

[0115]

[0116] in, Sampling points The first of the corresponding sampling point detection data The negative correlation components of the ecological data indicators, Sampling points The first of the corresponding sampling point detection data A set of negatively correlated indicators for various ecological data indicators. Sampling points The first of the corresponding sampling point detection data The number of negatively correlated ecological data indicators in the set of negatively correlated ecological data indicators. Sampling points The first of the corresponding sampling point detection data The first numerical sequence of various ecological data indicators across multiple consecutive historical sampling periods. For the first numerical sequence, the first... One element, Sampling points The first of the corresponding sampling point detection data The set of negatively correlated indicators of various ecological data indicators The third numerical sequence of negatively correlated ecological data indicators across multiple consecutive historical sampling periods. The third numerical sequence One element, The number of elements in the first (or third) numerical sequence.

[0117] Characterizing the first Ecological data indicators and their corresponding first The degree of consistency in the numerical changes of negatively correlated ecological data indicators across multiple consecutive historical sampling periods. Characterizing the first The degree of consistency in the numerical changes of various negatively correlated ecological data indicators within the set of ecological data indicators and their corresponding negatively correlated indicators. The absolute value and the first The degree of difference in the numerical changes of various negatively correlated ecological data indicators and their corresponding negatively correlated indicator sets is positively correlated. The larger the absolute value, the greater the degree of difference (i.e., the more opposite the direction of change).

[0118] Step 404: Based on the positive and negative correlation change components, determine the local change index of the ecological data indicators in the sampling point detection data corresponding to the sampling point.

[0119] For example, the local change index satisfies the following formula:

[0120]

[0121] in, Sampling points The first of the corresponding sampling point detection data Local change index of various ecological data indicators Sampling points The first of the corresponding sampling point detection data The positive correlation components of the changes in various ecological data indicators. Sampling points The first of the corresponding sampling point detection data The negative correlation components of the ecological data indicators, and The values ​​of all are in the range of 0 to 2, therefore, dividing by 4 will give us... Normalized to between 0 and 1, the larger the local change index, the stronger the consistency between the changing trend of the ecological data indicators and the relevant indicators, and the more likely the data is to be normal; the smaller the local change index, the weaker the consistency of the changing trend, and the more likely the data is to be abnormal.

[0122] Based on the above technical solution, this application identifies the positive and negative correlation index sets of each ecological data indicator through correlation analysis, calculates the positive and negative correlation change components respectively, and then integrates them to obtain the local change index. This can accurately assess the rationality of the data change trend, make full use of the coupling correlation between ecological data indicators, avoid misjudgment caused by single indicator analysis, complement the local deviation index, and further improve the comprehensiveness and accuracy of anomaly identification.

[0123] As a possible embodiment of this application, step 204 above can be implemented through the following steps:

[0124] Step 501: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, determine the local outlier factor of the ecological data indicator in the sampling point detection data corresponding to the sampling point based on the local deviation index and local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

[0125] In one possible implementation, this application can determine the initial local outlier factor of each ecological data indicator in the sampling point detection data corresponding to each sampling point, and determine the outlier factor correction coefficient based on the local deviation index and local change index of the ecological data indicator in the sampling point detection data corresponding to each sampling point.

[0126] For example, the initial local outlier factor can be calculated using a traditional local outlier detection algorithm. First, for each sampling point... Determine its The nearest neighbor sampling points ( The value range can be divided according to the number of sampling points (e.g., 10), forming a local neighborhood. Then, by comparing this... Calculate the difference between the corresponding ecological data indicators of the sampling point and its neighboring sampling points. The local reachability density of the corresponding ecological data indicators is used to quantify the sampling point. The degree of closeness between the corresponding ecological data indicators and their local neighborhood is then determined. This local reachability density is then compared with the average local reachability density of all its neighboring sampling points to obtain the initial local outlier factor.

[0127] Subsequently, the initial local outlier factors of the ecological data indicators in the sampling point detection data corresponding to the sampling point are corrected according to the outlier factor correction coefficient, so as to obtain the local outlier factors of the ecological data indicators in the sampling point detection data corresponding to the sampling point.

[0128] It should be noted that the above technical solution in this application takes a single sampling of various ecological data indicators in the sampling point detection data corresponding to each sampling point in the metadata set of the current period (such as the original monitoring data collected by each sampling point on the same day) as an example to illustrate the data cleaning scheme. If multiple samplings are carried out in the current period, the above data cleaning scheme can be carried out separately according to the original monitoring data collected by each sampling point obtained in each sampling.

[0129] For example, the local outlier factor satisfies the following formula:

[0130]

[0131] in, Sampling points The first of the corresponding sampling point detection data Sampling values ​​of various ecological data indicators, Sampling points The first of the corresponding sampling point detection data Local outlier factors in the sampled values ​​of various ecological data indicators. Sampling points The first of the corresponding sampling point detection data The initial local outlier factor of the sampled values ​​of various ecological data indicators. Sampling points The first of the corresponding sampling point detection data Local deviation index of various ecological data indicators. Sampling points The first of the corresponding sampling point detection data Local change index of various ecological data indicators It is a safety parameter used to correct denominators of 0 to avoid denominators of 0. Its value can be set by the implementer according to the implementation scenario, such as 0.001. The purpose is to use the hyperbolic tangent function. The weights are normalized to reflect the weights of local outliers.

[0132] in, It is positively correlated with the degree of data anomaly. It is negatively correlated with the degree of data anomaly. Based on spatiotemporal deviation and trend consistency, the anomaly weights are comprehensively evaluated by integrating the indices of the two dimensions. This provides a precise basis for correcting initial local outlier factors and avoids misjudgments caused by traditional local outlier detection algorithms that rely solely on density differences.

[0133] Step 502: Using the local outlier detection algorithm, anomaly detection is performed on the local outlier factors of each ecological data indicator in the sampling data corresponding to each sampling point, and the metadata set of the current period is cleaned based on the anomaly detection results.

[0134] This application constructs outlier correction coefficients using local deviation and local change indices to correct the initial local outlier factors in traditional local outlier detection algorithms, resulting in a more accurate comprehensive anomaly index. This effectively improves the distinction between abnormal and normal data. After detecting abnormal data in this way, this application can perform data cleaning by using spatial neighborhood weighted interpolation, ensuring the rationality and continuity of the cleaned data. This solves the problem of inaccurate identification of ecological metadata anomalies in traditional local outlier detection algorithms, further improving the overall quality of metadata cleaning on the LIMS platform.

[0135] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A data cleaning method for a LIMS platform based on metadata, characterized in that, include: Obtain the current metadata set and the historical metadata set from the LIMS platform; The metadata set includes sampling point detection data corresponding to multiple sampling points, and the sampling point detection data includes spatiotemporal information of the corresponding sampling points and various ecological data indicators. For each ecological data indicator in the sampling point detection data corresponding to each sampling point, a spatiotemporal deviation analysis is performed based on the metadata set of the current period and the metadata set of the historical period to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point. The local deviation index is used to characterize the degree of numerical deviation of the ecological data indicators in the sampling point detection data corresponding to the sampling point in the spatiotemporal neighborhood. For each ecological data indicator in the sampling point detection data corresponding to each sampling point, a trend consistency analysis is performed based on the metadata set of the historical period to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; the local change index is used to characterize the consistency of the change trend of the ecological data indicator in the sampling point detection data corresponding to the sampling point. Based on the local deviation index and the local change index, the metadata set of the current period is cleaned using a local outlier detection algorithm. Specifically, for each ecological data indicator in the sampling point detection data corresponding to each sampling point, spatiotemporal deviation analysis is performed based on the metadata set of the current period and the metadata set of historical periods to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point, including: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the spatiotemporal information of the sampling point, the spatial neighborhood set and temporal neighborhood set of the sampling point are extracted from the metadata set of the current period and the metadata set of the historical period, respectively. Based on the spatial neighborhood set, calculate the spatial deviation component of the ecological data indicator in the sampling point detection data corresponding to the sampling point; The time deviation component of the ecological data indicator in the sampling point detection data corresponding to the sampling point is calculated based on the time neighborhood set. Based on the spatial deviation component and the temporal deviation component, determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; Specifically, for each ecological data indicator in the sampling point detection data corresponding to each sampling point, trend consistency analysis is performed based on the metadata set of the historical period to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point, including: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, a correlation analysis is performed on the ecological data indicator in the sampling point detection data corresponding to the sampling point in the metadata set of the historical period and other ecological data indicators to determine the set of positively correlated indicators and the set of negatively correlated indicators in the sampling point detection data corresponding to the sampling point. Based on the set of positively correlated indicators, calculate the positively correlated change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point; Based on the set of negatively correlated indicators, calculate the negatively correlated change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point; Based on the positive and negative correlation change components, the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined.

2. The metadata-based LIMS platform data cleaning method according to claim 1, characterized in that, Based on the spatial neighborhood set, the spatial deviation components of the ecological data indicators in the sampling point detection data corresponding to the sampling point are calculated, including: Based on the spatial distance between each neighboring sampling point in the spatial neighborhood set and the sampling point, and the difference between each neighboring sampling point in the spatial neighborhood set and the ecological data index in the sampling point detection data corresponding to the sampling point, the spatial deviation component of the ecological data index in the sampling point detection data corresponding to the sampling point is calculated.

3. The metadata-based LIMS platform data cleaning method according to claim 1, characterized in that, The temporal deviation component of the ecological data indicator in the sampling point detection data corresponding to the sampling point is calculated based on the temporal neighborhood set, including: Based on the time interval between each historical sampling time in the time neighborhood set and the current sampling time of the sampling point, and the difference between the ecological data indicators in the sampling point detection data corresponding to each historical sampling time in the time neighborhood set and the current sampling time of the sampling point, the time deviation component of the ecological data indicators in the sampling point detection data corresponding to the sampling point is calculated.

4. The metadata-based LIMS platform data cleaning method according to claim 1, characterized in that, Based on the set of positively correlated indicators, the positively correlated change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point are calculated, including: Based on the metadata set of the historical period, a first numerical sequence of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined over multiple consecutive historical sampling times; the first numerical sequence is used to characterize the direction of change of the ecological data indicator over multiple consecutive historical sampling times. For each positively correlated ecological data indicator in the set of positively correlated indicators, a second numerical sequence of the positively correlated ecological data indicator is determined based on the metadata set of the historical period over multiple consecutive historical sampling times; the second numerical sequence is used to characterize the direction of change of the positively correlated ecological data indicator over multiple consecutive historical sampling times. Based on the first numerical sequence and the second numerical sequence, calculate the positive correlation change component of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

5. The metadata-based LIMS platform data cleaning method according to claim 1, characterized in that, Based on the set of negatively correlated indicators, the negatively correlated change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point are calculated, including: Based on the metadata set of the historical period, a first numerical sequence of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined over multiple consecutive historical sampling times; the first numerical sequence is used to characterize the direction of change of the ecological data indicator over multiple consecutive historical sampling times. For each negatively correlated ecological data indicator in the set of negatively correlated indicators, a third numerical sequence of the negatively correlated ecological data indicator is determined based on the metadata set of the historical period; the third numerical sequence is used to characterize the direction of change of the negatively correlated ecological data indicator over the consecutive historical sampling time. Based on the first numerical sequence and the third numerical sequence, the negative correlation change component of the ecological data indicator in the sampling point detection data corresponding to the sampling point is calculated.

6. The metadata-based LIMS platform data cleaning method according to claim 1, characterized in that, Based on the local deviation index and the local change index, the metadata set for the current period is cleaned using a local outlier detection algorithm, including: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, the local outlier factor of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined based on the local deviation index and local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point. The local outlier detection algorithm is used to detect anomalies in the local outliers of each ecological data indicator in the sampling data corresponding to each sampling point, and the metadata set of the current period is cleaned based on the anomaly detection results.

7. The metadata-based LIMS platform data cleaning method according to claim 6, characterized in that, For each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the local deviation index and local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point, the local outlier factor of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined, including: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, the initial local outlier factor of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined, and the outlier factor correction coefficient is determined based on the local deviation index and local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point. The initial local outlier factor of the ecological data indicator in the sampling point detection data corresponding to the sampling point is corrected according to the outlier factor correction coefficient to obtain the local outlier factor of the ecological data indicator in the sampling point detection data corresponding to the sampling point.

8. A metadata-based LIMS platform data cleaning system, characterized in that, include: The data acquisition module is used to acquire the current period's metadata set and the historical period's metadata set in the LIMS platform; The metadata set includes sampling point detection data corresponding to multiple sampling points, and the sampling point detection data includes spatiotemporal information of the corresponding sampling points and various ecological data indicators. The spatiotemporal deviation analysis module is used to perform spatiotemporal deviation analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the metadata set of the current period and the metadata set of the historical period, to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point. The local deviation index is used to characterize the degree of numerical deviation of the ecological data indicators in the sampling point detection data corresponding to the sampling point in the spatiotemporal neighborhood. The trend consistency analysis module is used to perform trend consistency analysis on each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the metadata set of the historical period, to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; the local change index is used to characterize the consistency of the change trend of the ecological data indicator in the sampling point detection data corresponding to the sampling point. The data cleaning module is used to clean the metadata set of the current period based on the local deviation index and the local change index, using a local outlier detection algorithm. Specifically, for each ecological data indicator in the sampling point detection data corresponding to each sampling point, spatiotemporal deviation analysis is performed based on the metadata set of the current period and the metadata set of historical periods to determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point, including: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, based on the spatiotemporal information of the sampling point, the spatial neighborhood set and temporal neighborhood set of the sampling point are extracted from the metadata set of the current period and the metadata set of the historical period, respectively. Based on the spatial neighborhood set, calculate the spatial deviation component of the ecological data indicator in the sampling point detection data corresponding to the sampling point; The time deviation component of the ecological data indicator in the sampling point detection data corresponding to the sampling point is calculated based on the time neighborhood set. Based on the spatial deviation component and the temporal deviation component, determine the local deviation index of the ecological data indicator in the sampling point detection data corresponding to the sampling point; Specifically, for each ecological data indicator in the sampling point detection data corresponding to each sampling point, trend consistency analysis is performed based on the metadata set of the historical period to determine the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point, including: For each ecological data indicator in the sampling point detection data corresponding to each sampling point, a correlation analysis is performed on the ecological data indicator in the sampling point detection data corresponding to the sampling point in the metadata set of the historical period and other ecological data indicators to determine the set of positively correlated indicators and the set of negatively correlated indicators in the sampling point detection data corresponding to the sampling point. Based on the set of positively correlated indicators, calculate the positively correlated change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point; Based on the set of negatively correlated indicators, calculate the negatively correlated change components of the ecological data indicators in the sampling point detection data corresponding to the sampling point; Based on the positive and negative correlation change components, the local change index of the ecological data indicator in the sampling point detection data corresponding to the sampling point is determined.

Images