Monitoring data optimization method for saline-alkali soil improvement equipment

Abstract

The invention provides a monitoring data optimization method for saline-alkali soil improvement equipment, and the method comprises the following steps: S1, collecting salinity concentration data through the saline-alkali soil improvement equipment, setting an advancing path and data collection frequency of the saline-alkali soil improvement equipment, carrying out the saline-alkali soil salinity data collection once every advancing preset distance, the position information and the time information of the data and the monitoring data of the salinity concentration sensor are recorded; s2, performing anomaly detection optimization on the collected salinity concentration data; and S3, storing and uploading the screened effective data. The invention relates to the technical field of data processing, solves the problem of misjudgment of a traditional COF algorithm in a saline-alkali land junction area through fusion of spatial junction evaluation and historical trend analysis, remarkably improves data anomaly detection precision, and provides efficient and reliable technical support for accurate improvement of saline-alkali land.

Description

Technical Field

[0001] The invention relates to the technical field of data processing, and in particular to a monitoring data optimization method for saline-alkali land improvement equipment. Background Art

[0002] The salinity of saline-alkali land is one of the important indicators for evaluating the effectiveness of improvement measures. Therefore, saline-alkali land improvement equipment needs to monitor and collect soil salt concentration data in real time when performing land improvement operations. Usually, saline-alkali land improvement equipment will collect data at fixed time or distance intervals along a preset path to form a data sequence with temporal and spatial associations.

[0003] For real-time collected saline-alkali land monitoring data, the COF (Connectivity-based Outlier Factor) anomaly detection algorithm is often used to identify data anomalies. The COF algorithm evaluates the connectivity distance of each data point in the local time series window to determine whether the data point is abnormal. However, the salinity of saline-alkali land has regional spatial heterogeneity, that is, the salt content in different regions may vary significantly, especially at the junction of regions, where the salt concentration changes more dramatically.

[0004] The existing COF anomaly detection method mainly evaluates the connectivity distance between each data point and its local neighboring points. If the salt concentration of a data point is significantly different from that of its neighboring data points, the COF score is high and the data point is easily judged as an anomaly. However, in the actual improvement process of saline-alkali land, there are reasonable and significant differences in salt content between different saline-alkali areas. When the saline-alkali land improvement equipment collects data at the junction of different salt concentration areas, normal salt changes can easily lead to abnormal misjudgment of the COF algorithm. This misjudgment causes normal data points to be mistakenly identified as anomalies, reducing the effectiveness of data collection and the reliability of subsequent data analysis. Summary of the invention

[0005] In view of this, the present invention aims to propose a monitoring data optimization method for saline-alkali land improvement equipment, which can effectively identify the data points collected by the saline-alkali land improvement equipment at the junction of different salinity areas to improve the spatial adaptability of abnormal data identification of the saline-alkali land improvement equipment.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A monitoring data optimization method for saline-alkali land improvement equipment comprises the following steps:

[0008] S1: Collect salt concentration data through saline-alkali land improvement equipment, set the travel path and data collection frequency of the saline-alkali land improvement equipment, collect saline-alkali land salt data once every preset travel distance, and record the location information, time information and monitoring data of the salt concentration sensor;

[0009] S2: Optimize anomaly detection on the collected salt concentration data;

[0010] The step S2 comprises:

[0011] S2.1: Based on the mean and standard deviation of the salt concentration of the spatial neighborhood historical data, the boundary degree factor of the data point is calculated through a nonlinear mapping function , identify data points at the boundaries of different salinity regions;

[0012] S2.2: Build a trend fitting model for historical data , a trend fitting model based on historical data , combining the residuals of the current data point and the historical trend and the time interval sensitivity, calculate the abnormal degree optimization factor ;

[0013] S2.3: By combining the above mentioned interface degree factors and abnormality optimization factor , adjust the local average link distance to obtain the optimized local average link distance , optimize the COF anomaly detection algorithm and filter out abnormal data;

[0014] S3: Stores and uploads filtered valid data.

[0015] Furthermore, in step S2.1, the junction degree factor The calculation formula is:

[0016] ;

[0017] Where λ is the sensitivity adjustment coefficient, For the The salt concentration value of each detection data point, For the first The detection data point is the center, and the spatial neighborhood radius The average salt concentration of historical test data within The standard deviation of the salt concentration of historical data in the spatial neighborhood, :for function, To represent the historical standard deviation of salt concentration in the entire monitoring area, is a very small positive number.

[0018] Furthermore, in step S2.2, the abnormality degree optimization factor The calculation formula is:

[0019] ;

[0020] in, For the The salt concentration value of each detection data point, is a very small positive number, is the predicted value of the fitted trend model, is the standard deviation of the residuals from the historical data fitting, is the time interval between the current data point and the most recent historical data, and △T is the average sampling interval of historical data.

[0021] Furthermore, in step S2.3, the optimized local average link distance Adjusted by the following formula:

[0022] ;

[0023] in, is the original local average link distance.

[0024] Furthermore, through the optimized average link distance Evaluate the COF outlier factor of the data point and obtain the optimized Outlier factor for each data point , the outlier factor is optimized by linear normalization , get the abnormality value , and according to the preset threshold Filter out abnormal data.

[0025] Furthermore, the trend fitting model is:

[0026] ;

[0027] in, , , Determined by least squares fitting.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] The monitoring data optimization method for saline-alkali land improvement equipment described in the present invention can effectively identify the data points collected by the saline-alkali land improvement equipment at the junction of different salinity areas by evaluating the degree of junction of the detection data points, thereby avoiding the misjudgment of abnormal situations caused by the normal and drastic changes in salt concentration at the spatial junction of the traditional COF algorithm, thereby significantly improving the spatial adaptability of abnormal data identification of saline-alkali land improvement equipment.

[0030] By analyzing the changing trends of salt concentration in historical data of sampling locations, the matching between current data points and historical trends is evaluated, thus effectively avoiding misjudging data that conforms to the normal improvement trend of saline-alkali land as abnormal, further improving the accuracy and reliability of anomaly detection, and ensuring that the data collection results are more in line with the actual saline-alkali land improvement process. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0032] Figure 1 A flow chart of a monitoring data optimization method for saline-alkali land improvement equipment according to an embodiment of the present invention; DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0034] In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper", "lower", "inner", "back", etc. are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0035] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0036] Example 1

[0037] This embodiment relates to a monitoring data optimization method for saline-alkali land improvement equipment, such as Figure 1 As shown, the following steps are included:

[0038] S1: Collect salt concentration data through saline-alkali land improvement equipment, set the travel path and data collection frequency of the saline-alkali land improvement equipment, collect saline-alkali land salt data once every preset travel distance, and record the location information, time information and monitoring data of the salt concentration sensor;

[0039] S2: Optimize anomaly detection on the collected salt concentration data;

[0040] The step S2 comprises:

[0041] S2.1: Based on the mean and standard deviation of the salt concentration of the spatial neighborhood historical data, the boundary degree factor of the data point is calculated through a nonlinear mapping function , identify data points at the boundaries of different salinity regions;

[0042] S2.2: Build a trend fitting model for historical data , a trend fitting model based on historical data , combining the residuals of the current data point and the historical trend and the time interval sensitivity, calculate the abnormal degree optimization factor ;

[0043] S2.3: By combining the above mentioned interface degree factors and abnormality optimization factor , adjust the local average link distance to obtain the optimized local average link distance , optimize the COF anomaly detection algorithm and filter out abnormal data;

[0044] S3: Stores and uploads filtered valid data.

[0045] Specifically, in step S1, the travel path of the saline-alkali land improvement equipment is set, and the data collection frequency is set. In this embodiment, the saline-alkali land improvement equipment is set to collect saline-alkali land salinity data every 20 meters. For each saline-alkali land salinity data collected, the location information, time information, and monitoring data of the salt concentration sensor are recorded. The location information is the coordinate data of the sampling position, and the time information is the time when the data is collected.

[0046] In step S2, the change pattern of the salt concentration detection data is first analyzed to obtain the boundary degree of the monitoring data. ; Then, by monitoring the intersection of data The historical trend of monitoring data of saline-alkali land improvement equipment is analyzed, so as to obtain the optimization factor of abnormal degree through trend consistency analysis of data points with high degree of intersection. ; Finally, the degree of intersection of the salt concentration data points and abnormality optimization factor The COF anomaly detection process of data points is optimized to obtain the optimized anomaly degree of each data point, and the salt concentration detection data of saline-alkali land is screened by setting the anomaly degree threshold, and the abnormal data is eliminated, thereby ensuring that the saline-alkali land improvement equipment can accurately monitor the data of the saline-alkali land improvement effect.

[0047] In this embodiment, during the data collection process of the saline-alkali land improvement equipment, when the data collected by the equipment exceeds When the COF anomaly detection method is used, the anomaly detection of the newly added data points can be started. Before the anomaly detection of the data points, the change pattern analysis is first performed through the spatial local historical monitoring data of the monitoring data points to identify the changes in each monitoring data point during the historical detection process. For the historical data of the saline-alkali land improvement process, the spatial location of the monitoring data point that is less than 100 km from the location is selected. All historical data points are taken as the neighboring area data points, and the degree of intersection of the data point is evaluated based on the monitoring numerical analysis of the neighboring area data points.

[0048] First test of saline-alkali land improvement equipment data points, the calculation formula for the intersection degree of the data point is:

[0049] ;

[0050] in, : It is the sensitivity adjustment coefficient, which is used to adjust the sensitivity of the algorithm to local data differences.

[0051] :The first equipment for improving saline-alkali land The salt concentration value of each detection data point.

[0052] :For the The detection data point is the center, and the spatial neighborhood radius The average salt concentration of the historical detection data within the area is calculated to reflect the overall level of the neighborhood data.

[0053] : is the standard deviation of the salt concentration of historical data in the spatial neighborhood, reflecting the fluctuation range of the neighborhood salinity value.

[0054] : is a natural constant.

[0055] :for Function is used to measure the relative relationship between the salinity fluctuation characteristics of the local area and the fluctuation characteristics of the overall monitoring area.

[0056] : It is the historical standard deviation of salt concentration in the entire monitoring area, which is used to reflect the background level of global salt fluctuations.

[0057] : is a very small positive number, set to , to prevent the denominator from being .

[0058] Based on the above, first, by evaluating the current data point The historical average salinity of the spatial neighborhood The difference between To indicate the significant difference between the current data point and the surrounding historical data in space. In order to evaluate this difference more flexibly, use The function maps the difference values ​​to form , which allows the magnitude of the difference to be emphasized in a nonlinear form and stabilized within a limited range when the data point differs greatly from the local historical data.

[0059] On the other hand, considering that the salt distribution patterns in different areas of saline-alkali land in actual scenarios are different, some areas are long-term stable, while others have drastic salt fluctuations, and these differences are difficult to express through a single local window. The function is used to measure the relative relationship between the local regional salinity fluctuation characteristics and the overall monitoring area fluctuation characteristics: if the historical salinity data fluctuation in the neighborhood is much higher than the global average fluctuation, it indicates that the salinity condition in this area is likely to be complex, with obvious spatial heterogeneity, and is more likely to belong to the boundary area.

[0060] Combining the above two perspectives, namely the difference in the current data points and the difference in historical regional fluctuations relative to the global level, an optimization factor is formed. The final formula is:

[0061] The first half The function maps local data differences stably in the interval (0,2) and smoothly adjusts the sensitivity to local anomalies;

[0062] Second half Based on the comparison of local fluctuations with overall fluctuations, the degree of intersection is further adjusted to ensure that the strength of the intersection judgment is only increased when the historical fluctuations in the region are large.

[0063] Therefore, the optimization factor The construction of the proposed method takes into account the specific details such as the spatial fluctuation characteristics of data in actual saline-alkali land improvement scenarios, historical regional stability differences, and global salinity fluctuation levels, making the boundary degree assessment more accurate and adaptable, thereby effectively reducing the misjudgment of the traditional COF algorithm at the regional boundary position and improving the accuracy of anomaly detection in saline-alkali land monitoring data.

[0064] After obtaining the degree of intersection of the saline-alkali land improvement equipment, the data points can be screened by the degree of intersection of the detection data, so as to evaluate the abnormality optimization factor of the data points in the boundary area based on the trend consistency of historical changes. For the detection data points in the boundary area of ​​different salt distributions in the saline-alkali land, because the local link distance will be larger due to different salt distributions in the continuous COF anomaly detection process, the abnormality of the data points in the boundary area will become larger. Therefore, for the data points in the boundary area, it is necessary to perform trend fitting through the historical data corresponding to the data points themselves, so as to evaluate the abnormality optimization factor through the difference between the newly detected data points and the historical trend fitting. Therefore, in this process, it is necessary to first determine the historical data used for fitting, and then perform least squares fitting on the historical data used for fitting to obtain the fitting function, and finally evaluate the abnormality optimization factor through the current newly detected data points.

[0065] First, determine the historical data window corresponding to the new detection data point. In this embodiment, the spatial neighborhood historical data set in the above steps is used as the historical data window. For multiple data points corresponding to each timestamp, the average of the multiple data points is used as the detection value corresponding to the timestamp, thereby ensuring that each historical detection timestamp corresponds to only one salt concentration data.

[0066] After determining the data for historical trend fitting, trend fitting can be performed using a regression model. In this embodiment, the fitting trend model is set as ,in , , are the regression coefficients fitted by the least square method, Is the timestamp.

[0067] Determining the polynomial fitting coefficient vector by the least square method and fitting the regression coefficient by the least square method are prior arts, so they will not be described in detail here.

[0068] After obtaining the regression model for trend assessment, the abnormality optimization factor can be evaluated by the difference between the predicted value of the trend fitting model and the actual measured value. The optimization factor of abnormal degree of detected data points :

[0069] ;

[0070] in:

[0071] :Indicates the first The salt concentration value of each detection data point.

[0072] : represents a natural constant.

[0073] : represents a very small positive number, set to , to prevent the denominator from being .

[0074] : Represents the standard deviation of the historical data fitting residuals.

[0075] : represents the predicted value of the current detection time fitted by the regression model, Indicates the timestamp of the current data point detection.

[0076] : Indicates the time interval between the current data point and the most recent historical data, reflecting the trend stability.

[0077] : Indicates the average interval of historical data sampling time.

[0078] Based on the above, the optimization factor The quantitative analysis of the deviation between the historical data trend fitting results and the current measured values ​​reflects the consistency of the data points with the historical data trends.

[0079] In particular, this formula design introduces two key factors:

[0080] Residual sensitivity: reflects whether the data point deviates significantly from the reasonable range of historical trends;

[0081] Time interval sensitivity: reflects the reliability of the data trend model. The newer the historical sampling data, the higher the reliability of the trend prediction value.

[0082] This more finely combines the salt concentration change trend in the actual saline-alkali land improvement process and the temporal characteristics of the data monitoring process.

[0083] Abnormality optimization factor designed by the present invention , a comprehensive evaluation is conducted through two key factors: First, based on the square of the residual between the current data point and the historical trend fitting forecast value, a nonlinear mapping is performed in the form of a Gaussian exponential function, so that the data points with smaller residuals obtain higher values, while data points that deviate significantly from the historical trend receive lower This method effectively quantifies the degree of deviation of the newly collected data point from the historical trend; secondly, it introduces the time interval between the current data point and the most recent historical data collection point. Adjust the time sensitivity. When the data collection time interval is small, the trend prediction reliability is higher. When the value of is higher, and when the interval is larger, the weight of trend reliability is moderately reduced, thereby improving the timeliness and trend adaptability of the optimization factor.

[0084] Through this dual consideration, the formula can accurately evaluate the consistency between new data points and historical change trends, effectively avoid misjudgment of data at the junction of normal areas, and improve the accuracy and reliability of data anomaly detection in saline-alkali land improvement equipment.

[0085] comprehensive Evaluate the optimized local average link distance:

[0086] ;

[0087] After obtaining the optimized local average link distance, the COF outlier factor of the data point can be evaluated by the optimized average link distance to obtain the optimized local average link distance. Outlier factor for each data point .

[0088] After obtaining the outlier factor of the detected data point, the abnormality degree of the data point is obtained through linear normalization mapping.

[0089] ;

[0090] After obtaining the abnormality of the data point, you can set the abnormality threshold Screening for abnormal data points.

[0091] The filtered data is stored and uploaded.

[0092] The present invention solves the problem of misjudgment of the traditional COF algorithm in the boundary area of ​​saline-alkali land by integrating spatial boundary assessment and historical trend analysis, significantly improves the accuracy of data anomaly detection, and provides efficient and reliable technical support for the precise improvement of saline-alkali land.

[0093] Example 2

[0094] A saline-alkali land improvement device includes a data acquisition module for collecting salt concentration data according to a preset path and interval; an abnormality detection optimization module for the abnormal data detection method in Example 1; and a data storage and transmission module for storing and uploading filtered valid data. Specifically, the data acquisition module integrates a salt concentration sensor, a GPS positioning device, and a clock module.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A monitoring data optimization method for saline-alkali land improvement equipment, characterized in that: The following steps are involved: S1: Collect salt concentration data through saline-alkali land improvement equipment, set the travel path and data collection frequency of the saline-alkali land improvement equipment, collect saline-alkali land salt data once every preset travel distance, and record the location information, time information and monitoring data of the salt concentration sensor; S2: Optimize anomaly detection on the collected salt concentration data; The step S2 comprises: S2.1: Based on the mean and standard deviation of the salt concentration of the spatial neighborhood historical data, the boundary degree factor of the data point is calculated through a nonlinear mapping function , identify data points at the boundaries of different salinity regions; S2.2: Build a trend fitting model for historical data , a trend fitting model based on historical data , combining the residuals of the current data point and the historical trend and the time interval sensitivity, calculate the abnormal degree optimization factor ; S2.3: By combining the above mentioned interface degree factors and abnormality optimization factor , adjust the local average link distance to obtain the optimized local average link distance , optimize the COF anomaly detection algorithm and filter out abnormal data; S3: Stores and uploads filtered valid data.

2. The monitoring data optimization method for saline-alkali land improvement equipment according to claim 1, characterized in that: In step S2.1, the junction degree factor The calculation formula is: ； Where λ is the sensitivity adjustment coefficient, For the The salt concentration value of each detection data point, For the first The detection data point is the center, and the spatial neighborhood radius The average salt concentration of historical test data within The standard deviation of the salt concentration of historical data in the spatial neighborhood, :for function, To represent the historical standard deviation of salt concentration in the entire monitoring area, is a very small positive number.

3. The monitoring data optimization method for saline-alkali land improvement equipment according to claim 2, characterized in that: In step S2.2, the abnormality degree optimization factor The calculation formula is: ； in, For the The salt concentration value of each detection data point, is a very small positive number, is the predicted value of the fitted trend model, is the standard deviation of the residuals from the historical data fitting, is the time interval between the current data point and the most recent historical data, and △T is the average sampling interval of historical data.

4. The monitoring data optimization method for saline-alkali land improvement equipment according to claim 3 is characterized in that: In step S2.3, the optimized local average link distance Adjusted by the following formula: ； in, is the original local average link distance.

5. The monitoring data optimization method for saline-alkali land improvement equipment according to claim 4, characterized in that: The average link distance after optimization Evaluate the COF outlier factor of the data point and obtain the optimized Outlier factor of data points , the outlier factor is optimized by linear normalization , get the abnormality value , and according to the preset threshold Filter out unusual data.

6. The monitoring data optimization method for saline-alkali land improvement equipment according to claim 1, characterized in that: The trend fitting model is: ； in, , , Determined by least squares fitting.

Images