A soil and water loss assessment method based on soil and water conservation supervision and inspection big data

Abstract

The present invention relates to the technical field of data model analysis, and in particular to a method for assessing soil and water loss based on soil and water conservation supervision and inspection big data. The method obtains soil and water conservation supervision and inspection big data at each location point as monitoring data; obtains abnormal values ​​of the monitoring data according to the fluctuation difference between each location point and the same type of monitoring data at other surrounding locations; obtains the weight of the monitoring data according to the abnormal values ​​and local outlier factors of the monitoring data; obtains the adjusted covariance matrix according to the matrix constructed by the weight and the matrix constructed by the monitoring data, determines the characteristic data through the principal component analysis algorithm, constructs a soil and water loss risk assessment model, and assesses soil and water loss. The present invention obtains the weight of each monitoring data participating in the principal component analysis algorithm, accurately adjusts the covariance matrix, avoids the influence of abnormal monitoring data on the principal component analysis algorithm, accurately obtains characteristic data, and accurately constructs a soil and water loss risk assessment model.

Description

Technical Field

[0001] The present invention relates to the technical field of data model analysis, and in particular to a soil and water loss assessment method based on soil and water conservation supervision and inspection big data. Background Art

[0002] As the country pays more attention to the ecological environment, soil and water conservation has become an important part of the construction of ecological civilization. Among them, soil and water loss will cause various ecological problems such as land resource destruction, deterioration of agricultural production environment, imbalance of ecological balance, frequent floods and droughts, etc., affecting the development of all walks of life. Therefore, it is necessary to supervise soil and water conservation to avoid the deterioration of soil and water loss.

[0003] In order to prevent soil erosion in time, the soil and water conservation supervision and inspection big data is obtained to build a soil and water loss risk assessment model to evaluate and analyze the soil and water loss situation. Among them, there are many types of soil and water conservation supervision and inspection big data, which is not conducive to accurately and efficiently building a soil and water loss risk assessment model. Therefore, in the existing method, the soil and water conservation supervision and inspection big data is reduced in dimension by the principal component analysis algorithm, and the soil and water loss risk assessment model is built according to the soil and water conservation supervision and inspection big data after the dimension reduction. However, in the actual process of principal component analysis, the possible existence of abnormal soil and water conservation supervision and inspection big data is not taken into account, resulting in an excessive proportion of abnormal soil and water conservation supervision and inspection big data in the principal component analysis process, which makes the soil and water conservation supervision and inspection big data after the dimension reduction inaccurate, and the soil and water loss risk assessment model cannot be accurately constructed, which affects the assessment accuracy of the soil and water loss risk assessment model and cannot prevent soil and water loss in time. Summary of the invention

[0004] In order to solve the technical problem that the soil and water conservation supervision and inspection big data that may exist abnormally is not taken into account in the process of principal component analysis, so that the soil and water conservation supervision and inspection big data after dimension reduction is inaccurate and the soil and water loss risk assessment model cannot be accurately constructed, the purpose of the present invention is to provide a soil and water loss assessment method based on soil and water conservation supervision and inspection big data, and the technical scheme adopted is as follows:

[0005] The present invention proposes a soil and water loss assessment method based on soil and water conservation supervision and inspection big data, the method comprising the following steps:

[0006] Obtain each type of soil and water conservation supervision and inspection big data at each location point at a fixed time every day within a preset time period as monitoring data;

[0007] Fit each monitoring data at each location point within a preset time period into a monitoring curve, divide each monitoring curve to obtain sub-curve segments; take any suspected abnormal monitoring data as the target data, take the monitoring curve of the same type as the target data at each preset neighborhood location point of the target data location point as the reference curve, take the time period corresponding to the preset neighborhood of the target data as the target time period, and obtain the matching curve segment of the sub-curve segment where the target data is located according to the distribution of each sub-curve segment of each reference curve within the target time period; obtain the matching data of the target data according to the distribution of the suspected abnormal monitoring data in each matching curve segment; obtain the abnormal value of the target data according to the difference between the target data and each matching data, and the difference between the sub-curve segment where the target data is located and each matching curve segment; wherein the abnormal value of the normal monitoring data is the first preset constant;

[0008] According to the outliers and local outlier factors of each monitoring data, the weight of each monitoring data is obtained; according to the matrix constructed by the weight and the matrix constructed by the monitoring data, the adjusted covariance matrix is ​​obtained, and the characteristic data is determined by the principal component analysis algorithm;

[0009] A soil and water loss risk assessment model is constructed based on the characteristic data to evaluate soil and water loss.

[0010] Furthermore, the method for obtaining the sub-curve segment is:

[0011] The curve segment formed by two adjacent extreme value points on each monitoring curve is taken as a sub-curve segment; wherein the extreme value point corresponding to the end of each sub-curve segment is divided into the next adjacent sub-curve segment.

[0012] Furthermore, the method for obtaining the matching curve segment is:

[0013] For any reference curve, obtain the duration of each sub-curve segment in the reference curve within the target time period as the first duration;

[0014] The sub-curve segment corresponding to the largest first duration is used as the matching curve segment of the sub-curve segment where the target data is located in the reference curve;

[0015] When there are at least two sub-curve segments corresponding to the largest first duration, the sub-curve segment that is earlier in time sequence is used as the matching curve segment of the sub-curve segment where the target data is located in the reference curve.

[0016] Furthermore, the method for obtaining the matching data is:

[0017] The time corresponding to the target data is taken as the target time. When there is suspected abnormal monitoring data in the matching curve segment, the suspected abnormal monitoring data whose corresponding time in the matching curve segment is closest to the target time is taken as the matching data of the target data.

[0018] When there is no suspected abnormal monitoring data in the matching curve segment, the monitoring data whose corresponding time in the matching curve segment is farthest from the target time is used as the matching data of the target data.

[0019] Furthermore, the method for obtaining the abnormal value is:

[0020] The average of the tangent slopes of all monitoring data in each sub-curve segment is taken as the overall change value of the corresponding sub-curve segment;

[0021] The overall change value difference between the sub-curve segment where the target data is located and each matching curve segment is taken as the first difference;

[0022] The shortest path value between the sub-curve segment where the target data is located and each matching curve segment is obtained through the dynamic time warping algorithm;

[0023] Acquire a reference abnormal value between the target data and each matching data according to the first difference, the shortest path value, the size difference between the target data and each matching data, and the corresponding time difference;

[0024] The result of normalizing the mean of the reference outliers between the target data and all matching data is taken as the outlier of the target data.

[0025] Furthermore, the calculation formula of the reference outlier value is:

[0026] ; ; ; ; In the formula, is the reference abnormal value between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point; is the first difference between the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located and the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; is the overall change value of the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located; is the overall change value of the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; The shortest path value between the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located and the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; is the absolute value of the difference between the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point and the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location; The matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the absolute value of the difference between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point at the corresponding time; is the time corresponding to the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point; The time corresponding to the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the absolute value function.

[0027] Furthermore, the weight is obtained by:

[0028] Obtain the local outlier factor of each monitoring data through the local outlier factor detection algorithm;

[0029] The weight of each monitoring data is obtained according to the difference between the outlier value of each monitoring data and the minimum outlier value, and the difference between the local outlier factor and the minimum local outlier factor.

[0030] Furthermore, the weight calculation formula is:

[0031] ; In the formula, is the weight of the mth monitoring data; is the abnormal value of the mth monitoring data; is the minimum outlier; is the second preset constant, which is greater than 0; is the local outlier factor of the mth monitoring data; is the minimum local outlier factor; is the absolute value function; norm is the normalization function; exp is the exponential function with a natural constant as the base.

[0032] Furthermore, the calculation formula of the adjusted covariance matrix is:

[0033] ; ; ; In the formula, is to adjust the covariance matrix; S is the total number of times monitoring data are collected at all locations within the preset time period; D is the total number of types of monitoring data; The matrix constructed for the weights; is the weight of the first type of monitoring data collected for the first time; is the weight of the Dth type of monitoring data collected for the first time; is the weight of the first type of monitoring data collected for the Sth time; is the weight of the D-th monitoring data collected for the S-th time; X is the matrix constructed by the monitoring data, The data corresponding to the first type of monitoring data in the first collection; is the data corresponding to the first type of monitoring data in the Sth collection; The data corresponding to the D-th monitoring data in the first collection; The data corresponding to the D-th monitoring data in the S-th collection; X is the transposed matrix.

[0034] Furthermore, the method for constructing a soil and water loss risk assessment model based on characteristic data is:

[0035] According to the characteristic data, the soil erodibility factor is obtained;

[0036] According to the soil erodibility factor and using the soil loss equation, a soil and water loss risk assessment model is constructed.

[0037] The present invention has the following beneficial effects:

[0038] Each monitoring data within a preset time period at each location point is fitted into a monitoring curve, so as to accurately analyze the fluctuation of the monitoring data; each monitoring curve is divided to obtain sub-curve segments, so as to facilitate the subsequent accurate analysis of the abnormality degree of the suspected abnormal monitoring data; any suspected abnormal monitoring data is taken as the target data, and the monitoring curve of the same type as the target data at each preset neighborhood location point of the target data location point is taken as the reference curve, so as to improve the efficiency and accuracy of the analysis of the abnormality degree of the target data; in order to more accurately analyze the abnormality degree of the target data, accurately judge whether the target data is truly abnormal, and then obtain the matching curve segment of the sub-curve segment where the target data is located and the matching data of the target data, and accurately obtain the target data according to the difference between the target data and each matching data, and the difference between the sub-curve segment where the target data is located and each matching curve segment. Outliers, accurately determine the degree of abnormality of the target data; in order to involve normal monitoring data, the outliers of the normal monitoring data are set as the first preset constant; in order to avoid the real abnormal monitoring data from affecting the accuracy of the principal component analysis algorithm, therefore, according to the outliers and local outlier factors of each monitoring data, the weight of each monitoring data is obtained, and the proportion of each monitoring data participating in the principal component analysis algorithm is accurately adjusted to avoid the interference of abnormal data on the principal component analysis algorithm; and then according to the matrix constructed by the weights and the matrix constructed by the monitoring data, the adjusted covariance matrix is ​​accurately obtained to improve the accuracy of the principal component analysis algorithm. Therefore, the principal component analysis algorithm is used to accurately obtain the feature data after dimensionality reduction, accurately construct a soil and water loss risk assessment model, accurately assess soil and water loss, and formulate measures in a timely manner to avoid the aggravation of soil and water loss, which is beneficial to the maintenance of the ecological environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flow chart of a method for assessing soil and water loss based on big data of soil and water conservation supervision and inspection is provided as an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a soil and water loss assessment method based on soil and water conservation supervision and inspection big data proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] The following is a detailed description of a specific scheme of a soil and water loss assessment method based on soil and water conservation supervision and inspection big data provided by the present invention in conjunction with the accompanying drawings.

[0044] The specific scenario of the embodiment of the present invention is: before constructing a soil and water loss risk assessment model, it is necessary to perform dimensionality reduction processing on the soil and water conservation supervision and inspection big data obtained at each location point to reduce the complexity of the data, extract the characteristic data in the soil and water conservation supervision big data, and then accurately and efficiently construct a soil and water loss risk assessment model based on the characteristic data.

[0045] The purpose of the embodiment of the present invention is: considering that each soil and water conservation supervision and inspection big data may be affected by noise data and have anomalies, if the principal component analysis is directly performed, the abnormal soil and water conservation supervision and inspection big data may affect the accuracy of the principal component analysis algorithm, resulting in inaccurate soil and water conservation supervision and inspection big data after dimensionality reduction, and unable to accurately construct a soil and water loss risk assessment model. Therefore, the embodiment of the present invention compares the soil and water conservation supervision and inspection big data of each location point in the same area with other surrounding location points, and then accurately obtains the abnormal value of each soil and water conservation supervision and inspection big data, obtains the weight of each soil and water conservation supervision and inspection big data according to the abnormal value, adjusts the covariance matrix in the principal component analysis algorithm, avoids the influence of abnormal soil and water conservation supervision and inspection big data on the principal component analysis algorithm, improves the accuracy of the principal component analysis algorithm, accurately reduces the dimension of soil and water conservation supervision and inspection big data, and accurately constructs a soil and water loss risk assessment model according to the soil and water conservation supervision and inspection big data after dimensionality reduction, and ensures the accuracy of soil and water loss assessment. Among them, the principal component analysis algorithm is a well-known technology and will not be repeated.

[0046] See also Figure 1 , which shows a flow chart of a method for soil and water loss assessment based on soil and water conservation supervision and inspection big data provided by an embodiment of the present invention, the method comprising the following steps:

[0047] Step S1: Obtain each type of soil and water conservation supervision and inspection big data at each location point at a fixed time every day within a preset time period as monitoring data.

[0048] Specifically, in order to evaluate the soil and water loss of production and construction projects, a location point is set every 50 meters in the area corresponding to the production and construction project, and each type of soil and water conservation supervision and inspection big data at a fixed time every day at each location point within the preset time period is obtained, wherein the soil and water conservation supervision and inspection big data includes permanent land area, temporary land area, disturbed land area, soil (stone) source, abandoned soil (stone) disposal, construction stage / construction period, new soil and water loss, soil and water loss hazards, soil and water conservation measures system, soil and water conservation effect, surrounding sensitive areas and other soil and water conservation supervision and inspection big data. In the embodiment of the present invention, the preset time period is set to 60 days, and the time for collecting soil and water conservation supervision and inspection big data at each location point every day is set to 18:00. The implementer can set the distance between two adjacent location points in the corresponding area of ​​the production and construction project, the size of the preset time period and the fixed time for collecting soil and water conservation supervision and inspection big data every day according to the actual situation, which is not limited here. For the convenience of description, the embodiment of the present invention uses soil and water conservation supervision and inspection big data as monitoring data.

[0049] The data types of different types of monitoring data may be different. For example, the data type corresponding to the disposal of abandoned soil (stone) is text. In order to better process the monitoring data of the disposal of abandoned soil (stone), the embodiment of the present invention encodes the monitoring data of the disposal of abandoned soil (stone) through one-hot encoding, and edits the backfill or deployment utilization corresponding to the disposal of abandoned soil (stone), the local landfill and the self-built abandoned soil yard into 001, 010 and 100 respectively. For other monitoring data of text data type, they are all encoded by one-hot encoding. Among them, one-hot encoding is a well-known technology and will not be repeated.

[0050] Step S2: Fit each type of monitoring data at each location point within a preset time period into a monitoring curve, divide each monitoring curve to obtain sub-curve segments; take any suspected abnormal monitoring data as the target data, take the monitoring curve of the same type as the target data at each preset neighborhood location point of the target data location point as the reference curve, take the time period corresponding to the preset neighborhood of the target data as the target time period, and obtain the matching curve segment of the sub-curve segment where the target data is located according to the distribution of each sub-curve segment of each reference curve within the target time period; obtain the matching data of the target data according to the distribution of the suspected abnormal monitoring data in each matching curve segment; obtain the abnormal value of the target data according to the difference between the target data and each matching data, and the difference between the sub-curve segment where the target data is located and each matching curve segment; wherein the abnormal value of the normal monitoring data is the first preset constant.

[0051] Specifically, under normal circumstances, the fluctuation degree of the same type of monitoring data at a certain location point and other surrounding location points is similar. When there is a large difference between the same type of monitoring data at a certain location point and other surrounding location points within a certain time period, it means that the monitoring data at the location point is more likely to be abnormal. In order to accurately compare the fluctuation of the same type of monitoring data at different location points, and then accurately detect abnormal monitoring data, the embodiment of the present invention uses the least squares method to perform curve fitting on each type of monitoring data at each location point within a preset time period according to the time sequence, and uses the fitted curve as the monitoring curve. Therefore, each location point corresponds to the same number of monitoring curves. Among them, the least squares method is a well-known technology and will not be described in detail. In order to improve the efficiency of obtaining abnormal monitoring data, the embodiment of the present invention first obtains suspected abnormal monitoring data in each type of monitoring data at each location point within a preset time period through a local outlier factor detection algorithm. Among them, the local outlier factor detection algorithm is a well-known technology and will not be described in detail. Each suspected abnormal monitoring data detected is analyzed, the abnormal value of each suspected abnormal monitoring data is obtained, and it is determined whether each suspected abnormal monitoring data is truly abnormal. In order to accurately analyze each suspected abnormal monitoring data, the embodiment of the present invention obtains the extreme value points on each monitoring curve through Newton's method, and takes the curve segment composed of two adjacent extreme value points on each monitoring curve as a sub-curve segment; wherein, the extreme value point corresponding to the end of each sub-curve segment is divided into the next adjacent sub-curve segment. It should be noted that the last extreme value point of each monitoring curve is divided into the last sub-curve segment in each monitoring curve. Among them, Newton's method is a well-known technology and will not be described in detail. At this point, the sub-curve segment where each suspected abnormal monitoring data is located is determined.

[0052] It is known that by comparing the differences between the monitoring curves of the same type between each position point and other surrounding position points, the abnormal degree of the suspected abnormal monitoring data at each position point can be accurately determined. Therefore, the embodiment of the present invention compares each position point with the monitoring data of the same type at its 8 neighboring position points. The implementer can set the number of neighborhood position points of each position point according to the actual situation, which is not limited here. When there are no 8 neighboring position points for a certain position point, only the existing neighborhood position points are analyzed. In order to clearly illustrate, the embodiment of the present invention takes any suspected abnormal monitoring data as the target data, and takes the target data as an example to illustrate the specific method of obtaining the abnormal value of the suspected abnormal monitoring data. The monitoring curve of the same type as the target data at each preset neighborhood position point of the target data position point is used as a reference curve. For example, if the target data position point is the nth position point and the type of the target data is d types, the monitoring curve corresponding to the d types of monitoring data at the 8 neighboring position points of the nth position point is used as a reference curve. In order to accurately analyze the degree of abnormality of the target data, and then obtain the matching curve segment of the sub-curve segment where the target data is located at the 8 neighborhood locations of the target data, taking into account the sequential reasons of soil and water loss, the monitoring data between the target data location and the 8 neighborhood locations will have deviations at the same time. Therefore, the embodiment of the present invention takes the 10 neighboring monitoring data forward and backward with the target data as the neighborhood monitoring data of the target data, wherein the target data and each neighborhood monitoring data are the same type of monitoring data at the same location point.For example, when the target data is the i-th monitoring data in the d-th monitoring data at the n-th location point, the neighborhood monitoring data of the target data are respectively: the i-1-th monitoring data in the d-th monitoring data at the n-th location point, the i-2-th monitoring data in the d-th monitoring data at the n-th location point, the i-3-th monitoring data in the d-th monitoring data at the n-th location point, the i-4-th monitoring data in the d-th monitoring data at the n-th location point, the i-5-th monitoring data in the d-th monitoring data at the n-th location point, the i-6-th monitoring data in the d-th monitoring data at the n-th location point, the i-7-th monitoring data in the d-th monitoring data at the n-th location point, the i-8-th monitoring data in the d-th monitoring data at the n-th location point, the i-9-th monitoring data in the d-th monitoring data at the n-th location point, the i-10-th monitoring data in the d-th monitoring data at the n-th location point. The target data is as follows: the i+1th monitoring data of the d-th type of monitoring data at the n-th location point, the i+2th monitoring data of the d-th type of monitoring data at the n-th location point, the i+3th monitoring data of the d-th type of monitoring data at the n-th location point, the i+4th monitoring data of the d-th type of monitoring data at the n-th location point, the i+5th monitoring data of the d-th type of monitoring data at the n-th location point, the i+6th monitoring data of the d-th type of monitoring data at the n-th location point, the i+7th monitoring data of the d-th type of monitoring data at the n-th location point, the i+8th monitoring data of the d-th type of monitoring data at the n-th location point, the i+9th monitoring data of the d-th type of monitoring data at the n-th location point and the i+10th monitoring data of the d-th type of monitoring data at the n-th location point, a total of 20 neighborhood monitoring data. The implementer can set the number of neighborhood monitoring data of the target data according to actual conditions, and there is no limitation here. It should be noted that when the target data is boundary monitoring data and its neighborhood monitoring data does not have 20, the number of neighborhood monitoring data is calculated based on the number of neighborhood monitoring data. The target data and all its neighborhood monitoring data are constructed as the preset neighborhood of the target data, and the time period corresponding to the preset neighborhood of the target data is used as the target time period. According to the distribution of each sub-curve segment of each reference curve in the target time period, the matching curve segment of the sub-curve segment where the target data is located is obtained.

[0053] Preferably, the method for obtaining the matching curve segment is as follows: for any reference curve, the duration of each sub-curve segment in the reference curve within the target time period is obtained as the first duration; wherein, the larger the first duration, the more likely the corresponding sub-curve segment is to match the sub-curve segment where the target data is located, therefore, the sub-curve segment corresponding to the largest first duration is used as the matching curve segment of the sub-curve segment where the target data is located in the reference curve; wherein, when there are at least two sub-curve segments corresponding to the largest first duration, the sub-curve segment that is closer in time sequence is used as the matching curve segment of the sub-curve segment where the target data is located in the reference curve. Thus, the matching curve segment of the sub-curve segment where the target data is located is obtained in each reference curve. When the sub-curve segment where the target data is located is more similar to each matching curve segment, it means that the sub-curve segment where the target data is located is less likely to be abnormal, which indirectly indicates that the degree of abnormality of the target data is smaller. In order to further analyze the degree of abnormality of the target data, the matching data of the target data is obtained according to the distribution of the suspected abnormal monitoring data in each matching curve segment. Preferably, the method for obtaining matching data is: taking the time corresponding to the target data as the target time, when there is suspected abnormal monitoring data in the matching curve segment, taking the suspected abnormal monitoring data closest to the target time at the corresponding time in the matching curve segment as the matching data of the target data, wherein, if there are two suspected abnormal monitoring data closest to the target time at the corresponding time in the matching curve segment, then the suspected abnormal monitoring data later in time sequence is selected as the matching data of the target data; when there is no suspected abnormal monitoring data in the matching curve segment, taking the monitoring data farthest from the target time at the corresponding time in the matching curve segment as the matching data of the target data, wherein, if there are two monitoring data farthest from the target time at the corresponding time in the matching curve segment, then the monitoring data later in time sequence is taken as the matching data of the target data. When the difference between the target data and the matching data is smaller, it means that the target data is more similar to the matching data in other position points, and the abnormal degree of the target data is smaller. Therefore, the embodiment of the present invention obtains the abnormal value of the target data according to the difference between the target data and each matching data, and the difference between the sub-curve segment where the target data is located and each matching curve segment.

[0054] Preferably, the method for obtaining the abnormal value is: taking the mean of the tangent slopes of all monitoring data in each sub-curve segment as the overall change value of the corresponding sub-curve segment; taking the overall change value difference between the sub-curve segment where the target data is located and each matching curve segment as the first difference; when the first difference is smaller, it means that the fluctuation degree of the sub-curve segment where the target data is located is more similar to that of each matching curve segment, and the abnormal degree of the sub-curve segment where the target data is located is smaller, which indirectly indicates that the abnormal degree of the target data is smaller. Through the dynamic time warping algorithm, the shortest path value between the sub-curve segment where the target data is located and each matching curve segment is obtained; when the shortest path value is smaller, it means that the sub-curve segment where the target data is located is more similar to the corresponding matching curve segment, and the abnormal degree of the sub-curve segment where the target data is located is smaller, which indirectly indicates that the abnormal degree of the target data is smaller. When the size difference between the target data and each matching data is smaller, it means that the abnormal degree of the target data is smaller; when the corresponding time difference between the target data and each matching data is smaller, it means that the target data and each matching data appear closer, and the target data is more likely to be a normal change, so the abnormal degree of the target data is smaller. Then, according to the first difference, the shortest path value, the size difference between the target data and each matching data, and the corresponding time difference, the reference outlier value between the target data and each matching data is obtained; the smaller the reference outlier value, the smaller the difference between the target data and the corresponding matching data, and the smaller the degree of abnormality of the target data. The result of normalizing the mean of the reference outlier value between the target data and all matching data is used as the outlier value of the target data. The larger the outlier value, the greater the possibility that the target data is abnormal. Among them, the dynamic time warping algorithm is a well-known technology and will not be repeated here. The specific method of obtaining the outlier value is as follows:

[0055] (1) Obtain reference outlier values.

[0056] As an example, taking the reference abnormal value between the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point and the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point as an example, the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point is the target data, and the calculation formula for obtaining the reference abnormal value between the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point and the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point is:

[0057] ; ; ; ; In the formula, is the reference abnormal value between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point; is the first difference between the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located and the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; is the overall change value of the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located; is the overall change value of the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; The shortest path value between the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located and the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; is the absolute value of the difference between the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point and the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location; The matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the absolute value of the difference between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point at the corresponding time; is the time corresponding to the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point; The time corresponding to the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the absolute value function.

[0058] It should be noted that The smaller, and The more similar, it means that the fluctuation degree of the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point is located is more similar to the sub-curve segment where the ith suspected abnormal monitoring data in the j-th preset neighborhood position point of the n-th position point is located, and the smaller the abnormal degree of the sub-curve segment where the ith suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point is, indirectly indicating that the abnormal degree of the ith suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point is smaller. The smaller; The smaller it is, the more similar the matching curve segment between the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is and the sub-curve segment where the ith suspected abnormal monitoring data in the jth preset neighborhood position point of the nth position point is, indicating that the abnormal degree of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is smaller. The smaller; The smaller, and The more equal they are, the smaller the abnormality of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point is. The smaller; The smaller, and The closer they are, the more likely the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point is a normal change, and the smaller the abnormality of the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point is. The smaller; therefore, The smaller it is, the less likely it is that the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location is abnormal. Is a non-negative number.

[0059] According to the method of obtaining the reference abnormal value between the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point and the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point, the reference abnormal value between each suspected abnormal monitoring data and each matching data is obtained.

[0060] (2) Obtain outliers.

[0061] As an example, taking the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point as an example, the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point is the target data, and the calculation formula for obtaining the abnormal value of the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point is:

[0062] ; In the formula, is the abnormal value of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point; J is the total number of preset neighborhood location points of the nth location point; It is the reference abnormal value between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point; norm is the normalization function.

[0063] It should be noted that The smaller it is, the smaller the abnormality of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point is. The smaller; therefore, The smaller it is, the more normal the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location is.

[0064] According to the method for obtaining the abnormal value of the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th position point, the abnormal value of each suspected abnormal monitoring data is obtained. It should be noted that in order to involve the normal monitoring data, the abnormal value of the normal monitoring data is set to the first preset constant. The embodiment of the present invention sets the first preset constant to 0.1. The implementer can set the size of the first preset constant according to the actual situation, which is not limited here. Among them, when the abnormal value of the suspected abnormal monitoring data is less than 0.1, the abnormal value of the suspected abnormal monitoring data is modified to 0.1 by default. At this point, the abnormal value of each monitoring data is obtained.

[0065] Step S3: Obtain the weight of each monitoring data according to the outliers and local outlier factors of each monitoring data; obtain the adjusted covariance matrix according to the matrix constructed by the weights and the matrix constructed by the monitoring data, and determine the characteristic data through the principal component analysis algorithm.

[0066] Specifically, the larger the outlier value of the monitoring data, the greater the degree of noise influence, the less reliable it is, and the smaller the proportion of participation in the principal component analysis should be, that is, the smaller the corresponding weight should be. At the same time, the local outlier factor of each monitoring data is obtained by the local outlier factor detection algorithm. When the local outlier factor is larger, it means that the degree of abnormality of the corresponding monitoring data is larger, and the proportion of participation in the principal component analysis should be smaller, that is, the corresponding weight should be smaller, so as to ensure the accuracy of the principal component analysis algorithm. Therefore, the embodiment of the present invention obtains the weight of each monitoring data according to the outlier value and local outlier factor of each monitoring data. In order to improve the accuracy of the principal component analysis algorithm, the weight of each monitoring data is constructed as a matrix, the covariance matrix is ​​adjusted, and the adjusted covariance matrix is ​​obtained. Then, the monitoring data is accurately reduced in dimension by the principal component analysis algorithm to obtain the characteristic data, so as to facilitate the subsequent accurate construction of the soil and water loss risk assessment model of the slope according to the characteristic data. Among them, the specific method of obtaining the adjusted covariance matrix is ​​as follows:

[0067] (1) Obtain weights.

[0068] Get the outlier value of each monitoring data. When the difference between the outlier value of a monitoring data and the minimum outlier value is smaller, it means that the degree of abnormality of the monitoring data is smaller, and the weight in the principal component analysis algorithm should be larger. Get the local outlier factor of each monitoring data. When the difference between the local outlier factor of a monitoring data and the minimum local outlier factor is smaller, it means that the monitoring data is more normal, and the weight in the principal component analysis algorithm should be larger. Therefore, the weight of each monitoring data is obtained according to the difference between the outlier value of each monitoring data and the minimum outlier value, and the difference between the local outlier factor and the minimum local outlier factor.

[0069] As an example, taking the mth monitoring data as an example, the calculation formula for obtaining the weight of the mth monitoring data is:

[0070] ; In the formula, is the weight of the mth monitoring data; is the abnormal value of the mth monitoring data; is the minimum outlier; is the second preset constant, which is greater than 0; is the local outlier factor of the mth monitoring data; is the minimum local outlier factor; is the absolute value function; norm is the normalization function; exp is the exponential function with a natural constant as the base.

[0071] The embodiment of the present invention will Set to 0.1 to avoid the denominator being 0. The implementer can set it according to the actual situation. The size is not limited here.

[0072] It should be noted that The smaller, and The closer they are, the smaller the abnormality of the mth monitoring data is, and the greater the weight in the principal component analysis algorithm is. The bigger, The bigger; The smaller, and The more equal they are, the more normal the mth monitoring data is, and the greater the weight in the principal component analysis algorithm. The bigger, The larger the The larger it is, the more normal the mth monitoring data is.

[0073] According to the method for obtaining the weight of the mth monitoring data, the weight of each monitoring data is obtained.

[0074] (2) Obtain the adjusted covariance matrix. Set the total number of monitoring data collected at all locations within the preset time period to S, set the total number of types of monitoring data to D, and construct the weights of all monitoring data into an S-row and D-column matrix , all monitoring data are constructed into a D-row and S-column matrix X, where Each row in corresponds to each column in X. Each column in corresponds to each row in X, ensuring that each monitoring data corresponds to its weight, accurately adjusting each monitoring data, and avoiding excessive proportion of abnormal monitoring data in the principal component analysis process, which affects the accuracy of the principal component analysis algorithm. The transposed matrix of X, X and X is used to obtain the adjusted covariance matrix. The methods for obtaining the transposed matrix and the covariance matrix are both well-known technologies and will not be described in detail. The calculation formula for obtaining the adjusted covariance matrix is:

[0075] ; ; ; In the formula, is to adjust the covariance matrix; S is the total number of times monitoring data are collected at all locations within the preset time period; D is the total number of types of monitoring data; The matrix constructed for the weights; is the weight of the first type of monitoring data collected for the first time; is the weight of the Dth type of monitoring data collected for the first time; is the weight of the first type of monitoring data collected for the Sth time; is the weight of the D-th monitoring data collected for the S-th time; X is the matrix constructed by the monitoring data, The data corresponding to the first type of monitoring data in the first collection; is the data corresponding to the first type of monitoring data in the Sth collection; The data corresponding to the D-th monitoring data in the first collection; The data corresponding to the D-th monitoring data in the S-th collection; X is the transposed matrix.

[0076] The final acquisition is a matrix with S rows and D columns. , the monitoring data is accurately reduced in dimension through the principal component analysis algorithm, and the characteristic data after the monitoring data is reduced in dimension is accurately obtained, so as to facilitate the subsequent accurate construction of the soil and water loss risk assessment model for the area corresponding to the production and construction project based on the characteristic data.

[0077] Step S4: construct a soil and water loss risk assessment model based on the characteristic data to assess soil and water loss.

[0078] Specifically, the soil erodibility factor of the area corresponding to the production and construction project is obtained by using the soil erodibility factor calculation formula through the acquired characteristic data. Then, according to the soil erodibility factor of the area corresponding to the production and construction project, the soil loss equation is used to construct a soil and water loss risk assessment model for the area corresponding to the production and construction project. Through the soil and water loss risk assessment model for the area corresponding to the production and construction project, the soil and water loss situation in the area corresponding to the production and construction project is accurately assessed, and then measures are formulated in time to avoid the aggravation of soil and water loss. Among them, the soil erodibility factor calculation formula and the soil loss equation are both well-known contents and will not be repeated here.

[0079] So far, the present invention is completed.

[0080] In summary, the embodiment of the present invention obtains the soil and water conservation supervision inspection big data at each location point as monitoring data; obtains the abnormal value of the monitoring data according to the fluctuation difference of the same type of monitoring data of each location point and other surrounding location points, and obtains the weight of the monitoring data according to the abnormal value and local outlier factor of the monitoring data; obtains the adjusted covariance matrix according to the matrix constructed by the weight and the matrix constructed by the monitoring data, determines the characteristic data through the principal component analysis algorithm, constructs a soil and water loss risk assessment model, and evaluates soil and water loss. The present invention obtains the weight of each monitoring data participating in the principal component analysis algorithm, accurately adjusts the covariance matrix, avoids the influence of abnormal monitoring data on the principal component analysis algorithm, accurately obtains characteristic data, and accurately constructs a soil and water loss risk assessment model.

[0081] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0082] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A soil and water loss assessment method based on soil and water conservation supervision and inspection big data, characterized in that: The method comprises the following steps: Obtain each type of soil and water conservation supervision and inspection big data at each location point at a fixed time every day within a preset time period as monitoring data; Fit each monitoring data at each location point within a preset time period into a monitoring curve, divide each monitoring curve to obtain sub-curve segments; take any suspected abnormal monitoring data as the target data, take the monitoring curve of the same type as the target data at each preset neighborhood location point of the target data location point as the reference curve, take the time period corresponding to the preset neighborhood of the target data as the target time period, and obtain the matching curve segment of the sub-curve segment where the target data is located according to the distribution of each sub-curve segment of each reference curve within the target time period; obtain the matching data of the target data according to the distribution of the suspected abnormal monitoring data in each matching curve segment; obtain the abnormal value of the target data according to the difference between the target data and each matching data, and the difference between the sub-curve segment where the target data is located and each matching curve segment; wherein the abnormal value of the normal monitoring data is the first preset constant; According to the outliers and local outlier factors of each monitoring data, the weight of each monitoring data is obtained; according to the matrix constructed by the weight and the matrix constructed by the monitoring data, the adjusted covariance matrix is ​​obtained, and the characteristic data is determined by the principal component analysis algorithm; Construct a soil and water loss risk assessment model based on characteristic data to assess soil and water loss; The method for obtaining the weight is: Obtain the local outlier factor of each monitoring data through the local outlier factor detection algorithm; Obtain the weight of each monitoring data according to the difference between the outlier value of each monitoring data and the minimum outlier value, and the difference between the local outlier factor and the minimum local outlier factor; The calculation formula of the weight is: ; In the formula, is the weight of the mth monitoring data; is the abnormal value of the mth monitoring data; is the minimum outlier; is the second preset constant, which is greater than 0; is the local outlier factor of the mth monitoring data; is the minimum local outlier factor; is the absolute value function; norm is the normalization function; exp is the exponential function with a natural constant as the base; The calculation formula of the adjusted covariance matrix is: ; ; ; In the formula, is to adjust the covariance matrix; S is the total number of times monitoring data are collected at all locations within the preset time period; D is the total number of types of monitoring data; The matrix constructed for the weights; is the weight of the first type of monitoring data collected for the first time; is the weight of the Dth type of monitoring data collected for the first time; is the weight of the first type of monitoring data collected for the Sth time; is the weight of the D-th monitoring data collected for the S-th time; X is the matrix constructed by the monitoring data, The data corresponding to the first type of monitoring data in the first collection; is the data corresponding to the first type of monitoring data in the Sth collection; The data corresponding to the D-th monitoring data in the first collection; The data corresponding to the D-th monitoring data in the S-th collection; X is the transposed matrix.

2. A soil and water loss assessment method based on soil and water conservation supervision and inspection big data as claimed in claim 1, characterized in that: The method for obtaining the sub-curve segment is: The curve segment formed by two adjacent extreme value points on each monitoring curve is taken as a sub-curve segment; wherein the extreme value point corresponding to the end of each sub-curve segment is divided into the next adjacent sub-curve segment.

3. A soil and water loss assessment method based on soil and water conservation supervision and inspection big data as claimed in claim 1, characterized in that: The method for obtaining the matching curve segment is: For any reference curve, obtain the duration of each sub-curve segment in the reference curve within the target time period as the first duration; The sub-curve segment corresponding to the largest first duration is used as the matching curve segment of the sub-curve segment where the target data is located in the reference curve; When there are at least two sub-curve segments corresponding to the largest first duration, the sub-curve segment that is earlier in time sequence is used as the matching curve segment of the sub-curve segment where the target data is located in the reference curve.

4. The method for evaluating soil and water loss based on soil and water conservation supervision and inspection big data as claimed in claim 1, characterized in that: The method for obtaining the matching data is: The time corresponding to the target data is taken as the target time. When there is suspected abnormal monitoring data in the matching curve segment, the suspected abnormal monitoring data whose corresponding time in the matching curve segment is closest to the target time is taken as the matching data of the target data. When there is no suspected abnormal monitoring data in the matching curve segment, the monitoring data whose corresponding time in the matching curve segment is farthest from the target time is used as the matching data of the target data.

5. The method for evaluating soil and water loss based on soil and water conservation supervision and inspection big data as claimed in claim 1, characterized in that: The method for obtaining the abnormal value is: The average of the tangent slopes of all monitoring data in each sub-curve segment is taken as the overall change value of the corresponding sub-curve segment; The overall change value difference between the sub-curve segment where the target data is located and each matching curve segment is taken as the first difference; The shortest path value between the sub-curve segment where the target data is located and each matching curve segment is obtained through the dynamic time warping algorithm; Acquire a reference abnormal value between the target data and each matching data according to the first difference, the shortest path value, the size difference between the target data and each matching data, and the corresponding time difference; The result of normalizing the mean of the reference outliers between the target data and all matching data is taken as the outlier of the target data.

6. A soil and water loss assessment method based on soil and water conservation supervision and inspection big data as claimed in claim 5, characterized in that: The calculation formula of the reference abnormal value is: ; ; ; ; In the formula, is the reference abnormal value between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point; is the first difference between the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located and the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; is the overall change value of the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located; is the overall change value of the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; The shortest path value between the sub-curve segment where the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth position point is located and the matching curve segment of the sub-curve segment where the ith suspected abnormal monitoring data at the jth preset neighborhood position point of the nth position point is located; is the absolute value of the difference between the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point and the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location; The matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the absolute value of the difference between the matching data of the ith suspected abnormal monitoring data in the dth type of monitoring data at the nth location point and the ith suspected abnormal monitoring data at the jth preset neighborhood location point of the nth location point at the corresponding time; is the time corresponding to the i-th suspected abnormal monitoring data in the d-th type of monitoring data at the n-th location point; The time corresponding to the matching data of the i-th suspected abnormal monitoring data at the j-th preset neighborhood location point of the n-th location point; is the absolute value function.

7. The method for evaluating soil and water loss based on soil and water conservation supervision and inspection big data as claimed in claim 1, characterized in that: The method for constructing a soil and water loss risk assessment model based on characteristic data is: According to the characteristic data, the soil erodibility factor is obtained; According to the soil erodibility factor and using the soil loss equation, a soil and water loss risk assessment model is constructed.

Images