Geological test data outlier detection method, system, device and medium

Abstract

The application provides a geological test data outlier detection method, system, device and medium. A three-dimensional space weight matrix is constructed according to collected spatial coordinates, and then the spatial correlation difference degrees of geological test data between each test monitoring point are determined. Spatial correlation screening is performed on the geological test data of each test monitoring point by combining each spatial correlation difference degree with a preset difference degree threshold, spatial correlation abnormal data and spatial correlation normal data are obtained. The spatial correlation normal data is input into an isolation forest model for local anomaly screening, and local abnormal data is output. The test abnormal data of the target area is generated by fusing the spatial correlation abnormal data and the local abnormal data. According to the scheme of the application, the abnormal data in the geological test data can be identified based on the three-dimensional space autocorrelation of the geological data combined with the geological mechanism rule.

Description

Technical Field

[0001] This application relates to the field of data anomaly detection technology, and in particular to a method, system, equipment and medium for detecting outliers in geological test data. Background Technology

[0002] Outlier detection in geological test data is an important part of geological exploration and engineering applications. Traditional methods for detecting outliers in geological data usually rely on statistical and mathematical models, such as box plots and standard deviation methods, to judge the abnormality of data based on numerical fluctuations in a single dimension.

[0003] The basic idea of ​​traditional geological data outlier detection is to analyze historical data or existing datasets to identify data points that significantly deviate from the expected range. These data points are usually considered outliers. It focuses on processing single-point data or two-dimensional data, lacking a deep understanding of the three-dimensional spatial correlation and complex geological mechanisms of geological data. As a result, it ignores the potential spatial dependencies and geological genesis factors between data. Therefore, how to identify outliers in geological test data based on the three-dimensional spatial autocorrelation of geological data and geological mechanism rules has become a difficult problem for the industry. Summary of the Invention

[0004] Based on this, this application provides a method and system for detecting outliers in geological test data by combining three-dimensional spatial autocorrelation of geological data with geological mechanism rules.

[0005] In a first aspect, this application provides a method for detecting outliers in geological test data, comprising the following steps:

[0006] Collect geological test data and corresponding spatial coordinates from various test and monitoring points in the target area;

[0007] A three-dimensional spatial weight matrix is ​​constructed based on all spatial coordinates. Then, based on the three-dimensional spatial weight matrix and all geological test data, the spatial correlation difference between geological test data at each test monitoring point is determined.

[0008] By combining the spatial correlation differences with the preset geological mechanism rules, the geological test data of each test monitoring point are spatially correlated and filtered to obtain spatially correlated abnormal data and spatially correlated normal data.

[0009] The spatially correlated normal data is input into the isolated forest model for local anomaly filtering, and the local anomaly data is output.

[0010] The spatial correlation anomaly data and the local anomaly data are fused to generate test anomaly data for the target area.

[0011] In some embodiments, constructing a three-dimensional spatial weight matrix based on all spatial coordinates specifically includes:

[0012] Calculate the spatial distance between each test monitoring point based on all spatial coordinates;

[0013] The spatial weight value between the corresponding test monitoring points is calculated based on the spatial distance between each test monitoring point. The spatial weight value is inversely proportional to the spatial distance between the corresponding test monitoring points.

[0014] Construct a three-dimensional spatial weight matrix by relating all the calculated spatial weight values ​​to spatial coordinates.

[0015] In some embodiments, determining the spatial correlation difference between geological test data at various test monitoring points based on the three-dimensional spatial weight matrix and all geological test data specifically includes:

[0016] The spatial dependency of each test monitoring point is determined based on all geological test data and the aforementioned three-dimensional spatial weight matrix.

[0017] Based on the spatial dependence of each test monitoring point and geological test data, the adjacent trend data of each test monitoring point are determined;

[0018] Determine the adjacent deviation of different monitoring parameters between adjacent trend data and corresponding geological test data for each test monitoring point;

[0019] The spatial correlation difference between geological test data at each test monitoring point is calculated based on all adjacent deviations.

[0020] In some embodiments, spatial correlation screening of geological test data at each test monitoring point is performed by combining spatial correlation differences with preset geological mechanism rules to obtain spatially correlated abnormal data and spatially correlated normal data, specifically including:

[0021] Obtain preset geological mechanism rules, and then determine the difference threshold based on the preset geological mechanism rules;

[0022] For each test monitoring point, the spatial correlation difference of the test monitoring point is compared with the difference threshold.

[0023] When the spatial correlation difference is greater than or equal to the difference threshold, the geological test data of the test monitoring point will be regarded as spatial correlation anomaly data.

[0024] When the spatial correlation difference is less than the difference threshold, the geological test data of the test monitoring point is taken as the normal spatial correlation data, thereby obtaining the abnormal spatial correlation data and the normal spatial correlation data.

[0025] In some embodiments, the spatially correlated normal data is input into the isolated forest model for local anomaly filtering, and the output local anomaly data specifically includes:

[0026] The spatially correlated normal data is preprocessed;

[0027] Construct an isolation forest model based on an attention mechanism, where the attention mechanism enhances the ability to identify abnormal data by weighting specific input features;

[0028] The preprocessed spatially correlated normal data is input into the attention-based isolation forest model, and the anomaly discrimination value of each test monitoring point in the spatially correlated normal data is output.

[0029] For each test monitoring point in the spatially correlated normal data, the geological test data corresponding to all test monitoring points whose anomaly discrimination value is greater than the preset anomaly discrimination threshold are regarded as local anomaly data.

[0030] In some embodiments, fusing the spatially correlated anomaly data and the local anomaly data to generate experimental anomaly data for the target region specifically includes:

[0031] The spatially correlated anomaly data and the local anomaly data are combined, and the combined anomaly data is visualized to obtain a three-dimensional visualization map.

[0032] The three-dimensional visualization map is stored as experimental anomaly data.

[0033] In some embodiments, the geological test data includes: water content, porosity, thermal conductivity, and chemical element content.

[0034] Secondly, this application provides a geological test data outlier detection system, the system comprising:

[0035] The data acquisition module is used to collect geological test data and corresponding spatial coordinates from various test monitoring points in the target area.

[0036] The processing module is used to construct a three-dimensional spatial weight matrix based on all spatial coordinates, and then determine the spatial correlation difference between geological test data of each test monitoring point based on the three-dimensional spatial weight matrix and all geological test data.

[0037] The processing module is also used to perform spatial correlation screening on the geological test data of each test monitoring point by combining the spatial correlation difference degree with the preset geological mechanism rules, so as to obtain spatial correlation abnormal data and spatial correlation normal data.

[0038] The processing module is also used to input the spatially correlated normal data into the isolated forest model for local anomaly filtering and output local anomaly data;

[0039] The execution module is used to fuse the spatial correlation anomaly data and the local anomaly data to generate test anomaly data for the target area.

[0040] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method for detecting outliers in geological test data.

[0041] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for detecting outliers in geological test data.

[0042] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0043] The geological test data anomaly detection method and system provided in this application first collects geological test data and corresponding spatial coordinates of each test monitoring point in the target area; constructs a three-dimensional spatial weight matrix based on all spatial coordinates, and then determines the spatial correlation difference between the geological test data of each test monitoring point based on the three-dimensional spatial weight matrix and all geological test data; performs spatial correlation filtering on the geological test data of each test monitoring point by combining each spatial correlation difference with a preset difference threshold to obtain spatially correlated abnormal data and spatially correlated normal data; inputs the spatially correlated normal data into an isolated forest model for local anomaly filtering and outputs local anomaly data; and merges the spatially correlated abnormal data and the local anomaly data to generate test anomaly data for the target area.

[0044] Therefore, this application first constructs a three-dimensional spatial weight matrix based on all spatial coordinates. This three-dimensional spatial weight matrix is ​​used to represent the spatial correlation between test monitoring points. Then, based on the three-dimensional spatial weight matrix and all geological test data, the spatial correlation difference between the geological test data of each test monitoring point is determined. The spatial correlation difference reflects the spatial heterogeneity of the geological test data between test monitoring points. Test monitoring points with larger spatial correlation differences have a weaker influence on other test monitoring points and can represent significant changes in the geological conditions of the test monitoring points corresponding to that spatial correlation difference. Then, each spatial correlation difference is combined with a preset difference threshold to determine the spatial correlation difference between the test monitoring points. The geological test data from the test monitoring points are spatially correlated and filtered to obtain spatially correlated anomalous data and spatially correlated normal data. Next, the spatially correlated normal data is input into an isolated forest model for local anomaly filtering, outputting local anomaly data. This step, by introducing an attention mechanism into the isolated forest model, can improve the detection accuracy of anomaly data and optimize small-sample adaptability when processing complex geological test data. Finally, the spatially correlated anomalous data and the local anomaly data are fused to generate test anomaly data for the target area. In summary, the solution of this application can identify anomalous data in geological test data based on the three-dimensional spatial autocorrelation of geological data combined with geological mechanism rules. Attached Figure Description

[0045] Figure 1 This is an exemplary flowchart of a geological test data outlier detection method according to some embodiments of this application;

[0046] Figure 2 This is a schematic diagram illustrating an application scenario of a geological test data outlier detection system according to some embodiments of this application;

[0047] Figure 3 This is a flowchart illustrating the process of determining spatial correlation differences according to some embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the structure of a geological test data outlier detection system according to some embodiments of this application;

[0049] Figure 5 This is a schematic diagram of the structure of a computer device for implementing a method for detecting outliers in geological test data, according to some embodiments of this application. Detailed Implementation

[0050] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0051] refer to Figure 1The figure is an exemplary flowchart of a geological test data outlier detection method according to some embodiments of this application. The geological test data outlier detection method mainly includes the following steps:

[0052] In step 101, geological test data and corresponding spatial coordinates of each test monitoring point in the target area are collected.

[0053] In practice, the collection of geological test data and corresponding spatial coordinates of various test monitoring points in the target area can be achieved in the following way: First, a series of geological monitoring sensors are deployed, such as moisture content monitoring sensors, porosity monitors, hot-wire sensors, and element content monitoring sensors, to different test monitoring points in the target area. These sensors transmit data to a central data storage device via wireless or wired networks. Then, a global positioning system (GPS) device is used to accurately locate each test monitoring point and obtain its spatial coordinates (i.e., longitude, latitude, and elevation information). The geological test data of each test monitoring point is then matched one-to-one with its corresponding spatial coordinates to ensure that the collected data accurately reflects the geological conditions and spatial distribution of each test monitoring point in the target area. Furthermore, when collecting the geological test data of each test monitoring point in the target area, timestamps can be added as needed to facilitate subsequent time-series analysis.

[0054] It should be noted that the geological test data described in this application includes different monitoring parameters such as water content, porosity, thermal conductivity and chemical element content. Other geological test data may also be collected in other embodiments.

[0055] In some embodiments, reference Figure 2 As shown in the figure, this figure is a schematic diagram of the application scenario of the geological test data outlier detection system shown in some embodiments of this application. The figure includes three main components: acquisition equipment, server and data storage equipment. The acquisition equipment is responsible for collecting geological test data and corresponding spatial coordinates of various test monitoring points in the target area, and sending the collected geological test data and corresponding spatial coordinates to the server through the communication network. The geological test data outlier detection system runs on the server, and the server stores the processing results in the data storage equipment and visualizes them.

[0056] In step 102, a three-dimensional spatial weight matrix is ​​constructed based on all spatial coordinates, and then the spatial correlation difference between geological test data at each test monitoring point is determined based on the three-dimensional spatial weight matrix and all geological test data.

[0057] In some embodiments, constructing a three-dimensional spatial weight matrix based on all spatial coordinates can be achieved using the following steps:

[0058] Calculate the spatial distance between each test monitoring point based on all spatial coordinates;

[0059] The spatial weight value between the corresponding test monitoring points is calculated based on the spatial distance between each test monitoring point. The spatial weight value is inversely proportional to the spatial distance between the corresponding test monitoring points.

[0060] Construct a three-dimensional spatial weight matrix by relating all the calculated spatial weight values ​​to spatial coordinates.

[0061] In specific implementation, the spatial distance between each test monitoring point can be calculated based on all spatial coordinates in the following ways: for example, the Euclidean distance formula can be used to calculate the spatial distance between any two test monitoring points; for three-dimensional spatial data containing elevation information, the three-dimensional Euclidean distance formula can be used to consider the influence of vertical distribution; in addition, appropriate spherical distance can be calculated based on the curvature of the Earth's surface to ensure the accuracy of the spatial distance. In other embodiments, other suitable spatial distance calculation methods can also be used, and this embodiment does not limit them.

[0062] In practical implementation, the spatial weight value between corresponding test monitoring points can be calculated based on the spatial distance between each pair of test monitoring points in the following way: The spatial weight value between corresponding test monitoring points can be calculated using an inverse proportional relationship based on the calculated spatial distance between each pair of test monitoring points. More specifically, for each pair of test monitoring points, the spatial weight value can be calculated as follows: the spatial weight value is inversely proportional to the distance, that is, the closer the distance, the larger the weight value. The optional calculation method is as follows: ,in, Indicates the spatial weight value. Indicates the spatial distance between two test monitoring points. It is an adjustment factor, usually with a value of 2 or higher. This inverse relationship ensures that the test monitoring points that are closer to each other have a greater impact on the other test monitoring points when calculating the weight, while the test monitoring points that are farther away have a smaller impact.

[0063] In specific implementation, the three-dimensional spatial weight matrix can be constructed by constructing all calculated spatial weight values ​​according to spatial coordinate relationships in the following way: all calculated spatial weight values ​​can be organized into a matrix according to the spatial coordinate relationships between the test monitoring points, where each matrix element represents a pair of spatial weight values ​​between test monitoring points, and the dimension of the matrix is ​​N×N (where N is the number of test monitoring points). It should be noted that this three-dimensional spatial weight matrix is ​​used to represent the spatial correlation between test monitoring points. In addition, other existing technologies can be used to construct the three-dimensional spatial weight matrix in other embodiments, which are not limited here.

[0064] In some embodiments, reference Figure 3 As shown in the figure, this is a flowchart illustrating the process of determining spatial correlation differences in some embodiments of this application. In this embodiment, determining the spatial correlation differences between geological test data at each test monitoring point based on the three-dimensional spatial weight matrix and all geological test data can be achieved through the following steps:

[0065] In step 1031, the spatial dependency of each test monitoring point is determined based on all geological test data and the three-dimensional spatial weight matrix.

[0066] In step 1032, adjacent trend data for each test monitoring point are determined based on the spatial dependence of each test monitoring point and geological test data;

[0067] In step 1033, the adjacent deviation of different monitoring parameters between adjacent trend data and corresponding geological test data at each test monitoring point is determined;

[0068] In step 1034, the spatial correlation difference of geological test data between each test monitoring point is calculated based on all adjacent deviations.

[0069] In specific implementation, the spatial dependency of each test monitoring point can be determined based on all geological test data and the three-dimensional spatial weight matrix in the following way: First, the covariance between each test monitoring point can be calculated based on all geological test data to obtain a covariance matrix. Then, the covariance matrix is ​​multiplied by the three-dimensional spatial weight matrix. Finally, the value of each element in the multiplied matrix is ​​used as the spatial dependency of the corresponding test monitoring point. The spatial dependency is used to quantify the spatial correlation between each test monitoring point.

[0070] It should be noted that in this application, in the process of calculating the covariance between each test monitoring point based on all geological test data, a weighted fusion method can be used to calculate the fusion value for different monitoring parameters within each test monitoring point, and then the fusion value is used to calculate the covariance.

[0071] In specific implementation, determining the adjacent trend data of each test monitoring point based on the spatial dependency of each test monitoring point and the geological test data can be achieved in the following way: For each test monitoring point, the geological test data of the neighboring test monitoring points of each test monitoring point can be weighted according to the spatial dependency of all test monitoring points to obtain the adjacent trend data of each test monitoring point, which reflects the geological characteristic change trend of the test monitoring point in the adjacent area; it should be noted that in the process of weighting the geological test data of the neighboring test monitoring points of each test monitoring point according to the spatial dependency of all test monitoring points, different monitoring parameters in each geological test data can be weighted separately, and then the weighted values ​​of different monitoring parameters in each geological test data can be used to form the adjacent trend data of the corresponding test monitoring point.

[0072] In specific implementation, the adjacent deviation of different monitoring parameters between adjacent trend data and corresponding geological test data of each test monitoring point can be achieved in the following way: For adjacent trend data of each test monitoring point, the weighted value of different monitoring parameters in the adjacent trend data is subtracted from the value of different monitoring parameters in the corresponding geological test data, and the subtracted value is used as the adjacent deviation of different monitoring parameters. Thus, the adjacent deviation of different monitoring parameters between adjacent trend data and corresponding geological test data of each test monitoring point is obtained. It should be noted that the adjacent deviation in this application reflects the deviation between the actual measured geological test data of the test monitoring point and its adjacent trend data. The larger the adjacent deviation, the more obvious the difference between the actual measured geological test data of the test monitoring point and its adjacent trend data, indicating an anomaly.

[0073] In practice, the spatial correlation difference between geological test data at each test monitoring point can be calculated by the following steps based on all adjacent deviations: For each test monitoring point, the spatial correlation difference between geological test data at each test monitoring point can be calculated by weighted averaging of all adjacent deviations corresponding to the test monitoring point. The spatial correlation difference reflects the spatial heterogeneity of geological test data between test monitoring points. Test monitoring points with larger spatial correlation differences have a weaker influence on other test monitoring points and can represent significant changes in the geological conditions of the test monitoring point corresponding to the spatial correlation difference.

[0074] In step 103, the geological test data of each test monitoring point are spatially correlated and screened by combining the spatial correlation difference degree with the preset geological mechanism rules to obtain spatial correlation abnormal data and spatial correlation normal data.

[0075] In some embodiments, spatial correlation screening of geological test data at each test monitoring point is performed by combining spatial correlation differences with preset geological mechanism rules to obtain spatially correlated abnormal data and spatially correlated normal data. This can be achieved through the following steps:

[0076] Obtain preset geological mechanism rules, and then determine the difference threshold based on the preset geological mechanism rules;

[0077] For each test monitoring point, the spatial correlation difference of the test monitoring point is compared with the difference threshold.

[0078] When the spatial correlation difference is greater than or equal to the difference threshold, the geological test data of the test monitoring point will be regarded as spatial correlation anomaly data.

[0079] When the spatial correlation difference is less than the difference threshold, the geological test data of the test monitoring point is taken as the normal spatial correlation data, thereby obtaining the abnormal spatial correlation data and the normal spatial correlation data.

[0080] In specific implementation, the preset geological mechanism rules can be obtained in the following ways: They can be based on rules derived from geological theoretical models. For example, one or more geological mechanism models (e.g., rock layer interaction models, groundwater flow models, etc.) can be selected according to the geological characteristics of the target area. Then, monitoring parameters such as rock layer interface strength, porosity, and permeability can be extracted from the geological mechanism models. Finally, different monitoring thresholds can be set for each monitoring parameter. For example, under specific geological conditions, if the rock layer strength is lower than 30 MPa, it may lead to instability. Therefore, a rule can be set: "If the rock layer interface strength is lower than 30 MPa, it is considered that there may be geological anomalies in this area, and the monitoring threshold for the rock layer interface strength is 30 MPa." In other embodiments, the preset geological mechanism rules can also be obtained based on rules derived from experience. For example, geological experts can be invited to extract geological mechanism rules based on the geological characteristics, historical data, and known geological events (such as landslides, groundwater leakage, etc.) of the target area. For example, the relationship between porosity and permeability can be set with a monitoring threshold based on geological experience. Other methods can also be used in other embodiments, which are not limited here.

[0081] In specific implementation, the difference threshold can be determined according to the preset geological mechanism rules in the following way: the difference threshold is calculated by weighting the monitoring thresholds in the preset geological mechanism rules. Other methods can also be used in other embodiments, which will not be elaborated here.

[0082] A set of geological test data from normal areas is obtained, and the spatial correlation difference of these geological test data is calculated as a background dataset. Then, based on the difference distribution of the background dataset, an appropriate difference threshold is selected. For example, statistical methods such as standard deviation or median can be used to determine the difference threshold. Other methods can also be used in other embodiments, which are not limited here.

[0083] It should be noted that, in this application, spatial correlation abnormal data refers to geological test data of the corresponding test monitoring point that differs significantly from the spatial relationship of other test monitoring points in the target area, indicating obvious abnormality; spatial correlation normal data refers to normal data where the spatial relationship of the corresponding test monitoring point is relatively consistent with that of other test monitoring points in the target area, showing no abnormal trend.

[0084] In step 104, the spatially correlated normal data is input into the isolated forest model for local anomaly filtering, and local anomaly data is output.

[0085] In some embodiments, the spatially correlated normal data is input into the isolated forest model for local anomaly filtering, and the output of local anomaly data can be achieved by the following steps:

[0086] The spatially correlated normal data is preprocessed;

[0087] Construct an isolation forest model based on an attention mechanism, where the attention mechanism enhances the ability to identify abnormal data by weighting specific input features;

[0088] The preprocessed spatially correlated normal data is input into the attention-based isolation forest model, and the anomaly discrimination value of each test monitoring point in the spatially correlated normal data is output.

[0089] For each test monitoring point in the spatially correlated normal data, the geological test data corresponding to all test monitoring points whose anomaly discrimination value is greater than the preset anomaly discrimination threshold are regarded as local anomaly data.

[0090] In some embodiments, constructing an attention-based isolated forest model can be achieved using the following steps:

[0091] The basic structure for constructing an isolated forest model;

[0092] By combining attention mechanisms, the correlation characteristics of each monitoring parameter in historical geological test data were determined among different test monitoring points;

[0093] The monitoring parameters in the historical geological test data are weighted according to all the correlation characteristics, and then the isolated forest model is trained using all the weighted monitoring parameters.

[0094] In practical implementation, constructing an isolated forest model based on an attention mechanism can be achieved as follows: First, construct the basic framework of the isolated forest model, including selecting basic model parameters (e.g., the number of trees, tree depth, etc.). Second, determining the correlation characteristics of each monitoring parameter in historical geological test data between different test monitoring points using the attention mechanism can be achieved as follows: Construct a self-attention layer. This self-attention layer dynamically adjusts the weights (i.e., correlation characteristics) of each parameter by calculating the relative importance between each parameter at each test monitoring point. More specifically, the self-attention mechanism calculates the similarity between each test monitoring point and other test monitoring points and assigns a correlation characteristic value to each monitoring parameter. This correlation characteristic value reflects the correlation characteristics of different monitoring parameters during the isolated forest training process. The influence of the data in the historical geological test data; in specific implementation, the monitoring parameters in the historical geological test data are weighted according to all the relevant features, and then the training of the isolated forest model is completed with all the weighted monitoring parameters. This can be achieved in the following way: the relevant features obtained through the self-attention mechanism are used to weight the monitoring parameters in the input historical geological test data. Then, these weighted monitoring parameters are used as input in the isolated forest training process. By generating multiple random trees, the abnormal data points are isolated, and the training of the isolated forest model is completed. It should be noted that during the training process, the isolated forest model determines whether a point is an anomaly by calculating the path length of each test monitoring point in the tree. The weighted features indirectly affect the path length by adjusting the path splitting method, thereby changing the anomaly score.

[0095] In addition, in specific implementation, the preprocessing of the spatially correlated normal data can be achieved in the following ways: for the spatially correlated normal data, a suitable interpolation method can be used to fill in the missing values ​​(e.g., nearest neighbor interpolation or linear interpolation); the spatially correlated normal data can also be normalized or standardized to unify the numerical range of all monitoring parameters to a standard range; the spatially correlated normal data can also be noise-processed, for example, by using filtering algorithms such as median filtering or Gaussian filtering to remove high-frequency noise; other methods can also be used for preprocessing in other embodiments, which will not be elaborated here.

[0096] In specific implementation, the preprocessed spatially correlated normal data is input into an attention-based isolation forest model. The output of the anomaly discrimination value for each test monitoring point in the spatially correlated normal data can be achieved in the following way: the preprocessed spatially correlated normal data is input into the attention-based isolation forest model, and the model evaluates each test monitoring point through the previously trained isolation forest and outputs the anomaly discrimination value for each test monitoring point. More specifically, the preprocessed data of each test monitoring point is input into the isolation forest model, and the model evaluates the degree of anomaly of each test monitoring point by means of path length. Due to the addition of the attention mechanism, the model will assign different weights according to the importance of each monitoring parameter, thereby more accurately evaluating the anomaly discrimination value of each test monitoring point. The anomaly discrimination value of each test monitoring point represents the probability or score of the test monitoring point being judged as anomaly by the isolation forest model. The higher the anomaly discrimination value, the more likely the test monitoring point is to be anomaly.

[0097] It should be noted that the anomaly detection threshold in this application can be determined by the anomaly score distribution of historical data. For example, the percentile of the anomaly score (such as 95%) can be selected as the anomaly detection threshold. Local anomaly data refers to data with a large deviation under different monitoring parameters, which needs further research and detection.

[0098] Additionally, it should be noted that this step, by introducing an attention mechanism into the isolated forest model, can improve the detection accuracy of anomaly data and optimize small sample adaptability when processing complex geological test data.

[0099] In step 105, the spatial correlation anomaly data and the local anomaly data are fused to generate test anomaly data for the target region.

[0100] In some embodiments, the fusion of the spatially correlated anomaly data and the local anomaly data to generate experimental anomaly data for the target region can be achieved by the following steps:

[0101] The spatially correlated anomaly data and the local anomaly data are combined, and the combined anomaly data is visualized to obtain a three-dimensional visualization map.

[0102] The three-dimensional visualization map is stored as experimental anomaly data.

[0103] In practice, the combined abnormal data can be visualized and transformed to obtain a three-dimensional visualization map. This can be achieved in the following way: a three-dimensional data visualization tool can be used to convert the combined abnormal data into a three-dimensional map. In the three-dimensional map, spatial coordinates can usually be used as the X, Y, and Z axes. The values ​​of each monitoring parameter in the geological test data corresponding to each test monitoring point can be used as the color values ​​of the three-dimensional visualization map. Thus, the abnormal density and intensity of different areas can be displayed intuitively in the map.

[0104] Furthermore, in another aspect of this application, in some embodiments, this application provides a geological test data outlier detection system, with reference to... Figure 4 The figure is a schematic diagram of the structure of a geological test data outlier detection system according to some embodiments of this application. The geological test data outlier detection system includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below:

[0105] The data acquisition module 401 in this application is mainly used to acquire geological test data and corresponding spatial coordinates of various test monitoring points in the target area.

[0106] Processing module 402 in this application is mainly used to construct a three-dimensional spatial weight matrix based on all spatial coordinates, and then determine the spatial correlation difference between geological test data of each test monitoring point based on the three-dimensional spatial weight matrix and all geological test data.

[0107] The processing module 402 described in this application is also used to perform spatial correlation screening on the geological test data of each test monitoring point by combining the spatial correlation difference degree with the preset geological mechanism rules, so as to obtain spatial correlation abnormal data and spatial correlation normal data.

[0108] The processing module 402 described in this application is also used to input the spatially correlated normal data into the isolated forest model for local anomaly filtering and output local anomaly data;

[0109] The execution module 403 in this application is mainly used to fuse the spatial correlation anomaly data and the local anomaly data to generate test anomaly data for the target area.

[0110] Each module in the aforementioned geological test data anomaly detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0111] In another embodiment, this application provides a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores geological test data anomaly detection data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for detecting geological test data anomalies.

[0112] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0113] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiment of the geological test data anomaly detection method.

[0114] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described embodiment of the geological test data outlier detection method.

[0115] In one embodiment, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps described in the embodiment of the geological test data outlier detection method.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0117] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting outliers in geological test data, characterized in that, Includes the following steps: Collect geological test data and corresponding spatial coordinates from various test and monitoring points in the target area; A three-dimensional spatial weight matrix is ​​constructed based on all spatial coordinates. Then, based on the three-dimensional spatial weight matrix and all geological test data, the spatial correlation difference between geological test data at each test monitoring point is determined. By combining the spatial correlation differences with the preset geological mechanism rules, the geological test data of each test monitoring point are spatially correlated and filtered to obtain spatially correlated abnormal data and spatially correlated normal data. The spatially correlated normal data is input into the isolated forest model for local anomaly filtering, and the local anomaly data is output. The spatial correlation anomaly data and the local anomaly data are fused to generate test anomaly data for the target area. Specifically, determining the spatial correlation difference between geological test data at each test monitoring point based on the three-dimensional spatial weight matrix and all geological test data includes: The spatial dependency of each test monitoring point is determined based on all geological test data and the aforementioned three-dimensional spatial weight matrix. Based on the spatial dependence of each test monitoring point and geological test data, the adjacent trend data of each test monitoring point are determined; Determine the adjacent deviation of different monitoring parameters between adjacent trend data and corresponding geological test data for each test monitoring point; Calculate the spatial correlation difference between geological test data at each test monitoring point based on all adjacent deviations; By combining spatial correlation differences with preset geological mechanism rules, spatial correlation screening is performed on geological test data from various test monitoring points to obtain spatially correlated abnormal data and spatially correlated normal data, specifically including: Obtain preset geological mechanism rules, and then determine the difference threshold based on the preset geological mechanism rules; For each test monitoring point, the spatial correlation difference of the test monitoring point is compared with the difference threshold. When the spatial correlation difference is greater than or equal to the difference threshold, the geological test data of the test monitoring point will be regarded as spatial correlation anomaly data. When the spatial correlation difference is less than the difference threshold, the geological test data of the test monitoring point is taken as the normal spatial correlation data, and then the abnormal spatial correlation data and the normal spatial correlation data are obtained. The difference threshold is determined according to the preset geological mechanism rules in the following way: the difference threshold is calculated by weighting the monitoring thresholds in the preset geological mechanism rules.

2. The method as described in claim 1, characterized in that, Constructing a three-dimensional spatial weight matrix based on all spatial coordinates specifically includes: Calculate the spatial distance between each test monitoring point based on all spatial coordinates; The spatial weight value between the corresponding test monitoring points is calculated based on the spatial distance between each test monitoring point. The spatial weight value is inversely proportional to the spatial distance between the corresponding test monitoring points. Construct a three-dimensional spatial weight matrix by relating all the calculated spatial weight values ​​to spatial coordinates.

3. The method as described in claim 1, characterized in that, The spatially correlated normal data is input into the isolated forest model for local anomaly filtering, and the output local anomaly data specifically includes: The spatially correlated normal data is preprocessed; Construct an isolation forest model based on an attention mechanism, where the attention mechanism enhances the ability to identify abnormal data by weighting specific input features; The preprocessed spatially correlated normal data is input into the attention-based isolation forest model, and the anomaly discrimination value of each test monitoring point in the spatially correlated normal data is output. For each test monitoring point in the spatially correlated normal data, the geological test data corresponding to all test monitoring points whose anomaly discrimination value is greater than the preset anomaly discrimination threshold are regarded as local anomaly data.

4. The method as described in claim 1, characterized in that, The specific steps for generating test anomaly data for the target region by fusing the spatial correlation anomaly data and the local anomaly data include: The spatially correlated anomaly data and the local anomaly data are combined, and the combined anomaly data is visualized to obtain a three-dimensional visualization map. The three-dimensional visualization map is stored as experimental anomaly data.

5. The method as described in claim 1, characterized in that, The geological test data includes: water content, porosity, thermal conductivity, and chemical element content.

6. A geological test data outlier detection system, which uses the method described in any one of claims 1 to 5 to detect geological test data outliers, characterized in that, The system includes: The data acquisition module is used to collect geological test data and corresponding spatial coordinates from various test monitoring points in the target area. The processing module is used to construct a three-dimensional spatial weight matrix based on all spatial coordinates, and then determine the spatial correlation difference between geological test data of each test monitoring point based on the three-dimensional spatial weight matrix and all geological test data. The processing module is also used to perform spatial correlation screening on the geological test data of each test monitoring point by combining the spatial correlation difference degree with the preset geological mechanism rules, so as to obtain spatial correlation abnormal data and spatial correlation normal data. The processing module is also used to input the spatially correlated normal data into the isolated forest model for local anomaly filtering and output local anomaly data; The execution module is used to fuse the spatial correlation anomaly data and the local anomaly data to generate test anomaly data for the target area.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the geological test data anomaly detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the geological test data outlier detection method as described in any one of claims 1 to 5.

Images