Abnormal data detection method and system for regional health big data

Abstract

The invention relates to the technical field of abnormal data detection, and discloses an abnormal data detection method and system for regional health big data, and the method comprises the steps: obtaining the mixed k-nearest neighbor of each data point based on a data set, and calculating the altitude of each data point; obtaining an absolute median difference variability factor by using an absolute median difference based on the data point and the k neighbor altitude thereof; based on the data points and the k-nearest neighbor altitude thereof, obtaining two-stage Gaussian variability factors by using overlapped Gaussian distribution for k-nearest neighbor; obtaining an angle variability factor based on the mixed k neighbor and the local angle variance; and on the basis of the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, obtaining an abnormal factor for detecting abnormal data, and obtaining an abnormal data point set according to the abnormal factor and a threshold value. According to the method, the defect and limitation that multiple abnormal factors are not considered in a traditional method are overcome, and the efficiency and quality of the abnormal data detection method are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal data detection, and in particular to an abnormal data detection method and system for regional health big data. Background Art

[0002] Abnormal data detection is one of the important tasks of data mining and technical data pattern recognition. Its purpose is to find targets or patterns that have significant abnormality or variability with most data objects or data patterns. Its related technologies have been widely used in finance, medical care, industry, transportation and other fields. Obviously, the data pattern analysis of regional health big data is also inseparable from the analysis and detection of abnormal data. For example, with the rapid development of the Internet and big data technology, the scale and types of medical and health data have grown rapidly, making the differences in pattern information presented by healthy people, sub-healthy people and diseased people, as well as different disease populations more significant. However, different patterns contain a lot of abnormal data, which affects people's analysis of health data and diagnosis of diseases. In addition, such abnormal data poses difficulties and challenges to regional health in data analysis and disease prevention and control. For example, new viruses produce symptoms similar to traditional diseases, and a small number of samples of multi-symptom complex diseases may affect the pattern analysis of health big data, thereby attracting researchers' attention and research on abnormal data. In the process of analyzing and detecting abnormal data, we found that its complexity in the feature space manifests itself in various forms: (1) located in the fuzzy boundary area of ​​multiple adjacent classes; (2) embedded in the sparse area at the edge of a class and having a similar distribution to non-abnormal points; (3) having similar local information features to data points in the weakly connected area within the class, etc.

[0003] In order to effectively and efficiently detect abnormal data, distance-based methods, density-based methods, distribution-based methods, angle-based methods, statistics-based methods, and graph-based methods have been proposed for detecting abnormal data in various neighborhoods. However, these methods mainly consider the abnormal characteristics of data from a single factor or a unique perspective, which makes them lack consideration of multiple abnormal factors and have poor performance. For example, the angle-based method uses angle information to detect abnormal data, but the computational time complexity of the global angle variance is high, and it lacks consideration of local information. The density-based method uses a unified density threshold to measure the abnormality of data, which is difficult to overcome the abnormality of multi-density and multi-distribution data. Summary of the invention

[0004] The purpose of the present invention is to provide an abnormal data detection method and system for regional health big data, so as to overcome the shortcomings and limitations of traditional methods that lack consideration of multiple abnormal factors, and significantly improve the efficiency and quality of the abnormal data detection method.

[0005] In order to achieve the above purpose, the technical solutions adopted are as follows:

[0006] In a first aspect, a method for detecting abnormal data for regional health big data is provided, the method comprising:

[0007] Acquire a data set, the data set comprising a plurality of data points, acquire a mixed k-nearest neighbor of each data point based on the data set, and calculate the altitude of each data point; wherein the data point is medical image data of the stomach, eye, blood vessel, heart, lung, liver or kidney;

[0008] Based on the data points and their k nearest neighbor elevations, the absolute median difference is used to obtain the absolute median difference variability factor;

[0009] Based on the elevation of the data point and its k nearest neighbors, the two-stage Gaussian variability factor is obtained using the overlapping Gaussian distribution of the k nearest neighbors.

[0010] Obtaining the angle variability factor based on hybrid k-nearest neighbors and local angle variance;

[0011] Based on the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, an abnormal factor for detecting abnormal data is obtained, and an abnormal data point set is obtained according to the abnormal factor and the threshold.

[0012] Optionally, calculate the elevation of each data point using the following formula:

[0013]

[0014] In the formula, α(x i ) represents the i-th data point x in the data set i The altitude, ρ(x i ) represents the data point x i The reverse k-nearest neighbor ratio coefficient, γ(x i ) represents the data point x i The shared neighbor ratio coefficient, d(x i ,x j ) represents the data point x i To its k nearest neighbor member x j Distance, N k (x i ) represents the data point x i The k nearest neighbors.

[0015] Optionally, based on the data point and its k nearest neighbor elevations, the absolute median difference is used to obtain an absolute median difference variability factor, including;

[0016] According to the elevation of each data point and its k nearest neighbors, the elevation of each member in the k nearest neighbors and the elevation of its k nearest neighbors, the forward absolute median difference variability factor and the backward absolute median difference variability factor of the local area elevation are calculated using the absolute median difference;

[0017] The forward absolute median difference variability factor and the backward absolute median difference variability factor are used to calculate the absolute median difference variability factor.

[0018] Optionally, the midpoint difference variability factor is calculated by the following formula:

[0019]

[0020] In the formula, FMADW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the forward absolute median difference variability factor value, RMADW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the backward absolute median difference variability factor value. MADW(x i , x j ) represents the data point pair (x i , x j ) between the absolute median difference variability factor value. When FMADW(α(x i ), α(x j )) and RMADW(α(x i ), α(x j ))When at least one of the conditions is met, MADW(x i , x j ) is logically true (True), otherwise, it is logically false (False).

[0021] Optionally, based on the elevations of the data points and their k nearest neighbors, a two-stage Gaussian variability factor is obtained using overlapping Gaussian distributions of k nearest neighbors, including:

[0022] According to the elevation of each data point and its k nearest neighbors, the elevation of each member in the k nearest neighbors and the elevation of its k nearest neighbors, the overlapping Gaussian distribution of the k nearest neighbors is used to calculate the first-stage 1σ Laida rule of the local area elevation set, and filter out abnormal data; the second-stage 1σ Laida rule of the remaining elevation set is calculated to obtain the local non-abnormal elevation set;

[0023] Using the local non-abnormal altitude set, calculating a forward two-stage Gaussian variability factor and a backward two-stage Gaussian variability factor;

[0024] The two-stage Gaussian variability factor is calculated based on the forward two-stage Gaussian variability factor and the backward two-stage Gaussian variability factor.

[0025] Optionally, the two-stage Gaussian variability factor is calculated by the following formula:

[0026]

[0027] In the formula, TTW(x i , x j ) represents the data point pair (x i , x j ) between the two-stage Gaussian variability factor values, FTTW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the forward two-stage Gaussian variability factor value, RTTW(α(x j ), α(x i )) represents the data point pair (x i , x j ) between the backward two-stage Gaussian variability factor values. i ), α(x j )) and RTTW(α(x j ), α(x i ))When at least one of the conditions is met, TTW(x i , x j ) is logically true (True), otherwise, it is logically false (False).

[0028] Optionally, the angle variability factor is calculated by the following formula:

[0029]

[0030] In the formula, θ(x i ) represents the data point x i The angle variability factor, Var() represents the variance operation, SNN k (x i , x j ) represents the data point pair (x i , x j ), acos() represents the arc cosine operation, τ jq The sum of the lengths of the two sides that represent the angle.

[0031] Optionally, the threshold includes a first threshold and a second threshold, and according to the abnormal factor and the threshold, the abnormal data point set is obtained by the following formula:

[0032]

[0033]

[0034] In the formula, ψ(x i ) takes a value of 0, indicating that the data point x i is an abnormal data point, ψ(x i ) takes a value of 1, indicating that the data point x i is a non-abnormal point, when and θ(x i )≥∈ 2 When at least one of the conditions is satisfied, ψ(x i ) takes the value of 0, otherwise it takes the value of 1. i , x j ) represents x i and the k nearest neighbor member x j Topological variability measure between RNN k (x i , x j ) represents x i With x j There is a reverse k-nearest neighbor relationship between them; ∈ 1 represents the first threshold, ∈ 2 represents the second threshold; when RNN k (x i , x j )、MADW(x i , x j ) and TTW(x i , x j ) satisfies at least one of the following conditions: OF(x i , x j ) takes the value 0, otherwise takes the value 1.

[0035] In a second aspect, a system for detecting abnormal data for regional health big data is provided, the system comprising:

[0036] A data acquisition module is configured to acquire a data set, the data set comprising a plurality of data points, acquire a mixed k-nearest neighbor of each data point based on the data set, and calculate the altitude of each data point; wherein the data point is medical image data of the stomach, eye, blood vessel, heart, lung, liver or kidney;

[0037] A first calculation module is configured to obtain an absolute median difference variability factor based on the data point and its k nearest neighbor elevations using the absolute median difference;

[0038] The second calculation module is configured to obtain a two-stage Gaussian variability factor based on the data point and its k-nearest neighbor elevations using overlapping Gaussian distributions of k-nearest neighbors;

[0039] A third calculation module is configured to obtain an angle variability factor based on a hybrid k-nearest neighbor and a local angle variance;

[0040] The anomaly detection module is configured to obtain an anomaly factor for detecting abnormal data based on the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, and obtain an abnormal data point set according to the anomaly factor and the threshold.

[0041] Optionally, the data acquisition module is further configured to:

[0042] The altitude of each data point is calculated using the following formula:

[0043]

[0044] In the formula, α(x i ) represents the i-th data point x in the data set i The altitude, ρ(x i ) represents the data point x i The reverse k-nearest neighbor ratio coefficient, γ(x i ) represents the data point x i The shared neighbor ratio coefficient, d(x i , x j ) represents the data point x i To its k nearest neighbor member x j Distance to data point x i The ratio of the distances to all k nearest neighbors of the data point, N k (x i ) represents the data point x i The k nearest neighbors.

[0045] The beneficial effects of the present invention are:

[0046] The present invention makes full use of mixed k-nearest neighbors, local terrain topology, overlapping Gaussian distribution, abnormal numerical detection method and local angle variance method, solves the shortcomings of traditional clustering methods that do not fully consider the structural complexity of data space and the low efficiency of existing watershed clustering methods, and can greatly improve the detection efficiency and quality of abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flow chart of an abnormal data detection method for regional health big data according to an embodiment of the present invention is shown.

[0048] Figure 2 A flow chart for implementing terrain topology according to an embodiment of the present invention is shown.

[0049] Figure 3 A flowchart for calculating the absolute median difference variability factor according to an embodiment of the present invention is shown.

[0050] Figure 4 A flowchart for calculating a two-stage Gaussian variability factor according to an embodiment of the present invention is shown.

[0051] Figure 5 A flowchart for calculating the angle variability factor according to an embodiment of the present invention is shown.

[0052] Figure 6 A structural diagram of an abnormal data detection system for regional health big data according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0054] The specific implementation of the present invention is further described in detail below in conjunction with the drawings and examples.

[0055] In view of the complex characteristics of abnormal data in the data space of regional health big data and the shortcomings of existing methods that fail to fully consider multiple complex factors, an embodiment of the present invention provides an abnormal data detection method and system for regional health big data to overcome the limitations of traditional abnormal data detection methods based on a single abnormal factor.

[0056] On the one hand, an embodiment of the present invention provides an abnormal data detection method for regional health big data, which is based on the watershed concept. First, the terrain topology of the data space is realized; then, the absolute median difference variability factor is established; secondly, the two-stage Gaussian threshold variability factor is realized; finally, the mixed k-nearest neighbor, terrain topology, absolute median difference variability factor and two-stage Gaussian variability factor are used to obtain the anomaly factor for detecting abnormal data, and the abnormal data point set is obtained based on the anomaly factor value and the threshold.

[0057] Specifically, see Figure 1 , Figure 1 FIG. 1 is a flow chart showing a method for detecting abnormal data for regional health big data according to an embodiment of the present invention. Figure 1 As shown, the method includes steps S100 to S500, which are described in detail as follows:

[0058] S100: Obtain a data set, which involves medical image data of the stomach, eyes, blood vessels, heart, lungs, liver, kidneys and other parts, obtain a mixed k-nearest neighbor of each data point based on the data set, and calculate the altitude of each data point.

[0059] In this embodiment, the purpose of step S100 is to calculate the altitude of each data point to achieve topological topology of the data space. Specifically, the altitude of the data point x is calculated based on the input data set and the similarity metric. i Mixed k-nearest neighbors (including k-nearest neighbors: N k (x i ), Reverse k-nearest neighbor: RNN k (x i , x j ), Shared k-nearest neighbors: SNN k (x i , x j ), density information and distribution information, calculate each data point x i The altitude α(x i ) to obtain the terrain topology of the data space.

[0060] In an exemplary embodiment, see Figure 2 , is a flowchart for realizing terrain topology, step S100 specifically includes steps S101 to S104, which are described in detail as follows:

[0061] S101: Calculate each data point x using Euclidean distance i The k nearest neighbors N k (x i ), Shared k-nearest neighbor SNN k (x i , x j ), reverse k-nearest-neighbor RNN k (x i , x j ), and obtain the mixed k nearest neighbors in the data space.

[0062] S102: Based on the mixed k-nearest neighbors, calculate the reverse k-nearest neighbor ratio, the formula is as follows:

[0063]

[0064] In the formula, ρ(x i ) represents the data point x i The reverse k-nearest neighbor ratio, k represents the number of neighbors, N k (x i ) represents the data point x i k nearest neighbors, RNN k (x i , x j ) represents the data point pair (x i , x j ) between two nodes, the return value is 1 if the relationship is true, otherwise it is 0.

[0065] S103: Calculate the shared k-nearest neighbor ratio, the formula is as follows:

[0066]

[0067] In the formula, γ(x i ) represents the data point x i The shared k-nearest neighbor ratio of |SNN k (x i , x j )| represents the data point pair (x i , x j ) is the number of common k nearest neighbors between them.

[0068] S104: Calculate the altitude of the data point using the following formula:

[0069]

[0070] In the formula, α(x i ) represents the data point x i The altitude, d(x i , x j ) represents x i To the nearest neighbor x j Distance and x i The ratio of the sum of the distances to its k nearest neighbors.

[0071] S200: Based on the data points and their k nearest neighbor elevations, the absolute median difference is used to obtain the absolute median difference variability factor.

[0072] In this embodiment, the data point x is obtained according to the k nearest neighbor altitude and absolute median difference method. i With neighbor x j The variability factor of the absolute median difference MADW(x i , x j ).

[0073] In an exemplary embodiment, see Figure 3 , which is a flow chart for calculating the absolute median difference variability factor, step S200 includes steps S201 to S203, which are described in detail as follows.

[0074] Step S201: Calculate ψ(·) based on the median data to obtain the data point x i The median of the k nearest neighbor elevations: ψ(α(N t (x i ))) Then get the median of the absolute value of the difference between a single altitude point and the median in the k nearest neighbor area: MAD i =1.4826*ψ(|α(N t (x i ))-ψ(α(N t (x i )))|), thus obtaining the data point x iWith neighbor x j The forward absolute median difference variability factor is as follows:

[0075]

[0076] In the formula, FMADW(x i , x i ) represents the data point x i For data point x j The forward absolute median difference variability factor, λ is 2, α(N k (x i )) represents x i The elevation set within the neighborhood of the k nearest neighbors, True represents the logical true value, and False represents the logical false value.

[0077] Step S202: According to the data point x j The median of the k nearest neighbor elevations ψ(α(N k (x j ))) Then get the median of the absolute value of the difference between a single altitude point and the median in the k nearest neighbor area: MAD j =1.4826*ψ(|α(N k (x j ))-ψ(α(N k (x j )))|), thus obtaining the data point x i With neighbor x j The backward absolute median difference variability factor is as follows:

[0078]

[0079] In the formula, RMADW(x i , x j ) represents the data point x i For data point x j The reverse absolute median difference variability factor, α(N k (x j )) represents x j The set of elevations within the neighborhood of the k nearest neighbors.

[0080] Step S203: According to the data point x i and its k nearest neighbor elevations, each member x in the k nearest neighbor point j The altitude and its k nearest neighbor altitudes are combined with the forward absolute median difference variability factor and the backward absolute median difference variability factor to obtain the absolute median difference variability factor, and the formula is as follows:

[0081]

[0082] In the formula, MADW(xi , x j ) represents the data point pair (x i , x j ) variability.

[0083] S300: Based on the data point and its k nearest neighbor elevations, a two-stage Gaussian variability factor is obtained using overlapping Gaussian distributions of k nearest neighbors.

[0084] In this embodiment, the k nearest neighbor altitude and overlapping Gaussian distribution method are used to obtain the data point x i With neighbor x j The two-stage Gaussian variability factor TTW(x i , x j ).

[0085] In an exemplary embodiment, see Figure 4 , which is a flowchart of a flowchart for calculating a two-stage Gaussian variability factor, step S300 includes steps S301 to S303, which are described in detail as follows.

[0086] Step S301: Using data point x i The elevation set α(N k (x i )), using the 1σ Laida rule, we get the data point set: ds i ={x j ||α(x j )-μ(α(N k (x i )))|≤σ(α(N k (x i ))), x j ∈N t (x i )}, and obtain x i The forward two-stage Gaussian variability factor is as follows:

[0087]

[0088] In the formula, FTTW(α(x i ), α(x j )) represents the data point x i With x j The forward two-stage Gaussian variability factor of .

[0089] Step S302: Using data point x j The elevation set α(N k (x j )), using the 1σ Laida rule, we get the data point set: ds j ={x i||α(x i )-μ(α(N k (x j )))|≤σ(α(N k (x j ))), x i ∈N k (x j )}, and obtain x i The forward two-stage Gaussian variability factor is as follows:

[0090]

[0091] In the formula, RTTW(α(x j ), α(x i )) represents the data point x i With x j The inverse two-stage Gaussian variability factor of .

[0092] Step S303: According to the data point x i and its k nearest neighbor elevations, each member x in the k nearest neighbor point j The elevation of and its k nearest neighbor elevations, combined with the forward two-stage Gaussian variability factor and the reverse two-stage Gaussian variability factor, is x i and the k nearest neighbor member x j The two-stage Gaussian variability factor TTW(x i , x j ), the formula is as follows:

[0093]

[0094] S400: Obtaining an angle variability factor based on a hybrid k-nearest neighbor and local angle variance.

[0095] In an exemplary embodiment, see Figure 5 , which is a flowchart of a flowchart for calculating the angle variability factor, step S400 includes steps S401 to S402, which are described in detail as follows.

[0096] Step S401: Based on the data point x i and its k nearest neighbors (N k (x i ))Member x j , shared k nearest neighbors (SNN k (x i , x j ))Member x q , calculate x i is the distance coefficient of the angle between the vertex vectors, and the formula is as follows:

[0097]

[0098] In the formula, || || represents the modular operation of the vector.

[0099] Step S402: Based on the distance coefficient and x i The arc cosine angle and variance of the angle between the vectors at the vertex, x i The angular variability factor θ(x i ), the formula is as follows:

[0100]

[0101] In the formula, acos() represents the arc cosine operation. i The angle variability factor value and threshold judgment x i Whether it is an abnormal data point.

[0102] S500: Based on the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, the abnormal factor for detecting abnormal data is obtained, and the abnormal data point set is obtained according to the abnormal factor and the threshold. The abnormal data mainly manifests as image data of non-significant lesion features with or without disease in the brain, cervical spine, lumbar spine, lungs, abdominal cavity, limbs and other parts.

[0103] In an exemplary embodiment, the threshold includes a first threshold and a second threshold, and the abnormal data point set is obtained by the following formula:

[0104]

[0105]

[0106] In the formula, ψ(x i ) takes a value of 0, indicating that the data point x i is an abnormal data point. and θ(x i )≥∈ 2 When at least one of the conditions is satisfied, ψ(x i ) takes the value of 0, otherwise it takes the value of 1. i , x j ) represents x i and the k nearest neighbor member x j The topological variability measure between (here can be used as an abnormal factor); RNN k (x i , x j ) represents x i With x j There is a reverse k-nearest neighbor relationship between them; ∈ 1 represents the first threshold, ∈ 2 represents the second threshold. k (xi , x j )、MADW(x i , x j ) and TTW(x i , x j ) satisfies at least one of the following conditions: OF(x i , x j ) takes the value 0, otherwise takes the value 1.

[0107] On the other hand, the embodiment of the present invention also provides an abnormal data detection system for regional health big data, see Figure 6 , the system comprises:

[0108] The data acquisition module 601 is configured to acquire a data set, the data set including medical image data of multiple body parts such as the brain, cervical spine, lumbar spine, lungs, abdomen, limbs, etc., acquire a mixed k-nearest neighbor of each data point based on the data set, and calculate the altitude of each data point;

[0109] A first calculation module 602 is configured to obtain an absolute median difference variability factor based on the data point and its k nearest neighbor elevations using the absolute median difference;

[0110] The second calculation module 603 is configured to obtain a two-stage Gaussian variability factor based on the data point and its k-nearest neighbor elevations using overlapping Gaussian distributions of k-nearest neighbors;

[0111] A third calculation module 604 is configured to obtain an angle variability factor based on a hybrid k-nearest neighbor and a local angle variance;

[0112] The anomaly detection module 605 is configured to obtain an anomaly factor for detecting abnormal data based on the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, and obtain an abnormal data point set according to the anomaly factor and the threshold, that is, an image data set of non-significant lesion features with or without disease.

[0113] In some embodiments, the data acquisition module 601 is further configured to:

[0114] The altitude of each data point is calculated using the following formula:

[0115]

[0116] In the formula, α(x i ) represents the i-th data point x in the data set i The altitude, ρ(x i ) represents the data point x i The reverse k-nearest neighbor ratio coefficient, γ(x i ) represents the data point x iThe shared neighbor ratio coefficient, d(x i , x j ) represents the data point x i To its k nearest neighbor member x j Distance to data point x i The ratio of the distances to all k nearest neighbors of the data point, N k (x i ) represents the data point x i The k nearest neighbors.

[0117] In some embodiments, the first computing module is further configured to:

[0118] According to the elevation of each data point and its k nearest neighbors, the elevation of each member in the k nearest neighbors and the elevation of its k nearest neighbors, the forward absolute median difference variability factor and the backward absolute median difference variability factor of the local area elevation are calculated using the absolute median difference;

[0119] The forward absolute median difference variability factor and the backward absolute median difference variability factor are used to calculate the absolute median difference variability factor.

[0120] In some embodiments, the first calculation module is further configured to calculate the median position difference variability factor by the following formula:

[0121]

[0122] In the formula, MADW(x i , x j ) represents the data point pair (x i , x j ) between the absolute median difference variability factor value, True means logical true, False means logical false, FMADW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the forward absolute median difference variability factor value, RMADW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the backward absolute median difference variability factor value. When FMADW(α(x i ), α(x j )) and RMADW(α(x i ), α(x j ))When at least one of the conditions is met, MADW(x i , x j ) is logically true (True), otherwise, it is logically false (False).

[0123] In some embodiments, the second computing module is further configured to:

[0124] According to the elevation of each data point and its k nearest neighbors, the elevation of each member in the k nearest neighbors and the elevation of its k nearest neighbors, the overlapping Gaussian distribution of the k nearest neighbors is used to calculate the first-stage 1σ Laida rule of the local area elevation set, and filter out abnormal data; the second-stage 1σ Laida rule of the remaining elevation set is calculated to obtain the local non-abnormal elevation set;

[0125] Using the local non-abnormal altitude set, calculating a forward two-stage Gaussian variability factor and a backward two-stage Gaussian variability factor;

[0126] The two-stage Gaussian variability factor is calculated based on the forward two-stage Gaussian variability factor and the backward two-stage Gaussian variability factor.

[0127] In some embodiments, the second calculation module is further configured to calculate the two-stage Gaussian variability factor by the following formula:

[0128]

[0129] In the formula, TTW(x i , x j ) represents the data point pair (x i , x j ) between the two-stage Gaussian variability factor values, FTTW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the forward two-stage Gaussian variability factor value, RTTW(α(x j ), α(x i )) represents the data point pair (x i , x j ) between the backward two-stage Gaussian variability factor values. i ), α(x j )) and RTTW(α(x j ), α(x i ))When at least one of the conditions is met, TTW(x i , x j ) is logically true (True), otherwise, it is logically false (False).

[0130] In some embodiments, the third calculation module is further configured to calculate the angle variability factor by the following formula:

[0131]

[0132] In the formula, θ(x i ) represents the data point x i The angle variability factor, Var() represents the variance operation, SNN k (x i , x j ) represents the data point pair (x i , x j ), acos() represents the arc cosine operation, τ jq The sum of the lengths of the two sides that represent the angle.

[0133] In some embodiments, the threshold includes a first threshold and a second threshold, and the abnormality detection module is further configured to obtain an abnormal medical image data set according to the abnormal factor and the threshold by the following formula.

[0134]

[0135]

[0136] In the formula, OF(x i , x j ) represents x i and the k nearest neighbor member x j Topological variability measure between RNN k (x i , x j ) represents x i With x j There is a reverse k-nearest neighbor relationship between them; ∈ 1 represents the first threshold, ∈ 2 represents the second threshold, ψ(x i ) takes a value of 0, indicating that the data point x i is an abnormal data point, ψ(x i ) takes a value of 1, indicating that the data point x i It is non-abnormal image data, that is, image data with non-significant lesion features that indicate disease or no disease.

[0137] It should be noted that the abnormal data detection system for regional health big data belongs to the same technical concept as the method described previously, and has the same technical principles and beneficial effects, so it will not be repeated here.

[0138] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The patent protection scope of the present invention should be defined by the claims.

Claims

1. A method for detecting abnormal data for regional health big data, characterized in that: The method comprises: Acquire a data set, the data set comprising a plurality of data points, acquire a mixed k-nearest neighbor of each data point based on the data set, and calculate the altitude of each data point; wherein the data point is medical image data of the stomach, eye, blood vessel, heart, lung, liver or kidney; Based on the data points and their k nearest neighbor elevations, the absolute median difference is used to obtain the absolute median difference variability factor; Based on the elevation of the data point and its k nearest neighbors, the two-stage Gaussian variability factor is obtained using the overlapping Gaussian distribution of the k nearest neighbors. Obtaining the angle variability factor based on hybrid k-nearest neighbors and local angle variance; Based on the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, an abnormal factor for detecting abnormal data is obtained, and an abnormal data point set is obtained according to the abnormal factor and the threshold.

2. The method according to claim 1, characterized in that The altitude of each data point is calculated using the following formula: In the formula, α(x i ) represents the i-th data point x in the data set i The altitude, ρ(x i ) represents the data point x i The reverse k-nearest neighbor ratio coefficient, γ(x i ) represents the data point x i The shared neighbor ratio coefficient, d(x i , x j ) represents the data point x i To its k nearest neighbor member x j The distance, N k (x i ) represents the data point x i The k nearest neighbors.

3. The method according to claim 1, characterized in that Based on the data points and their k nearest neighbor elevations, the absolute median difference is used to obtain the absolute median difference variability factors, including; According to the elevation of each data point and its k nearest neighbors, the elevation of each member in the k nearest neighbors and the elevation of its k nearest neighbors, the forward absolute median difference variability factor and the backward absolute median difference variability factor of the local area elevation are calculated using the absolute median difference; The forward absolute median difference variability factor and the backward absolute median difference variability factor are used to calculate the absolute median difference variability factor.

4. The method according to claim 2, characterized in that The center position variability factor is calculated using the following formula: In the formula, FMADW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the forward absolute median difference variability factor value, RMADW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the backward absolute median difference variability factor value. MADW(x i , x j ) represents the data point pair (x i , x j ) between the absolute median difference variability factor value; when FMADW(α(x i ), α(x j )) and RMADW(α(x i ), α(x j ))When at least one of the conditions is met, MADW(x i , x j ) is logically true (True), otherwise, it is logically false (False).

5. The method according to claim 1, characterized in that Based on the data points and their k nearest neighbor elevations, the overlapping Gaussian distribution of k nearest neighbors is used to obtain the two-stage Gaussian variability factors, including: According to the elevation of each data point and its k nearest neighbors, the elevation of each member in the k nearest neighbors and the elevation of its k nearest neighbors, the overlapping Gaussian distribution of the k nearest neighbors is used to calculate the first-stage 1σ Laida rule of the local area elevation set, and filter out abnormal data; the second-stage 1σ Laida rule of the remaining elevation set is calculated to obtain the local non-abnormal elevation set; Using the local non-abnormal altitude set, calculating a forward two-stage Gaussian variability factor and a backward two-stage Gaussian variability factor; The two-stage Gaussian variability factor is calculated based on the forward two-stage Gaussian variability factor and the backward two-stage Gaussian variability factor.

6. The method according to claim 5, characterized in that The two-stage Gaussian variability factor is calculated by the following formula: In the formula, FTTW(α(x i ), α(x j )) represents the data point pair (x i , x j ) between the forward two-stage Gaussian variability factor value, RTTW(α(x j ), α(x i )) represents the data point pair (x i , x j ) between the backward two-stage Gaussian variability factor value; TTW(x i , x j ) represents the data point pair (x i , x j ) between the two-stage Gaussian variability factor values. When FTTW(α(x i ), α(x j )) and RTTW(α(x j ), α(xi)) is satisfied, TTW(x i , x j ) is logically true (True), otherwise, it is logically false (False).

7. The method according to claim 6, characterized in that The angle variability factor is calculated using the following formula: In the formula, x j Represents x i k nearest neighbor members, SNN k (x i , x j ) represents the data point pair (x i , x j ), x q Represents x i With x j The members of the shared k nearest neighbors, θ(x i ) represents the data point x i The angle variability factor, Var() represents the variance operation, acos() represents the arc cosine operation, τ jq The sum of the lengths of the two sides that represent the angle.

8. The method according to claim 7, characterized in that The threshold includes a first threshold and a second threshold. According to the abnormal factor and the threshold, the abnormal data point set is obtained by the following formula: In the formula, ψ(x i ) takes a value of 0, indicating that the data point x i is an abnormal data point, ψ(x i ) takes a value of 1, indicating that the data point x i is a non-abnormal point, when and θ(x i )≥∈ 2 When at least one of the conditions is met, ψ(x i ) takes the value 0, otherwise takes the value 1. i , x j ) represents x i and the k nearest neighbor member x j Topological variability measure between RNN k (x i , x j ) represents x i With x j There is a reverse k-nearest neighbor relationship between them; ∈ 1 represents the first threshold, ∈ 2 represents the second threshold; when RNN k (x i , x j )、MADW(x i , x j ) and TTW(x i , x j ) satisfies at least one of the following conditions: OF(x i , x j ) takes the value 0, otherwise takes the value 1.

9. An abnormal data detection system for regional health big data, characterized in that: The system comprises: A data acquisition module is configured to acquire a data set, the data set comprising a plurality of data points, acquire a mixed k-nearest neighbor of each data point based on the data set, and calculate the altitude of each data point; wherein the data point is medical image data of the stomach, eye, blood vessel, heart, lung, liver or kidney; A first calculation module is configured to obtain an absolute median difference variability factor based on the data point and its k nearest neighbor elevations using the absolute median difference; The second calculation module is configured to obtain a two-stage Gaussian variability factor based on the data point and its k-nearest neighbor elevations using overlapping Gaussian distributions of k-nearest neighbors; A third calculation module is configured to obtain an angle variability factor based on a hybrid k-nearest neighbor and a local angle variance; The anomaly detection module is configured to obtain an anomaly factor for detecting abnormal data based on the absolute median difference variability factor, the two-stage Gaussian variability factor and the angle variability factor, and obtain an abnormal data point set according to the anomaly factor and the threshold.

10. The method according to claim 9, characterized in that The data acquisition module is further configured as follows: The altitude of each data point is calculated using the following formula: In the formula, α(x i ) represents the i-th data point x in the data set i The altitude, ρ(x i ) represents the data point x i The reverse k-nearest neighbor ratio coefficient, γ(x i ) represents the data point x i The shared neighbor ratio coefficient, d(x i , x j ) represents the data point x i To its k nearest neighbor member x j Distance, N k (x i ) represents the data point x i The k nearest neighbors.

Images