An abnormal data identification method, device and equipment and storage medium
Patent Information
- Application Number
- CN202410406777.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-03
- Publication Date
- 2026-10-30
- Estimated Expiration
- 2044-04-03
AI Technical Summary
[0004]针对现有的车辆出险异常数据识别方法存在识别效率低、质量不高等缺点,提出一种异常数据的识别方法
[0066]借由上述技术方案,本申请通过输入车辆出险时间序列数据;设定最大允许连续偏离分量数、最小相似分量数;计算任意两组数据的差异;计算最大连续偏离分量数和相似分量数;进行数据聚类;识别数据是否异常。由此可见,该方法能够有效地识别车辆出险异常数据,具有速度快、准确率高等特点。
Smart Images

Figure CN118260608B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of insurance data processing, and more specifically, to a method, apparatus, device, and storage medium for identifying abnormal data. Background Technology
[0002] Anomalies in vehicle accident data can have various causes, such as driver skill level, climate, geographical environment, driver experience, and road quality. Quickly and accurately identifying anomaly data supports insurance companies in conducting high-quality accident analysis, pinpointing the underlying causes, and providing decision support for claims processing, premium adjustments, and policy term modifications. Furthermore, appropriate processing of identified anomaly data can effectively improve the efficiency and quality of machine learning in the insurance field.
[0003] This application addresses the problem of identifying abnormal data related to vehicle accidents.
[0004] To address the shortcomings of existing methods for identifying abnormal vehicle accident data, such as low efficiency and poor quality, this paper proposes a new method for anomaly data identification. The specific steps are as follows: input vehicle accident time series data; set the maximum allowable number of consecutive deviation components and the minimum number of similar components; calculate the difference between any two sets of data; calculate the maximum number of consecutive deviation components and the number of similar components; perform data clustering; and identify whether the data is abnormal. This application can effectively identify abnormal vehicle accident data, featuring high speed and high accuracy. Summary of the Invention
[0005] In view of the above problems, this application provides a method, apparatus, device and storage medium for identifying abnormal data, so as to identify abnormal data of vehicle accidents more quickly and accurately.
[0006] To achieve the above objectives, the following specific solutions are proposed:
[0007] A method for identifying anomalous data includes:
[0008] S1. Input the time series data of vehicle accidents X i =(x i1 ,x i2 ,..,x im ), i=1,2,...,n; where X i The sampled data contains T time intervals, each containing data from L time points, x ij Let x be the incident rate of the i-th group at time j, 0 ≤ x ij ≤1, i=1,2,...,n, j=1,2,...,m, T, n, and L are all positive integers, m=L×T;
[0009] S2. Set the maximum allowable number of consecutive deviation components and the minimum number of similar components; further including:
[0010] S2-1, Set the maximum allowable number of continuous deviation components MaxD; MaxD is set according to formula (1):
[0011] (1)
[0012] Where α1 and α2 are pre-given real numbers, 0 < α1 < 0.5, 0 < α2 < 0.3, and [x] represents the integer part of x;
[0013] S2-2, Set the minimum number of similar components MinS; MinS is set according to formula (2):
[0014] (2)
[0015] Where β is a pre-given real number, 0 < β < 1;
[0016] S3. Calculate the difference between any two sets of data; further including: for all s, t=1,2,...,n, j=1,2,...,m, calculate X according to equation (3). s X t Differences
[0017] (3)
[0018] S4. Calculate the number of maximum consecutive deviation components and the number of similar components; further including: for all s, t = 1, 2, ..., n,
[0019] S4-1, Calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows:
[0020]
[0021] Where γ is a pre-given real number, 0≤γ≤1;
[0022] S4-2, Calculate X s X t The number of similar components n st ;n st To satisfy d stj The number of subscripts j ≤ γ, where j = 1, 2, ..., m;
[0023] S5. Perform data clustering; the specific steps are as follows:
[0024] S5-1, Let s=1;
[0025] S5-2: Let C(X s )={X s}, t=1;
[0026] S5-3: If t≠s and n st ≥MinS and m st ≤MaxD, then C(X s )=C(X s )∪{X t};
[0027] S5-4: If t<m, then t=t+1, go to S5-3;
[0028] S5-5: If s<n, then s=s+1, go to S5-2.
[0029] S6: Identify whether the vehicle accident data is abnormal, which further comprises: for s=1,2,…,n, sequentially check C(X s ), if the number of elements in C(X s ) is greater than δn, the data X s is normal, otherwise it is abnormal; where δ is a preset real number, 1 / n≤δ≤1.
[0030] An abnormal data identification apparatus, comprising:
[0031] a vehicle accident time series data input unit, configured to execute: S1: Input vehicle accident time series data X i =(x i1 ,x i2 ,..,x im ), i=1,2,…,n; where X i contains sampling data of T time intervals, each time interval contains data of L time points, x ij is the accident rate at the j-th time point of the i-th group, 0≤x ij ≤1, i=1,2,…,n, j=1,2,…,m, T, n, L are all positive integers, m=L×T;
[0032] a component number setting unit, configured to execute: S2: Set the maximum allowable continuous deviation component number and the minimum similar component number; the component number setting unit comprises a maximum allowable continuous deviation component number setting unit and a minimum similar component number setting unit:
[0033] the maximum allowable continuous deviation component number setting unit, configured to execute: S2-1: Set the maximum allowable continuous deviation component number MaxD;
[0034] the minimum similar component number setting unit, configured to execute: S2-2: Set the minimum similar component number MinS;
[0035] The data difference calculation unit is used to perform: S3, calculate the difference between any two sets of data;
[0036] The component number calculation unit is used to perform: S4, calculating the number of maximum continuous deviation components and the number of similar components; the component number calculation unit includes a maximum continuous deviation component number calculation unit and a similarity component number calculation unit.
[0037] The maximum continuous deviation component calculation unit is used to perform: S4-1, for all s,t=1,2,…,n, calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows:
[0038]
[0039] Where γ is a pre-given real number, 0≤γ≤1;
[0040] The similarity component calculation unit is used to perform: S4-2, calculate X s X t The number of similar components n st ;n st For equation (3) to satisfy d stj The number of subscripts j ≤ γ, where j = 1, 2, ..., m;
[0041] The data clustering unit is used to perform: S5, data clustering;
[0042] The data anomaly identification unit is used to perform: S6, identifying whether the data of the vehicle accident is abnormal.
[0043] Optionally, the maximum permissible number of consecutive deviation components setting unit includes:
[0044] The maximum permissible number of consecutive deviation components setting subunit is used to execute: MaxD is set according to the following formula:
[0045] (1)
[0046] Where α1 and α2 are pre-given real numbers, 0 < α1 < 0.5, 0 < α2 < 0.3, and [x] represents the integer part of x.
[0047] Optionally, the minimum similarity component number setting unit includes:
[0048] The minimum similarity component number setting subunit is used to execute MinS according to the following formula:
[0049] (2)
[0050] where β is a predefined real number, 0<β<1.
[0051] Optionally, the data difference calculation unit comprises:
[0052] A data difference calculation subunit, configured to perform: for all s, t=1,2,…,n, j=1,2,…,m, calculate X according to the following formula s , X t the difference of:
[0053] (3)
[0054] Optionally, the data clustering unit comprises:
[0055] A first data clustering subunit, configured to perform: S5-1, set s=1;
[0056] A second data clustering subunit, configured to perform: S5-2, set C(X s )={X s}, t=1;
[0057] A third data clustering subunit, configured to perform: S5-3, if t≠s and n st ≥MinS and m st ≤MaxD, then C(X s )=C(X s )∪{X t};
[0058] A fourth data clustering subunit, configured to perform: S5-4, if t<m, then t=t+1, and execute the third data clustering subunit;
[0059] A fifth data clustering subunit, configured to perform: S5-5, if s<n, then s=s+1, and execute the second data clustering subunit.
[0060] Optionally, the data abnormality identification unit comprises:
[0061] A data abnormality identification subunit, configured to perform: for s=1,2,…,n, sequentially check C(X s ), if the number of elements in C(X s ) is greater than δn, then the data X s is normal, otherwise it is abnormal; where δ is a predefined real number, 1 / n≤δ≤1.
[0062] An abnormal data identification device, comprising a memory and a processor;
[0063] the memory is configured to store a program;
[0064] The processor is used to execute the program to implement the various steps of the abnormal data identification method described above.
[0065] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the abnormal data identification method described above.
[0066] Using the above technical solution, this application inputs vehicle accident time series data; sets the maximum allowable number of consecutive deviation components and the minimum number of similar components; calculates the difference between any two sets of data; calculates the maximum number of consecutive deviation components and the number of similar components; performs data clustering; and identifies whether the data is abnormal. Therefore, this method can effectively identify abnormal vehicle accident data and has the characteristics of high speed and high accuracy. Attached Figure Description
[0067] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0068] Figure 1 This is a schematic diagram of a process for implementing abnormal data identification provided in an embodiment of this application;
[0069] Figure 2 A schematic diagram of a device structure for implementing abnormal data identification, provided in an embodiment of this application;
[0070] Figure 3 This is a schematic diagram of the structure of a device for identifying abnormal data, provided in an embodiment of this application. Detailed Implementation
[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0072] The proposed solution can be implemented based on a terminal with data processing capabilities, such as a computer, server, or cloud platform.
[0073] Next, the method for identifying abnormal data in this application will be introduced.
[0074] Specifically, this can be illustrated with a concrete example. For instance, an insurance company is analyzing claim rate data from the past seven years (each month representing a time point) across 310 regions involved in its auto insurance business to identify anomalies.
[0075] Based on the above examples, combined with Figure 1 Methods for identifying anomalous data may include the following steps:
[0076] Step S1: Input the time series data of vehicle accidents.
[0077] Specifically, the vehicle accident time series data is X i =(x i1 ,x i2 ,..,x im ), i=1,2,…,n.
[0078] Among them, X i The sampled data contains T time intervals, each containing data from L time points, x ij Let x be the incident rate of the i-th group at time j, 0 ≤ x ij ≤1, i=1,2,…,n, j=1,2,…,m, T, n, L are all positive integers, m=L×T.
[0079] Specifically, the values of n, L, and T can be n=310, L=12, and T=7, respectively, so m=L×T=84.
[0080] Step S2: Set the maximum number of allowed consecutive deviation components and the minimum number of similar components.
[0081] Specifically, step S2 further includes:
[0082] S2-1, Set the maximum number of allowed consecutive deviation components MaxD.
[0083] Specifically, MaxD is set according to formula (1):
[0084] (1)
[0085] Where α1 and α2 are pre-given real numbers, α1=0.3 and α2=0.1, and [x] represents the integer part of x.
[0086] S2-2, Set the minimum number of similar components MinS.
[0087] Specifically, MinS is set up according to the following formula:
[0088] (2)
[0089] Where β is a pre-given real number, β=0.8.
[0090] Step S3: Calculate the difference between any two sets of data.
[0091] Specifically, step S3, calculating the difference between any two sets of data, further includes:
[0092] For all s, t = 1, 2, ..., n, j = 1, 2, ..., m, calculate X according to equation (3). s X t Differences:
[0093] (3)
[0094] Step S4: Calculate the number of the largest consecutive deviation components and the number of similar components.
[0095] Specifically, step S4 further includes: for all s,t=1,2,…,n,
[0096] S4-1, Calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows:
[0097]
[0098] Where γ is a pre-given real number, γ=0.9.
[0099] S4-2, Calculate X s X t The number of similar components n st ;n st For equation (3) to satisfy d stj ≤ The number of subscripts j, j=1,2,…,m.
[0100] Step S5: Perform data clustering.
[0101] Specifically, step S5, the process of data clustering, involves the following steps:
[0102] S5-1, Let s=1;
[0103] S5-2, Let C(X) s )={X s}, t=1;
[0104] S5-3, If t≠s and n st ≥MinS and m st If ≤MaxD, then C(X) s )=C(X s )∪{Xt};
[0105] S5-4: If t<m, then t=t+1, and return to S5-3;
[0106] S5-5: If s<n, then s=s+1, and return to S5-2.
[0107] Step S6: identifying whether the vehicle accident data is abnormal.
[0108] Specifically, the process of identifying whether the data is abnormal in step S6 further comprises:
[0109] For s=1,2,...,n, sequentially check C(X s ), if C(X s ) has more than δn elements, then the data X s is normal, otherwise it is abnormal; where δ is a pre-given real number, 1 / n≤δ≤1, for example, δ=0.05.
[0110] In the method for identifying abnormal data provided in this embodiment, vehicle accident time series data is input; the maximum allowable number of consecutive deviation components and the minimum number of similar components are set; the difference between any two sets of data is calculated; the maximum number of consecutive deviation components and the number of similar components are calculated; data clustering is performed; and whether the data is abnormal is identified. It can be seen that this method can effectively identify abnormal vehicle accident data, and has the characteristics of high speed and high accuracy.
[0111] In the following, the apparatus for implementing abnormal data identification provided in the embodiments of the present application will be described, and the apparatus for implementing abnormal data identification described below may be cross-referenced with the method for identifying abnormal data described above.
[0112] Referring to Figure 2 , Figure 2 is a schematic structural diagram of an apparatus for implementing abnormal data identification disclosed in the embodiment of the present application.
[0113] As shown in Figure 2 , the apparatus may include:
[0114] Vehicle accident time series data input unit 11, configured to execute: S1: input vehicle accident time series data X i =(x i1 ,x i2 ,..,x im ), i=1,2,...,n; where X i includes sampling data of T time intervals, each time interval includes data of L time points, and x ij is the accident rate at the j-th time point of the i-th group, 0≤x ij≤1, i=1,2,…,n, j=1,2,…,m, T, n, L are all positive integers, m=L×T;
[0115] The component number setting unit 12 is used to perform: S2, setting the maximum permissible number of consecutive deviation components and the minimum number of similar components; the component number setting unit 12 includes a maximum permissible number of consecutive deviation components setting unit and a minimum number of similar components setting unit.
[0116] The maximum permissible number of consecutive deviation components setting unit is used to perform: S2-1, setting the maximum permissible number of consecutive deviation components MaxD;
[0117] The minimum similarity component number setting unit is used to perform: S2-2, setting the minimum similarity component number MinS;
[0118] The data difference calculation unit 13 is used to perform: S3, calculate the difference between any two sets of data;
[0119] The component number calculation unit 14 is used to perform: S4, calculating the number of maximum continuous deviation components and the number of similar components; the component number calculation unit includes a maximum continuous deviation component number calculation unit and a similarity component number calculation unit.
[0120] The maximum continuous deviation component calculation unit is used to perform: S4-1, for all s,t=1,2,…,n, calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows:
[0121]
[0122] Where γ is a pre-given real number, 0≤γ≤1;
[0123] The similarity component calculation unit is used to perform: S4-2, for all s,t=1,2,...,n, calculate X s X t The number of similar components n st ;n st For equation (3) to satisfy d stj The number of subscripts j ≤ γ, where j = 1, 2, ..., m;
[0124] Data clustering unit 15 is used to perform: S5, perform data clustering;
[0125] The data anomaly identification unit 16 is used to perform: S6, identify whether the data of the vehicle accident is abnormal.
[0126] Optionally, the maximum permissible number of consecutive deviation components setting unit includes:
[0127] Maximum allowable consecutive deviation component number setting subunit, configured to execute: MaxD is set according to the following formula:
[0128] (1)
[0129] where α1 and α2 are pre-given real numbers, 0<α1<0.5, 0<α2<0.3, and [x] represents the integer part of x.
[0130] Optionally, the minimum similar component number setting unit comprises:
[0131] Minimum similar component number setting subunit, configured to execute: MinS is set according to the following formula:
[0132] (2)
[0133] where β is a pre-given real number, 0<β<1.
[0134] Optionally, the data difference calculation unit comprises:
[0135] Data difference calculation subunit, configured to execute: for all s,t=1,2,...,n, j=1,2,...,m, calculate X according to the following formula s , X t :
[0136] (3).
[0137] Optionally, the data clustering unit comprises:
[0138] First data clustering subunit, configured to execute: S5-1, set s=1;
[0139] Second data clustering subunit, configured to execute: S5-2, set C(X s )={X s}, t=1;
[0140] Third data clustering subunit, configured to execute: S5-3, if t≠s and n st ≥MinS and m st ≤MaxD, then C(X s )=C(X s )∪{X t};
[0141] Fourth data clustering subunit, configured to execute: S5-4, if t<m, then t=t+1, and execute the third data clustering subunit;
[0142] The fifth data clustering sub-unit, configured to execute: S5-5. If s<n, then s=s+1, and execute the second data clustering sub-unit.
[0143] Optionally, the data abnormality identification unit includes:
[0144] A data abnormality identification sub-unit, configured to execute: sequentially check C(X for s=1,2,...,n s ), if C(X s ) has the number of elements greater than δn, then the data X s is normal, otherwise it is abnormal; where δ is a pre-given real number, and 1 / n≤δ≤1.
[0145] The abnormal data identification apparatus provided in the embodiments of the present application can be applied to an abnormal data identification device, such as a terminal: a mobile phone, a computer, etc. Optionally, Figure 3 shows a hardware structure block diagram of the abnormal data identification device, with reference to Figure 3 , the hardware structure of the abnormal data identification device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;
[0146] In the embodiments of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 complete mutual communication through the communication bus 4;
[0147] The processor 1 may be a central processing unit (CPU), or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;
[0148] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, etc., for example, at least one disk memory;
[0149] Wherein, the memory stores a program, the processor can call the program stored in the memory, and the program is configured to:
[0150] S1. Input the time series data X of vehicle accident occurrence i =(x i1 ,x i2 ,..,x im ), i=1,2,...,n; where X i includes sampling data of T time intervals, each time interval includes data of L time points, and x ij is the accident occurrence rate at the j-th time point of the i-th group, 0≤xij ≤1, i=1,2,...,n, j=1,2,...,m, T, n, and L are all positive integers, m=L×T;
[0151] S2. Set the maximum allowable number of consecutive deviation components and the minimum number of similar components; further including:
[0152] S2-1, Set the maximum allowable number of consecutive deviation components MaxD;
[0153] S2-2, Set the minimum number of similar components MinS;
[0154] S3. Calculate the difference between any two sets of data;
[0155] S4. Calculate the number of maximum consecutive deviation components and the number of similar components; further including: for all s, t = 1, 2, ..., n,
[0156] S4-1, Calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows:
[0157]
[0158] Where γ is a pre-given real number, 0≤γ≤1;
[0159] S4-2, Calculate X s X t The number of similar components n st ;n st For equation (3) to satisfy d stj The number of subscripts j ≤ γ, where j = 1, 2, ..., m;
[0160] S5. Perform data clustering;
[0161] S6. Identify whether the data regarding the vehicle accident is abnormal.
[0162] Optionally, the refined and extended functions of the program can be found in the description above.
[0163] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:
[0164] S1. Input the time series data of vehicle accidents X i =(x i1 ,x i2 ,..,x im ), i=1,2,...,n; where X iThe sampled data contains T time intervals, each containing data from L time points, x ij Let x be the incident rate of the i-th group at time j, 0 ≤ x ij ≤1, i=1,2,...,n, j=1,2,...,m, T, n, and L are all positive integers, m=L×T;
[0165] S2. Set the maximum allowable number of consecutive deviation components and the minimum number of similar components; further including:
[0166] S2-1, Set the maximum allowable number of consecutive deviation components MaxD;
[0167] S2-2, Set the minimum number of similar components MinS;
[0168] S3. Calculate the difference between any two sets of data;
[0169] S4. Calculate the number of maximum consecutive deviation components and the number of similar components; further including: for all s, t = 1, 2, ..., n,
[0170] S4-1, Calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows:
[0171]
[0172] Where γ is a pre-given real number, 0≤γ≤1;
[0173] S4-2, Calculate X s X t The number of similar components n st ;n st For equation (3) to satisfy d stj The number of subscripts j ≤ γ, where j = 1, 2, ..., m;
[0174] S5. Perform data clustering;
[0175] S6. Identify whether the data regarding the vehicle accident is abnormal.
[0176] Optionally, the refined and extended functions of the program can be found in the description above.
[0177] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0178] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0179] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for identifying abnormal data, characterized in that, comprising: S1. Input the time series data of vehicle accidents X i =(x i1 ,x i2 ,..,x im ), i=1,2,...,n; where X i The sampled data contains T time intervals, each containing data from L time points, x ij Let x be the incident rate of the i-th group at time j, 0 ≤ x ij ≤1, i=1,2,...,n, j=1,2,...,m, T, n, and L are all positive integers, m=L×T; S2. setting the maximum allowable number of consecutive deviation components and the minimum number of similar components; further comprising: S2-1. setting the maximum allowable number of consecutive deviation components MaxD; S2-2. setting the minimum number of similar components MinS; S3. calculating the difference between any two sets of data; S4. calculating the maximum number of consecutive deviation components and the number of similar components; further comprising: for all s, t=1,2,...,n, S4-1, Calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows: Where, d stk ,d st(k+1) ,...,d st(k+u-1) Substituting j=k,k+1,...,k+u-1 into the formula respectively , calculated in, where γ is a pre-given real number, 0≤γ≤1; S4-2, Calculate X s X t The number of similar components n st ;n st To satisfy d stj The number of subscripts j ≤ γ, where, j=1,2,...,m; S5. performing data clustering; The specific steps of S5. performing data clustering are as follows: S5-1. let s=1; S5-2, Let C(X) s )={X s }, t=1; S5-3, If t≠s and n st ≥MinS and m st If ≤MaxD, then C(X) s )=C(X s )∪{X t }; S5-4. if t<m, then t=t+1, go to S5-3; S5-5. if s<n, then s=s+1, go to S5-2; S6. identifying whether the vehicle accident data is abnormal.
2. The method for identifying abnormal data according to claim 1, characterized in that, The step S2-1 of setting the maximum allowable number of consecutive deviation components MaxD further comprises: MaxD is set according to the following formula: (1) wherein α1 and α2 are preset real numbers, 0<α1<0.5, 0<α2<0.3, and [x] represents the integer part of x.
3. The method for identifying abnormal data according to claim 1, characterized in that, The step S2-2 of setting the minimum number of similar components MinS further comprises: MinS is set according to the following formula: (2) wherein β is a preset real number, 0<β<1.
4. The method for identifying abnormal data according to claim 1, characterized in that, The step S3 of calculating the difference between any two sets of data further comprises: For all s, t = 1, 2, ..., n, j = 1, 2, ..., m, calculate X according to the following formula. s X t Differences: (3)。 5. The method for identifying abnormal data according to claim 1, characterized in that, S6, identifying whether the vehicle accident data is abnormal, further includes: for s=1,2,...,n, sequentially checking C(X) s If C(X) s If the number of elements in X is greater than δn, then the data X... s If it is normal, then it is abnormal; where δ is a pre-given real number, 1 / n≤δ≤1.
6. A device for identifying abnormal data, characterized in that, Applied to the abnormal data identification method as claimed in claim 1, the apparatus comprises: The vehicle accident time series data input unit is used to perform: S1, inputting the vehicle accident time series data X. i =(x i1 ,x i2 ,..,x im ), i=1,2,...,n; where X i The sampled data contains T time intervals, each containing data from L time points, x ij Let x be the incident rate of the i-th group at time j, 0 ≤ x ij ≤1, i=1,2,...,n, j=1,2,...,m, T, n, and L are all positive integers, m=L×T; A component number setting unit, configured to perform: S2. setting the maximum allowable number of consecutive deviation components and the minimum number of similar components; the component number setting unit comprises a maximum allowable consecutive deviation component number setting unit and a minimum similar component number setting unit: The maximum allowable consecutive deviation component number setting unit is configured to perform: S2-1. setting the maximum allowable number of consecutive deviation components MaxD; The minimum similar component number setting unit is configured to perform: S2-2. setting the minimum number of similar components MinS; A data difference calculation unit, configured to perform: S3. calculating the difference between any two sets of data; A component number calculation unit, configured to perform: S4. calculating the maximum number of consecutive deviation components and the number of similar components; the component number calculation unit comprises a maximum consecutive deviation component number calculation unit and a similar component number calculation unit; The maximum continuous deviation component calculation unit is used to perform: S4-1, for all s,t=1,2,...,n, calculate X s X t Maximum number of consecutive deviation components m st The specific calculation formula is as follows: wherein γ is a preset real number, 0≤γ≤1; The similarity component calculation unit is used to perform: S4-2, for all s,t=1,2,...,n, calculate X s X t The number of similar components n st ;n st To satisfy d stj The number of subscripts j ≤ γ j=1,2,...,m; A data clustering unit, configured to perform: S5. performing data clustering; A data abnormality identification unit, configured to perform: S6. identifying whether the vehicle accident data is abnormal.
7. The abnormal data identification device according to claim 6, characterized in that, The maximum allowable consecutive deviation component number setting unit comprises: A maximum allowable consecutive deviation component number setting subunit, configured to perform: MaxD is set according to the following formula: (1) wherein α1 and α2 are preset real numbers, 0<α1<0.5, 0<α2<0.3, and [x] represents the integer part of x.
8. An anomalous data identification device, characterized in that, comprising a memory and a processor; the memory is configured to store a program; the processor is configured to execute the program to implement each step of the abnormal data identification method as described in any one of claims 1 to 5.
9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, each step of the abnormal data identification method as described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Big-data analyzing method and mass spectrometric system using same method
CN107679052A
Highway traffic flow prediction method and device
CN110263860A
