Engineering data anomaly recognition system and method based on big data

Abstract

The invention discloses an engineering data anomaly recognition system and method based on big data, and relates to the technical field of engineering data management.The method includes the steps that keywords are obtained, data ripple anomaly recognition components are set for the keywords for recognition, and critical values in a range are input step by step for judgment; the large fluctuation influence of data exception identification on the whole engineering data is reduced, keywords are confirmed in real time, a data exception threshold value is set, a reference identifier is set, the identification frequency of safe and reliable data is reduced, the identification precision and efficiency are improved, the threshold value of the keywords is dynamically adjusted and updated, and the identification efficiency is improved. The engineering data can respond to data changes more timely, the accuracy of the engineering data is ensured, fine fluctuation of key data is recognized and adjusted in time, small errors are effectively prevented from being gradually accumulated and amplified in long-term operation, the overall accuracy of the engineering data is improved, the potential data exception risk is reduced, and the engineering quality is improved. And the reliability of engineering data is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering data management, and in particular to a system and method for identifying anomalies in engineering data based on big data. Background Art

[0002] Engineering data is the digital cornerstone of engineering projects. It not only records the entire process from design to construction, but also carries the key decision-making basis for project quality, safety and sustainability. Accurate and comprehensive engineering data can form a feedback loop in the project, timely reveal potential risks and efficiency improvement points, and ensure that every decision is based on facts, thereby maximizing resource utilization and reducing long-term operation and maintenance costs. Through engineering data, every detail of the project can be quantified and analyzed, allowing managers to make more informed decisions, promote continuous optimization of the project, and achieve higher benefits and quality standards.

[0003] Today, engineering data anomaly identification faces obvious problems, especially the phenomenon that small errors gradually amplify over time. Due to the lack of sensitivity of the identification model, the difficulty in ensuring both real-time and accuracy, the lack of monitoring of cumulative effects, and the imperfect feedback mechanism, small anomalies are often ignored. Over time, these small problems may lead to serious systemic failures, endangering the stability and reliability of the engineering system. Summary of the invention

[0004] The purpose of the present invention is to provide a system and method for identifying anomalies in engineering data based on big data to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for identifying anomalies in engineering data based on big data, the method for identifying anomalies in engineering data comprising the following steps:

[0006] Step S1, obtaining and classifying engineering data, and setting an engineering data abnormality threshold according to the maximum and minimum values ​​of the fluctuation of the classification standard value;

[0007] Step S1-1, obtaining engineering data, classifying the data according to data features to establish a target data set, and setting a standard value and a maximum and minimum value of the fluctuation of the target data set according to the standard value set by the classification;

[0008] Step S1-2: setting an engineering data anomaly threshold according to the maximum and minimum values ​​of the target data set standard value fluctuation, wherein the engineering data anomaly threshold includes a minimum engineering data anomaly threshold Hmin and a maximum engineering data anomaly threshold Hmax.

[0009] In the engineering data, a range value of the allowable fluctuation around the standard value is set for the corresponding classification, specifically a maximum value and a minimum value of the allowable fluctuation;

[0010] Step S2, obtaining historical abnormal data in the engineering data, performing keyword extraction on the obtained engineering data and the historical abnormal data in the engineering data, and recording them as pre-ripple data;

[0011] Step S2-1, record the data in the classification that exceeds the maximum value of the standard value fluctuation and is lower than the minimum value of the standard value fluctuation as abnormal data, obtain the abnormal data in the historical engineering data and organize them to generate an abnormal data set;

[0012] Step S2-2: extract keywords from the abnormal data set and the acquired engineering data. The keyword extraction uses the following formula:

[0013]

[0014] Where K(x, y) represents the probability that word x and category y appear simultaneously in the engineering data; L(x) represents the probability that a word appears alone in the engineering data; L(y) represents the probability that a category appears alone in the engineering data; a key threshold Q is set, and keywords with K values ​​greater than Q in the extraction calculation are extracted and recorded as pre-ripple data.

[0015] Extract the keywords in the engineering data that have the greatest impact on the overall data fluctuation, filter out unimportant engineering data for identification, and improve the accuracy and reliability of identification.

[0016] Step S3: construct a corresponding data ripple anomaly identification component and a ripple feedback component according to the pre-ripple data, add the corresponding data ripple anomaly identification component to the pre-ripple data to identify anomalies in the data, and feed back the abnormal value of the pre-ripple data through the ripple feedback component;

[0017] Step S3-1, obtain pre-ripple data, extract each keyword in the pre-ripple data, set the parameters of the data ripple anomaly recognition component according to the standard value of the corresponding keyword and the engineering data anomaly threshold, set the standard value of the pre-ripple data as the recognition reference value, and set the engineering data anomaly threshold as the recognition boundary value to construct the data ripple anomaly recognition component;

[0018] Step S3-2: According to the keywords corresponding to the pre-ripple data, add the corresponding data ripple anomaly recognition component for recognition.

[0019] By selecting the critical value corresponding to the keyword, we can more efficiently identify the impact of the critical value on the overall engineering data and amplify the identification of data anomalies.

[0020] Step S3-3, setting a ripple feedback component, obtaining a recognized keyword, and setting the keyword to the number of the corresponding ripple feedback component;

[0021] Step S3-4, using the feedback component corresponding to the number, the data ripple anomaly recognition component feeds back the abnormal value identified by the pre-ripple data, and calculates the abnormal value according to the degree of deviation of the data point from the average value. The abnormal value calculation formula is as follows:

[0022]

[0023] In the formula, P represents the outlier value of the identified keyword; P sta It is expressed as the standard value for identifying keywords; P new It is represented as the adjustment value of the identified keyword; N is represented as the total number of data points; V i a is the value of each data point; b is the mean of the data points.

[0024] By setting the ripple feedback component, the status of the data ripple anomaly recognition component can be fed back in real time, thereby improving the accuracy and reliability of the engineering data anomaly recognition method.

[0025] Step S4, obtaining the recognition points for data ripple anomaly recognition by using keywords, collecting data from each recognition point to generate an engineering data trend graph, and calculating the fluctuation degree value generated by each recognition point, and judging the recognition condition by the engineering data trend graph and the fluctuation degree value generated by a single recognition;

[0026] Step S4-1, set the fluctuation trend threshold Fmax and the fluctuation degree threshold Zmax, calculate the median value of the standard value of the recognition keyword and the engineering data anomaly threshold, use it as the recognition point for the first data ripple anomaly recognition, and generate the engineering data trend chart according to the results of each median recognition;

[0027] Step S4-2, assuming that the engineering data fluctuation value is Z, the fluctuation trend value in the engineering data trend chart is F, and the initial value is 0, the calculation formula of the engineering data fluctuation value Z is as follows:

[0028]

[0029] In the formula, Z represents the fluctuation degree of engineering data when the corresponding median value is input; E represents the absolute value of the difference between the engineering data after the median value is input and the original engineering data; q represents the standard deviation of the input median value; R represents the average value of the input median value; u represents the influence coefficient of the median value corresponding to the engineering data; P sta It is expressed as the standard value for identifying keywords;

[0030] The identification conditions are judged based on the engineering data trend chart and the fluctuation value generated by a single identification:

[0031] When Z≤Zmax, F≤Fmax, re-identification is performed, and the identification point is the median value of the engineering data abnormal threshold and the last identification point;

[0032] When Z > Zmax or F > Fmax, stop the recognition by adjusting the abnormal threshold of the engineering data;

[0033] Step S4-3: In generating the engineering data trend chart, use the degree of fluctuation value Z of the engineering data as the ordinate and the time interval T of the engineering data fluctuation as the abscissa to calculate the slope of the trend chart. The slope calculation of the trend chart uses the following formula:

[0034]

[0035] In the formula, F represents the change slope of the engineering data trend chart; Z1 and Z0 represent the fluctuation values of the engineering data; T1 and T0 respectively represent the times of Z1 and Z0.

[0036] Avoid adjusting the corresponding weights after data update, increasing the weights of the original data, and the originally set abnormal threshold exceeding the tolerance range of the overall fluctuation of the engineering data. Timely confirm and update the data abnormal threshold corresponding to each keyword to avoid the risk of data anomalies in advance. Adjusting the abnormal threshold of the engineering data means that after inputting the keyword for recognition, setting the values that satisfy Z ≤ Zmax and F ≤ Fmax as the new data abnormal threshold of the recognized keyword.

[0037] Step S5: Construct a reference identifier. The reference identifier is provided with a recognition confirmation threshold C. When using the abnormal threshold of the engineering data for recognition, for keywords that satisfy Z < Zmax and F < Fmax when the recognition uses the abnormal threshold of the engineering data, set the reference identifier. When a keyword with a reference identifier is recognized, judge the recognition confirmation threshold C. When the recognition confirmation threshold C is equal to 0, perform the recognition, and reset the recognition confirmation threshold C after the recognition is completed; otherwise, skip the recognition and subtract 1 from the recognition confirmation threshold C.

[0038] Keywords with a reference identifier indicate that the influence of the data abnormal threshold on the overall fluctuation of the engineering data is small, and the fluctuation range of the set standard value is small, with high data security and reduced frequency of data abnormal recognition.

[0039] An engineering data abnormal recognition system based on big data includes an engineering data acquisition module, a ripple data extraction module, a data ripple abnormal recognition module, a ripple feedback module, a recognition judgment module, and a reference identifier module;

[0040] The engineering data acquisition module includes a real-time engineering data acquisition unit, a historical engineering data acquisition unit, and an engineering data sorting unit; the real-time engineering data acquisition unit is used to acquire data information in real-time engineering; the historical engineering data acquisition unit is used to acquire historical data of the engineering data; the engineering data sorting unit is used to unify the format and clean and sort the acquired engineering data;

[0041] The ripple data extraction module includes a keyword extraction unit and a keyword distribution storage unit; the keyword extraction unit is used to calculate the weight of the keywords in the engineering data and extract the data whose calculated values ​​meet the conditions; the keyword distribution storage unit is used to store different types of keyword data;

[0042] The data ripple anomaly recognition module includes a fluctuation value calculation unit, a recognition point positioning unit and a trend graph generation unit; the fluctuation value calculation unit is used to calculate the fluctuation value calculation generated by the recognition input value, including the calculation of its own fluctuation value and the calculation of the overall fluctuation impact value of the engineering data; the recognition point positioning unit is used to find and locate the recognition point of each keyword anomaly recognition; the trend graph generation unit is used to generate a trend graph of engineering data during continuous recognition;

[0043] The ripple feedback module includes an abnormal recording unit and an abnormal feedback unit; the abnormal recording unit is used to record the abnormal value of the corresponding recognition point in the keyword recognition process; the abnormal feedback unit is used to feedback the information recorded by the abnormal recording unit;

[0044] The recognition and judgment module includes a keyword threshold judgment unit and a keyword re-recognition judgment unit; the keyword threshold judgment unit is used to judge the threshold adjustment condition of the keyword; the keyword re-recognition judgment unit is used to judge whether the recognized keyword meets the re-recognition condition;

[0045] The reference identification module includes a reference identification confirmation unit and a reference identification setting unit; the reference identification confirmation unit is used to confirm the keyword to be set with the reference identification; the reference identification setting unit is used to set the reference identification for the keyword after confirming the keyword to be set with the reference identification.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows: after extracting keywords from engineering data, accurate data anomaly identification is performed, a data ripple anomaly identification component is set to identify critical values ​​within the range of allowable fluctuations of keyword standard values, and judgment is made by gradually and continuously inputting critical values ​​within the range, thereby reducing the large fluctuation impact of data anomaly identification on the overall engineering data, confirming the accuracy of the data anomaly threshold set for the corresponding keyword in real time, setting a benchmark identifier to reduce the frequency of safe and reliable data being identified, and improving the accuracy and efficiency of identification;

[0047] 1. Dynamically adjust and update keyword thresholds so that engineering data can respond to data changes more promptly and ensure the accuracy of engineering data.

[0048] 2. This technical solution can timely identify and adjust subtle fluctuations in key data, and can effectively prevent small errors from gradually accumulating and amplifying during long-term operation. This not only improves the overall accuracy of engineering data, but also reduces potential data anomaly risks, ensuring that the data used in the project has long-term stability and reliability.

[0049] 3. Through real-time monitoring and feedback adjustment of keywords, engineering data can automatically adapt to environmental changes more quickly and reduce human intervention. This intelligent data adjustment method can improve overall operation and maintenance efficiency and reduce the possibility of human errors.

[0050] 4. Through more accurate anomaly identification and threshold adjustment, potential risk signals can be captured at an early stage in the engineering data operation and maintenance process to prevent the problem from further deteriorating. This forward-looking measure can significantly improve the stability of the engineering system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a process of identifying anomalies in engineering data based on big data according to the present invention;

[0052] Figure 2 The present invention is a structural schematic diagram of an engineering data anomaly identification system based on big data. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution, a method for identifying anomalies in engineering data based on big data, and the method for identifying anomalies in engineering data includes the following steps:

[0055] Step S1, obtaining and classifying engineering data, and setting an engineering data abnormality threshold according to the maximum and minimum values ​​of the fluctuation of the classification standard value;

[0056] Step S1-1, obtaining engineering data, classifying the data according to data features to establish a target data set, and setting a standard value and a maximum and minimum value of the fluctuation of the target data set according to the standard value set by the classification;

[0057] Step S1-2: setting an engineering data anomaly threshold according to the maximum and minimum values ​​of the target data set standard value fluctuation, wherein the engineering data anomaly threshold includes a minimum engineering data anomaly threshold Hmin and a maximum engineering data anomaly threshold Hmax.

[0058] Step S2, obtaining historical abnormal data in the engineering data, performing keyword extraction on the obtained engineering data and the historical abnormal data in the engineering data, and recording them as pre-ripple data;

[0059] Step S2-1, record the data in the classification that exceeds the maximum value of the standard value fluctuation and is lower than the minimum value of the standard value fluctuation as abnormal data, obtain the abnormal data in the historical engineering data and organize them to generate an abnormal data set;

[0060] Step S2-2: extract keywords from the abnormal data set and the acquired engineering data. The keyword extraction uses the following formula:

[0061]

[0062] Where K(x, y) represents the probability that word x and category y appear simultaneously in the engineering data; L(x) represents the probability that a word appears alone in the engineering data; L(y) represents the probability that a category appears alone in the engineering data; a key threshold Q is set, and keywords with K values ​​greater than Q in the extraction calculation are extracted and recorded as pre-ripple data.

[0063] For example, the engineering data includes ("overheat", "vibration", and "short circuit"), and the categories are ("fault", "normal"). "Overheat" appears 200 times, of which 150 times in the "fault" category. "Vibration" appears 100 times, of which 90 times in the "fault" category. "Short circuit" appears 50 times, of which 60 times in the "fault" category. The total number of "fault" categories is 600, and the total number of "normal" categories is 400. The key threshold Q is 5%, and the data is extracted for keywords; the formula is used for calculation:

[0064] K("overheat", "failure") = log(0.15 / 0.2*0.6) = 0.097

[0065] K("vibration", "fault") = log(0.09 / 0.1*0.6) = 0.176

[0066] K("short circuit", "fault") = log(0.06 / 0.05*0.6) = 2.000

[0067] Among them, "overheating", "vibration" and "short circuit" each account for 33.333% of the engineering data. According to the 5% keyword extraction, the qualified data is "short circuit", that is, "open circuit" is extracted as the keyword and recorded as pre-ripple data.

[0068] Step S3: construct a corresponding data ripple anomaly identification component and a ripple feedback component according to the pre-ripple data, add the corresponding data ripple anomaly identification component to the pre-ripple data to identify anomalies in the data, and feed back the abnormal value of the pre-ripple data through the ripple feedback component;

[0069] Step S3-1, obtain pre-ripple data, extract each keyword in the pre-ripple data, set the parameters of the data ripple anomaly recognition component according to the standard value of the corresponding keyword and the engineering data anomaly threshold, set the standard value of the pre-ripple data as the recognition reference value, and set the engineering data anomaly threshold as the recognition boundary value to construct the data ripple anomaly recognition component;

[0070] Step S3-2: According to the keywords corresponding to the pre-ripple data, add the corresponding data ripple anomaly recognition component for recognition.

[0071] For example, according to the pre-ripple data extracted by calculation in step S2, the parameters associated with "short circuit" are obtained, and the data standards and ranges corresponding to the keywords related to "short circuit" in the engineering data including current intensity, voltage level, resistance and value temperature are obtained, and the corresponding data ripple anomaly recognition components are added for identification.

[0072] Step S3-3, setting a ripple feedback component, obtaining a recognized keyword, and setting the keyword to the number of the corresponding ripple feedback component;

[0073] Step S3-4, using the feedback component corresponding to the number, the data ripple anomaly recognition component feeds back the abnormal value identified by the pre-ripple data, and calculates the abnormal value according to the degree of deviation of the data point from the average value. The abnormal value calculation formula is as follows:

[0074]

[0075] In the formula, P represents the outlier value of the identified keyword; P sta It is expressed as the standard value for identifying keywords; P new It is represented as the adjustment value of the identified keyword; N is represented as the total number of data points; V i a is the value of each data point; b is the mean of the data points.

[0076] For example, according to step S3, the temperature data is identified. The temperature data set includes 39 degrees Celsius, 42 degrees Celsius, 41 degrees Celsius, 40 degrees Celsius, 43 degrees Celsius, and 44 degrees Celsius. The standard temperature value is 40 degrees Celsius, and the allowable fluctuation range is 35 degrees Celsius to 45 degrees Celsius. The identified input temperature is 42 degrees Celsius. The abnormal value is calculated according to the deviation degree of the data point from the average value. The calculation formula for the abnormal value of temperature is as follows:

[0077] P = (42-20) / 41.5 = 0.048

[0078] That is, for temperature recognition, when 42 degrees Celsius is input, the abnormal value of the data is 0.048.

[0079] Step S4, obtaining the recognition points for data ripple anomaly recognition by using keywords, collecting data from each recognition point to generate an engineering data trend graph, and calculating the fluctuation degree value generated by each recognition point, and judging the recognition condition by the engineering data trend graph and the fluctuation degree value generated by a single recognition;

[0080] Step S4-1, set the fluctuation trend threshold Fmax and the fluctuation degree threshold Zmax, calculate the median value of the standard value of the recognition keyword and the engineering data anomaly threshold, use it as the recognition point for the first data ripple anomaly recognition, and generate the engineering data trend chart according to the results of each median recognition;

[0081] Step S4-2, assuming that the engineering data fluctuation value is Z, the fluctuation trend value in the engineering data trend chart is F, and the initial value is 0, the calculation formula of the engineering data fluctuation value Z is as follows:

[0082]

[0083] In the formula, Z represents the fluctuation degree of engineering data when the corresponding median value is input; E represents the absolute value of the difference between the engineering data after the median value is input and the original engineering data; q represents the standard deviation of the input median value; R represents the average value of the input median value; u represents the influence coefficient of the median value corresponding to the engineering data; P sta It is expressed as the standard value for identifying keywords;

[0084] The identification conditions are judged based on the engineering data trend chart and the fluctuation value generated by a single identification:

[0085] When Z≤Zmax, F≤Fmax, re-identification is performed, and the identification point is the median value of the engineering data abnormal threshold and the last identification point;

[0086] When Z>Zmax or F>Fmax, adjust the abnormal threshold of engineering data to stop identification;

[0087] Step S4-3, in generating the engineering data trend graph, the engineering data fluctuation value Z is the ordinate, and the time interval T at which the engineering data fluctuates is the abscissa. The slope of the trend graph is calculated. The slope of the trend graph is calculated using the following formula:

[0088]

[0089] Where F represents the slope of the engineering data trend graph; Z1 and Z0 represent the fluctuation values ​​of the engineering data; T1 and T0 represent the Z1 and Z0 times, respectively.

[0090] For example, according to step S4, the temperature data is identified, and its engineering data includes the current intensity, 10A, the influence coefficient is 0.4; the voltage level is 220V, the influence coefficient is 0.45; the resistance value is 22 ohms, the influence coefficient is 0.2; the temperature is 40 degrees Celsius, the influence coefficient is 0.05; set Zmax to 50%, Fmax to 1.5;

[0091] According to the recognized input temperature value of 42 degrees Celsius, the overall fluctuation value is calculated according to the fluctuation degree value calculation formula:

[0092] Z = (2 + 1.2) / 1 * 100 * 0.05 = 16

[0093] Through the formula calculation, it can be obtained that when the temperature data is abnormal, when the temperature is adjusted to 42 degrees Celsius, the fluctuation value of the overall engineering data is 16%;

[0094] The temperature value entered for 5 consecutive times is 42 degrees Celsius, and it is entered once every 10 minutes. The overall fluctuation value of the engineering data from the first to the fifth time is calculated as follows: Z1 is 16%, Z2 is 22.5%, Z3 is 30.5%, Z4 is 46.8%, and Z5 is 46.8%; according to the slope calculation formula:

[0095] F 1-2 =0.65, F 2-3 =0.7, F 3-4 =1.63, F 4-5 =0

[0096] To judge the identification,

[0097] Z1 <Zmax,Z2<Zmax,Z3<Zmax,Z4<Zmax,Z5<Zmax;

[0098] F 1-2 <Fmax,F 2-3 <Fmax,F 3-4 >Fmax,F 4-5 >Fmax;

[0099] According to the judgment conditions:

[0100] When Z ≤ Zmax and F ≤ Fmax, perform re-identification, and the identification point is the median value of the engineering data anomaly threshold and the previous identification point.

[0101] When Z > Zmax or F > Fmax, adjust the engineering data anomaly threshold to stop identification.

[0102] F 3-4 > Fmax and F 4-5 > Fmax satisfies F > Fmax, adjust the fluctuation range of the temperature standard value to 38 degrees Celsius to 42 degrees Celsius, and stop identification.

[0103] Step S5: Construct a reference identifier. The reference identifier is provided with an identification confirmation threshold C. When the identification value is the engineering data anomaly threshold, when identifying using the engineering data anomaly threshold, set the reference identifier for the keywords that satisfy Z < Zmax and F < Fmax. When a keyword with a reference identifier set is identified, judge the identification confirmation threshold C. When the identification confirmation threshold C is equal to 0, perform identification on it. After the identification is completed, reset the identification confirmation threshold C. Otherwise, skip the identification, and subtract 1 from the identification confirmation threshold C.

[0104] During the identification process, set the reference identifier for the keywords corresponding to Z < Zmax and F < Fmax when the input value is the engineering data anomaly threshold.

[0105] For example, the standard value of the current intensity is 10 A, the allowable fluctuation range is 9.5 A to 10.5 A, the Zmax corresponding to the current intensity is 30%, the Fmax is 0.8, and the identification confirmation threshold C is 7. Continuously input 10.5 A for the current intensity 5 times for anomaly identification. Through step S4 calculation, it is obtained that Z < Zmax and F < Fmax. Set the reference identifier for the current intensity data, skip the identification, and change the identification confirmation threshold C from 7 to 6.

[0106] An engineering data anomaly identification system based on big data includes an engineering data acquisition module, a ripple data extraction module, a data ripple anomaly identification module, a ripple feedback module, an identification judgment module, and a reference identifier module.

[0107] The engineering data acquisition module includes a real-time engineering data acquisition unit, a historical engineering data acquisition unit, and an engineering data sorting unit; the real-time engineering data acquisition unit is used to collect data information in real-time engineering; the historical engineering data acquisition unit is used to collect historical data of engineering data; the engineering data sorting unit is used to unify the format and clean and sort the collected engineering data.

[0108] The ripple data extraction module includes a keyword extraction unit and a keyword distribution storage unit; the keyword extraction unit is used to calculate the weight of the keywords in the engineering data and extract the data whose calculated values ​​meet the conditions; the keyword distribution storage unit is used to store different types of keyword data;

[0109] The data ripple anomaly recognition module includes a fluctuation value calculation unit, a recognition point positioning unit and a trend graph generation unit; the fluctuation value calculation unit is used to calculate the fluctuation value calculation generated by the recognition input value, including the calculation of its own fluctuation value and the calculation of the overall fluctuation impact value of the engineering data; the recognition point positioning unit is used to find and locate the recognition point of each keyword anomaly recognition; the trend graph generation unit is used to generate a trend graph of engineering data during continuous recognition;

[0110] The ripple feedback module includes an abnormal recording unit and an abnormal feedback unit; the abnormal recording unit is used to record the abnormal value of the corresponding recognition point in the keyword recognition process; the abnormal feedback unit is used to feedback the information recorded by the abnormal recording unit;

[0111] The recognition and judgment module includes a keyword threshold judgment unit and a keyword re-recognition judgment unit; the keyword threshold judgment unit is used to judge the threshold adjustment condition of the keyword; the keyword re-recognition judgment unit is used to judge whether the recognized keyword meets the re-recognition condition;

[0112] The reference identification module includes a reference identification confirmation unit and a reference identification setting unit; the reference identification confirmation unit is used to confirm the keyword to be set with the reference identification; the reference identification setting unit is used to set the reference identification for the keyword after confirming the keyword to be set with the reference identification.

[0113] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for identifying anomalies in engineering data based on big data, characterized in that: The engineering data anomaly identification method comprises the following steps: Step S1, obtaining and classifying engineering data, and setting an engineering data abnormality threshold according to the maximum and minimum values ​​of the fluctuation of the classification standard value; Step S2, obtaining historical abnormal data in the engineering data, performing keyword extraction on the obtained engineering data and the historical abnormal data in the engineering data, and recording them as pre-ripple data; Step S3: construct a corresponding data ripple anomaly identification component and a ripple feedback component according to the pre-ripple data, add the corresponding data ripple anomaly identification component to the pre-ripple data to identify anomalies in the data, and feed back the abnormal value of the pre-ripple data through the ripple feedback component; Step S4, obtaining the recognition points for data ripple anomaly recognition by using keywords, collecting data from each recognition point to generate an engineering data trend graph, and calculating the fluctuation degree value generated by each recognition point, and judging the recognition condition by the engineering data trend graph and the fluctuation degree value generated by a single recognition; Step S5, construct a benchmark identifier, the benchmark identifier is set with an identification confirmation threshold, and when the data anomaly threshold is used for identification, the fluctuation value of the engineering data generated is less than the fluctuation threshold, and the trend value in the generated engineering data trend chart is also less than the trend threshold. The benchmark identifier is set for the keyword. When the keyword with the benchmark identifier is identified, the identification confirmation threshold C is judged. When the identification confirmation threshold C is equal to 0, it is identified. After the identification is completed, the identification confirmation threshold C is reset, otherwise the identification is skipped and the identification confirmation threshold C is reduced by 1.

2. The method for identifying anomalies in engineering data based on big data according to claim 1, characterized in that: In step S1, the specific steps of setting the engineering data abnormality threshold are as follows: Step S1-1, obtaining engineering data, classifying the data according to data features to establish a target data set, and setting a standard value and a maximum and minimum value of the fluctuation of the target data set according to the standard value set by the classification; Step S1-2: setting an engineering data anomaly threshold according to the maximum and minimum values ​​of the target data set standard value fluctuation, wherein the engineering data anomaly threshold includes a minimum engineering data anomaly threshold Hmin and a maximum engineering data anomaly threshold Hmax.

3. The method for identifying anomalies in engineering data based on big data according to claim 1, characterized in that: In step S2: Step S2-1, record the data in the classification that exceeds the maximum value of the standard value fluctuation and is lower than the minimum value of the standard value fluctuation as abnormal data, obtain the abnormal data in the historical engineering data and organize them to generate an abnormal data set; Step S2-2: extract keywords from the abnormal data set and the acquired engineering data. The keyword extraction uses the following formula: Where K(x, y) represents the probability that word x and category y appear simultaneously in the engineering data; L(x) represents the probability that a word appears alone in the engineering data; L(y) represents the probability that a category appears alone in the engineering data; Set the key threshold Q, extract the keywords whose K value is greater than Q in the extraction calculation, and record them as pre-ripple data.

4. The method for identifying anomalies in engineering data based on big data according to claim 3 is characterized in that: In step S3, the specific steps of constructing a data ripple anomaly recognition component to recognize pre-ripple data are as follows: Step S3-1: Obtain pre-ripple data, extract each keyword in the pre-ripple data, set the parameters of the data ripple anomaly recognition component through the standard value of the corresponding keyword and the engineering data anomaly threshold, set the standard value of the pre-ripple data as the recognition reference value, and set the engineering data anomaly threshold as the recognition boundary value to construct the data ripple anomaly recognition component; Step S3-2: According to the keywords corresponding to the pre-ripple data, add the corresponding data ripple anomaly recognition components for recognition; Step S3-3: Set the ripple feedback component, obtain the recognized keywords, and set the keywords as the numbers of the corresponding ripple feedback components; Step S3-4: Use the feedback component corresponding to the number to feedback the anomaly values recognized by the data ripple anomaly recognition component for the pre-ripple data, calculate the anomaly values according to the deviation degree of the data points from the average value, and the anomaly value calculation formula is as follows: In the formula, P represents the outlier value of the identified keyword; P sta It is expressed as the standard value for identifying keywords; P new It is represented as the adjustment value of the identified keyword; N is represented as the total number of data points; V i a is the value of each data point; b is the mean of the data points.

5. The method for identifying anomalies in engineering data based on big data according to claim 4 is characterized in that: In the said Step S4, the specific recognition steps are as follows: Step S4-1: Set the fluctuation trend threshold Fmax and the fluctuation degree threshold Zmax, calculate the median value of the standard value of the recognized keyword and the engineering data anomaly threshold, use it as the recognition point for the first data ripple anomaly recognition, and generate an engineering data trend chart according to the recognition results of each median value; Step S4-2: Let the engineering data fluctuation value be Z, the fluctuation trend value in the engineering data trend chart be F, and the initial value be 0. The calculation formula of the engineering data fluctuation value Z is as follows: In the formula, Z represents the fluctuation degree of engineering data when the corresponding median value is input; E represents the absolute value of the difference between the engineering data after the median value is input and the original engineering data; q represents the standard deviation of the input median value; R represents the average value of the input median value; u represents the influence coefficient of the median value corresponding to the engineering data; P sta It is expressed as the standard value for identifying keywords; Judge the recognition conditions according to the engineering data trend chart and the fluctuation degree value generated by a single recognition: When Z≤Zmax and F≤Fmax, perform re-recognition, and the recognition point is the median value of the engineering data anomaly threshold and the previous recognition point; When Z>Zmax or F>Fmax, adjust the engineering data anomaly threshold to stop recognition; Step S4-3: In the generated engineering data trend chart, use the degree of the engineering data fluctuation value Z as the ordinate and the time T interval of the engineering data fluctuation as the abscissa to calculate the slope of the trend chart. The slope calculation of the trend chart uses the following formula: In the formula, F represents the change slope of the engineering data trend chart; Z1 and Z0 represent the fluctuation values of the engineering data; T1 and T0 respectively represent the times of Z1 and Z0.

6. The method for identifying anomalies in engineering data based on big data according to claim 5 is characterized in that: In the said Step S5, construct a reference identifier. The reference identifier is provided with a recognition confirmation threshold C. When the recognition value is the engineering data anomaly threshold, when using the engineering data anomaly threshold for recognition, set the reference identifier for the keywords that satisfy Z<Zmax and F<Fmax. When a keyword with a reference identifier set is recognized, judge the recognition confirmation threshold C. When the recognition confirmation threshold C is equal to 0, perform recognition on it. After the recognition is completed, reset the recognition confirmation threshold C, otherwise skip the recognition, and subtract 1 from the recognition confirmation threshold C.

7. A system for identifying anomalies in engineering data based on big data, which is applied to the method for identifying anomalies in engineering data based on big data according to any one of claims 1 to 6, characterized in that: The said engineering data anomaly recognition system includes an engineering data acquisition module and a ripple data extraction module; The engineering data acquisition module includes a real-time engineering data acquisition unit, a historical engineering data acquisition unit and an engineering data collation unit; the real-time engineering data acquisition unit is used to collect data information in real-time engineering; the historical engineering data acquisition unit is used to collect historical data of engineering data; the engineering data collation unit is used to unify the format of the collected engineering data and clean and collate the data; The ripple data extraction module includes a keyword extraction unit and a keyword distribution storage unit; the keyword extraction unit is used to calculate the weights of the keywords in the engineering data and extract the data whose calculated values ​​meet the conditions; the keyword distribution storage unit is used to store different types of keyword data.

8. The engineering data anomaly identification system based on big data according to claim 7 is characterized by: The engineering data anomaly identification system also includes a data ripple anomaly identification module and a ripple feedback module; The data ripple anomaly recognition module includes a fluctuation value calculation unit, a recognition point positioning unit and a trend graph generation unit; the fluctuation value calculation unit is used to calculate the fluctuation value calculation generated by the recognition input value, including the calculation of its own fluctuation value and the calculation of the overall fluctuation impact value of the engineering data; the recognition point positioning unit is used to find and locate the recognition point of each keyword anomaly recognition; the trend graph generation unit is used to generate a trend graph of engineering data during continuous recognition; The ripple feedback module includes an abnormality recording unit and an abnormality feedback unit; the abnormality recording unit is used to record the abnormal value of the corresponding recognition point in the keyword recognition process; The abnormality feedback unit is used to feed back the information recorded by the abnormality recording unit.

9. The engineering data anomaly identification system based on big data according to claim 7 is characterized by: The engineering data anomaly identification system also includes an identification and judgment module and a reference identification module; The recognition and judgment module includes a keyword threshold judgment unit and a keyword re-recognition judgment unit; the keyword threshold judgment unit is used to judge the threshold adjustment condition of the keyword; the keyword re-recognition judgment unit is used to judge whether the recognized keyword meets the re-recognition condition; The reference identification module includes a reference identification confirmation unit and a reference identification setting unit; The reference identifier confirmation unit is used to confirm the keyword that needs to be set with a reference identifier; the reference identifier setting unit is used to set the reference identifier for the keyword after confirming it.

Images