Project quality early warning method and system based on data mining

Abstract

The invention relates to the technical field of data processing, in particular to an engineering quality early warning method and system based on data mining, and the method comprises the steps: obtaining a historical parameter time sequence of an engineering transformer, and inputting the historical parameter time sequence into a time sequence model to obtain a prediction parameter time sequence; taking a plurality of dimension parameters at the same moment as a data point, calculating a Manhattan distance between any data point and other data points, and taking the data points with the Manhattan distances smaller than a set distance threshold as neighborhood data points of the any data point; and calculating an abnormal score of the any data point by using an improved local abnormal factor algorithm, and if the data point whose abnormal score is higher than a set score threshold is an abnormal data point, determining that the moment corresponding to the abnormal data point is an abnormal moment, and performing early warning. According to the invention, the problem of insufficient accuracy and applicability during engineering quality supervision is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a project quality early warning method and system based on data mining. Background Art

[0002] During the construction process, quality issues are one of the key factors affecting project safety, economic benefits and social benefits. Due to the long construction period and complex factors involved, traditional quality control methods usually rely on manual experience and post-inspection, which is not only inefficient, but also has a large lag and subjectivity, making it difficult to detect and prevent potential quality problems in a timely manner. In addition, with the expansion of project scale and the improvement of technical requirements, traditional methods have been unable to adapt to the real-time quality monitoring and risk prediction needs in complex projects.

[0003] In recent years, with the rapid development of big data technology and data mining algorithms, the ability to collect, store and analyze a large amount of engineering construction related data has been significantly improved. These data include material properties, construction environment, operating parameters and quality inspection results, etc., which not only contain important laws in the engineering construction process, but also contain potential quality risk information. However, how to effectively extract valuable information from massive data, identify key influencing factors, and achieve early warning of quality problems is still a technical problem that needs to be solved urgently.

[0004] The patent application document with application publication number CN109784758A discloses a project quality supervision and early warning system and method based on the BIM model. This patent application document realizes visual supervision of core businesses by combining the BIM model with the needs of project quality supervision, significantly improving supervision efficiency and coverage. However, the above technical solution is mainly based on the simple combination of the BIM model and the needs of project quality supervision to supervise and warn the project quality. It fails to deeply explore the limitations of the BIM model in complex scenarios, and does not fully consider the potential impact of the temporal characteristics of historical parameters on the quality supervision results, resulting in insufficient accuracy and applicability when conducting project quality supervision in complex engineering environments. Summary of the invention

[0005] In order to solve the problem of insufficient accuracy and applicability when conducting engineering quality supervision in a complex engineering environment raised in the above background technology, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a project quality early warning method based on data mining, comprising: obtaining a historical parameter time series of an engineering transformer, wherein the historical parameter time series comprises a multi-dimensional parameter time series at different times; inputting the historical parameter time series into a time series model to obtain a predicted parameter time series; taking multiple dimensional parameters at the same time as a data point, calculating the Manhattan distance between any data point and other data points, and taking the data point whose Manhattan distance is less than a set distance threshold as a neighborhood data point of the any data point; calculating the abnormal score of any data point using an improved local anomaly factor algorithm, responding to a data point whose abnormal score is higher than the set score threshold as an abnormal data point, and issuing an early warning; wherein the improved local anomaly factor algorithm comprises a local reachable density, wherein the local reachable density is the product of an initial value and a local reachable density weight, and the local reachable density weight is the product of a neighborhood similarity and a neighborhood abnormality; the neighborhood abnormality characterizes the degree of abnormality of a data point; the neighborhood similarity for: ,in, For the The total number of neighboring data points of a data point, For the The neighboring data points and The cosine similarity of the neighboring data points is For the The neighboring data points and Manhattan distance to neighboring data points.

[0007] The above technical solution realizes comprehensive analysis and accurate anomaly detection of multi-dimensional historical parameter time series of engineering transformers by combining time series prediction with improved local anomaly factor algorithm. The time series model can effectively capture the trends and laws of historical data and provide reliable prediction parameters; Manhattan distance is used to screen neighborhood data points, enhancing sensitivity to local characteristics; the improved local anomaly factor algorithm further integrates neighborhood similarity and anomaly, and improves the accuracy and robustness of anomaly detection by optimizing the calculation weight of local reachable density. In complex engineering scenarios, this solution can quickly locate anomalies, reduce false alarms and missed alarms, and provide more intelligent and efficient technical support for engineering quality supervision through real-time early warning mechanisms.

[0008] Furthermore, the neighborhood abnormality is: ; In the formula, For the The neighborhood abnormality of a data point is For the data points and The Manhattan distance between neighboring data points is For the The total number of neighboring data points of a data point, For the The first of the neighboring data points The value of the dimension parameter, For the The first of all neighboring data points of a data point The median of the dimension parameter values, is the total number of dimension parameters, and ∏ represents the product.

[0009] The above technical solution introduces the calculation of neighborhood anomaly, combines the Manhattan distance and the normalized difference of multidimensional parameters, and accurately measures the degree of anomaly of each data point in its neighborhood. The deviation degree of the neighborhood data point parameters from the median is used as the measurement benchmark to avoid the interference of extreme values ​​on the results. At the same time, Manhattan distance weighting ensures that the closer the point is, the greater the impact on the neighborhood anomaly, which is more in line with the actual data distribution characteristics. This method can effectively capture small anomalies in multidimensional data space, improve the sensitivity and reliability of complex engineering quality issues, and provide stronger adaptability and accuracy support for subsequent anomaly detection and early warning.

[0010] Furthermore, the historical parameter time series includes: a temperature parameter sequence, a vibration parameter sequence, a voltage parameter sequence, a current parameter sequence and a settlement parameter sequence of the engineering transformer.

[0011] Furthermore, a temperature sensor is used to collect a temperature parameter sequence of the engineering transformer, a vibration sensor is used to collect a vibration parameter sequence of the engineering transformer, a voltage sensor is used to collect a voltage parameter sequence of the engineering transformer, a current sensor is used to collect a current parameter sequence of the engineering transformer, and a settlement meter is used to collect a settlement parameter sequence of the engineering transformer.

[0012] Furthermore, the time series model is a long short-term memory network model.

[0013] The above technical solution uses the long short-term memory network as a time series model, which can effectively capture the long-term dependencies and short-term fluctuation characteristics in the historical parameter time series of the engineering transformer, and solve the gradient vanishing and information loss problems that are prone to occur in traditional time series models when processing long-span data, thereby improving the accuracy and stability of predictions. The model can dynamically adapt to the complex change patterns of multi-dimensional parameters, accurately generate prediction parameter time series, provide high-quality data support for subsequent anomaly detection, and significantly improve the application performance of the system in complex scenarios.

[0014] Furthermore, it also includes data cleaning and data standardization processing of the historical parameter time series.

[0015] The above technical solution can effectively remove outliers, missing values ​​and noise data by cleaning and standardizing the historical parameter time series, ensuring the data quality of the input model, and converting parameters of different dimensions to a unified scale to eliminate the impact of dimensional differences on the calculation results. This process not only improves the accuracy and robustness of time series model predictions, but also lays a solid foundation for subsequent anomaly detection based on multidimensional parameters, thereby significantly enhancing the adaptability and reliability of the overall system in complex engineering scenarios.

[0016] Furthermore, it also includes training the time series model, specifically: inputting the training set into a pre-built time series model for training, and during the training process, calculating the loss between the output prediction value and the label; adjusting the model parameters using the gradient descent method to minimize the prediction error; iteratively adjusting the parameters of the time series model until the loss is less than a certain value or reaches a set number of training times, and finally obtaining a trained time series model.

[0017] The above technical solution can effectively improve the model's learning ability and prediction accuracy of historical parameter time series characteristics. This process ensures that the model fully fits the data characteristics and effectively captures key rules. At the same time, by setting the loss threshold or the upper limit of the number of training times, it avoids overfitting or insufficient training problems, and finally obtains the optimized time series model. This solution lays the foundation for accurately generating prediction parameter time series, thereby improving the reliability of anomaly detection and the accuracy of engineering quality warning.

[0018] In a second aspect, the present invention provides an engineering quality early warning system based on data mining, comprising a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, any one of the above-described engineering quality early warning methods based on data mining is implemented.

[0019] The beneficial effects of the present invention are: The present invention combines time series analysis, neighborhood calculation and improved local anomaly factor algorithm to achieve accurate prediction and anomaly detection of multi-dimensional parameters of engineering transformers, effectively improving the real-time and intelligent level of complex engineering quality monitoring; using long-term and short-term memory network models to capture the rules in historical time series and generate high-precision prediction parameters; further through the screening of neighborhood data points based on Manhattan distance and the calculation of local anomaly factors that comprehensively consider neighborhood similarity and anomaly, the robustness and sensitivity of anomaly detection are enhanced. Combined with the collaborative design of software and hardware, the system realizes full-process automated processing and a flexible early warning mechanism, providing a more efficient and reliable technical means for engineering quality management. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 is a flow chart schematically illustrating a method for early warning of engineering quality based on data mining according to an embodiment of the present invention; Figure 2 The figure schematically shows a structural block diagram of an engineering quality early warning system based on data mining according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] An implementation of an engineering quality early warning method based on data mining.

[0024] like Figure 1 As shown, a flow chart of a method for early warning of engineering quality based on data mining in an embodiment of the present invention includes the following steps: S1: Obtain the historical parameter time series of the engineering transformer, and input the historical parameter time series into the time series model to obtain the predicted parameter time series.

[0025] In one embodiment, the historical parameter time series of the engineering transformer includes a temperature parameter sequence, a vibration parameter sequence, a voltage parameter sequence, a current parameter sequence, and a settlement parameter sequence. To ensure the integrity and accuracy of the input data, the historical parameter time series is subjected to data cleaning and data standardization. The data cleaning process includes removing missing values, outliers, and redundant data to eliminate the interference of noise on subsequent modeling; the data standardization process normalizes or standardizes the parameters to unify the dimensions of different parameters, thereby ensuring that the model can fairly handle parameter sequences of different physical meanings and improve modeling accuracy.

[0026] In one embodiment, the historical parameter time series is collected through multiple sensors: the temperature parameter sequence is collected through a high-precision temperature sensor, the vibration parameter sequence is monitored in real time through a vibration sensor, the voltage parameter sequence is obtained through a voltage sensor, the current parameter sequence is captured through a current sensor, and the settlement parameter sequence is collected through a settlement meter. This multi-source collection method can fully cover the key parameters in the operation process of the engineering transformer, ensure the multidimensionality and high resolution of the input data, and thus provide sufficient data support for subsequent modeling and prediction.

[0027] In one embodiment, the time series model can be a long short-term memory network model. Due to its excellent time-dependency modeling ability and the ability to capture nonlinear sequence relationships, the model can effectively overcome the limitations of traditional time series models in long-term dependency feature extraction. The long short-term memory network model can filter and store key information in historical data through its unique gating mechanism, such as input gate, forget gate and output gate, while ignoring irrelevant or redundant information, thereby improving the accuracy and stability of the prediction results.

[0028] In another embodiment, the time series model can be a gated recurrent unit model, which effectively simplifies the network structure by introducing update gates and reset gates. Compared with the long short-term memory network gated recurrent unit, it has higher computational efficiency while still having the ability to handle long-term dependencies of time series. The update gate can filter the part of historical information that is most relevant to the current prediction, while the reset gate can dynamically forget irrelevant historical data, thereby reducing the interference of redundant information and highlighting key features. This high efficiency and flexibility make the GRU model perform particularly well in scenarios that require fast predictions, while maintaining a high prediction accuracy, providing reliable support for parameter timing prediction of engineering transformers.

[0029] Taking the long short-term memory network model as an example, the time series model is trained as follows: the training set is input into the pre-built time series model for training. During the training process, the loss between the output prediction value and the label is calculated; the model parameters are adjusted using the gradient descent method to minimize the prediction error; the parameters of the time series model are iteratively adjusted until the loss is less than a certain value or the set number of training times is reached, and finally a trained time series model is obtained.

[0030] S2: Treat multiple dimensional parameters at the same time as one data point, calculate its Manhattan distance with other data points, and regard data points less than the set distance threshold as its neighborhood data points.

[0031] For example, assume that the standardized parameters at a certain moment include temperature, vibration, voltage, current and sedimentation, which are recorded as data points , another data point , calculate the Manhattan distance between the two: ,like is less than the distance threshold, then for A neighborhood data point is a data point in the neighborhood of a target data point. A smaller distance threshold will strictly limit the selection of neighborhood data points, and only include points that are very close to the target data point in the neighborhood, avoiding too many data points in the neighborhood, thereby improving the efficiency and pertinence of the algorithm. A larger distance threshold will relax the neighborhood screening criteria, allowing more data points that are farther away to be included in the neighborhood, effectively reducing local fluctuations caused by scarce data points and enhancing the stability of anomaly detection.

[0032] S3: Calculate the anomaly score of any data point using the improved local anomaly factor algorithm.

[0033] In one embodiment, the improved local anomaly factor algorithm includes a local reachable density, wherein the local reachable density is the product of an initial value and a local reachable density weight, wherein the local reachable density weight is the product of a neighborhood similarity and a neighborhood anomaly degree; the neighborhood anomaly degree represents the degree of anomaly of a data point; The neighborhood abnormality is: ; In the formula, For the The neighborhood abnormality of a data point is For the data points and The Manhattan distance between neighboring data points is For the The total number of neighboring data points of a data point, For the The first of the neighboring data points The value of the dimension parameter, For the The first of all neighboring data points of a data point The median of the dimension parameter values, is the total number of dimension parameters, and ∏ represents the product.

[0034] In another embodiment, the neighborhood abnormality is: ; In the formula, For the The neighborhood abnormality of a data point is For the data points and The Manhattan distance between neighboring data points is For the The total number of neighboring data points of a data point, For the The first of the neighboring data points The value of the dimension parameter, For the The first of all neighboring data points of a data point The mean of the dimension parameter values, is the total number of dimension parameters. When the mean and median are the same, the median is the first choice. ∏ means multiplication.

[0035] The neighborhood similarity for: ,in, For the The total number of neighboring data points of a data point, For the The neighboring data points and The cosine similarity of the neighboring data points is For the The neighboring data points and Manhattan distance to neighboring data points.

[0036] S4: In response to a data point having an abnormal score higher than a set score threshold being an abnormal data point, an early warning is issued.

[0037] The score threshold value may be set to 1.0, and may also be determined based on actual conditions.

[0038] In one embodiment, when the system detects an abnormality in the engineering quality parameters through data mining technology, it will immediately trigger a real-time warning signal to ensure that relevant personnel or systems can respond quickly. The warning mechanism can be flexibly configured according to the needs of the engineering site, such as highlighting the abnormal area on the monitoring screen and superimposing the engineering drawings to accurately locate the abnormal point; sounding an alarm through sound and light equipment to remind on-site personnel to pay attention to potential safety hazards; or pushing abnormal information to management personnel via SMS, email, etc., with parameter details, timestamps and risk assessment levels, so as to achieve comprehensive and efficient engineering quality warning and response.

[0039] The solution of the present invention can accurately monitor the state changes of engineering transformers and timely warn of potential quality problems by combining the processing of multi-dimensional parameter time series data with an improved local anomaly factor algorithm. The use of long short-term memory networks to model historical data can effectively capture complex time series dependencies and nonlinear laws. At the same time, data cleaning and standardization processing improve the quality of the data and the reliability of the model. By calculating the Manhattan distance and local anomaly factors between data points, the system can sensitively identify abnormal patterns and trigger alarms based on preset thresholds. This solution provides intelligent and efficient technical support for real-time monitoring of engineering quality, can detect potential risks at an early stage, reduce the burden of manual monitoring, and improve engineering safety and maintenance efficiency.

[0040] An embodiment of an engineering quality early warning system based on data mining: like Figure 2 As shown, a structural block diagram of an engineering quality early warning system based on data mining in an embodiment of the present invention includes a processor and a memory.

[0041] The present invention also provides a project quality early warning system based on data mining. Figure 2 As shown, the system includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an engineering quality early warning method based on data mining according to the present invention is implemented.

[0042] The engineering quality early warning system based on data mining also includes other components familiar to those skilled in the art, such as communication interfaces, and their settings and functions are known in the art, so they will not be described in detail here.

[0043] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0044] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0045] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A project quality early warning method based on data mining, characterized in that: include: Obtaining a historical parameter time series of an engineering transformer, wherein the historical parameter time series includes a multi-dimensional parameter time series at different times; Input the historical parameter time series into the time series model to obtain the predicted parameter time series; Taking multiple dimension parameters at the same time as a data point, calculating the Manhattan distance between any data point and other data points, and taking the data points whose Manhattan distance is less than the set distance threshold as the neighborhood data points of any data point; Calculating the abnormal score of any of the data points using an improved local abnormal factor algorithm, responding to a data point whose abnormal score is higher than a set score threshold as an abnormal data point, and issuing an early warning; The improved local anomaly factor algorithm includes local reachable density, which is the product of the initial value and the local reachable density weight, and the local reachable density weight is the product of the neighborhood similarity and the neighborhood anomaly; The neighborhood anomaly degree represents the degree of anomaly of the data point; The neighborhood similarity for: ,in, For the The total number of neighboring data points of a data point, For the The neighboring data points and The cosine similarity of the neighboring data points is For the The neighboring data points and Manhattan distance to neighboring data points.

2. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: The neighborhood abnormality is: ； In the formula, For the The neighborhood abnormality of a data point is For the data points and The Manhattan distance between neighboring data points is For the The total number of neighboring data points of a data point, For the The first of the neighboring data points The value of the dimension parameter, For the The first of all neighboring data points of a data point The median of the dimension parameter values, is the total number of dimension parameters, and ∏ represents the product.

3. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: The historical parameter time series includes: a temperature parameter sequence, a vibration parameter sequence, a voltage parameter sequence, a current parameter sequence and a settlement parameter sequence of the engineering transformer.

4. The engineering quality early warning method based on data mining according to claim 3 is characterized in that: The temperature parameter sequence of the engineering transformer is collected by the temperature sensor, the vibration parameter sequence of the engineering transformer is collected by the vibration sensor, the voltage parameter sequence of the engineering transformer is collected by the voltage sensor, the current parameter sequence of the engineering transformer is collected by the current sensor, and the settlement parameter sequence of the engineering transformer is collected by the settlement meter.

5. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: The time series model is a long short-term memory network model.

6. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: It also includes data cleaning and data standardization processing of the historical parameter time series.

7. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: It also includes training the time series model, specifically: The training set is input into the pre-built time series model for training. During the training process, the loss between the output prediction value and the label is calculated; Use gradient descent to adjust model parameters to minimize prediction error; Iteratively adjust the parameters of the time series model until the loss is less than a certain value or the set number of training times is reached, and finally a trained time series model is obtained.

8. A project quality early warning system based on data mining, characterized in that: It comprises a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an engineering quality early warning method based on data mining as described in any one of claims 1 to 7 is implemented.

Images