An engineering quality early warning method and system based on data mining

Abstract

The present invention relates to the technical field of data processing, and particularly relates to an engineering quality early warning method and system based on data mining, including: obtaining the historical parameter time series of an engineering transformer, and inputting the historical parameter time series into a time series model to obtain a predicted parameter time series; taking multiple-dimensional parameters at the same moment as a data point, calculating the Manhattan distance between any data point and other data points, and taking the data points with the Manhattan distance less than a set distance threshold as the neighborhood data points of the any data point; calculating the anomaly score of the any data point by using an improved local outlier factor algorithm, and in response to the data points with the anomaly score higher than a set score threshold being anomaly data points, determining the moment corresponding to the anomaly data points as an anomaly moment and giving an early warning. The present invention solves the problems of insufficient accuracy and applicability in engineering quality supervision.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing. More specifically, the present invention relates to an engineering quality early warning method and system based on data mining. Background Art

[0002] During the process of engineering construction, quality problems are one of the key factors affecting engineering safety, economic benefits, and social benefits. Due to the long construction period and complex involved factors of engineering construction, traditional quality control methods usually rely on manual experience and post-inspection, which are not only inefficient but also have a large lag and subjectivity, making it difficult to detect and prevent potential quality problems in a timely manner. In addition, with the expansion of project scale and the improvement of technical requirements, traditional methods have been difficult to meet the needs of real-time quality monitoring and risk prediction in complex projects.

[0003] In recent years, with the rapid development of big data technology and data mining algorithms, the ability to collect, store, and analyze a large amount of engineering construction-related data has been significantly improved. These data include material properties, construction environment, operating parameters, and quality inspection results, etc., which not only contain important laws in the engineering construction process but also potential quality risk information. However, how to effectively extract valuable information from massive data, identify key influencing factors, and achieve early warning of quality problems is still a technical problem that needs to be solved urgently at present.

[0004] The patent application document with the publication number of CN109784758A discloses an engineering quality supervision and early warning system and method based on a BIM model. By combining the BIM model with the requirements of engineering quality supervision, this patent application document realizes the visual supervision of core business and significantly improves the supervision efficiency and coverage. However, the above technical solution mainly simply combines the BIM model with the requirements of engineering quality supervision to supervise and early warn of engineering quality, fails to deeply explore the limitations of the BIM model in complex scenarios, and does not fully consider the potential impact of the time-series characteristics of historical parameters on the quality supervision results, resulting in problems of insufficient accuracy and applicability when conducting engineering quality supervision in complex engineering environments. Summary of the Invention

[0005] To solve the problems of insufficient accuracy and applicability when conducting engineering quality supervision in complex engineering environments as mentioned in the above background art, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a method for early warning of engineering quality based on data mining, including: obtaining the historical parameter time series of an engineering transformer, where the historical parameter time series includes multi-dimensional parameter time series at different times; inputting the historical parameter time series into a time series model to obtain a predicted parameter time series; taking multiple-dimensional parameters at the same time as a data point, calculating the Manhattan distance between any data point and other data points, and taking the data points with the Manhattan distance less than the set distance threshold as the neighborhood data points of the any data point; calculating the anomaly score of any data point using an improved local outlier factor algorithm, and in response to the data points with the anomaly score higher than the set score threshold being anomaly data points, and giving an early warning; where the improved local outlier factor algorithm includes local reachability density, and the local reachability density is the product of an initial value and a local reachability density weight, and the local reachability density weight is the product of neighborhood similarity and neighborhood anomaly degree; the neighborhood anomaly degree characterizes the anomaly degree of the data point; the neighborhood similarity is: , where is the total number of neighborhood data points of the th data point, is the th neighborhood data point and the th neighborhood data point's cosine similarity, is the th neighborhood data point and the th neighborhood data point's Manhattan distance.

[0007] Through the above technical solution, by combining time series prediction with an improved local outlier factor algorithm, a comprehensive analysis of the multi-dimensional historical parameter time series of the engineering transformer and accurate anomaly detection are realized. The time series model can effectively capture the trends and laws of historical data and provide reliable predicted parameters; the Manhattan distance is used to screen neighborhood data points, enhancing the sensitivity to local characteristics; the improved local outlier factor algorithm further combines neighborhood similarity and anomaly degree, and by optimizing the calculation weight of local reachability density, the accuracy and robustness of anomaly detection are improved. In complex engineering scenarios, this solution can quickly locate anomaly points, reduce false alarms and missed detections, and through a real-time early warning mechanism, provide more intelligent and efficient technical support for engineering quality supervision.

[0008] Furthermore, the neighborhood anomaly degree is:

[0009] ;

[0010] In the formula, is the neighborhood anomaly degree of the th data point, is the th data point and the The Manhattan distance between adjacent neighborhood data points, is the total number of neighborhood data points of the th data point, is the value of the th dimension parameter among the th neighborhood data points, is the median of the values of the th dimension parameter among all neighborhood data points of the th data point, is the total number of dimension parameters, and ∏ represents the product.

[0011] Through the introduction of the calculation of neighborhood anomaly degree, the above technical solution combines the Manhattan distance and the normalized difference degree of multi-dimensional parameters to accurately measure the anomaly degree of each data point within its neighborhood. Using the deviation degree between the parameters of neighborhood data points and the median as the measurement benchmark, it avoids the interference of extreme values on the results. At the same time, through the Manhattan distance weighting, it ensures that the closer points have a greater impact on the neighborhood anomaly degree, thus better conforming to the characteristics of the actual data distribution. This method can effectively capture minor anomalies in the multi-dimensional data space, improve the sensitivity and reliability for complex engineering quality problems, and provide stronger adaptability and accuracy support for subsequent anomaly detection and early warning.

[0012] Further, the historical parameter time series includes: the temperature parameter sequence, vibration parameter sequence, voltage parameter sequence, current parameter sequence, and settlement parameter sequence of the engineering transformer.

[0013] Further, the temperature parameter sequence of the engineering transformer is collected by a temperature sensor, the vibration parameter sequence is collected by a vibration sensor, the voltage parameter sequence is collected by a voltage sensor, the current parameter sequence is collected by a current sensor, and the settlement parameter sequence is collected by a settlement gauge.

[0014] Further, the time series model is a long short-term memory network model.

[0015] By adopting the long short-term memory network as the time series model, the above technical solution can effectively capture the long-term dependence relationship and short-term fluctuation characteristics in the historical parameter time series of the engineering transformer, solve the problems of gradient disappearance and information loss that are prone to occur in traditional time series models when dealing with data with a long time span, and thus improve the accuracy and stability of prediction. This model can dynamically adapt to the complex change laws of multi-dimensional parameters, accurately generate the predicted parameter time series, provide high-quality data support for subsequent anomaly detection, and significantly improve the application performance of the system in complex scenarios.

[0016] Further, it also includes data cleaning and data standardization processing of the historical parameter time series.

[0017] Through data cleaning and standardization processing of the historical parameter time series, the above technical solution can effectively remove outliers, missing values, and noise data, ensure the data quality of the input model, and at the same time convert parameters of different dimensions to a unified scale to eliminate the influence of dimension differences on the calculation results. This process not only improves the accuracy and robustness of time series model prediction, but also lays a solid foundation for subsequent anomaly detection based on multi-dimensional parameters, thus significantly enhancing the adaptability and reliability of the overall system in complex engineering scenarios.

[0018] Furthermore, it also includes training the time series model, specifically: inputting the training set into the pre-constructed time series model for training. During the training process, calculate the loss between the predicted value output and the label; use the gradient descent method to adjust the model parameters to minimize the prediction error; iteratively adjust the parameters of the time series model until the loss is less than a certain value or reaches the set number of training times, and finally obtain the trained time series model.

[0019] The above technical solution can effectively improve the model's learning ability and prediction accuracy for historical parameter time series features. This process ensures that the model fully fits the data characteristics and effectively captures key patterns. At the same time, by setting a loss threshold or an upper limit on the number of training times, problems of overfitting or under-training are avoided, and finally an optimized time series model is obtained. This solution lays a foundation for accurately generating the predicted parameter time series, thereby improving the reliability of anomaly detection and the accuracy of engineering quality warning.

[0020] In the second aspect, the present invention provides an engineering quality warning system based on data mining, including a memory and a processor. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, it realizes an engineering quality warning method based on data mining as described in any one of the above.

[0021] The beneficial effects of the present invention are as follows:

[0022] By combining time series analysis, neighborhood calculation, and an improved local outlier factor algorithm, the present invention realizes accurate prediction and anomaly detection of multi-dimensional parameters of engineering transformers, effectively improving the real-time performance and intelligent level of complex engineering quality monitoring; uses a long short-term memory network model to capture patterns in historical time series and generate high-precision prediction parameters; further enhances the robustness and sensitivity of anomaly detection through neighborhood data point screening based on Manhattan distance and local outlier factor calculation considering neighborhood similarity and anomaly degree. Combining the collaborative design of software and hardware, the system realizes full-process automated processing and a flexible warning mechanism, providing a more efficient and reliable technical means for engineering quality management. Description of the Drawings

[0023] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0024] Figure 1 is a flowchart schematically showing a method for engineering quality early warning based on data mining according to an embodiment of the present invention;

[0025] Figure 2 is a block diagram schematically showing the structure of an engineering quality early warning system based on data mining according to an embodiment of the present invention. Detailed Embodiments

[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present invention.

[0027] The following will describe in detail the specific embodiments of the present invention in conjunction with the accompanying drawings.

[0028] An embodiment of a method for engineering quality early warning based on data mining.

[0029] As Figure 1 shown, a flowchart of a method for engineering quality early warning based on data mining according to an embodiment of the present invention includes the following steps:

[0030] S1: Obtain the historical parameter time series of the engineering transformer, and input the historical parameter time series into the time series model to obtain the predicted parameter time series.

[0031] In one embodiment, the historical parameter time series of the engineering transformer includes a temperature parameter sequence, a vibration parameter sequence, a voltage parameter sequence, a current parameter sequence, and a settlement amount parameter sequence. To ensure the integrity and accuracy of the input data, data cleaning and data standardization processing are performed on the historical parameter time series. Among them, the process of data cleaning includes removing missing values, outliers, and redundant data to eliminate the interference of noise on subsequent modeling; the process of data standardization unifies the dimensions of different parameters by normalizing or standardizing the parameters, ensuring that the model can fairly process parameter sequences with different physical meanings and improving the modeling accuracy.

[0032] In one embodiment, the acquisition of historical parameter time series is achieved through multiple sensors: the temperature parameter series is acquired by a high-precision temperature sensor, the vibration parameter series is monitored in real time by a vibration sensor, the voltage parameter series is obtained by a voltage sensor, the current parameter series is captured by a current sensor, and the settlement parameter series is acquired by a settlement gauge. This multi-source acquisition method can comprehensively cover the key parameters during the operation of the engineering transformer, ensuring the multi-dimensionality and high resolution of the input data, thus providing sufficient data support for subsequent modeling and prediction.

[0033] In one embodiment, the time series model can be a long short-term memory network model. Due to its excellent time-dependent modeling ability and the ability to capture non-linear sequence relationships, this model can effectively overcome the limitations of traditional time series models in extracting long-term dependence features. Through its unique gating mechanism, such as input gate, forget gate, and output gate, the long short-term memory network model can filter and store key information in historical data while ignoring irrelevant or redundant information, thereby improving the accuracy and stability of the prediction results.

[0034] In another embodiment, the time series model can be a gated recurrent unit model. By introducing an update gate and a reset gate, the model effectively simplifies the network structure. Compared with the long short-term memory network gated recurrent unit, it has higher computational efficiency and still has the ability to handle the long-term dependence of time series. The update gate can filter out the part of historical information that is most relevant to the current prediction, while the reset gate can dynamically forget irrelevant historical data, thereby reducing the interference of redundant information and highlighting key features. This high efficiency and flexibility make the GRU model perform particularly well in scenarios where rapid prediction is required, while maintaining a high prediction accuracy, providing reliable support for the parameter time series prediction of the engineering transformer.

[0035] Taking the long short-term memory network model as an example, the time series model is trained as follows: the training set is input into the pre-constructed time series model for training. During the training process, the loss between the predicted value of the output and the label is calculated; the gradient descent method is used to adjust the model parameters to minimize the prediction error; the parameters of the time series model are iteratively adjusted until the loss is less than a certain value or the set number of training times is reached, and finally the trained time series model is obtained.

[0036] S2: Regarding multiple-dimensional parameters at the same moment as a data point, calculate its Manhattan distance from other data points, and consider the data points with a distance less than the set distance threshold as its neighborhood data points.

[0037] Exemplarily, assume that the standardized parameters at a certain moment include temperature, vibration, voltage, current, and settlement, denoted as data point , and the data point at another moment. Calculate the Manhattan distance between the two: , if is less than the distance threshold, then is a neighborhood data point of . Among them, a smaller distance threshold will strictly limit the selection of neighborhood data points, only classifying points very close to the target data point into the neighborhood, avoiding including too many data points in the neighborhood, thereby improving the efficiency and pertinence of the algorithm; a larger distance threshold will relax the neighborhood screening criteria, allowing more data points at a greater distance to be included in the neighborhood, effectively reducing local fluctuations caused by sparse data points and enhancing the stability of anomaly detection.

[0038] S3: Calculate the anomaly score of any data point using the improved local outlier factor algorithm.

[0039] In one embodiment, the improved local outlier factor algorithm includes local reachability density, and the local reachability density is the product of an initial value and a local reachability density weight, and the local reachability density weight is the product of neighborhood similarity and neighborhood anomaly degree; the neighborhood anomaly degree characterizes the anomaly degree of a data point.

[0040] The neighborhood anomaly degree is:

[0041] ;

[0042] In the formula, is the neighborhood anomaly degree of the th data point, is the Manhattan distance between the th data point and the th neighborhood data point, is the total number of neighborhood data points of the th data point, is the value of the th dimension parameter among the th neighborhood data points, is the median of the values of the th dimension parameter among all neighborhood data points of the th data point, is the total number of dimension parameters, and ∏ represents taking the product.

[0043] In another embodiment, the neighborhood anomaly degree is:

[0044] ;

[0045] In the formula, is the neighborhood anomaly degree of the th data point, is the Manhattan distance between the th data point and the th neighborhood data point, is the total number of neighborhood data points for the th data point, is the th value of the th dimensional parameter among the th neighborhood data points, is the mean value of the th dimensional parameter value among all neighborhood data points of the th data point. When the mean value is the same as the median value, the median value is selected as the first choice. ∏ represents the product.

[0046] S4: In response to the data points with abnormal scores higher than the set score threshold being abnormal data points, a warning is issued.

[0047] The value of the above set score threshold can be 1.0. Of course, it can also be determined according to the actual situation.

[0048] In one embodiment, when the system detects an abnormality in the engineering quality parameters through data mining technology, a real-time warning signal will be immediately triggered to ensure that relevant personnel or systems can respond quickly. The warning mechanism can be flexibly configured according to the needs of the engineering site. For example, the abnormal area is highlighted on the monitoring large screen and the engineering drawing is superimposed to accurately locate the abnormal point; an alarm is issued through the sound and light equipment to remind the on-site personnel to pay attention to potential safety hazards; or the abnormal information is pushed to the management personnel in the form of text messages, emails, etc., with parameter details, time stamps, and risk assessment levels attached, so as to achieve comprehensive and efficient engineering quality warning and response.

[0049] The solution of the present invention can accurately monitor the state change of the engineering transformer and timely warn of potential quality problems by combining the processing of multi-dimensional parameter time series data and the improved local outlier factor algorithm. Using the long short-term memory network to model historical data can effectively capture complex time series dependencies and non-linear patterns. At the same time, data cleaning and standardization processing improve the quality of data and the reliability of the model. By calculating the Manhattan distance and local outlier factor between data points, the system can sensitively identify abnormal patterns and trigger an alarm based on a preset threshold. This solution provides intelligent and efficient technical support for the real-time monitoring of engineering quality, can detect potential risks at an early stage, reduce the burden of manual monitoring, and improve engineering safety and maintenance efficiency.

[0050] An embodiment of an engineering quality warning system based on data mining:

[0051] As Figure 2 shown, the structural block diagram of an engineering quality warning system based on data mining according to an embodiment of the present invention includes a processor and a memory.

[0052] The present invention also provides an engineering quality early warning system based on data mining. As Figure 2 shown, the system includes a processor and a memory, and the memory stores computer program instructions, which, when executed by the processor, implement an engineering quality early warning method based on data mining according to the present invention.

[0053] The engineering quality early warning system based on data mining further includes other components well-known to those skilled in the art, such as a communication interface, and its settings and functions are known in the art, so they will not be elaborated here.

[0054] In the present invention, the aforementioned memory can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or apparatus. For example, a computer-readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc., or any other medium that can be used to store the required information and can be accessed by an application program, module, or both. Any such computer storage medium can be part of the device or accessible or connectable to the device. Any application or module described in the present invention can be implemented using computer-readable / executable instructions that can be stored or otherwise held by such a computer-readable medium.

[0055] In the description of this specification, the meanings of "a plurality of" and "several" are at least two, such as two, three, or more, unless otherwise specifically defined.

[0056] Although this specification has shown and described multiple embodiments of the present invention, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art will think of many ways of modification, change, and substitution without departing from the idea and spirit of the present invention. It should be understood that various alternative solutions to the embodiments of the present invention described herein can be adopted in the process of practicing the present invention.

[0057] The neighborhood similarity is: , where is the total number of neighborhood data points of the th data point, is the cosine similarity between the th neighborhood data point and the th neighborhood data point, is the Manhattan distance between the th neighborhood data point and the th neighborhood data point.

[0058] The neighborhood similarity is: , where is the total number of neighborhood data points of the th data point, is the cosine similarity between the th neighborhood data point and the th neighborhood data point, is the Manhattan distance between the th neighborhood data point and the th neighborhood data point.

Claims

1. A project quality early warning method based on data mining, characterized in that: include: Obtaining a historical parameter time series of an engineering transformer, wherein the historical parameter time series includes a multi-dimensional parameter time series at different times; Input the historical parameter time series into the time series model to obtain the predicted parameter time series; Taking multiple dimension parameters at the same time as a data point, calculating the Manhattan distance between any data point and other data points, and taking the data points whose Manhattan distance is less than the set distance threshold as the neighborhood data points of any data point; Calculating the abnormal score of any of the data points using an improved local abnormal factor algorithm, responding to a data point whose abnormal score is higher than a set score threshold as an abnormal data point, and issuing an early warning; The improved local anomaly factor algorithm includes local reachable density, which is the product of the initial value and the local reachable density weight, and the local reachable density weight is the product of the neighborhood similarity and the neighborhood anomaly; The neighborhood anomaly degree represents the degree of anomaly of the data point; The neighborhood similarity S i for: Among them, B i is the total number of neighborhood data points of the ith data point, s j,j′ is the cosine similarity between the jth neighborhood data point and the j′th neighborhood data point, d j,j′ is the Manhattan distance between the jth neighborhood data point and the j′th neighborhood data point.

2. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: The neighborhood abnormality is: In the formula, F i is the neighborhood abnormality of the i-th data point, d i,j is the Manhattan distance between the i-th data point and the j-th neighboring data point, B i is the total number of neighborhood data points of the ith data point, G j,a is the value of the ath dimension parameter in the jth neighborhood data point, X i,a is the median of the a-th dimension parameter values ​​of all neighboring data points of the ith data point, N is the total number of dimension parameters, and ∏ represents the product.

3. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: The historical parameter time series includes: a temperature parameter sequence, a vibration parameter sequence, a voltage parameter sequence, a current parameter sequence and a settlement parameter sequence of the engineering transformer.

4. The engineering quality early warning method based on data mining according to claim 3 is characterized in that: The temperature parameter sequence of the engineering transformer is collected by the temperature sensor, the vibration parameter sequence of the engineering transformer is collected by the vibration sensor, the voltage parameter sequence of the engineering transformer is collected by the voltage sensor, the current parameter sequence of the engineering transformer is collected by the current sensor, and the settlement parameter sequence of the engineering transformer is collected by the settlement meter.

5. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: The time series model is a long short-term memory network model.

6. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: It also includes data cleaning and data standardization processing of the historical parameter time series.

7. The engineering quality early warning method based on data mining according to claim 1 is characterized in that: It also includes training the time series model, specifically: The training set is input into the pre-built time series model for training. During the training process, the loss between the output prediction value and the label is calculated; Use gradient descent to adjust model parameters to minimize prediction error; Iteratively adjust the parameters of the time series model until the loss is less than a certain value or the set number of training times is reached, and finally a trained time series model is obtained.

8. A project quality early warning system based on data mining, characterized in that: The invention comprises a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an engineering quality early warning method based on data mining as described in any one of claims 1 to 7 is implemented.

Images