Abnormal data processing method, computing device and machine readable storage medium

Abstract

The invention discloses an abnormal data processing method, computing equipment and a machine readable storage medium, and relates to the technical field of data processing. The abnormal data processing method comprises the steps of performing time clustering on historical operation data of at least one engineering machine to obtain a plurality of sample operation data sets; training a preset model based on the historical operation data of the target engineering machinery to obtain an operation data prediction model; determining the duration corresponding to the abnormal data segment; under the condition that the duration corresponding to the abnormal data segment is greater than or equal to a preset duration, performing similarity matching on the plurality of sample operation data sets and the abnormal data segment, and performing supplementary processing on the abnormal data based on the target sample operation data with the highest matching degree to obtain processed real-time operation data; and under the condition that the duration corresponding to the abnormal data segment is smaller than a preset duration, performing supplementary processing on the abnormal data segment based on the operation data prediction model to obtain processed real-time operation data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an abnormal data processing method, a computing device, and a machine-readable storage medium. Background Art

[0002] With the rapid development of mechanical manufacturing technology, construction machinery such as tower cranes are widely used in a variety of engineering construction scenarios. Usually, data is collected through sensors during the operation of construction machinery, and the structural design of the construction machinery is analyzed based on the collected operation data. High-quality operation data can be used to optimize and improve the structural design of construction machinery and improve the operation efficiency of construction machinery. In actual data collection scenarios, abnormal data such as missing data or abnormal mutation of data values ​​are prone to occur. Interpolation filling or mean filling can be used to process abnormal data.

[0003] However, the quality of operating data is affected by multiple factors such as timeliness, environment, working conditions and loads. Construction machinery such as tower cranes are usually in harsh working environments and complex working conditions, which leads to a large amount of continuous abnormal data in the collected operating data. Even if interpolation filling or mean filling is used to process the abnormal data, high-quality operating data cannot be obtained, which in turn affects the structural design of construction machinery. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an abnormal data processing method, a computing device, and a machine-readable storage medium to solve the problem that high-quality operation data cannot be obtained in the prior art.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a method for processing abnormal data, and the method for processing abnormal data includes:

[0006] Performing time clustering on historical operation data of at least one construction machine to obtain multiple sample operation data sets;

[0007] Training a preset model based on historical operation data of the target construction machinery to obtain an operation data prediction model;

[0008] Determine whether there is an abnormal data segment in the real-time operation data of the target engineering machinery;

[0009] When there is an abnormal data segment in the real-time operation data, determine the duration corresponding to the abnormal data segment;

[0010] When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data is supplemented based on the target sample operation data with the highest matching degree to obtain processed real-time operation data;

[0011] When the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is supplemented based on the operation data prediction model to obtain processed real-time operation data.

[0012] In an embodiment of the present application, determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery includes:

[0013] Acquire multiple pieces of normal operation data corresponding to the target construction machinery;

[0014] Linearly combine multiple pieces of normal operation data to obtain a linear expression between the sampling value of the operation data and time;

[0015] Based on the linear expression, determine whether there is an abnormal data segment in the real-time operation data of the target engineering machinery.

[0016] In an embodiment of the present application, when the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data is supplemented based on the target sample operation data with the highest matching degree, including:

[0017] When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the target sample operation data with the highest matching degree is determined;

[0018] determining a first similarity between the abnormal data segment and the target sample operation data;

[0019] When the first similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

[0020] In an embodiment of the present application, the abnormal data processing method further includes:

[0021] When the first similarity is less than the similarity threshold, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model;

[0022] determining a second similarity between the abnormal data segment and the predicted operational data;

[0023] When the second similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0024] In an embodiment of the present application, multiple sample operation data sets are matched with abnormal data segments for similarity, and target sample operation data with the highest matching degree is determined, including:

[0025] Determine a target sample running data set corresponding to the abnormal data segment from a plurality of sample running data sets;

[0026] The abnormal data segment is matched with each running data in the target sample running data set for similarity, and the target sample running data with the highest matching degree with the abnormal data segment is determined.

[0027] In an embodiment of the present application, when the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is supplemented with processing based on the operation data prediction model to obtain processed real-time operation data, including:

[0028] When the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model;

[0029] determining a third similarity between the abnormal data segment and the predicted operational data;

[0030] When the third similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0031] In an embodiment of the present application, the abnormal data processing method further includes:

[0032] When the third similarity is less than the similarity threshold, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the target sample operation data with the highest matching degree is determined;

[0033] determining a fourth similarity between the abnormal data segment and the target sample operation data;

[0034] When the fourth similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

[0035] In an embodiment of the present application, the abnormal data processing method further includes:

[0036] When both the first similarity and the second similarity are less than the similarity threshold and the first similarity is greater than the second similarity, or when both the third similarity and the fourth similarity are less than the similarity threshold and the fourth similarity is greater than the third similarity, supplementary processing is performed on the abnormal data segment based on the target sample operation data to obtain processed real-time operation data;

[0037] When both the first similarity and the second similarity are less than the similarity threshold and the first similarity is less than or equal to the second similarity, or when both the third similarity and the fourth similarity are less than the similarity threshold and the fourth similarity is less than or equal to the third similarity, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0038] In an embodiment of the present application, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model, including:

[0039] Obtain the normal data segment corresponding to the abnormal data segment;

[0040] Update the operation data prediction model based on the normal data segment to obtain an updated operation data prediction model;

[0041] The abnormal data segment is input into the updated operation data prediction model to obtain the predicted operation data output by the updated operation data prediction model.

[0042] In an embodiment of the present application, time clustering is performed on historical operation data of at least one engineering machine to obtain multiple sample operation data sets, including:

[0043] Dividing historical operation data of at least one construction machine according to a preset period to obtain a plurality of pieces of operation data of the preset period;

[0044] Based on the dynamic time warping algorithm and the preset time clustering algorithm, time clustering is performed on multiple preset period operation data to obtain multiple sample operation data sets.

[0045] In an embodiment of the present application, a preset model is trained based on the historical operation data of the target engineering machinery to obtain an operation data prediction model, including:

[0046] Build a preset model based on long short-term memory network;

[0047] Obtaining operation data of a target duration in historical operation data of a target engineering machinery, wherein the target duration is a duration corresponding to a start time of the historical operation data to a preset time;

[0048] The preset model is trained based on the operation data of the target duration to obtain the operation data prediction model.

[0049] In an embodiment of the present application, the abnormal data segment includes a continuous first data segment and a second data segment, the first data segment is located before the second data segment, and the first data segment is normal data, and the second data segment is abnormal data.

[0050] A second aspect of the present application provides a computing device, including:

[0051] a memory configured to store instructions;

[0052] The processor is configured to call instructions from the memory and implement the above-mentioned abnormal data processing method when executing the instructions.

[0053] A third aspect of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned abnormal data processing method.

[0054] The present application provides an abnormal data processing method, including: performing time clustering on the historical operation data of at least one engineering machinery to obtain multiple sample operation data sets; training a preset model based on the historical operation data of the target engineering machinery to obtain an operation data prediction model; determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery; in the case that there is an abnormal data segment in the real-time operation data, determining the duration corresponding to the abnormal data segment; in the case that the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, performing similarity matching on multiple sample operation data sets and the abnormal data segment, supplementing the abnormal data based on the target sample operation data with the highest matching degree, and obtaining the processed real-time operation data; in the case that the duration corresponding to the abnormal data segment is less than the preset duration, supplementing the abnormal data segment based on the operation data prediction model, and obtaining the processed real-time operation data. By analyzing the duration of the abnormal data segment, and providing two different abnormal data supplementation processing methods based on the historical operation data and the prediction model according to the duration, it is possible to perform abnormal processing according to the actual situation of the abnormal data and obtain higher quality operation data. Through high-quality operation data, it is possible to make better improvements to the structural design of engineering machinery and improve the operation efficiency of engineering machinery. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0056] Figure 1 A schematic diagram of a process flow of an abnormal data processing method according to an embodiment of the present application is schematically shown;

[0057] Figure 2 An example diagram of historical operation data according to an embodiment of the present application is schematically shown;

[0058] Figure 3 An example diagram of a plurality of sample operation data sets according to an embodiment of the present application is schematically shown;

[0059] Figure 4An example diagram of real-time operation data according to an embodiment of the present application is schematically shown;

[0060] Figure 5 An example diagram of normal data according to an embodiment of the present application is schematically shown;

[0061] Figure 6 An example diagram of abnormal data according to an embodiment of the present application is schematically shown;

[0062] Figure 7 An example diagram of processed real-time operating data according to an embodiment of the present application is schematically shown.

[0063] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0064] It should be noted that if the embodiments of the present application involve directional indications (such as front and back), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0065] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0066] Figure 1 The following schematically shows a flow chart of an abnormal data processing method according to an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a method for processing abnormal data, and the method for processing abnormal data includes:

[0067] S110, performing time clustering on historical operation data of at least one construction machine to obtain multiple sample operation data sets.

[0068] The historical operation data of at least one engineering machine of the same model and configuration is obtained from the Internet of Things platform, wherein the number of engineering machines is set according to actual needs and is not limited here. The type of engineering machine is set according to actual needs and can be a tower crane, etc., which is not limited here. For ease of understanding, the engineering machine in the embodiment of the present application is a tower crane. In the case where there is abnormal data in the collected data of the tower crane, the abnormal data can be matched and supplemented through the historical operation data.

[0069] In fact, the data collection scenario will obtain a large amount of historical operation data of construction machinery. When the original historical operation data is used directly, due to the large amount of data, it is easy to have a long matching time and reduced matching accuracy. In this embodiment, the historical operation data of at least one construction machinery is time clustered to obtain multiple sample operation data sets, and each sample operation data set includes similar operation data.

[0070] In an embodiment of the present application, time clustering is performed on historical operation data of at least one engineering machine to obtain multiple sample operation data sets, including:

[0071] Dividing historical operation data of at least one construction machine according to a preset period to obtain a plurality of pieces of operation data of the preset period;

[0072] Based on the dynamic time warping algorithm and the preset time clustering algorithm, time clustering is performed on multiple preset period operation data to obtain multiple sample operation data sets.

[0073] See also Figure 2 , Figure 2 An example diagram of historical operation data according to an embodiment of the present application is schematically shown.

[0074] Since the amount of historical operation data is too large, directly using the original historical operation data is prone to long matching time and reduced matching accuracy. In this embodiment, the historical operation data of at least one construction machinery is divided according to a preset period to obtain multiple preset period operation data. It should be understood that the value of the preset period is set according to actual needs, which can be 24 hours or any lifting period, and is not limited here.

[0075] See also Figure 3 , Figure 3 An example diagram of historical operation data according to an embodiment of the present application is schematically shown.

[0076] Based on the dynamic time warping algorithm as the data similarity judgment criterion, a preset time clustering algorithm is used to perform time clustering on multiple preset period operation data to obtain multiple categories of sample operation data sets. As shown in the figure, the sample operation data set includes multiple high-similarity sample operation data. Matching historical operation data through the sample operation data set can reduce the matching time and provide matching accuracy. For ease of understanding, the figure shows 13 different categories of sample operation data sets.

[0077] S120, training a preset model based on historical operation data of the target construction machinery to obtain an operation data prediction model.

[0078] The target engineering machinery is an engineering machinery for which it is necessary to determine whether there is abnormal data in the real-time operation data. The preset model is trained based on the historical operation data of the target engineering machinery. Specifically, the historical operation data is used as the input of the preset model, and the predicted operation data is used as the output of the preset model. The preset model is trained to obtain the operation data prediction model.

[0079] In an embodiment of the present application, a preset model is trained based on the historical operation data of the target engineering machinery to obtain an operation data prediction model, including:

[0080] Build a preset model based on long short-term memory network;

[0081] Obtaining operation data of a target duration in historical operation data of a target engineering machinery, wherein the target duration is a duration corresponding to a start time of the historical operation data to a preset time;

[0082] The preset model is trained based on the operation data of the target duration to obtain the operation data prediction model.

[0083] Based on the Long Short-Term Memory (LSTM) network, a preset model is constructed. At the beginning of operation of construction machinery such as tower cranes, normal data is usually collected, and the operation data of the target duration in the historical operation data of the target construction machinery is obtained, wherein the target duration is the duration corresponding to the start time of the historical operation data to the preset time. The value of the preset time is set according to actual needs and is not limited here. For ease of understanding, in the embodiment of the present application, the operation data of the target duration is the operation data of the first 100 nodes, and the preset time is the time corresponding to the operation data of the 100th node. The operation data of the target duration is used as the training sample data for the normal operation of the tower crane, and the preset model is trained based on the training sample data to obtain an operation data prediction model.

[0084] S130, determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery.

[0085] Obtain the real-time operation data of the target engineering machinery in the operating state, and determine whether there are abnormal data segments in the real-time operation data of the target engineering machinery. Tower cranes are usually in harsh working environments and complex working conditions, which leads to a large number of continuous abnormal data segments in the collected operation data. It should be understood that the abnormal data segment can be data missing or abnormal mutation values, which will not be elaborated here.

[0086] In an embodiment of the present application, determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery includes:

[0087] Acquire multiple pieces of normal operation data corresponding to the target construction machinery;

[0088] Linearly combine multiple pieces of normal operation data to obtain a linear expression between the sampling value of the operation data and time;

[0089] Based on the linear expression, determine whether there is an abnormal data segment in the real-time operation data of the target engineering machinery.

[0090] See also Figure 4 , Figure 4 An example diagram of real-time operation data according to an embodiment of the present application is schematically shown.

[0091] As shown in the figure, the area marked in red is the area where abnormal data exists. Normal data in the operating data usually conforms to autoregression, while abnormal data does not conform to autoregression. Obtain multiple pieces of non-abnormal operating data corresponding to the target engineering machinery, and use the non-abnormal operating data as test data, perform n-order linear combination on the test data, and then use the least squares method to quickly determine the coefficient matrix, and then the linear expression between the sampling value of the operating data and time is:

[0092]

[0093] Among them, t is the tth time, y t is the sampling value of the running data at time t, a n is the nth coefficient of the matrix.

[0094] See also Figure 5 , Figure 5 An example diagram of normal data according to an embodiment of the present application is schematically shown.

[0095] Based on the linear expression, determine whether there is an abnormal data segment in the real-time operation data of the target engineering machinery. Specifically, substitute the current time into the linear expression to obtain the predicted collection value corresponding to the current time predicted by the linear expression, and compare the predicted collection value with the actual collection value corresponding to the current time in the actual operation data.

[0096] See also Figure 6 , Figure 6 An example diagram of abnormal data according to an embodiment of the present application is schematically shown.

[0097] When the difference between the predicted collected value and the actual collected value is large, the actual collected value is determined to be abnormal data, and the abnormal data in the real-time operation data is extracted to obtain a continuous abnormal data segment. By determining the abnormal data through data autoregression, abnormal data detection can be performed without the need for a large amount of historical operation data, and the accuracy of abnormal data detection can be improved.

[0098] S140, when there is an abnormal data segment in the real-time operation data, determining a duration corresponding to the abnormal data segment;

[0099] When there is an abnormal data segment in the real-time operation data, the duration corresponding to the abnormal data segment is determined; the abnormal data segment includes multiple continuous abnormal data. In this embodiment, it is necessary to determine whether the engineering machinery has abnormal data for a long time or a short time, and then select different abnormal data processing methods according to the duration corresponding to the abnormal data segment.

[0100] S150, when the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data is supplemented based on the target sample operation data with the highest matching degree to obtain processed real-time operation data.

[0101] When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data is supplemented based on the target sample operation data with the highest matching degree. When a large amount of historical operation data is collected, since the duration corresponding to the abnormal data segment is long, similar historical operation data with the highest matching degree with the abnormal data segment can be matched from the historical operation data, and then the abnormal data segment is supplemented with the high-similarity historical operation data to obtain the processed real-time operation data.

[0102] In an embodiment of the present application, the abnormal data segment includes a continuous first data segment and a second data segment, the first data segment is located before the second data segment, and the first data segment is normal data, and the second data segment is abnormal data.

[0103] In this embodiment, the abnormal data segment includes a first data segment corresponding to the normal data, and a second data segment corresponding to the abnormal data. It should be understood that the duration corresponding to the first data segment is set according to actual needs and is not limited here. For ease of understanding, in the embodiment of the present application, the first data segment is the normal data of the 10 loading cycles before the abnormal data. By analyzing the continuous first data segment and the second data segment, the accuracy of exception handling can be improved.

[0104] S160, when the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is supplemented with processing based on the operation data prediction model to obtain processed real-time operation data.

[0105] When the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is supplemented based on the operation data prediction model. The predicted operation data is output based on the operation data prediction model, and the abnormal data is compensated by the predicted operation data, so that higher quality real-time operation data can be obtained.

[0106] By analyzing the duration of the abnormal data segment and providing two different abnormal data supplementation processing methods based on historical operation data and prediction model, it is possible to handle the abnormalities according to the actual situation of the abnormal data and obtain higher quality operation data. With high quality operation data, it is possible to make better improvements to the structural design of engineering machinery and improve the operation efficiency of engineering machinery.

[0107] In an embodiment of the present application, when the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data is supplemented based on the target sample operation data with the highest matching degree, including:

[0108] When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the target sample operation data with the highest matching degree is determined;

[0109] determining a first similarity between the abnormal data segment and the target sample operation data;

[0110] When the first similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

[0111] The value of the preset duration is set according to actual needs, and can be 24 hours, etc., which is not limited here. For ease of understanding, the preset duration in the embodiment of the present application is 2 hoisting cycles of the tower crane. When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, that is, the duration corresponding to the abnormal data segment is greater than or equal to 2 hoisting cycles, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the target sample operation data with the highest matching degree is determined.

[0112] Determine a first similarity between the abnormal data segment and the target sample operation data. When the first similarity is greater than or equal to a similarity threshold, since the target sample operation data is data with a high similarity to the abnormal data segment, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

[0113] In an embodiment of the present application, the abnormal data processing method further includes:

[0114] When the first similarity is less than the similarity threshold, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model;

[0115] determining a second similarity between the abnormal data segment and the predicted operational data;

[0116] When the second similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0117] When the first similarity is less than the similarity threshold, it is determined that data anomaly processing is performed based on historical operation data. In the case where the data accuracy is poor due to the small amount of historical operation data, the model output can be used to predict the operation data and perform supplementary processing. Specifically, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model.

[0118] A second similarity between the abnormal data segment and the predicted operation data is determined. When the second similarity is greater than or equal to the similarity threshold, since the predicted operation data is data with a high similarity to the abnormal data segment, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0119] In an embodiment of the present application, multiple sample operation data sets are matched with abnormal data segments for similarity, and target sample operation data with the highest matching degree is determined, including:

[0120] Determine a target sample running data set corresponding to the abnormal data segment from a plurality of sample running data sets;

[0121] The abnormal data segment is matched with each running data in the target sample running data set for similarity, and the target sample running data with the highest matching degree with the abnormal data segment is determined.

[0122] Based on the time clustering algorithm, a target sample operation data set corresponding to the abnormal data segment is determined from multiple sample operation data sets. The abnormal data segment is matched with each operation data in the target sample operation data set for similarity, and the target sample operation data with the highest matching degree with the abnormal data segment is determined. For ease of understanding, in the embodiments of the present application, the dynamic time warping (Dynamic Time Warping, DTW) distance is used as the data similarity judgment criterion. Specifically, the DTW distance between the abnormal data segment and each operation data in the target sample operation data set is determined, and the target sample operation data is the sample operation data corresponding to the minimum DTW distance.

[0123] In an embodiment of the present application, when the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is supplemented with processing based on the operation data prediction model to obtain processed real-time operation data, including:

[0124] When the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model;

[0125] determining a third similarity between the abnormal data segment and the predicted operational data;

[0126] When the third similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0127] When the duration corresponding to the abnormal data segment is less than the preset duration, that is, the duration corresponding to the abnormal data segment is less than 2 loading cycles, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model.

[0128] See also Figure 7 , Figure 7 An example diagram of processed real-time operating data according to an embodiment of the present application is schematically shown.

[0129] Determine the third similarity between the abnormal data segment and the predicted operation data. When the third similarity is greater than or equal to the similarity threshold, since the predicted operation data is data with high similarity to the abnormal data segment, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data. As shown in the figure, the blue curve is the real-time operation data before processing, and the red curve is the real-time operation data after processing. Perform abnormal processing according to the actual situation of the abnormal data to obtain higher quality operation data.

[0130] In an embodiment of the present application, the abnormal data processing method further includes:

[0131] When the third similarity is less than the similarity threshold, multiple sample operation data sets are matched with the abnormal data segment for similarity, and the target sample operation data with the highest matching degree is determined;

[0132] determining a fourth similarity between the abnormal data segment and the target sample operation data;

[0133] When the fourth similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

[0134] When the third similarity is less than the similarity threshold, the operation data prediction model is determined to perform data anomaly processing. If the data accuracy is poor due to insufficient training data, historical operation data can be used for supplementary processing. Specifically, multiple sample operation data sets are matched with the abnormal data segments for similarity, and the target sample operation data with the highest matching degree is determined.

[0135] Determine a fourth similarity between the abnormal data segment and the target sample operation data. When the fourth similarity is greater than or equal to the similarity threshold, since the target sample operation data is data with a high similarity to the abnormal data segment, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

[0136] In an embodiment of the present application, the abnormal data processing method further includes:

[0137] When both the first similarity and the second similarity are less than the similarity threshold and the first similarity is greater than the second similarity, or when both the third similarity and the fourth similarity are less than the similarity threshold and the fourth similarity is greater than the third similarity, supplementary processing is performed on the abnormal data segment based on the target sample operation data to obtain processed real-time operation data;

[0138] When both the first similarity and the second similarity are less than the similarity threshold and the first similarity is less than or equal to the second similarity, or when both the third similarity and the fourth similarity are less than the similarity threshold and the fourth similarity is less than or equal to the third similarity, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0139] It should be understood that the first similarity and the fourth similarity are both similarities between the abnormal data segment and the target sample running data, which is P1=1 / (1+d1 / n), wherein P1 is the first similarity and / or the fourth similarity, d1 is the DTW distance between the abnormal data segment and the sample running data, and n is the number of collection points of the abnormal data segment.

[0140] The second similarity and the third similarity are both similarities between the abnormal data segment and the predicted running data, which is P2=1 / (1+d2 / n), where P2 is the second similarity and / or the third similarity, d2 is the DTW distance between the abnormal data segment and the sample running data, and n is the number of collection points of the abnormal data segment.

[0141] When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, the first similarity between the abnormal data segment and the target sample operation data is calculated first, and then the second similarity between the abnormal data segment and the predicted operation data is calculated. When the duration corresponding to the abnormal data segment is less than the preset duration, the third similarity between the abnormal data segment and the predicted operation data is calculated, and then the fourth similarity between the abnormal data segment and the target sample operation data is calculated.

[0142] In actual abnormal data processing scenarios, there are situations where the similarity between the abnormal data segment and the predicted operating data, and the similarity between the abnormal data segment and the target sample operating data are both less than the similarity threshold. It is necessary to determine the abnormal data supplementary processing method based on the size relationship between the similarities. Specifically, when the first similarity and the second similarity are both less than the similarity threshold, and the first similarity is greater than the second similarity, or when the third similarity and the fourth similarity are both less than the similarity threshold, and the fourth similarity is greater than the third similarity, the abnormal data segment is supplemented based on the target sample operating data to obtain processed real-time operating data;

[0143] When both the first similarity and the second similarity are less than the similarity threshold and the first similarity is less than or equal to the second similarity, or when both the third similarity and the fourth similarity are less than the similarity threshold and the fourth similarity is less than or equal to the third similarity, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

[0144] In an embodiment of the present application, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model, including:

[0145] Obtain the normal data segment corresponding to the abnormal data segment;

[0146] Update the operation data prediction model based on the normal data segment to obtain an updated operation data prediction model;

[0147] The abnormal data segment is input into the updated operation data prediction model to obtain the predicted operation data output by the updated operation data prediction model.

[0148] The normal data segment corresponding to the abnormal data segment is obtained. In this embodiment, the normal data segment of 1 to 2 hoisting cycles before the abnormal data segment is obtained. The normal data segment is constructed as a temporary training sample, and the operation data prediction model is updated based on the normal data segment to obtain an updated operation data prediction model to improve the prediction accuracy of the operation data prediction model. The abnormal data segment is input into the updated operation data prediction model to obtain the predicted operation data output by the updated operation data prediction model.

[0149] The present application provides an abnormal data processing method, including: performing time clustering on the historical operation data of at least one engineering machinery to obtain multiple sample operation data sets; training a preset model based on the historical operation data of the target engineering machinery to obtain an operation data prediction model; determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery; in the case that there is an abnormal data segment in the real-time operation data, determining the duration corresponding to the abnormal data segment; in the case that the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, performing similarity matching on multiple sample operation data sets and the abnormal data segment, supplementing the abnormal data based on the target sample operation data with the highest matching degree, and obtaining the processed real-time operation data; in the case that the duration corresponding to the abnormal data segment is less than the preset duration, supplementing the abnormal data segment based on the operation data prediction model, and obtaining the processed real-time operation data. By analyzing the duration of the abnormal data segment, and providing two different abnormal data supplementation processing methods based on the historical operation data and the prediction model according to the duration, it is possible to perform abnormal processing according to the actual situation of the abnormal data and obtain higher quality operation data. Through high-quality operation data, it is possible to make better improvements to the structural design of engineering machinery and improve the operation efficiency of engineering machinery.

[0150] The present application also provides a computing device, including:

[0151] a memory configured to store instructions;

[0152] The processor is configured to call instructions from the memory and implement the above-mentioned abnormal data processing method when executing the instructions.

[0153] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned abnormal data processing method.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0155] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0159] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0160] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0161] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0162] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for processing abnormal data, characterized in that: The abnormal data processing method comprises: Performing time clustering on historical operation data of at least one construction machine to obtain multiple sample operation data sets; Training a preset model based on historical operation data of the target construction machinery to obtain an operation data prediction model; Determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery; In the case where there is an abnormal data segment in the real-time operation data, determining a duration corresponding to the abnormal data segment; When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, the plurality of sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data are supplemented based on the target sample operation data with the highest matching degree to obtain processed real-time operation data; When the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is supplementarily processed based on the operation data prediction model to obtain processed real-time operation data.

2. The abnormal data processing method according to claim 1, characterized in that: The determining whether there is an abnormal data segment in the real-time operation data of the target engineering machinery includes: Acquire a plurality of pieces of normal operation data corresponding to the target engineering machinery; Linearly combining the plurality of pieces of normal operation data to obtain a linear expression between the sampling value of the operation data and time; Based on the linear expression, it is determined whether there is an abnormal data segment in the real-time operation data of the target engineering machinery.

3. The abnormal data processing method according to claim 1, characterized in that: When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, the plurality of sample operation data sets are matched with the abnormal data segment for similarity, and the abnormal data is supplemented based on the target sample operation data with the highest matching degree, including: When the duration corresponding to the abnormal data segment is greater than or equal to the preset duration, the plurality of sample operation data sets are matched with the abnormal data segment for similarity, and the target sample operation data with the highest matching degree is determined; determining a first similarity between the abnormal data segment and the target sample operation data; When the first similarity is greater than or equal to a similarity threshold, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

4. The abnormal data processing method according to claim 3, characterized in that: The abnormal data processing method further includes: When the first similarity is less than the similarity threshold, inputting the abnormal data segment into the operation data prediction model to obtain predicted operation data output by the operation data prediction model; determining a second similarity between the abnormal data segment and the predicted operation data; When the second similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

5. The abnormal data processing method according to claim 3, characterized in that: The performing similarity matching between the plurality of sample operation data sets and the abnormal data segment to determine the target sample operation data with the highest matching degree includes: Determine a target sample operation data set corresponding to the abnormal data segment from the multiple sample operation data sets; The abnormal data segment is matched with each operation data in the target sample operation data set for similarity, and the target sample operation data with the highest matching degree with the abnormal data segment is determined.

6. The abnormal data processing method according to claim 1, characterized in that: When the duration corresponding to the abnormal data segment is less than the preset duration, supplementary processing is performed on the abnormal data segment based on the operation data prediction model to obtain processed real-time operation data, including: When the duration corresponding to the abnormal data segment is less than the preset duration, the abnormal data segment is input into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model; determining a third similarity between the abnormal data segment and the predicted operation data; When the third similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the predicted operation data to obtain processed real-time operation data.

7. The abnormal data processing method according to claim 6, characterized in that: The abnormal data processing method further includes: When the third similarity is less than the similarity threshold, performing similarity matching between the plurality of sample operation data sets and the abnormal data segment to determine the target sample operation data with the highest matching degree; determining a fourth similarity between the abnormal data segment and the target sample operation data; When the fourth similarity is greater than or equal to the similarity threshold, the abnormal data segment is supplemented based on the target sample operation data to obtain processed real-time operation data.

8. The abnormal data processing method according to claim 4 or 6, characterized in that: The abnormal data processing method further includes: When both the first similarity and the second similarity are less than the similarity threshold and the first similarity is greater than the second similarity, or when both the third similarity and the fourth similarity are less than the similarity threshold and the fourth similarity is greater than the third similarity, supplementary processing is performed on the abnormal data segment based on the target sample operation data to obtain processed real-time operation data; When the first similarity and the second similarity are both smaller than the similarity threshold, and the first similarity is smaller than or equal to the second similarity, or when the third similarity and the fourth similarity are both smaller than the similarity threshold, and the fourth similarity is smaller than or equal to the third similarity, the abnormal data segment is supplementarily processed based on the predicted operation data to obtain processed real-time operation data.

9. The abnormal data processing method according to claim 6, characterized in that: The step of inputting the abnormal data segment into the operation data prediction model to obtain the predicted operation data output by the operation data prediction model includes: Acquire a normal data segment corresponding to the abnormal data segment; updating the operation data prediction model based on the normal data segment to obtain an updated operation data prediction model; The abnormal data segment is input into the updated operation data prediction model to obtain the predicted operation data output by the updated operation data prediction model.

10. The abnormal data processing method according to claim 1, characterized in that: The historical operation data of at least one engineering machine is subjected to time clustering to obtain multiple sample operation data sets, including: Dividing the historical operation data of the at least one engineering machine according to a preset period to obtain a plurality of pieces of operation data of the preset period; Based on a dynamic time warping algorithm and a preset time clustering algorithm, the plurality of preset period operation data are time clustered to obtain a plurality of sample operation data sets.

11. The abnormal data processing method according to claim 1, characterized in that: The method of training a preset model based on the historical operation data of the target engineering machinery to obtain an operation data prediction model includes: Build a preset model based on long short-term memory network; Acquire operation data of a target duration in historical operation data of a target engineering machinery, wherein the target duration is a duration corresponding to a start time of the historical operation data to a preset time; The preset model is trained based on the operation data of the target duration to obtain an operation data prediction model.

12. The abnormal data processing method according to claim 1, characterized in that: The abnormal data segment includes a continuous first data segment and a second data segment, the first data segment is located before the second data segment, and the first data segment is normal data, and the second data segment is abnormal data.

13. A computing device, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instruction from the memory and implement the abnormal data processing method according to any one of claims 1 to 12 when executing the instruction.

14. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions, which are used to enable a machine to execute the abnormal data processing method according to any one of claims 1 to 12.

Images