Equipment operation data processing method and computer equipment

Abstract

The invention relates to an equipment operation data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: obtaining original equipment operation data obtained by target equipment in an operation process; based on a data correlation relationship between the original equipment operation data, performing first-layer redundancy elimination on the original equipment operation data to obtain intermediate equipment operation data; performing data reconstruction according to the data features of the intermediate equipment operation data to obtain reconstructed equipment operation data; and according to the data difference between the operation data of the reconstruction equipment and the operation data of the intermediate equipment, performing second-layer redundancy elimination on the operation data of the intermediate equipment to obtain operation data of the target equipment so as to ensure that redundant data in the operation data of the original equipment is effectively eliminated.

Description

Technical Field

[0001] The present application relates to the technical field of industrial data processing, and in particular to a method and apparatus for processing equipment operation data, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the increasing automation of industrial equipment, real-time monitoring and data collection of industrial equipment status have become core components of industrial production. The collection and analysis of industrial data not only monitors and maintains equipment status but also provides critical data support for optimizing production processes and improving equipment efficiency. Industrial automation and equipment monitoring require the collection of equipment operating data through various data collection methods, such as the use of various sensing devices. The accuracy and real-time nature of the operating data collected by these sensing devices are crucial for optimizing production efficiency and ensuring equipment safety.

[0003] However, there is often redundant data in the data collected by various sensing devices for industrial equipment, and traditional data de-redundancy processing methods cannot effectively remove redundant data. Summary of the Invention

[0004] Based on this, it is necessary to provide a device operation data processing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can effectively remove redundant data in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for processing device operation data, the method comprising:

[0006] Obtaining the original device operation data obtained during the operation of the target device;

[0007] Based on the data correlation relationship between the original device operation data, performing first-layer redundancy removal on the original device operation data to obtain intermediate device operation data;

[0008] Reconstructing data according to the data characteristics of the intermediate device operation data to obtain reconstructed device operation data;

[0009] According to the data difference between the reconstructed device operation data and the intermediate device operation data, the intermediate device operation data is subjected to second-layer de-redundancy to obtain target device operation data, where the target device operation data is used to characterize the operation status of the target device.

[0010] In one embodiment, performing first-layer redundancy removal on the original device operation data based on the data correlation relationship between the original device operation data to obtain the intermediate device operation data includes:

[0011] constructing each data pair based on each of the original device operation data;

[0012] For each of the data pairs, performing data correlation analysis on the original device operation data included in the data pair to obtain a correlation analysis result of the data pair;

[0013] Based on the correlation analysis results of each of the data pairs, redundancy removal is performed on the original device operation data included in each of the data pairs to obtain intermediate device operation data.

[0014] In one embodiment, performing data correlation analysis on the original device operation data included in the targeted data pair to obtain the correlation analysis result of the targeted data pair includes:

[0015] Determine a first statistical parameter corresponding to each of the original device operation data included in the data pair;

[0016] determining a linear relationship parameter based on the first statistical parameter and the original device operation data included in the targeted data pair;

[0017] According to the linear relationship parameter, the correlation analysis result of the data pair is obtained.

[0018] In one embodiment, performing data correlation analysis on the original device operation data included in the targeted data pair to obtain the correlation analysis result of the targeted data pair includes:

[0019] Determining probability distribution parameters corresponding to the original equipment operation data included in the data pair;

[0020] determining a nonlinear relationship parameter according to the probability distribution parameter;

[0021] The correlation analysis result of the targeted data pair is obtained according to the nonlinear relationship parameter.

[0022] In one embodiment, based on the correlation analysis results of the respective data pairs, de-redundancy is performed on the original device operation data included in each data pair to obtain the intermediate device operation data, including:

[0023] determining a target data pair from each of the data pairs based on a correlation threshold and a correlation analysis result of each of the data pairs;

[0024] determining, based on a second statistical parameter of the original device operating data included in the target data pair and a correlation analysis result corresponding to the target data pair, a comprehensive weight of each of the original device operating data included in the target data pair;

[0025] determining the cumulative weight of each of the original device operation data included in the target data pair according to the respective comprehensive weights and respective frequencies of the original device operation data included in the target data pair;

[0026] Based on the cumulative weight, de-redundancy is performed on the original device operation data included in the target data pair to obtain retained device operation data;

[0027] The intermediate device operation data is obtained according to the original device operation data included in the retained device operation data and other data pairs; the other data pairs include data pairs in each of the data pairs except the target data pair.

[0028] In one embodiment, the second statistical parameter includes a variance, the correlation analysis result includes a linear relationship parameter, and the linear relationship parameter is determined by the original device operation data included in the data pair and the respective first statistical parameters;

[0029] Determining the comprehensive weight of each of the original device operating data included in the target data pair based on the second statistical parameter of the original device operating data included in the target data pair and the correlation analysis result corresponding to the target data pair includes:

[0030] Determining first linear weights of the original device operation data included in each target data pair according to the variances corresponding to the original device operation data included in the target data pair;

[0031] Determining, according to the linear relationship parameters corresponding to the target data pairs, respective second linear weights of the original device operation data included in each of the target data pairs;

[0032] Based on the first linear weight and the second linear weight, the linear comprehensive weight of each of the original device operation data included in the target data pair is obtained, and the comprehensive weight is obtained according to the linear comprehensive weight.

[0033] In one embodiment, the second statistical parameter includes a variance, the correlation analysis result includes a nonlinear relationship parameter, and the nonlinear relationship parameter is determined by a probability distribution parameter corresponding to each of the original device operation data included in the data pair;

[0034] Determining the comprehensive weight of each of the original device operating data included in the target data pair based on the second statistical parameter of the original device operating data included in the target data pair and the correlation analysis result corresponding to the target data pair includes:

[0035] Determining first nonlinear weights for the original device operating data included in each target data pair according to the variances corresponding to the original device operating data included in the target data pair;

[0036] Determining, according to the nonlinear relationship parameters corresponding to the target data pairs, respective second nonlinear weights of the original device operation data included in each of the target data pairs;

[0037] Based on the first nonlinear weight and the second nonlinear weight, the nonlinear comprehensive weight of each of the original equipment operation data included in the target data pair is obtained, and the comprehensive weight is obtained according to the nonlinear comprehensive weight.

[0038] In one embodiment, based on the correlation analysis results of the respective data pairs, de-redundancy is performed on the original device operation data included in each data pair to obtain the intermediate device operation data, including:

[0039] Determining a first target data pair from each of the data pairs based on a first correlation threshold and a correlation analysis result of each of the data pairs;

[0040] De-redundancy is performed on the original device operation data included in the first target data pair to obtain intermediate de-redundancy data;

[0041] constructing respective intermediate data pairs based on the respective intermediate de-redundant data;

[0042] For each of the intermediate data pairs, performing data correlation analysis on the intermediate de-redundant data included in the intermediate data pair to obtain a correlation analysis result of the intermediate data pair;

[0043] Based on the correlation analysis results of the intermediate data pairs, de-redundancy is performed on the intermediate de-redundancy data included in each of the intermediate data pairs to obtain intermediate device operation data.

[0044] In one embodiment, reconstructing data based on the data characteristics of the intermediate device operation data to obtain reconstructed device operation data includes:

[0045] Performing feature encoding on the intermediate device operation data through a pre-trained data reconstruction model to obtain encoding features of the intermediate device operation data;

[0046] The data reconstruction model is used to decode and reconstruct the intermediate device operation data based on the coding features to obtain reconstructed device operation data.

[0047] In one embodiment, performing layer 2 redundancy removal on the intermediate device operating data based on the data difference between the reconstructed device operating data and the intermediate device operating data to obtain the target device operating data includes:

[0048] determining a data difference between the reconstructed device operating data and the intermediate device operating data;

[0049] When the data difference meets the data filtering condition, the intermediate device operation data is filtered to obtain the target device operation data.

[0050] In one embodiment, the method further comprises:

[0051] determining an abnormality discrimination parameter, a first abnormality discrimination interval, and a second abnormality discrimination interval of the original equipment operation data;

[0052] When the abnormality discrimination parameter is within the first abnormality discrimination interval, correcting the original equipment operation data corresponding to the abnormality discrimination parameter to obtain corrected original equipment operation data;

[0053] When the abnormality determination parameter is within the second abnormality determination interval, the raw device operation data corresponding to the abnormality determination parameter is filtered to obtain filtered raw device operation data.

[0054] In a second aspect, the present application further provides a device operation data processing apparatus, the apparatus comprising:

[0055] A data acquisition module is used to obtain the original device operation data obtained by the target device during operation;

[0056] a first redundancy removal module, configured to perform first-layer redundancy removal on the original device operation data based on a data correlation relationship between the original device operation data to obtain intermediate device operation data;

[0057] A data reconstruction module, configured to reconstruct the data according to the data characteristics of the intermediate device operation data to obtain reconstructed device operation data;

[0058] The second de-redundancy module is used to perform a second-layer de-redundancy on the intermediate device operating data according to the data difference between the reconstructed device operating data and the intermediate device operating data to obtain target device operating data, where the target device operating data is used to represent the operating status of the target device.

[0059] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-described method when executing the computer program.

[0060] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described above when the computer program is executed by a processor.

[0061] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which implements the steps of the method described above when executed by a processor.

[0062] The device operation data processing method, apparatus, computer device, computer-readable storage medium, and computer program product described above obtain original device operation data obtained during the operation of a target device; perform a first-level de-redundancy process on the original device operation data based on data correlations between the original device operation data to obtain intermediate device operation data; reconstruct the data based on data characteristics of the intermediate device operation data to obtain reconstructed device operation data; and perform a second-level de-redundancy process on the intermediate device operation data based on data differences between the reconstructed device operation data and the intermediate device operation data to obtain target device operation data. When processing the device operation data, two levels of de-redundancy process are performed on the original device operation data from a coarse to fine level based on the correlations between the original device operation data and the characteristics of the original device operation data. In the first-level de-redundancy process, correlations between the data are analyzed to reduce duplicate information in the original device operation data, thereby reducing data dimensionality and improving the efficiency of subsequent data processing. In the second-level de-redundancy process, data features within the data are extracted and reconstructed to effectively identify data in the intermediate device operation data that contributes significantly to the analysis of the target device's operating status, thereby further streamlining the data and ensuring the effective removal of redundant data in the original device operation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 A diagram of an application environment for a device to execute a data processing method in one embodiment;

[0065] Figure 2A schematic flow chart of a device operation data processing method in one embodiment;

[0066] Figure 3 A schematic flow chart of the steps for obtaining operation data of an intermediate device in one embodiment;

[0067] Figure 4 It is a structural block diagram of a device operation data processing apparatus in one embodiment;

[0068] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0070] The device operation data processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. After obtaining the device operation data obtained by the target device during operation, the server 104 analyzes the correlation between the various original device operation data, and can analyze the data correlation relationship between the various original device operation data, so as to perform the first layer of de-redundancy on the original device operation data according to the data correlation relationship, and obtain the intermediate device operation data after preliminary de-redundancy; then, the server 104 reconstructs each intermediate device operation data according to the data characteristics of each intermediate device operation data, and obtains the reconstructed device operation data corresponding to each intermediate device operation data, and performs the second layer of de-redundancy on the intermediate device operation data according to the data difference between the reconstructed device operation data and the corresponding intermediate device operation data, and can obtain the target operation data after final de-redundancy.

[0071] Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, and projectors. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Head-mounted devices may include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. Server 104 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0072] In an exemplary embodiment, Figure 2 As shown, a method for processing device operation data is provided, which is applied to Figure 1 The server 104 in FIG is used as an example to illustrate, and it is understood that the method can also be applied to Figure 1 The terminal 102 in the embodiment may also be applied to a system including the terminal 102 and the server 104, and implemented through interaction between the terminal 102 and the server 104. The method of this embodiment includes:

[0073] Step 201: Acquire original device operation data obtained during the operation of the target device.

[0074] The target device refers to a device that needs to process data; the target device can be any type of mechanical equipment, electronic equipment, system or other type of industrial equipment, such as a motor, engine, computer server, etc.

[0075] Raw device operating data refers to data generated by the target device during operation. This data may include, but is not limited to, operating data collected by data collection devices (such as sensors), recorded operation logs, and performance indicators. Operational data includes pressure, temperature, voltage, and current, and operation logs include startup and shutdown data. In specific implementations, raw device operating data can be a single data point, a sequence, or a collection of multiple data points.

[0076] The original device operation data usually contains a large amount of redundant data or noise. When using the original device operation data to analyze the operating status of the target device, it is usually necessary to perform a de-redundancy operation on the original device operation data. On the one hand, the de-redundant data can better reflect the operating characteristics of the target device, so as to improve the accuracy of the analysis of indicators such as the operating status of the target device. On the other hand, the de-redundant data contains less data, which can not only improve the efficiency of the analysis of indicators such as the operating status of the operating device, but also reduce data storage costs.

[0077] In this embodiment, the target devices primarily include industrial equipment in the fields of industrial automation and the Industrial Internet of Things. By acquiring the raw device operating data obtained during the operation of the industrial equipment, the operating status of the industrial equipment can be analyzed based on the raw device operating data, thereby understanding the operating status of the industrial equipment.

[0078] Exemplarily, during the operation of the target device, when the collection device collects original device operation data of the target device, the server obtains the original device operation data.

[0079] Step 202 : Based on the data correlation relationship between the original device operation data, perform first-layer redundancy removal on the original device operation data to obtain intermediate device operation data.

[0080] The data correlation relationship refers to the association relationship between the original device operation data obtained by performing data correlation analysis on at least two original device operation data.

[0081] In an exemplary embodiment, when performing data correlation analysis, two pieces of original device operating data are taken as an example. The two pieces of original device operating data may be original device operating data collected at different times using the same data acquisition method, for example, temperature data collected from a target device at time t and time t+1 using a temperature sensor. The two pieces of original device operating data may also be original device operating data collected at the same or different times using different data acquisition methods, for example, using different acquisition devices, where the difference here includes the same type but different specific devices or different types. In addition, the two pieces of original device operating data for data correlation analysis generally have the same data meaning. For example, both pieces of original device operating data are temperature, which may be the internal temperature of the industrial device, the external temperature, or the temperature at any location. Of course, it is understandable that in some other embodiments, the two pieces of original device operating data may also have different data meanings, for example, velocity and acceleration. Although the two pieces of data are different, they are generally related and the method of this embodiment may also be used to perform data correlation analysis.

[0082] Among them, the first-level de-redundancy refers to the process of removing duplicate or highly correlated data in the original equipment operation data based on the data correlation relationship between the original equipment operation data. The first-level de-redundancy can reduce the amount of data, facilitate the improvement of the processing efficiency of subsequent data, and retain as much useful information as possible.

[0083] Among them, the intermediate device operation data refers to the data set obtained after the first layer of de-redundancy processing. In this data set, some redundancy in the original device operation data has been preliminarily removed, but sufficient information is still retained to reflect the operating status of the device.

[0084] Exemplarily, after obtaining the original device operation data of the target device, the server performs a data correlation analysis on each original device operation data to obtain a data correlation relationship between each original device operation data. Subsequently, the server performs a first-layer de-redundancy on the original device operation data based on the obtained data correlation relationship to remove redundant data in the original device operation data, and obtains the intermediate device operation data based on the original device operation data after removing the redundant data.

[0085] Step 203: reconstruct the data according to the data characteristics of the intermediate device operation data to obtain reconstructed device operation data.

[0086] Data features refer to the characteristics or attributes of intermediate device operating data. They reflect the latent representation of this data, which may contain valuable information for analyzing the operating status of the target device. Data features may include, but are not limited to, data distribution, changing trends, and periodicity.

[0087] Among them, data reconstruction refers to reconstructing data using algorithms or models based on the data characteristics of the intermediate device operation data to reproduce the intermediate device operation data. The data reconstruction process includes but is not limited to at least one of the operations such as data aggregation, dimensionality reduction, and transformation.

[0088] The reconstructed device operation data is the data obtained after data reconstruction. Since the reconstructed device operation data is reconstructed based on the data features of the intermediate device operation data, if the data features are key features of the intermediate device operation data, it will contain more information that is more valuable or has a higher contribution to the status analysis of the target device, and the intermediate device operation data can be reproduced more easily after reconstruction, that is, the difference between the reconstructed device operation data and the intermediate device operation data is smaller; conversely, if the data features are non-key features, it will contain less information that is less valuable or has a lower contribution to the status analysis of the target device, and the probability of reconstructing the intermediate device operation data is lower, that is, the difference between the reconstructed device operation data and the intermediate device operation data is larger. In this way, redundant data in the intermediate device operation data can be further determined to further reduce the amount of data.

[0089] Exemplarily, the server extracts data features of the intermediate device operation and reconstructs data according to the data features of the intermediate device operation data to obtain reconstructed device operation data corresponding to the intermediate device operation data.

[0090] Step 204 : Based on the data difference between the reconstructed device operating data and the intermediate device operating data, perform second-layer redundancy removal on the intermediate device operating data to obtain the target device operating data.

[0091] Among them, data difference refers to the difference between the reconstructed device operation data and the intermediate device operation data. If the data difference is larger, it means that the data feature when reconstructing the data is a non-critical feature, and it contains less information that can reflect the characteristics of the intermediate device operation data. Therefore, the intermediate device operation data cannot be accurately reproduced. Therefore, the contribution of this data to the subsequent analysis of the operation status of the target device is low, and it can be determined as redundant data or irrelevant data and can be removed; on the contrary, if the data difference is smaller, it means that the data feature when reconstructing the data is a critical feature, and it contains more information that can reflect the characteristics of the intermediate device operation data. Therefore, the intermediate device operation data can be reproduced more accurately. Therefore, the contribution of this data to the subsequent analysis of the operation status of the target device is high, and it can be determined as not redundant data or relevant data and can be retained.

[0092] Among them, the second-level de-redundancy refers to the process of further removing data that is irrelevant or redundant to the analysis of the target device based on the data differences between the reconstructed device operation data and the intermediate device operation data, so as to further reduce the amount of data and ensure that the target device operation data finally obtained is both concise and accurate.

[0093] The target device operation data is a data set obtained after two layers of redundancy removal, and the target device operation data can be used for the operation status of the target device.

[0094] Exemplarily, the server determines the data difference between the reconstructed device operation data and the intermediate device operation data, and performs a second-layer de-redundancy on the intermediate device operation data based on the data difference. If the data difference is small, the intermediate device operation data is retained; otherwise, the intermediate device operation data is removed, thereby obtaining the final target device operation data.

[0095] In the above-mentioned device operation data processing method, raw device operation data obtained during the operation of the target device is obtained; based on the data correlation relationship between each raw device operation data, a first-level de-redundancy is performed on the raw device operation data to obtain intermediate device operation data; data is reconstructed based on the data characteristics of the intermediate device operation data to obtain reconstructed device operation data; and based on the data differences between the reconstructed device operation data and the intermediate device operation data, a second-level de-redundancy is performed on the intermediate device operation data to obtain the target device operation data. When processing the device operation data, two levels of de-redundancy are performed on the raw device operation data from coarse to fine levels based on the correlation relationship between the raw device operation data and the characteristics of the raw device operation data itself. In the first-level de-redundancy process, by analyzing the correlation between the data, it is possible to reduce duplicate information in the raw device operation data, which is conducive to reducing data dimensionality and improving the efficiency of subsequent data processing. In the second-level de-redundancy process, by extracting data characteristics within the data and reconstructing the data, it is possible to effectively identify data in the intermediate device operation data that has a high contribution to the analysis of the target device's operating status, thereby further streamlining the data and ensuring the effective removal of redundant data in the raw device operation data.

[0096] In one embodiment, the step of performing first-layer redundancy removal to obtain intermediate device operation data includes:

[0097] Each data pair is constructed based on the original device operation data; for each data pair, the original device operation data included in the data pair is subjected to data correlation analysis to obtain the correlation analysis result of the data pair; based on the respective correlation analysis results of each data pair, the original device operation data included in each data pair is de-redundant to obtain intermediate device operation data.

[0098] A data pair refers to a data combination formed by pairing two or more pieces of original device operation data. In this embodiment, data pairs are constructed using two pieces of original device operation data as an example to facilitate analysis of the correlation between the two pieces of original device operation data in each data pair.

[0099] Among them, data correlation analysis refers to the process of analyzing the correlation or association between the two original device operating data in the data pair. In this embodiment, the correlation between the two original device operating data may include but is not limited to linear correlation, nonlinear correlation, etc. Correspondingly, data correlation analysis may include but is not limited to linear correlation analysis, nonlinear correlation analysis, etc. In specific implementation, data correlation analysis can be implemented through statistical methods or machine learning algorithms. The analysis methods include but are not limited to one or more methods such as Pearson correlation coefficient, Spearman rank correlation coefficient, mutual information, etc., to quantify the data correlation relationship between the two original device data, thereby providing data support for the subsequent de-redundancy process.

[0100] The correlation analysis result refers to the result obtained after performing data correlation analysis on the two original device operation data in each data pair. The correlation analysis result can be expressed as a numerical value (such as a correlation coefficient) to indicate the degree of correlation between the two original device operation data.

[0101] Exemplarily, the server combines the original device operation data in pairs to construct multiple data pairs. For each data pair, the server performs data correlation analysis on the original device operation data included in each targeted data pair to obtain the correlation analysis results of each targeted data pair. Subsequently, the server performs de-redundancy on the original device operation data included in each data pair based on the respective correlation analysis results of each data pair to preliminarily remove the redundant data contained in the original device operation data, obtain the original device operation data after removing the redundant data, and then obtain the intermediate device operation data.

[0102] In this embodiment, by constructing data pairs, correlation analysis is performed on the original device operation data in pairs to obtain correlation analysis results, and the original device operation data is de-redundant based on the correlation analysis results. According to the correlation between the original device data, redundant information in the original device operation data can be effectively identified, thereby obtaining streamlined intermediate device operation data, which is conducive to optimizing data quality and providing a more accurate and efficient data basis for subsequent data analysis or equipment monitoring.

[0103] In one embodiment, the data correlation analysis includes linear correlation analysis, and the process of performing data correlation analysis on the data pair includes:

[0104] Determine the first statistical parameters corresponding to the original device operation data included in the data pair; determine the linear relationship parameters based on the first statistical parameters and the original device operation data included in the data pair; and obtain the correlation analysis results of the data pair based on the linear relationship parameters.

[0105] The targeted data pair refers to the data pair for which the linear correlation analysis is performed, and the original equipment operation data included in the targeted data pair refers to the original equipment operation data for which the linear correlation analysis is performed.

[0106] Among them, the first statistical parameter refers to a parameter that describes the data distribution or data characteristics of the original equipment operation data. The first statistical parameter may include but is not limited to parameters such as mean, standard deviation, variance, maximum value, minimum value, etc., to reflect the overall distribution and degree of dispersion of the original equipment operation data.

[0107] Linear relationship parameters are parameters that describe the linear relationship between two pieces of raw device operating data. These parameters include, but are not limited to, slope, intercept, and correlation coefficient, and are used to quantify the linear relationship between the two pieces of raw device operating data, thereby reflecting the existence and strength of a linear relationship between the two pieces of raw device operating data.

[0108] In an exemplary embodiment, the first statistical parameter is determined based on the mean, and the linear correlation analysis mainly refers to the use of the Pearson correlation coefficient to perform linear analysis on the original equipment operation data included in the data pair. Correspondingly, the linear relationship parameter is the Pearson correlation coefficient, and the correlation analysis result refers to the linear correlation analysis result of the data pair determined based on the Pearson correlation coefficient.

[0109] Exemplarily, first, the server determines the corresponding means according to the original device operation data included in the targeted data pair; then, the server determines the Pearson correlation coefficient between the original device operation data included in each targeted data pair according to the corresponding means and the original device operation data included in the targeted data pair; finally, the server obtains the corresponding linear correlation analysis result for each targeted data pair according to the Pearson correlation coefficient between the original device operation data included in each targeted data pair.

[0110] The Pearson correlation coefficient between the original equipment operation data included in the data pair is expressed as:

[0111] (1)

[0112] in, For the data pair The Pearson correlation coefficient of the original equipment operating data included in the data pair is The correlation coefficient between To run data for the original equipment separately The mean of is the index of the data pair. For the The raw device operating data included in the targeted data pair.

[0113] It is understandable that in some other embodiments, when determining the linear relationship parameters of the data pair, the Spearman rank correlation coefficient and the Kendall rank correlation coefficient may also be used.

[0114] In this embodiment, by determining the first statistical parameter of the original device operation data and further calculating the linear relationship parameter based on the first statistical parameter, the linear correlation between the original device operation data included in the data pair can be accurately reflected, thereby obtaining a scientific and objective linear correlation analysis result.

[0115] In one embodiment, the data correlation analysis includes nonlinear correlation analysis, and the process of performing data correlation analysis on the data pair includes:

[0116] Determine the probability distribution parameters corresponding to the original equipment operation data included in the targeted data pair; determine the nonlinear relationship parameters based on the probability distribution parameters; and obtain the correlation analysis results of the targeted data pair based on the nonlinear relationship parameters.

[0117] The targeted data pair refers to the data pair for which nonlinear correlation analysis is performed, and the original equipment operation data included in the targeted data pair refers to the original equipment operation data for which nonlinear correlation analysis is performed.

[0118] Among them, the probability distribution parameters refer to parameters that describe the probability distribution possibility of the original equipment operation data. The probability distribution parameters may include but are not limited to at least one of normal distribution, joint probability distribution, marginal probability distribution, binomial distribution, Poisson distribution, etc., to reflect the statistical characteristics of the original equipment operation data.

[0119] Nonlinear relationship parameters are parameters that describe the nonlinear relationship between two pieces of raw device operating data. Nonlinear parameters include, but are not limited to, mutual information, polynomial functions, exponential functions, logarithmic functions, and distance correlation coefficients. These parameters quantify the nonlinear relationship between the two pieces of raw device operating data, thereby reflecting the presence and strength of a nonlinear relationship between the two pieces of raw device operating data.

[0120] In an exemplary embodiment, the probability distribution parameters are determined based on the joint probability distribution and the marginal probability distribution. The nonlinear correlation analysis mainly refers to the use of mutual information to perform nonlinear analysis on the original equipment operation data included in the targeted data pair. Correspondingly, the nonlinear relationship parameters are the coefficients after the mutual information calculation. The correlation analysis results refer to the nonlinear correlation analysis results of the targeted data pair determined based on the mutual information analysis.

[0121] Exemplarily, first, the server determines the joint probability distribution parameters and marginal probability distribution parameters of the original device operation data included in the targeted data pair based on the original device operation data included in the targeted data pair; then, the server determines the mutual information between the original device operation data included in each targeted data pair based on the joint probability distribution parameters and the marginal probability distribution parameters; finally, the server obtains the nonlinear correlation analysis results corresponding to each targeted data pair based on the mutual information between the original device operation data included in each targeted data pair.

[0122] The mutual information between the original device operation data included in the data pair is expressed as:

[0123] (2)

[0124] in, For the data pair The mutual information of the original equipment operation data included in the data pair In this embodiment, An array or collection of multiple raw device operation data, including multiple corresponding raw device operation data ,when For independent data points, That is ; Run data for raw equipment The joint probability distribution parameters of Original equipment operation data The corresponding marginal probability distribution parameters.

[0125] It is understandable that in some other embodiments, when determining the nonlinear relationship parameters of the data pair, polynomial functions, exponential functions, logarithmic functions, distance correlation coefficients, etc. can also be used.

[0126] In this embodiment, by determining the probability distribution parameters of the original device operation data and further determining the nonlinear relationship parameters based on the probability distribution parameters, it is possible to accurately reflect the nonlinear correlation between the original device operation data included in the targeted data pair, thereby obtaining a scientific and objective nonlinear correlation analysis result.

[0127] In one embodiment, Figure 3 As shown, based on the correlation analysis results of each data pair, the original device operation data included in each data pair is de-redundant to obtain intermediate device operation data, including:

[0128] Step 301 : determining a target data pair from each data pair based on a correlation threshold and the correlation analysis results of each data pair.

[0129] Among them, the correlation threshold refers to a numerical standard that is pre-set and used to determine whether there is a data correlation relationship between various data pairs, and is used to screen data pairs with correlation as target data pairs. The target data pair refers to the data pair with linear correlation or nonlinear correlation screened out from various data pairs. In specific implementation, the correlation threshold can be set to different thresholds according to different correlation analysis results. For example, for linear correlation analysis results, the correlation threshold can be set to 0.9 or other values. When the linear correlation parameter exceeds 0.9, it means that the operating data of the two original devices are highly linearly correlated, and there may be redundancy. Otherwise, it means that the linear correlation of the operating data of the two original devices is low.

[0130] Exemplarily, the server obtains a pre-set correlation threshold and the correlation analysis results of each data pair, and compares the correlation analysis results of each data pair with the correlation threshold to determine the target data pair that meets the linear correlation condition or the nonlinear correlation condition from each data pair.

[0131] Step 302 : Determine the comprehensive weight of each of the original device operation data included in the target data pair based on the second statistical parameter of the original device operation data included in the target data pair and the correlation analysis result corresponding to the target data pair.

[0132] The second statistical parameter refers to a parameter that describes the data distribution or data characteristics of the original device operation data included in the filtered target data pair; the second statistical parameter may include, but is not limited to, parameters such as mean, standard deviation, variance, maximum value, and minimum value to reflect the overall distribution and dispersion of the original device operation data included in the target data pair. In specific implementations, the second statistical parameter may be a statistical parameter of the original device operation data included in the target data pair in a data set consisting of all the original device operation data of the filtered target data pair, or a statistical parameter of the data set consisting of all the original device operation data of the filtered target data pair after deduplication.

[0133] The comprehensive weight refers to the weight coefficient of each raw device operating data item in each target data pair, determined based on the second statistical parameter of the raw device operating data in the target data pair and the results of the correlation analysis. It is used to reflect the relative importance of each raw device operating data item in each target data pair within all the raw device operating data items corresponding to the target data pair. A raw device operating data item with a greater comprehensive weight indicates that it is more important in the target data pair and may be considered for retention during subsequent redundancy removal. Conversely, a raw device operating data item with a smaller comprehensive weight indicates that it is less important in the target data pair and may be considered for removal during subsequent redundancy removal.

[0134] Exemplarily, the server determines the second statistical parameters of each original device operation data based on the original device operation data included in the target data pair, and determines the comprehensive weight of each original device operation data included in each target data pair based on the second statistical parameters and the corresponding correlation analysis results of the target data pair.

[0135] Step 303 : determining the cumulative weight of each of the original device operation data included in the target data pair according to the comprehensive weight and frequency of each of the original device operation data included in the target data pair.

[0136] Frequency refers to the number of times a piece of raw device operating data appears in the filtered target data pairs, that is, the number of target data pairs that include the raw device operating data. A higher frequency of a piece of raw device operating data indicates that there are more other raw device operating data related to it, which means that the raw device operating data has a greater influence. Conversely, a lower frequency of a piece of raw device operating data indicates that there are fewer other raw device operating data related to it, which means that the raw device operating data has a smaller influence.

[0137] The cumulative weight refers to the cumulative weight coefficient of each piece of raw device operating data, calculated based on the combined weight of each piece of raw device operating data in the target data pair and its frequency of occurrence in the target data pair. This coefficient comprehensively reflects the relative importance of each piece of raw device operating data in the target data pair. A greater cumulative weight for a piece of raw device operating data indicates a greater relevance to the analysis of the target device's operating status, and therefore, more important, and therefore suitable for retention during the de-redundancy process. Conversely, a smaller cumulative weight for a piece of raw device operating data indicates a weaker relevance to the analysis of the target device's operating status, and therefore, less important, and therefore suitable for removal during the de-redundancy process.

[0138] In specific implementation, the cumulative weight can be obtained by fusing the comprehensive weights of the operating data of each original device. The fusion methods include but are not limited to simple addition, weighted addition, etc. When the weighted addition method is adopted, the comprehensive weight of another original device operating data belonging to the same data pair as the original device operating data can be comprehensively determined.

[0139] Exemplarily, the server determines the frequency of each original device operation data based on the filtered target data pairs, and determines the cumulative weight of each original device operation data included in each target data pair based on the respective comprehensive weights of the original device operation data included in the target data pairs and the respective corresponding frequencies of each original device operation data in the target data pairs.

[0140] The cumulative weight of each original device operation data included in each target data pair is expressed as:

[0141] (3)

[0142] in, Run data for raw equipment The cumulative weight of Indicates that the original equipment operation data The target data of the sum of the comprehensive weights, That is, original equipment operating data The comprehensive weight of .

[0143] Step 304 : De-redundancy is performed on the original device operation data included in the target data pair based on the cumulative weight to obtain the retained device operation data.

[0144] De-redundancy refers to filtering raw device data based on the cumulative weight of each raw device data in each target data pair to remove redundant data, thereby achieving the goal of streamlining data and improving data quality. Specifically, raw device operation data can be filtered by setting a retention threshold. That is, when the cumulative weight of a piece of raw device operation data exceeds the retention threshold, the raw device operation data is retained. Conversely, when the cumulative weight of a piece of raw device operation data does not reach the retention threshold, it can be determined that the raw device operation data is non-redundant and can be removed.

[0145] The retained device operation data refers to the original device operation data remaining after all the original device operation data in the filtered target data pairs are de-redundanted.

[0146] Exemplarily, the server compares the cumulative weight with the retention threshold according to the cumulative weight and a preset retention threshold, so as to perform de-redundancy on the original device operation data included in the target data pair and obtain the retained device operation data.

[0147] Step 305: Obtain intermediate device operation data based on the original device operation data included in the pair of retained device operation data and other data.

[0148] The other data pairs refer to the data pairs other than the target data pair in all data pairs. The intermediate device operation data includes all the original device operation data included in the other data pairs and the original device operation data remaining after removing redundant data in the target data pair.

[0149] Exemplarily, the server obtains the intermediate device operation data based on all original device operation data included in other data pairs and the original device operation data remaining after redundant data is removed from the target data pair.

[0150] In this embodiment, the target data pair is determined through correlation analysis, and the comprehensive weight of the original device operation data is determined based on the second statistical parameter and the correlation analysis result, and then the cumulative weight of each original device operation data is determined according to the comprehensive weight and the frequency of occurrence of the original device operation data in the target data pair. The original device operation data included in the target data pair is de-redundanted by using the cumulative weight, which can effectively identify and remove redundant data in the original device operation data, so as to provide more accurate and critical data support for subsequent analysis.

[0151] In one embodiment, taking the second statistical parameter as the variance and the correlation analysis result as the linear relationship parameter as an example, determining the comprehensive weight of each of the original device operating data included in the target data pair based on the second statistical parameter of the original device operating data included in the target data pair and the correlation analysis result corresponding to the target data pair includes:

[0152] According to the variances corresponding to the original device operation data included in the target data pair, the first linear weights of the original device operation data included in each target data pair are determined respectively; according to the linear relationship parameters corresponding to the target data pair, the second linear weights of the original device operation data included in each target data pair are determined respectively; based on the first linear weight and the second linear weight, the linear comprehensive weights of the original device operation data included in the target data pair are obtained, and the comprehensive weight is obtained according to the linear comprehensive weight.

[0153] The variance refers to a parameter that describes the degree of dispersion of the original device operating data among all the original device operating data in the target data pair, so as to determine the importance of the original device operating data in the target data pair. If the variance of a certain original device operating data is larger, it is more likely to be retained in the process of de-redundancy; conversely, if the variance of a certain original device operating data is smaller, it is more likely to be removed in the process of de-redundancy. In specific implementation, when determining the variance of the original device operating data, it is possible to deduplicate all the original device operating data in the target data pair and then determine the variance for each original device operating data, or it is possible to directly determine the variance for each original device operating data based on all the original device operating data in the target data pair.

[0154] Among them, the first linear weight refers to the weight determined based on the variance corresponding to each original equipment operation data during the process of performing linear correlation analysis on the original equipment operation data, which is used to reflect the impact of the discrete degree of the original equipment operation data on the comprehensive weight. The larger the variance, the greater the first linear weight.

[0155] Among them, the second linear weight refers to the weight determined based on the corresponding linear relationship parameters of the target data during the linear correlation analysis of the original equipment operation data. It is used to reflect the impact of the strength of the linear relationship between the data on the comprehensive weight. The stronger the linear relationship, the greater the second linear weight.

[0156] The linear comprehensive weight refers to a comprehensive weight calculated based on the first linear weight and the second linear weight. The linear comprehensive weight comprehensively considers the degree of dispersion and the strength of the linear relationship of the raw equipment operating data, and can provide a more comprehensive evaluation of the raw equipment operating data. In one exemplary embodiment, the linear comprehensive weight is the final comprehensive weight. Of course, it is understandable that in other embodiments, when determining the final comprehensive weight based on the linear comprehensive weight, the linear comprehensive weight can also be processed using weighting, correction, or other methods to obtain the final comprehensive weight.

[0157] Exemplarily, first, the server determines the corresponding variance for each piece of original device operation data based on the data obtained after deduplication processing of the original device operation data included in the target data pair, and then determines the first linear weight of each piece of original device operation data included in each target data pair based on the corresponding variance. The first linear weight is expressed as:

[0158] (4)

[0159] in, Run data for raw equipment The first linear weight of Run data for raw equipment The variance of Indicates all original device operation data after deduplication of target data Sum the variances of , The number of deduplicated data runs across all raw devices for the target data pair.

[0160] Subsequently, the server determines the second linear weight of the original device operation data included in each target data pair according to the linear relationship parameter corresponding to the target data pair. The second linear weight is expressed as:

[0161] (5)

[0162] in, Run data for raw equipment and original equipment operating data The second linear weight of .

[0163] Finally, the server obtains the linear comprehensive weights of the original device operation data included in the target data pair based on the first linear weight and the second linear weight, and obtains the comprehensive weight based on the linear comprehensive weight. When determining the linear comprehensive weight, the corresponding coefficients can be determined based on the degree of influence of the first linear weight and the second linear weight relative to the linear comprehensive weight to obtain the linear comprehensive weight. The linear comprehensive weight is expressed as:

[0164] (6)

[0165] in, Run data for raw equipment The linear comprehensive weight of Run data for raw equipment The first linear weight of Run data for raw equipment The second linear weight of , is the coefficient of the proportion of the first linear weight and the second linear weight in the linear comprehensive weight, .

[0166] In this embodiment, by comprehensively considering the variance of the original equipment operation data and the corresponding linear relationship parameters of the target data, the linear comprehensive weight of each original equipment operation data can be determined more accurately, and then a more reasonable comprehensive weight can be obtained, which is conducive to improving the accuracy and reliability of data analysis and helping to more accurately determine redundant data.

[0167] In one embodiment, taking the case where the second statistical parameter includes the variance and the correlation analysis result as the bracketed nonlinear relationship parameter, the comprehensive weight of each of the original device operating data included in the target data pair is determined based on the second statistical parameter of the original device operating data included in the target data pair and the correlation analysis result corresponding to the target data pair, including:

[0168] According to the variances corresponding to the original equipment operation data included in the target data pair, the first nonlinear weights of the original equipment operation data included in each target data pair are determined respectively; according to the nonlinear relationship parameters corresponding to the target data pair, the second nonlinear weights of the original equipment operation data included in each target data pair are determined respectively; based on the first nonlinear weight and the second nonlinear weight, the nonlinear comprehensive weights of the original equipment operation data included in the target data pair are obtained, and the comprehensive weight is obtained according to the nonlinear comprehensive weight.

[0169] The variance refers to a parameter that describes the degree of dispersion of the original device operating data among all the original device operating data in the target data pair, so as to determine the importance of the original device operating data in the target data pair. If the variance of a certain original device operating data is larger, it is more likely to be retained in the process of de-redundancy; conversely, if the variance of a certain original device operating data is smaller, it is more likely to be removed in the process of de-redundancy. In specific implementation, when determining the variance of the original device operating data, it is possible to deduplicate all the original device operating data in the target data pair and then determine the variance for each original device operating data, or it is possible to directly determine the variance for each original device operating data based on all the original device operating data in the target data pair.

[0170] Among them, the first nonlinear weight refers to the weight determined based on the variance corresponding to each original equipment operation data during the process of performing nonlinear correlation analysis on the original equipment operation data, which is used to reflect the impact of the discrete degree of the original equipment operation data on the comprehensive weight. The larger the variance, the greater the first nonlinear weight.

[0171] Among them, the second nonlinear weight refers to the weight determined based on the corresponding nonlinear relationship parameters of the target data during the nonlinear correlation analysis of the original equipment operation data. It is used to reflect the impact of the strength of the nonlinear relationship between the data on the comprehensive weight. The stronger the nonlinear relationship, the greater the second nonlinear weight.

[0172] The nonlinear comprehensive weight refers to a comprehensive weight calculated based on the first nonlinear weight and the second nonlinear weight. The nonlinear comprehensive weight comprehensively considers the degree of discreteness and the strength of the nonlinear relationship of the original equipment operating data, and can provide a more comprehensive evaluation of the original equipment operating data. In an exemplary embodiment, the nonlinear comprehensive weight is the final comprehensive weight. Of course, it is understandable that in other embodiments, when determining the final comprehensive weight based on the nonlinear comprehensive weight, the nonlinear comprehensive weight can also be processed by weighting, correction, etc. to obtain the final comprehensive weight.

[0173] Exemplarily, first, the server determines the corresponding variance for each piece of original device operation data based on the data obtained after deduplication processing of the original device operation data included in the target data pair, and then determines the first nonlinear weight of each piece of original device operation data included in each target data pair based on the corresponding variance. The first nonlinear weight is expressed as:

[0174] (7)

[0175] in, Run data for raw equipment The first nonlinear weight, Run data for raw equipment The variance of Indicates all original device operation data after deduplication of target data Sum the variances of , The number of deduplicated data runs across all raw devices for the target data pair.

[0176] Subsequently, the server determines the second nonlinear weight of the original device operation data included in each target data pair according to the nonlinear relationship parameter corresponding to the target data pair. The second nonlinear weight is expressed as:

[0177] (8)

[0178] in, Run data for raw equipment and original equipment operating data The second nonlinear weight of .

[0179] Finally, the server obtains the nonlinear comprehensive weight of each of the original device operating data included in the target data pair based on the first nonlinear weight and the second nonlinear weight, and obtains the comprehensive weight based on the nonlinear comprehensive weight. When determining the nonlinear comprehensive weight, the corresponding coefficient can be determined based on the degree of influence of the first nonlinear weight and the second nonlinear weight relative to the nonlinear comprehensive weight to obtain the nonlinear comprehensive weight. The nonlinear comprehensive weight is expressed as:

[0180] (9)

[0181] in, Run data for raw equipment The nonlinear comprehensive weight of Run data for raw equipment The first nonlinear weight, Run data for raw equipment The second nonlinear weight of , is the coefficient of the proportion of the first nonlinear weight and the second nonlinear weight in the nonlinear comprehensive weight, .

[0182] In this embodiment, by comprehensively considering the variance of the original equipment operation data and the corresponding nonlinear relationship parameters of the target data, the nonlinear comprehensive weight of each original equipment operation data can be determined more accurately, and then a more reasonable comprehensive weight can be obtained, which is conducive to improving the accuracy and reliability of data analysis and helping to more accurately determine redundant data.

[0183] In an optional embodiment, based on the correlation analysis results of each data pair, redundancy removal is performed on the original device operation data included in each data pair to obtain intermediate device operation data, including:

[0184] Based on a first correlation threshold and the correlation analysis results of each data pair, a first target data pair is determined from each data pair; redundancy is performed on the original device operation data included in the first target data pair to obtain intermediate de-redundant data; based on each intermediate de-redundant data, each intermediate data pair is constructed; for each intermediate data pair, a data correlation analysis is performed on the intermediate de-redundant data included in the intermediate data pair to obtain a correlation analysis result of the intermediate data pair; based on the correlation analysis result of the intermediate data pair, redundancy is performed on the intermediate de-redundant data included in each intermediate data pair to obtain intermediate device operation data.

[0185] In an exemplary embodiment, the server may first perform a correlation analysis on the original device operation data and perform preliminary de-redundancy on the original device operation data. After obtaining the intermediate de-redundant data, the server may perform another correlation analysis based on the intermediate de-redundant data using the correlation analysis process corresponding to the above embodiment to further de-redundantize the intermediate de-redundant data, thereby improving the accuracy of de-redundancy of the original device operation data.

[0186] For example, the first correlation analysis may be a linear correlation analysis performed on the original device operation data to remove redundant data with high linear correlation in the original device operation data, thereby obtaining intermediate de-redundant data after the redundant data are removed; the second correlation analysis may be a nonlinear correlation analysis performed on the intermediate de-redundant data to remove redundant data with high nonlinear correlation in the intermediate de-redundant data, thereby obtaining intermediate device operation data after the redundant data are removed. The specific correlation analysis process can be found in the description of the above embodiment, and will not be described in detail in this embodiment.

[0187] For another example, the first correlation analysis can be a nonlinear correlation analysis performed on the original device operation data to obtain intermediate de-redundant data; the second correlation analysis can be a linear correlation analysis performed on the intermediate de-redundant data to obtain the final intermediate device operation data. For another example, both the first correlation analysis and the second correlation analysis can be linear correlation analysis or nonlinear correlation analysis.

[0188] It is understood that in other embodiments, more than two correlation analyses may be performed based on the actual de-redundancy accuracy required, so that a relatively accurate preliminary de-redundancy result can be obtained after multiple analyses and de-redundancy. In still other embodiments, each correlation analysis may be performed simultaneously, and the results of multiple correlation analyses may be fused to obtain the final intermediate device operation data. The fusion methods include but are not limited to splicing, de-duplication, weighting, and difference calculation.

[0189] In this embodiment, by setting a first correlation threshold to filter the first data pair and performing preliminary de-redundancy processing on the original device operation data in the filtered first data pair, data redundancy can be initially reduced; then, the intermediate de-redundant data obtained after de-redundancy is subjected to another correlation analysis, and further de-redundancy is performed based on the analysis results, thereby obtaining more refined intermediate device operation data, which is beneficial to reducing the cost of data storage and processing and ensuring the accuracy and validity of the data.

[0190] In one embodiment, data reconstruction is performed based on the data characteristics of the intermediate device operation data to obtain the reconstructed device operation data, including:

[0191] Through the pre-trained data reconstruction model, the intermediate device operation data is feature-encoded to obtain the encoding features of the intermediate device operation data; through the data reconstruction model, based on the encoding features, the intermediate device operation data is decoded and reconstructed to obtain the reconstructed device operation data.

[0192] A pre-trained data reconstruction model is one that has been trained with a large amount of data and is capable of encoding data features and decoding them to reconstruct the data. Specifically, the data reconstruction model can be implemented using algorithms such as autoencoders and machine learning.

[0193] Among them, feature coding refers to the process of extracting features from the intermediate device operation data to obtain the potential representation of the intermediate device operation data and obtaining the coding features of the intermediate device operation data. The coding features refer to the features that can more accurately reflect the operating status and key information of the target device.

[0194] Decoding refers to the process in which a data reconstruction model converts and processes coding features to establish a new data representation, so as to obtain new reconstructed device operation data having the characteristics of the original intermediate device operation data.

[0195] The reconstructed equipment operation data refers to the equipment operation data processed by the data reconstruction model.

[0196] Exemplarily, when the server calls the autoencoder for data reconstruction, it uses the pre-trained autoencoder to perform feature encoding on the intermediate device operation data layer by layer. During the encoding process, the dimension of the hidden layer is reduced layer by layer to obtain the encoding features of the intermediate device operation data; then, the server calls the autoencoder again, and decodes and reconstructs the intermediate device operation data based on the encoding features. During the decoding process, the dimension of the output layer is reduced layer by layer to obtain the reconstructed device operation data.

[0197] In this embodiment, by utilizing a pre-trained data reconstruction model, the intermediate device operation data can be efficiently feature encoded and decoded and reconstructed, thereby obtaining reconstructed device operation data that retains the key information of the intermediate device operation data. This is conducive to determining the contribution of the intermediate device operation data to the operation status analysis of the target device, and further screening the intermediate device operation data based on this, thereby further reducing the amount of data.

[0198] In one embodiment, based on the data difference between the reconstructed device operating data and the intermediate device operating data, performing second-layer redundancy removal on the intermediate device operating data to obtain the target device operating data includes:

[0199] Determine the data difference between the reconstructed device operation data and the intermediate device operation data; if the data difference meets the data filtering condition, filter the intermediate device operation data to obtain the target device operation data.

[0200] The data filtering condition refers to a condition set based on data variance to further filter the intermediate device operating data. Typically, the data filtering condition can be a set data filtering threshold. If the data variance is greater than or equal to the data filtering threshold, the intermediate device operating data is considered to contain critical information representing the target device's operating status and can be retained. Conversely, if the data variance is less than the data filtering threshold, the intermediate device operating data is considered to contain non-critical information representing the target device's operating status and can be removed.

[0201] For example, the data error can be determined based on the second-order norm. First, the server determines the data difference between the reconstructed device operation data and the intermediate device operation data corresponding to the reconstructed device operation data. The data difference is expressed as:

[0202] (10)

[0203] in, To reconstruct the data difference between the device operation data and the intermediate device operation data, Run data for the middleware, Reconstruct device operation data.

[0204] Subsequently, the server obtains a pre-set data filtering threshold and compares the data difference with the data filtering threshold. If the data difference is greater than or equal to the data filtering threshold, the intermediate device operation data is retained. If the data difference is less than the data filtering threshold, the intermediate device operation data is filtered to obtain the target device operation data.

[0205] In this embodiment, by comparing the reconstructed device operation data with the intermediate device operation data, the intermediate device operation data with large differences after reconstruction can be accurately identified and filtered out, thereby accurately extracting the target device operation data that has a high contribution to the operation status analysis of the target device, which is conducive to further improving the data de-redundancy effect.

[0206] In one embodiment, the device operation data processing method further includes:

[0207] Determine the abnormality discrimination parameter, the first abnormality discrimination interval and the second abnormality discrimination interval of the original equipment operation data; when the abnormality discrimination parameter is within the first abnormality discrimination interval, correct the original equipment operation data corresponding to the abnormality discrimination parameter to obtain the corrected original equipment operation data; when the abnormality discrimination parameter is within the second abnormality discrimination interval, filter the original equipment operation data corresponding to the abnormality discrimination parameter to obtain the filtered original equipment operation data.

[0208] The abnormality discrimination parameter refers to a parameter used to determine whether the raw device operating data contains abnormalities, thereby reflecting whether the target device's operating status deviates from normal conditions. In this embodiment, the abnormality discrimination parameter can be determined based on the standard score or Z-score (i.e., Z value) of the raw device operating parameter. This not only standardizes the raw device operating data and unifies its scale to eliminate differences between different dimensions, but also serves as a unified abnormality discrimination parameter to determine whether the raw device operating data is abnormal.

[0209] The first abnormality discrimination interval and the second abnormality discrimination interval refer to preset data intervals related to the abnormality discrimination parameter, and are used to determine whether the abnormality discrimination parameter is within an acceptable slight abnormality range or a severe abnormality range.

[0210] In an exemplary embodiment, the first abnormality discrimination interval is determined as the discrimination interval for minor abnormalities, and the second abnormality discrimination interval is determined as the discrimination interval for major abnormalities. When the abnormality discrimination parameter falls within the first abnormality discrimination interval, it is considered that the original equipment operation data corresponding to the abnormality discrimination parameter has a minor deviation, but its accuracy can still be restored through correction. When the abnormality discrimination parameter falls within the second abnormality discrimination interval, it is considered that the original equipment operation data corresponding to the abnormality discrimination parameter has a serious deviation and is not suitable for use. The original equipment operation data is then filtered. When the abnormality discrimination parameter falls within both the first abnormality discrimination interval and the second abnormality discrimination interval, the original equipment operation data corresponding to the abnormality discrimination parameter is considered normal data.

[0211] Correcting the raw equipment operating data corresponding to the abnormality discrimination parameter refers to processing the slightly abnormal raw equipment operating data to normalize it when the raw equipment operating data is slightly abnormal. In specific implementations, the slightly abnormal raw equipment operating data can be corrected using methods including, but not limited to, mean smoothing, regression analysis, and interpolation, so that the slightly abnormal raw equipment operating data is replaced with the corrected data to obtain the corrected raw equipment operating data.

[0212] Among them, filtering the original equipment operation data corresponding to the abnormality discrimination parameter means that when the original equipment operation data is seriously abnormal, the seriously abnormal original equipment operation data is eliminated from the acquired data set to obtain filtered original equipment operation data.

[0213] Exemplarily, the server determines an abnormality discrimination parameter of the original device operation data. The abnormality discrimination parameter is expressed as:

[0214] (11)

[0215] in, is the abnormality discrimination parameter, For raw device operation data, is the mean of all original equipment running data, expressed as , , The number of runs for the original device; is the standard deviation of the original equipment operation data, expressed as .

[0216] The server determines the first abnormality discrimination interval as [2,3]∪[-2,-3], the second abnormality discrimination interval as (-∞, -3)∪(3, +∞), and the interval (-2, 2) other than this is determined as the normal interval.

[0217] When the abnormality discrimination parameter falls into the first abnormality discrimination interval [2,3]∪[-2,-3], the original equipment operating parameter corresponding to the abnormality discrimination parameter is smoothed to obtain the corrected original equipment operating parameter. The corrected original equipment operating parameter is expressed as:

[0218] (12)

[0219] in, are the corrected original equipment operating parameters, represents the mean value of all original equipment operating parameters, There are slight abnormalities with the original equipment operating parameters.

[0220] When the abnormality discrimination parameter falls into the second abnormality discrimination interval (-∞, -3)∪(3, +∞), the original device operation data is filtered to obtain filtered original device operation data.

[0221] When the abnormality discrimination parameter falls within the interval (-2, 2), the original equipment operation data corresponding to the abnormality discrimination parameter is considered to be normal data, and no abnormality processing is performed.

[0222] In this embodiment, by setting abnormality discrimination parameters and the corresponding first and second abnormality discrimination intervals, abnormal situations in the data can be accurately identified, and corresponding processing can be performed according to the different abnormality levels of the original equipment operation data, which can avoid the interference of abnormal data on subsequent analysis or decision-making, and can both ensure the integrity of the data and improve the reliability and availability of the data.

[0223] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0224] Based on the same inventive concept, embodiments of the present application also provide a device operation data processing apparatus for implementing the device operation data processing method described above. The solution provided by this apparatus is similar to the solution described in the method described above. Therefore, the specific limitations in one or more embodiments of the device operation data processing apparatus provided below can be found in the limitations of the device operation data processing method described above and will not be further elaborated here.

[0225] In an exemplary embodiment, Figure 4 As shown, a device operation data processing apparatus is provided, comprising: a data acquisition module 401, a first de-redundancy module 402, a data reconstruction module 403 and a second de-redundancy module 404, wherein:

[0226] The data acquisition module 401 is used to acquire the original device operation data obtained during the operation of the target device;

[0227] A first de-redundancy module 402 is configured to perform first-layer de-redundancy on the original device operation data based on the data correlation relationship between the original device operation data to obtain intermediate device operation data;

[0228] The data reconstruction module 403 is used to reconstruct the data according to the data characteristics of the intermediate device operation data to obtain the reconstructed device operation data;

[0229] The second de-redundancy module 404 is configured to perform a second-layer de-redundancy on the intermediate device operating data according to the data difference between the reconstructed device operating data and the intermediate device operating data to obtain target device operating data, which is used to represent the operating status of the target device.

[0230] In an optional embodiment, the first de-redundancy module 402 is also used to construct each data pair based on each original device operation data; for each data pair, the original device operation data included in the data pair is subjected to data correlation analysis to obtain the correlation analysis result of the data pair; based on the respective correlation analysis results of each data pair, the original device operation data included in each data pair is de-redundant to obtain intermediate device operation data.

[0231] In an optional embodiment, the first de-redundancy module 402 is further used to determine the first statistical parameters corresponding to each of the original device operation data included in the data pair; determine the linear relationship parameters based on the first statistical parameters and the original device operation data included in the data pair; and obtain the correlation analysis results of the data pair based on the linear relationship parameters.

[0232] In an optional embodiment, the first de-redundancy module 402 is further used to determine the probability distribution parameters corresponding to the original device operation data included in the data pair; determine the nonlinear relationship parameters based on the probability distribution parameters; and obtain the correlation analysis results of the data pair based on the nonlinear relationship parameters.

[0233] In an optional embodiment, the first de-redundancy module 402 is further used to determine a target data pair from each data pair based on a correlation threshold and the correlation analysis results of each data pair; determine the comprehensive weight of each original device operation data included in the target data pair according to the second statistical parameter of the original device operation data included in the target data pair and the correlation analysis results corresponding to the target data pair; determine the cumulative weight of each original device operation data included in the target data pair according to the comprehensive weight and frequency of each original device operation data included in the target data pair; based on the cumulative weight, perform de-redundancy on the original device operation data included in the target data pair to obtain retained device operation data; obtain intermediate device operation data based on the retained device operation data and the original device operation data included in other data pairs; the other data pairs include data pairs in each data pair except the target data pair.

[0234] In an optional embodiment, the first de-redundancy module 402 is further used to determine the first linear weight of each original device operation data included in each target data pair based on the variance corresponding to each original device operation data included in the target data pair; determine the second linear weight of each original device operation data included in each target data pair based on the linear relationship parameters corresponding to the target data pair; obtain the linear comprehensive weight of each original device operation data included in the target data pair based on the first linear weight and the second linear weight, and obtain the comprehensive weight based on the linear comprehensive weight.

[0235] In an optional embodiment, the first de-redundancy module 402 is further used to determine the first nonlinear weight of the original device operation data included in each target data pair based on the variance corresponding to the original device operation data included in the target data pair; determine the second nonlinear weight of the original device operation data included in each target data pair based on the nonlinear relationship parameters corresponding to the target data pair; obtain the nonlinear comprehensive weight of the original device operation data included in the target data pair based on the first nonlinear weight and the second nonlinear weight, and obtain the comprehensive weight based on the nonlinear comprehensive weight.

[0236] In an optional embodiment, the first de-redundancy module 402 is further used to determine a first target data pair from each data pair based on a first correlation threshold and the correlation analysis results of each data pair; perform de-redundancy on the original device operation data included in the first target data pair to obtain intermediate de-redundant data; construct each intermediate data pair based on each intermediate de-redundant data; for each intermediate data pair, perform data correlation analysis on the intermediate de-redundant data included in the intermediate data pair to obtain a correlation analysis result of the intermediate data pair; and based on the correlation analysis result of the intermediate data pairs, perform de-redundancy on the intermediate de-redundant data included in each intermediate data pair to obtain intermediate device operation data.

[0237] In an optional embodiment, the data reconstruction module 403 is also used to perform feature encoding on the intermediate device operation data through a pre-trained data reconstruction model to obtain the encoding features of the intermediate device operation data; and to decode and reconstruct the intermediate device operation data based on the encoding features through the data reconstruction model to obtain reconstructed device operation data.

[0238] In an optional embodiment, the second de-redundancy module 404 is further configured to determine the data difference between the reconstructed device operating data and the intermediate device operating data; if the data difference meets the data filtering condition, the intermediate device operating data is filtered to obtain the target device operating data.

[0239] In an optional embodiment, the device operation data processing apparatus further includes:

[0240] The abnormal data processing module is used to determine the abnormality discrimination parameter, the first abnormality discrimination interval and the second abnormality discrimination interval of the original equipment operation data; when the abnormality discrimination parameter is within the first abnormality discrimination interval, the original equipment operation data corresponding to the abnormality discrimination parameter is corrected to obtain the corrected original equipment operation data; when the abnormality discrimination parameter is within the second abnormality discrimination interval, the original equipment operation data corresponding to the abnormality discrimination parameter is filtered to obtain the filtered original equipment operation data.

[0241] Each module in the aforementioned device operation data processing apparatus may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0242] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, memory and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store device operation data, data pairs, various statistical parameters, various relationship parameters, probability distribution parameters and various weight data. The I / O interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for processing device operation data is implemented.

[0243] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0244] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the device operation data processing method of the above embodiment when executing the computer program.

[0245] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the device operation data processing method of the above embodiment are implemented.

[0246] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the device operation data processing method of the above embodiment when executed by a processor.

[0247] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0248] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0249] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0250] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for processing equipment operation data, characterized in that: The method comprises: Obtaining the original device operation data obtained during the operation of the target device; Based on the data correlation relationship between the original device operation data, performing first-layer redundancy removal on the original device operation data to obtain intermediate device operation data; Reconstructing data according to the data characteristics of the intermediate device operation data to obtain reconstructed device operation data; According to the data difference between the reconstructed device operation data and the intermediate device operation data, the intermediate device operation data is subjected to second-layer de-redundancy to obtain target device operation data, where the target device operation data is used to characterize the operation status of the target device.

2. The method according to claim 1, characterized in that The step of performing first-layer redundancy removal on the original device operation data based on the data correlation relationship between the original device operation data to obtain the intermediate device operation data includes: constructing each data pair based on each of the original device operation data; For each of the data pairs, performing data correlation analysis on the original device operation data included in the data pair to obtain a correlation analysis result of the data pair; Based on the correlation analysis results of each of the data pairs, redundancy removal is performed on the original device operation data included in each of the data pairs to obtain intermediate device operation data.

3. The method according to claim 2, characterized in that The performing of data correlation analysis on the original device operation data included in the targeted data pair to obtain a correlation analysis result of the targeted data pair includes at least one of the following: Determining first statistical parameters corresponding to the respective original device operation data included in the targeted data pairs; determining linear relationship parameters based on the first statistical parameters and the original device operation data included in the targeted data pairs; Obtaining a correlation analysis result of the data pair according to the linear relationship parameter; Determine probability distribution parameters corresponding to the original equipment operation data included in the targeted data pair; determine nonlinear relationship parameters based on the probability distribution parameters; and obtain correlation analysis results of the targeted data pair based on the nonlinear relationship parameters.

4. The method according to claim 2, characterized in that The method of performing redundancy removal on the original device operation data included in each data pair based on the respective correlation analysis results of each data pair to obtain intermediate device operation data includes: determining a target data pair from each of the data pairs based on a correlation threshold and a correlation analysis result of each of the data pairs; determining, based on a second statistical parameter of the original device operating data included in the target data pair and a correlation analysis result corresponding to the target data pair, a comprehensive weight of each of the original device operating data included in the target data pair; determining the cumulative weight of each of the original device operation data included in the target data pair according to the respective comprehensive weights and respective frequencies of the original device operation data included in the target data pair; Based on the cumulative weight, de-redundancy is performed on the original device operation data included in the target data pair to obtain retained device operation data; The intermediate device operation data is obtained according to the original device operation data included in the retained device operation data and other data pairs; the other data pairs include data pairs in each of the data pairs except the target data pair.

5. The method according to claim 4, characterized in that The second statistical parameter includes a variance, and the correlation analysis result includes at least one of a linear relationship parameter and a nonlinear relationship parameter, wherein the linear relationship parameter is determined by the original device operation data included in the targeted data pair and the respective first statistical parameters, and the nonlinear relationship parameter is determined by the probability distribution parameter corresponding to the respective original device operation data included in the targeted data pair; Determining the comprehensive weight of each of the original device operating data included in the target data pair based on the second statistical parameter of the original device operating data included in the target data pair and the correlation analysis result corresponding to the target data pair includes at least one of the following: Determining first linear weights of the original device operation data included in each target data pair according to the variances corresponding to the original device operation data included in the target data pair; Determining, according to the linear relationship parameters corresponding to the target data pairs, respective second linear weights of the original device operation data included in each of the target data pairs; Based on the first linear weight and the second linear weight, obtaining a linear comprehensive weight of each of the original device operation data included in the target data pair, and obtaining a comprehensive weight according to the linear comprehensive weight; Determining first nonlinear weights for the original device operating data included in each target data pair according to the variances corresponding to the original device operating data included in the target data pair; Determining, according to the nonlinear relationship parameters corresponding to the target data pairs, respective second nonlinear weights of the original device operation data included in each of the target data pairs; Based on the first nonlinear weight and the second nonlinear weight, the nonlinear comprehensive weight of each of the original equipment operation data included in the target data pair is obtained, and the comprehensive weight is obtained according to the nonlinear comprehensive weight.

6. The method according to claim 2, characterized in that The method of performing redundancy removal on the original device operation data included in each data pair based on the respective correlation analysis results of each data pair to obtain intermediate device operation data includes: Determining a first target data pair from each of the data pairs based on a first correlation threshold and a correlation analysis result of each of the data pairs; De-redundancy is performed on the original device operation data included in the first target data pair to obtain intermediate de-redundancy data; constructing respective intermediate data pairs based on the respective intermediate de-redundant data; For each of the intermediate data pairs, performing data correlation analysis on the intermediate de-redundant data included in the intermediate data pair to obtain a correlation analysis result of the intermediate data pair; Based on the correlation analysis results of the intermediate data pairs, de-redundancy is performed on the intermediate de-redundancy data included in each of the intermediate data pairs to obtain intermediate device operation data.

7. The method according to claim 1, characterized in that The step of reconstructing the data according to the data characteristics of the intermediate device operation data to obtain the reconstructed device operation data includes: Performing feature encoding on the intermediate device operation data through a pre-trained data reconstruction model to obtain encoding features of the intermediate device operation data; The data reconstruction model is used to decode and reconstruct the intermediate device operation data based on the coding features to obtain reconstructed device operation data.

8. The method according to claim 1, characterized in that The step of performing layer 2 redundancy removal on the intermediate device operating data according to a data difference between the reconstructed device operating data and the intermediate device operating data to obtain target device operating data includes: determining a data difference between the reconstructed device operating data and the intermediate device operating data; When the data difference meets the data filtering condition, the intermediate device operation data is filtered to obtain the target device operation data.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: determining an abnormality discrimination parameter, a first abnormality discrimination interval, and a second abnormality discrimination interval of the original equipment operation data; When the abnormality discrimination parameter is within the first abnormality discrimination interval, correcting the original equipment operation data corresponding to the abnormality discrimination parameter to obtain corrected original equipment operation data; When the abnormality determination parameter is within the second abnormality determination interval, the raw device operation data corresponding to the abnormality determination parameter is filtered to obtain filtered raw device operation data.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images