Industrial algorithm real-time test method and system based on data playback

Abstract

The invention belongs to the field of algorithm testing, and discloses an industrial algorithm real-time testing method and system based on data playback. The method comprises the following steps: collecting field data, and preprocessing the field data to obtain a simulation data set; using the simulation data set to operate a user algorithm, obtaining an operation result of the user algorithm in real time, and calculating an algorithm score of the user algorithm; keeping real-time operation of the algorithm, calculating scores, collecting field data again, preprocessing the field data, screening the preprocessed data based on clustering analysis and information entropy calculation to obtain an updated data set, and performing hot update on the simulated data set by using the updated data set; and judging the simulation data set after hot updating, if a stop condition is met, taking the newly calculated algorithm score as the final algorithm score, and if the condition is not met, performing hot updating again until the stop condition is met, and taking the newly calculated user algorithm score as the final algorithm score.

Description

Technical Field

[0001] The present invention belongs to the technical field of algorithm testing, and in particular relates to a real-time testing method and system for industrial algorithms based on data playback. Background Art

[0002] With the continuous development of industrial technology, the intelligence of algorithms in industrial fields such as intelligent manufacturing, chemical production, metallurgical industry, transportation and logistics, construction and infrastructure is becoming more and more intelligent. The testing and verification of various algorithms is becoming more and more important. In traditional benchmark tests, algorithm verification can only be performed offline, which makes it difficult to monitor and analyze data in real time and cannot adapt well to changes in industrial needs. Moreover, over time, equipment performance may degrade and the type of fault may change, resulting in errors in the test results of the algorithm. In order to meet this challenge, a real-time testing method and system for industrial algorithms is urgently needed to effectively and accurately verify the algorithm while running it in real time, and comprehensively evaluate the comprehensive performance of industrial algorithms in a real-time environment, so as to provide effective support for industrial intelligence. Summary of the invention

[0003] The technical problem to be solved by the present invention is that traditional industrial algorithm testing does not have real-time performance and how real-time testing of industrial algorithms can ensure the rationality of scoring and the authenticity and diversity of data sets. The present invention overcomes the deficiencies and defects mentioned in the above background technology and provides a real-time testing method and system for industrial algorithms based on data playback.

[0004] In order to solve the above technical problems, the present invention proposes a real-time testing method for industrial algorithms based on data playback, comprising the following steps:

[0005] S1: Collect the field data set, obtain the original data set, preprocess the original data set, and obtain the simulated data set;

[0006] S2: Use the simulated data set to run the algorithm to be tested uploaded by the user, obtain the running results of the algorithm to be tested uploaded by the user in real time, evaluate the algorithm to be tested uploaded by the user, and calculate the algorithm score of the algorithm to be tested uploaded by the user in real time;

[0007] S3: While S2 is continuously running in real time, field data of a specific time length is collected again to obtain a new original data set, the new original data set is preprocessed, and the preprocessed new original data set is screened based on cluster analysis and information entropy calculation to obtain an updated data set, and the updated data set is inserted into the simulation data set to perform hot update on the simulation data set;

[0008] S4: Judge the simulated data set after hot update. If the set stop condition is met, the algorithm score obtained by the last calculation is used as the final algorithm score. If the condition is not met, repeat S3 until the simulated data set meets the stop condition, and the user algorithm score obtained by the last calculation is used as the final algorithm score.

[0009] Preferably, the step of evaluating the algorithm to be tested uploaded by the user and calculating the algorithm score of the algorithm to be tested uploaded by the user in real time includes:

[0010] Calculate the scores and weights of the performance indicators, complexity indicators, and stability indicators of the algorithms uploaded by users;

[0011] Calculate the algorithm score of the algorithm to be tested uploaded by the user 现场录波回放 :

[0012] Score 现场录波回放 =w performance Score performance +w complexity Score complexity +w runtime Score runtime (1);

[0013] Among them, w performance 、w complexity 、w runtime are the weights of performance index, complexity index and stability index, Score performance 、Score complexity 、Score runtime They are the scores of performance index, complexity index and stability index respectively.

[0014] Preferably, the performance indicator includes at least one performance parameter;

[0015] Performance Index Score performance The calculation formula is as follows:

[0016]

[0017] Among them, p i represents the i-th performance parameter, w pi represents the weight of the i-th performance parameter, and n is the number of performance parameters.

[0018] Preferably, the complexity index includes:

[0019] Time complexity, space complexity;

[0020] Complexity index score Score complexity The calculation formula is as follows:

[0021] Score complexity =w space Score space +w time Score time (3);

[0022]

[0023] Among them, w space 、w time is the weight corresponding to the space complexity and time complexity, Score space 、Score time They are the space complexity score and time complexity score respectively, T represents the time complexity of the current algorithm, T max represents the maximum allowed time complexity, S represents the space complexity of the current algorithm, and S max Indicates the maximum allowed space complexity.

[0024] Preferably, the stability index includes:

[0025] Stable running time of the algorithm;

[0026] Stability Index Score runtime The calculation is as follows:

[0027]

[0028] Among them, β is the adjustment parameter of the fractional decay rate.

[0029] Preferably, the step S3 of screening the preprocessed new original data set based on cluster analysis and information entropy calculation to obtain an updated data set includes:

[0030] S31: Perform cluster analysis on the new original data set after preprocessing to obtain the center point of the cluster analysis:

[0031] Randomly select K initial cluster analysis center points for the new original data set after preprocessing;

[0032] Assign data points, assign each data point to the cluster formed by its nearest center point according to the distance;

[0033] By recalculating the mean of all data points in each cluster, the cluster analysis center point is updated. The mean of all data points in the cluster is calculated as shown in formula (6):

[0034]

[0035] Among them, μ i is the center point of the ith cluster, Ci are all the data points of the ith cluster, x j is a data point;

[0036] Repeat the process of assigning data points and updating cluster analysis center points until the cluster analysis center points no longer change or the specified number of iterations is reached;

[0037] S32: Calculate the information entropy and information gain of the new original data set after preprocessing. The information entropy is calculated as follows:

[0038]

[0039] Among them, H(D) is the information entropy of data set D, p i is the probability of category i;

[0040] The information gain is calculated as follows:

[0041] IG(D,x j )=H(D)-H(D\{x j}) (8);

[0042] Among them, IG(D,x j ) is the sample x j The information gain of the data set D, H(D\{x j}) is to remove the sample x j The conditional entropy of the posterior dataset, p i′ is the probability of category i';

[0043] According to the calculated information gain, find the sample points with larger information gain in different categories of data;

[0044] S33: Compare the center point of the cluster analysis and the sample points with large information gain obtained in S32 and S33 with the simulated data set, calculate the vector similarity between the samples, delete the samples with existing working conditions or high vector similarity, and update the data set with the remaining samples.

[0045] Based on a general inventive concept, the present invention also provides an industrial algorithm real-time testing system based on data playback, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention performs real-time verification and evaluation of the algorithm by means of on-site data playback, thereby simulating the operation of the algorithm model in an industrial environment, so that users can verify the algorithm in real time, collect real-time data at the same time, monitor equipment performance changes, and extract typical data with richness and representativeness based on cluster analysis and information gain, and perform hot updates on the simulated data set used in the algorithm verification process, so that the data set can reflect the latest on-site conditions and performance changes of the algorithm. In addition, evaluation indicators such as algorithm performance and complexity are constructed, and algorithms of different users are comprehensively evaluated from aspects such as accuracy, stability, and complexity, thereby achieving the effect of real-time monitoring and analysis of the algorithm, and ensuring the fairness and effectiveness of algorithm evaluation. The real-time testing method of the fault diagnosis algorithm based on data playback provided by the present invention can be widely used in industrial fields such as intelligent manufacturing, chemical production, metallurgical industry, transportation and logistics, construction and infrastructure, and tests and evaluates algorithms such as soft measurement, health monitoring, fault diagnosis, and process optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a flow chart of a real-time testing method for industrial algorithms based on data playback according to a preferred embodiment of the present invention;

[0050] Figure 2 It is a comparison diagram of the test results of two test methods: the traditional benchmark test method and the industrial algorithm real-time test method based on data playback;

[0051] Figure 3 It is a data chart of a simulated data set that has not been subjected to data hot update;

[0052] Figure 4 This is a data chart of the simulated data set after the data is hot updated. DETAILED DESCRIPTION

[0053] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively and meticulously below in conjunction with the accompanying drawings and preferred embodiments of the present invention, but the protection scope of the present invention is not limited to the following specific embodiments.

[0054] Unless otherwise defined, all professional terms used below have the same meanings as those generally understood by those skilled in the art. The professional terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention.

[0055] Unless otherwise specified, various raw materials, reagents, instruments and equipment used in the present invention can be purchased from the market or prepared by existing methods.

[0056] Example:

[0057] like Figure 1 As shown, this embodiment provides an industrial algorithm real-time testing method based on data playback, comprising the following steps:

[0058] S1: Collect field data to obtain the original data set, preprocess the original data set to obtain the simulated data set;

[0059] S2: Use the simulated data set to run the algorithm to be tested uploaded by the user, obtain the running results of the algorithm to be tested uploaded by the user in real time, evaluate the algorithm to be tested uploaded by the user, and calculate the algorithm score of the algorithm to be tested uploaded by the user in real time;

[0060] S3: While S2 is running continuously in real time, field data of a specific time length is collected again to obtain a new original data set, and the new original data set is preprocessed. The preprocessing method includes: timestamp alignment, data screening, and screening the preprocessed new original data set based on cluster analysis and information entropy calculation to obtain an updated data set, and inserting the updated data set into the simulation data set to perform hot update on the simulation data set;

[0061] S4: Judge the simulated data set after hot update. If the set stop condition is met, the algorithm score obtained by the last calculation is used as the final algorithm score. If the condition is not met, repeat S3 until the simulated data set meets the stop condition, and the user algorithm score obtained by the last calculation is used as the final algorithm score.

[0062] In this embodiment, the algorithm to be tested uploaded by the user is a fault detection algorithm for a central air-conditioning system; the field data include: cooling capacity of the chiller, host load rate, evaporator approach temperature, condenser approach temperature, chilled water outlet temperature, cooling water inlet temperature, condensing pressure, and the fault category corresponding to each data; the preprocessing method includes: timestamp alignment and data selection. By preprocessing the collected field data, the data is made to correspond in time, and invalid data is filtered out.

[0063] The above-mentioned industrial algorithm real-time testing method based on data playback verifies and evaluates the algorithm in real time by replaying the on-site data, thereby simulating the operation of the algorithm model in an industrial environment, allowing users to verify the algorithm in real time. At the same time, by hot updating the simulated data set, the simulated data set is updated without affecting the real-time operation of the algorithm, ensuring that the simulated data set can reflect the latest on-site conditions and performance changes of the algorithm, ensuring the diversity and authenticity of the simulated data set, and making the algorithm verification more reliable. Compared with the traditional benchmark test method, this method has the advantages of fair and accurate evaluation results, reliable verification process, high data authenticity, and good real-time and adaptability of algorithm verification.

[0064] As a preferred implementation of this embodiment, the step of evaluating the algorithm to be tested uploaded by the user and calculating the algorithm score of the algorithm to be tested uploaded by the user in real time includes:

[0065] Calculate the scores and weights of the performance indicators, complexity indicators, and stability indicators of the algorithms uploaded by users; calculate the algorithm score of the algorithms uploaded by users 现场录波回放 :

[0066] Score 现场录波回放 =w performance Score performance +w complexity Score complexity +w runtime Score runtime (1);

[0067] Among them, w performance 、w complexity 、w runtime are the weights of performance index, complexity index and stability index, Score performance 、Score complexity 、Score runtime They are the scores of performance index, complexity index and stability index respectively.

[0068] It is worth noting that, unlike the traditional benchmark evaluation method, the present invention constructs performance indicators, complexity indicators and stability indicators to comprehensively evaluate user algorithms from different aspects, ensuring the fairness and effectiveness of algorithm scoring, and through dynamically adjustable evaluation indicator weights, it can adapt to different operating environments and user needs.

[0069] As a preferred implementation of this embodiment, the performance indicator includes at least one performance parameter;

[0070] Performance Index Score performance The calculation formula is as follows:

[0071]

[0072] Among them, p i represents the i-th performance parameter, w pi represents the weight of the i-th performance parameter, and n is the number of performance parameters.

[0073] In this embodiment, four performance parameters are selected for the fault diagnosis algorithm of the central air conditioning system: accuracy, fault detection rate, fault isolation accuracy, and false alarm rate. The above performance parameters are representative performance parameters selected based on the characteristics of the algorithm to be tested "fault diagnosis algorithm for central air conditioning system" uploaded by the user in this embodiment; therefore, in this embodiment, the performance index score Score performance The calculation formula is as follows:

[0074] Score performance =w accuracy accuracy+w FDR ·FDR+w FIP ·FIP+w FAR FAR (3);

[0075] Among them, accuracy, FDR, FIP, and FAR are accuracy, fault detection rate, fault isolation accuracy, and false alarm rate, respectively. accuracy 、w FDR 、w FIP 、w FAR are the weights of accuracy, fault detection rate, fault isolation accuracy, and false alarm rate, respectively;

[0076] The calculation methods of accuracy, fault detection rate FDR, fault isolation precision FIP, and false alarm rate FAR are as follows:

[0077]

[0078] Among them, TP represents the number of samples that are actually faulty and predicted to be faulty, TN represents the number of samples that are actually not faulty and predicted to be not faulty, FP represents the number of samples that are actually not faulty but predicted to be faulty, and FN represents the number of samples that are actually faulty but predicted to be not faulty.

[0079] As a preferred implementation of this embodiment, the complexity index includes:

[0080] Time complexity, space complexity;

[0081] Complexity index score Score complexity The calculation formula is as follows:

[0082] Scorecomplexity =w space Score space +w time Score time (8);

[0083]

[0084] Among them, w space 、w time is the weight corresponding to the space complexity and time complexity, Score space 、Score time They are the space complexity score and time complexity score respectively, T represents the time complexity of the current algorithm, T max represents the maximum allowed time complexity, S represents the space complexity of the current algorithm, and S max Indicates the maximum allowed space complexity.

[0085] As a preferred implementation of this embodiment, the stability index includes:

[0086] Stable running time of the algorithm;

[0087] Stability Index Score runtime The calculation is as follows:

[0088]

[0089] Among them, β is the adjustment parameter of the fractional decay rate.

[0090] As a preferred implementation of this embodiment, the method of screening the preprocessed new original data set based on cluster analysis and information entropy calculation to obtain an updated data set includes:

[0091] S31: Perform cluster analysis on the new original data set after preprocessing to obtain the center point of the cluster analysis:

[0092] Randomly select K initial cluster analysis center points for the new original data set after preprocessing;

[0093] Assign data points, assign each data point to the cluster formed by its nearest center point according to the distance;

[0094] By recalculating the mean of all data points in each cluster, the cluster analysis center point is updated. The mean of all data points in the cluster is calculated as shown in formula (11):

[0095]

[0096] Among them, μ i is the center point of the ith cluster, C iare all the data points of the ith cluster, x j is a data point;

[0097] Repeat the process of assigning data points and updating cluster analysis center points until the cluster analysis center points no longer change or the specified number of iterations is reached;

[0098] S32: Calculate the information entropy and information gain of the new original data set after preprocessing. The information entropy is calculated as follows:

[0099]

[0100] Among them, H(D) is the information entropy of data set D, p i is the probability of category i;

[0101] The information gain is calculated as follows:

[0102] IG(D,x j )=H(D)-H(D\{x j}) (13);

[0103] Among them, IG(D,x j ) is the sample x j The information gain of the data set D, H(D\{x j}) is to remove the sample x j The conditional entropy of the posterior dataset, p i′ is the probability of category i';

[0104] According to the calculated information gain, find the sample points with larger information gain in different categories of data;

[0105] In this embodiment, the field data include: the cooling capacity of the chiller, the host load rate, the evaporator approach temperature, the condenser approach temperature, the chilled water outlet temperature, the cooling water inlet temperature, the condensing pressure and the fault category corresponding to each data; therefore, the above steps of finding sample points with larger information gain in data of different categories are to find sample points with larger information gain in data of different fault categories.

[0106] S33: Compare the center point of the cluster analysis obtained in S32 and S33 and the sample points with larger information gain in the data of different fault categories with the simulated data set, calculate the vector similarity between the samples, delete the samples with existing working conditions or high vector similarity, and update the data set with the remaining samples.

[0107] In this step, this embodiment uses cosine similarity to calculate the vector similarity between samples. The cosine similarity algorithm has the characteristics of simple calculation, normalized measurement, and wide applicability. The calculation formula of cosine similarity is as follows:

[0108]

[0109] Among them, x is a simulated data set without hot update, and x' is the new data selected above, that is, the center point of cluster analysis and the sample points with large information gain in different categories of data.

[0110] It is worth noting that the data screening and hot update method based on cluster analysis and information gain provided by the present invention can supplement the simulated data set with representative new sample data without affecting the real-time operation of the algorithm, so that the algorithm data is close to the actual operation situation, thereby ensuring the diversity and authenticity of the simulated data set used for data playback. Figure 3 , Figure 4 As shown in the figure, there are obvious changes in the distribution of the simulated data set before and after the hot update. These changes represent changes in the on-site operating conditions. Through hot updates, the simulated data set is closer to the actual situation, thereby improving the diversity, representativeness, and authenticity of the verification data, and ensuring the accuracy and fairness of the algorithm scoring.

[0111] The present application also provides a data playback-based algorithm real-time testing system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program. The data playback-based industrial algorithm real-time testing system can implement various embodiments of the data playback-based industrial algorithm real-time testing method, and can achieve the same beneficial effects, which will not be described in detail here.

[0112] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A real-time testing method for industrial algorithms based on data playback, characterized in that: include: S1: Collect field data to obtain the original data set, preprocess the original data set to obtain the simulated data set; S2: Use the simulated data set to run the algorithm to be tested uploaded by the user, obtain the running results of the algorithm to be tested uploaded by the user in real time, evaluate the algorithm to be tested uploaded by the user, and calculate the algorithm score of the algorithm to be tested uploaded by the user in real time; S3: While S2 is running continuously in real time, field data of a specific time length is collected again to obtain a new original data set, the new original data set is preprocessed, and the preprocessed new original data set is screened based on cluster analysis and information entropy calculation to obtain an updated data set, and the updated data set is inserted into the simulation data set to perform hot update on the simulation data set; S4: Judge the simulated data set after hot update. If the set stop condition is met, the algorithm score obtained by the last calculation is used as the final algorithm score. If the condition is not met, repeat S3 until the simulated data set meets the stop condition, and the user algorithm score obtained by the last calculation is used as the final algorithm score.

2. The real-time testing method for industrial algorithms based on data playback according to claim 1, characterized in that: The step of evaluating the algorithm to be tested uploaded by the user and calculating the algorithm score of the algorithm to be tested uploaded by the user in real time includes: Calculate the scores and weights of the performance indicators, complexity indicators, and stability indicators of the algorithms uploaded by users; Calculate the algorithm score of the algorithm to be tested uploaded by the user 现场录波回放 : Score 现场录波回放 =w performance ·Score performance +w complexity ·Score complexity +w runtime ·Score runtime (1); Among them, w performance 、w complexity 、w runtime are the weights of performance index, complexity index and stability index, Score performance 、Score complexity 、Score runtime They are the scores of performance index, complexity index and stability index respectively.

3. The real-time testing method for industrial algorithms based on data playback according to claim 2, characterized in that: The performance indicator includes at least one performance parameter; Performance Index Score performance The calculation formula is as follows: Among them, p i represents the i-th performance parameter, w pi represents the weight of the i-th performance parameter, and n is the number of performance parameters.

4. The real-time testing method for industrial algorithms based on data playback according to claim 2, characterized in that: The complexity indicators include: Time complexity, space complexity; Complexity index score Score complexity The calculation formula is as follows: Score complexity =w space ·Score space +w time ·Score time (3); Among them, w space 、w time is the weight corresponding to the space complexity and time complexity, Score space 、Score time They are the space complexity score and time complexity score respectively, T represents the time complexity of the current algorithm, T max represents the maximum allowed time complexity, S represents the space complexity of the current algorithm, and S max Indicates the maximum allowed space complexity.

5. The real-time testing method for industrial algorithms based on data playback as claimed in claim 2, characterized in that: The stability indicators include: Stable running time of the algorithm; Stability Index Score runtime The calculation is as follows: Among them, β is the adjustment parameter of the fractional decay rate.

6. The real-time testing method for industrial algorithms based on data playback according to claim 1, characterized in that: S3 screens the preprocessed new original data set based on cluster analysis and information entropy calculation to obtain an updated data set, including: S31: Perform cluster analysis on the new original data set after preprocessing to obtain the center point of the cluster analysis: Randomly select K initial cluster analysis center points for the new original data set after preprocessing; Assign data points, assign each data point to the cluster formed by its nearest center point according to the distance; By recalculating the mean of all data points in each cluster, the cluster analysis center point is updated. The mean of all data points in the cluster is calculated as shown in formula (6): Among them, μ i is the center point of the ith cluster, C i are all the data points of the ith cluster, x j is a data point; Repeat the process of assigning data points and updating cluster analysis center points until the cluster analysis center points no longer change or the specified number of iterations is reached; S32: Calculate the information entropy and information gain of the new original data set after preprocessing. The information entropy is calculated as follows: Among them, H(D) is the information entropy of data set D, p i is the probability of category i; The information gain is calculated as follows: IG(D,x j )=H(D)-H(D\{x j }) (9); Among them, IG(D,x j ) is the sample x j The information gain of the data set D, H(D\{x j }) is to remove the sample x j The conditional entropy of the posterior dataset, p i′ is the probability of category i'; According to the calculated information gain, find the sample points with larger information gain in different categories of data; S33: Compare the center point of the cluster analysis and the sample points with large information gain obtained in S32 and S33 with the simulated data set, calculate the vector similarity between the samples, delete the samples with existing working conditions or high vector similarity, and update the data set with the remaining samples.

7. A real-time testing system for industrial algorithms based on data playback, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 6 are implemented.

Images