A performance evaluation method and system for fault detection and diagnosis algorithm

Abstract

The present invention belongs to the technical field of building chillers, and provides a performance evaluation method and system for a fault detection and diagnosis algorithm. The method includes: constructing an input sample set according to an input scenario, and specifying real data information related to each input sample; based on the input sample set, using a fault detection and diagnosis algorithm to obtain fault detection and fault diagnosis outputs; comparing the fault detection and fault diagnosis outputs corresponding to each input sample with the real data information related to the input sample, and the obtained evaluation results are used to evaluate the performance indicators of the fault detection and diagnosis algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of building chillers, and in particular relates to a performance evaluation method and system for a fault detection and diagnosis algorithm. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Inefficient chiller fault diagnosis is prevalent in today’s commercial buildings. Fault detection and diagnostics (FDD) tools use building operational data to identify the presence of a fault and isolate its root cause. Widespread adoption of such tools and correction of the faults they identify would result in energy savings of approximately 5% to 15% in the commercial building sector.

[0004] Fault detection is the process of detecting faulty behavior, and fault diagnosis is the process of isolating the cause of a fault once it has been detected. Fault detection and diagnosis are sometimes performed separately, but are often combined in a single step. The development of automatic fault detection and diagnosis (AFDD) methods for building heating, ventilation, air conditioning (HVAC) and control systems has been an active area of ​​research over the past three decades.

[0005] The reality is that there is no universally accepted standard for evaluating FDD algorithms. There is an urgent need to develop a broadly applicable evaluation procedure for existing and next-generation FDD tools. Such a procedure would provide a trusted, standard approach for the validation and comparison of FDD tools at all stages of development, from early research to mature commercial products. Given the diversity of chiller FDD use cases and competing technologies, establishing a standard evaluation methodology is a daunting challenge. The building industry and other industries have made significant progress in establishing chiller FDD test procedures and metrics. However, existing evaluation methods vary significantly in terms of the specific evaluation parameters within a given general approach and how these choices affect the evaluation results.

[0006] Therefore, in response to the above problems, a performance evaluation method for the fault detection and diagnosis algorithm of building chillers is urgently needed. This general and systematic method can be used to evaluate the performance of the chiller FDD algorithm. Summary of the invention

[0007] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a performance evaluation method and system for a fault detection and diagnosis algorithm, which realizes the performance evaluation of a fault detection and diagnosis algorithm for a chiller in a building by comprehensively considering fault definition, evaluation input sample definition, performance indicators, etc.

[0008] Chiller fault detection and diagnosis (FDD) algorithms are one of the most active areas of research and commercial product development in the building industry. However, evaluating their performance has taken more effort than developing these algorithms. As a result, there is still considerable uncertainty about the accuracy and effectiveness of research FDD algorithms and commercial products. Together, this situation has hampered the widespread adoption of FDD tools. This paper proposes a general and systematic approach to evaluate the performance of FDD algorithms.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] A first aspect of the present invention provides a method for evaluating the performance of a fault detection and diagnosis algorithm.

[0011] A performance evaluation method for a fault detection and diagnosis algorithm, comprising:

[0012] Construct an input sample set based on the input scenario and specify the real data information related to each input sample; in order to evaluate whether the output of the AFDD algorithm is correct for a given input sample, it is first necessary to establish the state of the system represented by the sample: faulty or not, and if there is a fault, what are the causes of the fault. In this step, each input sample generated is assigned a true value state (state label).

[0013] Based on the input sample set, the fault detection and diagnosis algorithm is applied to obtain fault detection and fault diagnosis outputs; wherein the input samples are taken from the input scenarios that constitute the automatic fault detection and diagnosis (AFDD) algorithm evaluation data set. Each input sample is a data set for which the FDD algorithm performance evaluation should produce a single result. The input samples may include system information (metadata) and time series trend data from building sensors and control systems.

[0014] The fault detection and fault diagnosis outputs corresponding to each input sample are compared with the real data information related to the input sample, and the evaluation results are used to evaluate the performance indicators of the fault detection and diagnosis algorithm. Specifically, for each input sample, the AFDD algorithm will produce a detection result to indicate whether a fault exists and a diagnosis result to further explain the exact nature or root cause of the fault. Combined, the detection and diagnosis results produce a single output. The evaluation results are generated by comparing the output of the FDD algorithm for each sample with the real data to produce a set of raw evaluation results. These raw results are then aggregated to generate one or more FDD performance metrics.

[0015] Furthermore, the input scenarios include: building type, equipment type, fault type, fault intensity, fault prevalence, environmental conditions and data used by the fault detection and diagnosis algorithm.

[0016] Furthermore, the input samples include: system information and time series trend data from building sensors, and system information and time series trend data from control systems.

[0017] Further, based on the input sample set, the process of using the fault detection and diagnosis algorithm to obtain the fault detection and fault diagnosis output includes: obtaining one or several analysis elements by preprocessing the input sample, analyzing the one or several analysis elements by the fault detection and diagnosis algorithm to obtain several intermediate results, and aggregating the several intermediate results into one or several analysis elements by the fault detection and diagnosis algorithm to obtain fault detection and fault diagnosis. Specifically, in this step, the FDD algorithm is first initialized and then executed on each input sample. Initialization may include the input of metadata specific to the selected input scenario, supervised learning using a training data set (input samples are labeled with real data), or parameter adjustment to adjust the sensitivity of the algorithm. In the FDD algorithm, the input sample is preprocessed into one or several required analysis elements. The FDD algorithm analyzes each analysis element to obtain intermediate FDD results. Then, the FDD algorithm aggregates these intermediate results into all analysis elements to produce detection and diagnosis results displayed to the user.

[0018] Furthermore, the process of using the obtained evaluation results for evaluating the performance indicators of fault detection and diagnosis algorithms includes: comparing the fault detection and fault diagnosis outputs corresponding to each input sample with the real data information related to the input sample to obtain the evaluation results, and constructing an evaluation result set; aggregating the evaluation result set to generate performance metrics of one or more fault detection and diagnosis algorithms.

[0019] Furthermore, the performance indicators of the fault detection and diagnosis algorithm include: cross entropy loss function and accuracy.

[0020] Furthermore, the fault detection and diagnosis algorithm adopts a model constructed by a long short-term memory network algorithm.

[0021] A second aspect of the present invention provides a performance evaluation system for a fault detection and diagnosis algorithm.

[0022] A performance evaluation system for a fault detection and diagnosis algorithm, comprising:

[0023] An input information determination module is configured to: construct an input sample set according to an input scenario, and specify real data information related to each input sample;

[0024] A fault detection and diagnosis module is configured to: obtain fault detection and fault diagnosis outputs based on an input sample set using a fault detection and diagnosis algorithm;

[0025] The evaluation module is configured to compare the fault detection and fault diagnosis output corresponding to each input sample with the real data information related to the input sample, and the obtained evaluation result is used to evaluate the performance index of the fault detection and diagnosis algorithm.

[0026] A third aspect of the present invention provides a computer-readable storage medium.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the performance evaluation method and steps of the fault detection and diagnosis algorithm as described in the first aspect above.

[0028] A fourth aspect of the present invention provides a computer device.

[0029] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the performance evaluation method of the fault detection and diagnosis algorithm as described in the first aspect above are implemented.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention discloses a performance evaluation method and system for a fault detection and diagnosis algorithm, which comprehensively considers fault definition, evaluation input sample definition, performance indicators, etc. to achieve performance evaluation of a fault detection and diagnosis algorithm for a building chiller. It is worth noting that there is no universally accepted standard to evaluate chiller FDD algorithms, and there is an urgent need to develop a widely applicable evaluation method for existing and next-generation FDD tools. Considering the diversity of FDD use cases and competing technologies, the present invention proposes a general and standard chiller FDD performance evaluation method and documents the design decisions required to implement the method.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 It is a framework diagram of the performance evaluation method of the fault detection and diagnosis algorithm of the present invention;

[0035] Figure 2 is an execution flow chart of a performance evaluation method of a fault detection and diagnosis algorithm of the present invention;

[0036] Figure 3 is a graph of the execution results of the performance evaluation method using the fault detection and diagnosis algorithm of the present invention;

[0037] Figure 4 It is a performance evaluation index result diagram before and after the performance evaluation method of the fault detection and diagnosis algorithm of the present invention is executed. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0041] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0042] Embodiment 1

[0043] like Figure 1As shown, this embodiment provides a performance evaluation method for a fault detection and diagnosis algorithm. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a terminal, a server, and a system, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0044] Construct an input sample set based on the input scenario and specify the real data information related to each input sample;

[0045] Based on the input sample set, a fault detection and diagnosis algorithm is used to obtain fault detection and fault diagnosis outputs;

[0046] The fault detection and fault diagnosis output corresponding to each input sample is compared with the real data information related to the input sample, and the obtained evaluation results are used to evaluate the performance indicators of the fault detection and diagnosis algorithm.

[0047] Specifically, the implementation process of the technical solution of this embodiment can adopt the following steps:

[0048] S1: Identify a set of input scenarios that define environmental conditions, fault types, and fault intensity (fault severity relative to measurables);

[0049] Specifically, in S1, the input scenario is determined as follows:

[0050] Each input scenario defines a test case consisting of one or more input samples. An input scenario can specify:

[0051] Building type and features: An office building in Jinan;

[0052] Equipment type: HVAC chiller: A 90-ton centrifugal chiller, with shell and tube heat exchangers for the evaporator and condenser, R134a as the refrigerant and water as the coolant;

[0053] Fault type:

[0054] Table 1 Fault types and severity levels

[0055]

[0056] Environmental conditions: The whole equipment is installed in a constant temperature environment of about 22°C;

[0057] Data available for the FDD algorithm (such as sensors, instruments or control systems): The FDD algorithm uses the long short-term memory network algorithm, and the data acquisition method is as follows:

[0058] The energy optimization system host "Energy Efficiency Gateway" is set up at the user's site. Through the "Energy Efficiency Gateway", cloud-edge integration is realized to realize the local collection, edge computing and local control of electromechanical equipment unit information, and meet the user's energy management needs such as metering, monitoring, analysis and control of central air-conditioning systems and terminal air-conditioning systems. Real-time monitoring of the operating status data of the chiller equipment realizes the functions of operating parameter setting, remote equipment control, statistical analysis and historical query.

[0059] S2: Create a set of input samples drawn from the input dataset, each of which is a set of test datasets, and the performance evaluation will produce a single result;

[0060] Specifically, in S2, the input sample is determined as follows:

[0061] The input samples are taken from the input scenarios that constitute the automatic fault detection and diagnosis (AFDD) algorithm evaluation dataset. Each input sample is a set of data for which the FDD algorithm performance evaluation should produce a single result. The input samples may include system information (metadata) and time series trend data from building sensors and control systems. The input samples include 6,000 sets of data, which are divided into training sets and test sets in a ratio of 8:2. The data is shown in the table (first 10 sets of data):

[0062] Table 2 Chiller time series data

[0063] Serial number kW TCO TEI TCI TEA Evap Tons 1 1.68E-45 66.7 64.45 66 0.1 9.85E-47 2 1.68E-45 66.66 64.45 66 0.1 1.04E-46 3 1.68E-45 66.98 65.63 66.7 0 2.787 4 1.68E-45 67.02 65.72 66.74 0 2.429 5 1.68E-45 67.02 65.8 66.82 0 2.436 6 1.68E-45 67.02 65.88 66.9 0 2.07 7 1.68E-45 67.11 66.04 66.9 0.2 0.9692 8 1.68E-45 67.11 66.17 66.9 0.4 0.9696 9 1.68E-45 67.11 66.33 66.9 0.2 0.9692 10 1.68E-45 67.11 66.49 66.99 0 0.6004

[0064] S3: specifies the real data information associated with each input sample;

[0065] Specifically, in S3, the specific content of the real data is as follows:

[0066] In order to evaluate whether the output of the AFDD algorithm is correct for a given input sample, it is first necessary to establish the state of the system represented by the sample: faulty or not, and if there is a fault, what are the causes of the fault. In this step, each input sample generated in step 2 is assigned a ground truth state, as shown in the following table:

[0067] Table 3 Fault types and one-hot codes

[0068] Fault type Fault Tags One-hot encoding normal 0 100 000 Condenser water flow reduction 1 010 000 Evaporator water flow reduction 2 001 000 Refrigerant leak 3 000 100 Excessive refrigerant 4 000 010 Condenser fouling 5 000 001

[0069] S4: executing the fault detection and diagnosis (FDD) algorithm being evaluated for each input sample, the FDD algorithm receiving the input sample and generating fault detection and fault diagnosis outputs;

[0070] Specifically, in S4, the algorithm performs the following specific contents:

[0071] In this step, the FDD algorithm is first initialized and then executed for each input sample. Initialization can include the input of metadata specific to the selected input scenario, supervised learning using a training dataset (input samples are labeled with real data), or tuning (parameter tuning) to adjust the sensitivity of the algorithm. Figure 2 The workflow of the FDD algorithm to convert input samples (Step 2) into algorithm outputs (Step 5) is given. In the FDD algorithm, the input samples are preprocessed into one or several required analysis elements. The FDD algorithm analyzes each analysis element to obtain intermediate FDD results. Then, the FDD algorithm aggregates these intermediate results into all analysis elements to produce detection and diagnosis results displayed to the user (Step 5).

[0072] The FDD algorithm uses the long short-term memory network (LSTM) algorithm, and the algorithm execution is as follows:

[0073] Firstly, an LSTM fault recognition model is constructed. The model structure mainly consists of an input layer, a hidden layer, and an output layer. The loss function uses the cross entropy loss function. The model training process is optimized using the Nadam algorithm optimizer, and the network model is built under the Keras framework.

[0074] The specific steps of model training are as follows:

[0075] Step 1: Import the sample data after data preprocessing and feature selection into the input layer.

[0076] Step 2: The hidden layer contains two LSTM layers. The LSTM is trained layer by layer using the data transmitted from the input layer. The output of the neurons in the hidden layer of the previous LSTM network is sent to the next layer for calculation.

[0077] Step 3: Feed the final result of the output sequence into the Softmax classifier as the output layer.

[0078] Step 4: Compare the output predicted fault labels with the actual labels, continuously optimize the model through loss function calculation, perform network training based on GPU parallel computing, and realize the rapid construction of LSTM model.

[0079] S5: Retrieve the fault detection and fault diagnosis output of the FDD algorithm;

[0080] Specifically, in S5, the specific content of the algorithm output is:

[0081] For each input sample, the AFDD algorithm will produce a detection result to indicate whether a fault exists and a diagnosis result to further explain the exact nature or root cause of the fault. Combined, the detection and diagnosis results produce a single output for use in step 6. The fault diagnosis result is shown in Figure 3 shown.

[0082] S6: Evaluate FDD performance indicators.

[0083] Specifically, in S6, the specific contents of the evaluation results and performance indicators are:

[0084] The evaluation results are generated by comparing the output of the FDD algorithm for each sample (Step 5) with the real data (Step 3) to produce a set of raw evaluation results. These raw results are then aggregated to generate one or more FDD performance metrics (Step 6). This embodiment uses two performance indicators, namely the cross entropy loss function and accuracy.

[0085] Regarding the two probability distributions p and q of the sample set, let p be the true distribution and q be the fitted distribution. The expected encoding length required to identify a sample is measured according to the true distribution p, that is, the information entropy:

[0086]

[0087] If the fitted distribution q is used to represent the expectation of the encoding length from the true distribution p, that is, the cross entropy:

[0088]

[0089] For the difference between the true distribution p and the fitted distribution q, we can measure it with KL divergence, also called relative entropy:

[0090]

[0091] In the classification problem of machine learning, we hope to narrow the gap between the model prediction and the label. The label set remains unchanged, so we only need to focus on the cross entropy during the optimization process. In the multi-classification task of this study, the cross entropy loss function used to describe the training performance of the three fault classification models can be calculated as:

[0092]

[0093] Where p = [p0, ..., pC-1 ] is a probability distribution, each element p i represents the probability that the sample belongs to the i-th category; y = [y0, ..., y C-1 ] is the one-hot representation of the sample label.

[0094] In the test samples, five faults with four severity levels were used as examples to compare the chiller fault diagnosis performance of the three comparison models. When a single fault occurs, Cross entropy can also be used to evaluate the recognition performance corresponding to a specific fault. At the same time, the fault recognition accuracy (Accuracy) is used as another performance indicator. For the test time series of each type of fault, Accuracy is defined as the ratio of the number of correctly diagnosed samples to the number of diagnosed samples, expressed as a percentage, and the calculation formula is as follows:

[0095]

[0096] Among them, n sample is the number of samples; y i ' is the predicted label; y i is the true label.

[0097] The fault labels output by the model are more likely to be identical to the actual fault labels. The more accurate the classification, the higher the accuracy.

[0098] The results of the FDD algorithm for chiller fault diagnosis after evaluation index calculation are as follows Figure 4 As shown in the figure, we can see that the fault diagnosis performance of the long short-term memory network is good. It has good fault diagnosis effects in data samples with different faults and different severity levels, high fault diagnosis accuracy, and excellent model training performance.

[0099] Embodiment 2

[0100] This embodiment provides a performance evaluation system for a fault detection and diagnosis algorithm.

[0101] A performance evaluation system for a fault detection and diagnosis algorithm, comprising:

[0102] An input information determination module is configured to: construct an input sample set according to an input scenario, and specify real data information related to each input sample;

[0103] A fault detection and diagnosis module is configured to: obtain fault detection and fault diagnosis outputs based on an input sample set using a fault detection and diagnosis algorithm;

[0104] The evaluation module is configured to compare the fault detection and fault diagnosis output corresponding to each input sample with the real data information related to the input sample, and the obtained evaluation result is used to evaluate the performance index of the fault detection and diagnosis algorithm.

[0105] It should be noted that the examples and application scenarios implemented by the input information determination module, the fault detection and diagnosis module, and the evaluation module are the same as those of the corresponding steps in Embodiment 1, but are not limited to the contents disclosed in the above Embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0106] Embodiment 3

[0107] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the performance evaluation method of the fault detection and diagnosis algorithm described in the first embodiment are implemented.

[0108] Embodiment 4

[0109] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the performance evaluation method of the fault detection and diagnosis algorithm described in the first embodiment are implemented.

[0110] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0114] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A performance evaluation method for a fault detection and diagnosis algorithm, characterized in that: include: Construct an input sample set based on the input scenario and specify the real data information related to each input sample; The input scenarios include: building type, equipment type, fault type, fault intensity, fault prevalence, environmental conditions, and data used by fault detection and diagnosis algorithms; Based on the input sample set, a fault detection and diagnosis algorithm is used to obtain fault detection and fault diagnosis outputs; The fault detection and fault diagnosis output corresponding to each input sample is compared with the real data information related to the input sample, and the evaluation results obtained are used to evaluate the performance indicators of the fault detection and diagnosis algorithm; The process of using the obtained evaluation results for evaluating the performance indicators of the fault detection and diagnosis algorithm includes: comparing the fault detection and fault diagnosis output corresponding to each input sample with the real data information related to the input sample to obtain the evaluation results and construct an evaluation result set; aggregating the evaluation result set to generate one or more performance metrics of the fault detection and diagnosis algorithm; Specific content of evaluation results and performance indicators: The evaluation results are obtained by comparing the output of the FDD algorithm for each sample with the real data to produce a set of raw evaluation results, and then aggregating these raw results to generate one or more FDD performance metrics, using two performance indicators, namely the cross entropy loss function and accuracy; Regarding the two probability distributions p and q of the sample set, let p be the true distribution and q be the fitted distribution; the expectation of the encoding length required to identify a sample is measured according to the true distribution p, that is, the information entropy: If the fitted distribution q is used to represent the expectation of the encoding length from the true distribution p, that is, the cross entropy: The difference between the true distribution p and the fitted distribution q is measured by KL divergence, also called relative entropy: In the classification problem of machine learning, the gap between model prediction and label is narrowed. The label set remains unchanged. In the optimization process, only the cross entropy needs to be paid attention to. In the multi-classification task, the cross entropy loss function used to describe the training performance of the three fault classification models is calculated as: Where p = [p0, ..., p C-1 ] is a probability distribution, each element p i represents the probability that the sample belongs to the i-th category; y = [y0, ..., y C-1 ] is the one-hot representation of the sample label; In the test samples, five faults with four severity levels were taken as examples to compare the chiller fault diagnosis performance of the three comparison models. When a single fault occurs, Cross entropy can also be used to evaluate the recognition performance corresponding to a specific fault. At the same time, the fault recognition accuracy is used as another performance indicator. For the test time series of each type of fault, Accuracy is defined as the ratio of the number of correctly diagnosed samples to the number of diagnosed samples, expressed as a percentage, and the calculation formula is as follows: Among them, n sample is the number of samples; y i ' is the predicted label; y i is the true label; The fault labels output by the model are more similar to the actual fault labels. The more accurate the classification, the higher the accuracy. The results of the FDD algorithm for chiller fault diagnosis after the evaluation index calculation show that the long short-term memory network has better fault diagnosis performance, and has better fault diagnosis effects in data samples with different faults and different severities. The fault diagnosis accuracy is high and the model training performance is excellent.

2. The performance evaluation method of the fault detection and diagnosis algorithm according to claim 1, characterized in that: The input samples include: system information and time series trend data from building sensors, and system information and time series trend data from control systems.

3. The performance evaluation method of the fault detection and diagnosis algorithm according to claim 1, characterized in that: The process of obtaining fault detection and fault diagnosis outputs based on an input sample set by using a fault detection and diagnosis algorithm includes: obtaining one or several analysis elements by preprocessing the input samples, analyzing the one or several analysis elements by using a fault detection and diagnosis algorithm to obtain several intermediate results, and aggregating the several intermediate results into one or several analysis elements by using a fault detection and diagnosis algorithm to obtain fault detection and fault diagnosis.

4. The performance evaluation method of the fault detection and diagnosis algorithm according to claim 1, characterized in that: The fault detection and diagnosis algorithm adopts a model constructed by a long short-term memory network algorithm.

5. A performance evaluation system for a fault detection and diagnosis algorithm, characterized in that: A method for implementing a performance evaluation method of a fault detection and diagnosis algorithm as claimed in any one of claims 1 to 4, comprising: An input information determination module is configured to: construct an input sample set according to an input scenario, and specify real data information related to each input sample; A fault detection and diagnosis module is configured to: obtain fault detection and fault diagnosis outputs based on an input sample set using a fault detection and diagnosis algorithm; The evaluation module is configured to compare the fault detection and fault diagnosis output corresponding to each input sample with the real data information related to the input sample, and the obtained evaluation result is used to evaluate the performance index of the fault detection and diagnosis algorithm.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the performance evaluation method of the fault detection and diagnosis algorithm as described in any one of claims 1 to 4 are implemented.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the performance evaluation method of the fault detection and diagnosis algorithm according to any one of claims 1 to 4 are implemented.

Images