Interactive computer-implemented method for testing an HVAC system

Abstract

An interactive computer-implemented method for testing an HVAC system includes: - input of at least one document (11) relating to the HVAC system by a user into a data processing system (4), the documents containing schematic representations of the HVAC system; - in the data processing system (4), extracting a plurality of functional blocks (880) (106) from the schematic representations by a machine learning system (8); - detecting inconsistencies, including missing documents or information, within the data processing system depending on the result of the extraction (108); - a conversation agent (5) in the data processing system interacting with the user to request missing documents or information relating to the HVAC system (107, 101); - Creating a report based on the extraction and detection of discrepancies (109).

Description

Technical area

[0001] The present invention relates to a computer-implemented method for reviewing technical documents describing an HVAC (heating, ventilation and air conditioning) system. State of the art

[0002] Modern heating, ventilation, and air conditioning (HVAC) systems are incredibly sophisticated, but rarely live up to their potential. A significant energy performance gap (EPG) exists between the energy performance calculated at the time of design and the actual energy performance of the building once the system is in operation. According to some estimates, this energy performance gap is as high as 30% and is primarily caused by errors and other discrepancies during the planning and design, implementation, commissioning, or operation of the system.

[0003] The term "HVAC documentation" refers to a collection of documents and records related to the design, installation, operation, and maintenance of HVAC systems in a building. The purpose of HVAC documentation is to ensure that HVAC systems are effectively designed, installed, and maintained to create a comfortable and healthy indoor environment while ensuring energy efficiency and compliance with regulatory requirements. Reliable HVAC documentation is therefore critical for reducing EPG.

[0004] HVAC documentation typically includes design documentation, including schematics and / or branch diagrams that show the layout of ducts, pipes, and various equipment such as furnaces, air conditioners, boilers, and ventilation systems used to distribute and treat air and fluids within a building or series of buildings. Other elements of the documentation may include a building description, load calculations to determine heating and cooling loads for various areas within a building, and installation documents, such asInstallation plans, for contractors to correctly install HVAC equipment and components; manufacturer-provided equipment manuals that contain instructions for the proper installation of HVAC equipment; test and commissioning reports that document tests performed to verify that the HVAC system is functioning as intended; and compliance and regulatory documentation, including certificates of compliance, inspection reports, and so on. Documentation may also describe or illustrate the connection to the global information system (GIS). Some schematic diagrams may include dimensions or minimum clearances between components or functional blocks.

[0005] Overall, HVAC documentation plays a critical role in ensuring the efficient and safe operation of HVAC systems, meeting regulatory requirements, and assisting with system troubleshooting and maintenance. It helps building owners, operators, and maintenance personnel effectively manage HVAC systems throughout their lifecycle.

[0006] For large HVAC systems, HVAC documentation often becomes quite extensive. Analysis and review of this documentation is often required not only during planning and installation, but also during operation to predict, prevent, detect, and / or analyze discrepancies. This method is typically performed manually, i.e., by human specialists. This makes the process time-consuming, laborious, and error-prone.

[0007] In the field of building automation, the process has changed little since the advent of digital controls in the 1980s. Initially, all HVAC systems were programmed by hand in a structured language (e.g., Pascal). Then, graphical editors emerged, allowing programmers to use function blocks and link their inputs and outputs to create the desired information flow.

[0008] To accelerate development, some hardware vendors have developed their own function blocks to facilitate programming. Function blocks are black boxes that contain multiple functions and represent inputs / outputs, such as a fan controller with its monitoring, or a heater start. They represent a good step toward standardization, but are rarely used in practice because they are unfamiliar. Instead, the programmer relies on implicit notions of function blocks that are defined "ad hoc" during programming.

[0009] Automating and standardizing the process would be of great value in significantly reducing discrepancies in HVAC design and control, thereby improving the energy efficiency of buildings. However, there is one major hurdle to overcome: digitizing HVAC system schematics. A simple first step is to scan the printed schematics to create digital images. In some cases, the schematics already exist in digital form as images, PDFs, or vector graphics. The difficult step then lies in analyzing the documents to identify graphical symbols, text, functional blocks, and their connections. This is typically done manually.

[0010] It would be desirable to develop a computer-implemented system to support this task of analyzing and reviewing HVAC documentation to make it more efficient and to reduce the risk of undetected discrepancies in the documentation.

[0011] The state of the art has already proposed using machine learning technologies to enable document review.

[0012] US2017235705 B2 discloses an automated analysis of audit files in which the text content is extracted from the audit documents in order to accelerate the auditing process.

[0013] CN112464036 A discloses the verification of data contents by first detecting a data type and then applying a corresponding target verification method.

[0014] CN116306563 A discloses a machine learning-based method for reviewing a set of documents, using an existing knowledge graph as a reference for reviewing the documents to be reviewed.

[0015] CN109146419 A discloses an inspection method that analyzes drawings to classify them and performs an inspection using a database of rules. The inspection method is self-improving through machine learning with a human-in-the-loop scheme. Furthermore, the inspection report is transmitted via online communication.

[0016] None of the above documents contain a test method suitable for certifying an HVAC system. Furthermore, the quality of the test depends on the documents submitted. For example, a user could choose not to submit documentation regarding faulty blocks or components and receive a positive test report that ignores the defects in those faulty blocks.

[0017] The consequences of such discrepancies in HVAC documentation are usually problems during installation, poorly functioning HVAC systems, over- or under-sized systems, or failures during operation.

[0018] From a technical perspective, it is therefore crucial to be able to efficiently detect discrepancies in HVAC documents. Brief description of the invention

[0019] An object of the present invention is to provide a method that overcomes or reduces the deficiencies and limitations of the prior art.

[0020] According to the invention, these objects are achieved by an interactive, computer-implemented method for testing an HVAC system, comprising: - input of at least one document relating to the HVAC system by a user into a data processing system, the documents containing schematic representations of the HVAC system; - in the data processing system, extracting a plurality of functional blocks from the schematic representations by a machine learning system; - Detecting inconsistencies, including missing documents or information, within the data processing system depending on the extraction result; - Interaction of a conversational agent in the data processing system with the user to request missing documents or information related to the HVAC system; - Create a report based on the extraction and detection of discrepancies.

[0021] The method can thus analyze schematic representations by extracting various functional blocks. The extraction is performed with the help of a machine learning system, so that even new and previously unseen schematic representations can be analyzed using the machine learning system's prior training.

[0022] The method for testing HVAC software and hardware is therefore based on machine learning, specifically using an AI-based conversational agent that interacts with a user performing the interactive test. The method may include the steps of interaction, document capture, document analysis, and the generation of a test report, with the goal of creating a test that indicates whether the HVAC documentation contains discrepancies.

[0023] The conversational agent can be controlled by the HLK's Large Document Model (LDM) understanding.

[0024] The method may include a hierarchical decomposition of the schematic representation into functional blocks. These functional blocks may be compared with a reference database of functional blocks known to be free of discrepancies.

[0025] The machine learning system can perform iterative extraction of functional blocks at different hierarchy levels.

[0026] Any combination of more than one elementary component that performs a function is defined as a functional block. Furthermore, a functional block can consist of further smaller functional blocks. Ultimately, the schematic representation of a complete HVAC system corresponds to a functional block, which in turn is hierarchically structured from smaller, underlying functional blocks.

[0027] The method can detect when documents or information are missing and interact through a conversational agent to request these additional documents.

[0028] The detection of missing documents or information depends on the extraction of the functional blocks. For example, an extracted functional block may refer to a specific component that requires technical specifications, calculations, and / or detailed schematics. The data processing system can then detect that such a component is depicted in a schematic, determine that some of the information or documentation related to this functional block or its installation is missing, and request the missing information or documentation.

[0029] For example, a user may be asked to provide additional information about a specific functional block, such as a manual or specifications for a component within that block. A user may also be asked to provide other missing documentation elements, such as text or PDF documents related to the building, calculations, installation plans, or any other type of document that might be expected in a particular context. Inconsistencies in HVAC documentation within the scope of this application may include errors or suboptimal configurations, such as one or more of the following: • Malfunctioning function blocks, e.g., function blocks with intrinsic design flaws such as missing internal or external connections, incorrect connections, missing elementary component, incorrect elementary component, function block that does not comply with regulatory or user-specific requirements / specifications, function block with associated functional or control software that does not comply with the reference database and / or insufficiently documented function blocks, etc. • Incorrect arrangements of functional blocks, such as design errors, arrangements of functional blocks with inherent design discrepancies that do not meet regulatory or user requirements / specifications, incorrect or illogical connections between functional blocks, incorrect assignment of functional blocks to the arrangement, missing connections or elements, incorrect or poor selection of components within the arrangement, incorrect connection of components and / or inadequately documented arrangements of functional blocks, hydraulic failures, electrical failures, etc. • Discrepancies in the textual documentation, such as illogical or incomprehensible documentation, discrepancies such as errors in the text and / or calculations, missing documentation, information in the documentation that does not comply with the schematic representations and / or the regulatory requirements, errors in temperatures, load, power, etc. or errors in the cadastral numbers. • Missing documents; • Etc

[0030] A functional block is considered likely to contain discrepancies if an internal connection is incorrect, if a specific elementary component is missing, or if the correct functional or control software does not match a reference database of certified software.

[0031] The report may include a list of functional blocks that match previously known discrepancy-free (i.e., certified) functional blocks and / or a list of functional blocks that are likely to contain discrepancies.

[0032] The machine learning system may include a transformer.

[0033] The transformer can be based on an attention mechanism to focus attention on elementary components or functional blocks near an elementary component or functional block (local neighborhood) to ensure efficient computation.

[0034] The transformer neural network can be trained with specific synthetic schematics to improve its internal representation of HVAC systems. For example, it can be trained with schematics containing automatically generated discrepancies, such as removed components or connections, added components, replaced components, modified text, etc.

[0035] The machine learning system may include a recurrent neural network (RNN), such as an LSTM network. RNNs are networks in which a result or intermediate result is fed back into the network. RNNs are often used for the analysis of temporal and sequential data, such as texts, videos, flowcharts, computer code, document sets, etc. In practice, RNNs can be used for the iterative extraction of functional blocks consisting of functional sub-blocks as components.

[0036] The machine learning system may comprise a convolutional neural network (CNN), preferably a deep convolutional neural network controlled by an attention mechanism. CNNs are feedforward neural networks that use filters and pooling layers; they are well suited for solving problems with spatial data, such as images, and are therefore well suited, within the scope of the invention, to solving the problem of extracting input blocks and classifications into schematic representations.

[0037] The machine learning system can be trained or have been trained with a large document model (LDM).

[0038] The machine learning system can use a large document model (LDM), also known as a generative pre-trained transformer (GPT) model.

[0039] The LDM can be trained on a comprehensive dataset of HVAC documents, possibly supplemented by synthesized documents.

[0040] The machine learning system can be used to classify the extracted functional blocks as working or non-working.

[0041] The method may include a step of vectorizing and OCR-recognizing the schematic representations. The vectorized and OCR-recognized schematic representations may be input into the machine learning system.

[0042] The document set may contain text descriptions and / or calculations related to the HVAC system.

[0043] The method may include a step of interpreting the text descriptions and / or calculations.

[0044] The extracted functional blocks can be transmitted to a software module that detects discrepancies such as errors and / or malfunctions.

[0045] The method may include a step of analyzing the interactions between said functional blocks.

[0046] The documents may contain a description of a building in which the HVAC system is to be installed. The data processing system can determine whether the HVAC system is suitable for the specified building.

[0047] The interactions between the conversational agent and the user are preferably multimodal and include the input and display of images and text.

[0048] Some of the documents relate to hardware components of the HVAC system. In one embodiment, some other documents relate to software components of the HVAC system. Short description of the characters

[0049] Exemplary embodiments of the invention are disclosed in the description and illustrated by the figures, in which: Fig. 1 schematically illustrates a system according to an embodiment of the invention; Fig. 2 schematically illustrates the workflow of a method according to an embodiment of the invention; Fig. 3 schematically illustrates a CNN that can be used as a machine learning system for extracting feature blocks from documents; Fig. 4 different steps of a transformer-based method for automatically detecting functional blocks and design discrepancies in HVAC plans are illustrated; Fig. 5 illustrates an example of a schematic representation of an HVAC system that can be input into the data processing system of the invention; Fig. 6A is an example of a functional block derived from the schematic diagram of Fig. 5 can be extracted; Fig. 6B shows an example of a reference function block that is linked to the extracted function block from Fig. can be compared; and Figure 7 shows an example of a prefabricated transformer that can be used in the context of the invention. Examples of embodiments of the present invention

[0050] Fig. 1 shows an example of a system that can be used to carry out the method according to the invention.

[0051] In this embodiment, a human user 2 uses a user device 20, such as a PC, mobile phone, tablet, etc., to input documents 11A, 11B related to an HVAC system. The documents include schematic diagrams 11B of the HVAC system, such as schematic diagrams showing the layout of ducts, pipes, and various devices, such as furnaces, air conditioners, boilers, ventilation systems, etc., for distributing and conditioning air and fluids in a building or group of buildings.Other elements of 11A documentation may include text and / or plans describing the building, load calculations to determine heating and cooling loads for various areas within a building, installation documents such as installation plans for contractors to properly install HVAC equipment and components, manufacturers' equipment manuals with instructions for proper HVAC equipment installation, test and commissioning reports that document tests performed to verify the HVAC system is functioning as intended, and code and regulation compliance documents, including certificates of compliance, inspection reports, etc.

[0052] The user device 20 further serves to transmit said documents 11A, 11B in digital form via the Internet 3 to a remote server 4, display said documents, interact with a conversational agent 5 in a remote server 4, and display a report based on the analysis and classification of said documents. A software program may be executed by the user device 20 to perform some of these tasks. Alternatively or additionally, some or all of these tasks are performed by the remote server 4, with the user 2 using a web browser to communicate with this remote server.

[0053] Remote server 4 may be a single remote server accessible over the Internet, a virtual server, a series of servers, or any machine or series of machines that offer a certain amount of processing power and storage capacity. In the example shown, remote server 4 may execute a number of software modules, including software modules 5-9 and 90 described in the following sections.

[0054] A conversational agent 5 running on the remote server 5 interacts with the user 2. The agent may include a module on the remote server 4.

[0055] A generative machine learning system 6, preferably using a large language model (LLM), is used by the conversational agent 5 to interpret the user's responses to prompts and generate further prompts, sentences, and, in one embodiment, images or diagrams. The generative machine learning system 6 uses an LLM tailored to the testing task and a reference database for HVAC systems.

[0056] The conversational agent 5 is preferably capable of conducting a multimodal dialogue with the user 2. In the context of this application, a dialogue is referred to as multimodal if the conversational agent 5 can receive and send text and images as part of the interaction with the user. The dialogue may also include the exchange of audio data, e.g., a text dialogue with a speech synthesizer and a speech recognition module on the server side. The dialogue may, for example, include a visual question-and-answer interaction in which a user interacts with an image using a pointing device as a haptic modality and receives a text-based response from the conversational agent.

[0057] The conversational agent 5 prompts user 2 to send documents to the HVAC system in digital format and prompts user 2 to send additional documents if some documents are missing. As part of the interaction, a report on the HVAC documents is also sent or provided. An image processing module 7 receives images in documents sent by user 2 via the conversational agent 5 and preprocesses these images. Image preprocessing may include, for example, one or more of the following methods: cropping, noise reduction, sharpening, brightness correction, white balance correction, contrast enhancement, etc. Image preprocessing also includes graphic symbols, context, and text extraction.

[0058] Another machine learning system 8 receives the preprocessed graphical symbols, connections, and text along with their position in the documents, extracts functional blocks from these graphical symbols, connections, and text, and classifies these functional blocks. The machine learning system 8 may include a deep neural network trained on a large set of HVAC reference schematics and possibly HVAC-related text. The schematic representations of the HVAC system used for training have preferably been annotated to mark functional blocks so that the machine learning system can extract such functional blocks from other schematic representations of the HVAC system.

[0059] A discrepancy detection module 9 can be used to detect discrepancies in the functional blocks extracted by the system 8. The discrepancy detection module can access a reference database 80 containing functioning or non-functioning functional blocks.

[0060] A report generator 90 creates a report which preferably contains a list of the extracted and identified label blocks, as well as text indicating the discrepancies found (if any), and, if appropriate, some recommendations.

[0061] An example of the workflow for a method according to the invention is shown in Fig. 2 shown.

[0062] In step 100, a user 2 logs on to his user device 20 and to the remote server 4.

[0063] In step 101, the user receives a request generated by conversational agent 5 to send documents about an HVAC system and sends these documents 11A, 11B in digital form. The documents can be sent in any suitable electronic format, such as a PDF image, text, pixel image format (e.g., TIF, JPEG, etc.), vector image format (e.g., DWG, AI, etc.), a zip file, or any combination thereof. Some of the documents may contain a name and / or metadata indicating, for example, the content of the document. Some documents may contain images, such as schematic diagrams of the HVAC system. Other documents may optionally contain text, such as a description, instructions, or calculations. Some documents may also contain a mixture of text and images. The user may be prompted to send documents from a specific list, or they may send any documents.The documents can be scanned by the user or retrieved from a file repository.

[0064] Fig. 5 is an image showing an example of an HVAC diagram 11B that may be entered into the data processing system 4 along with other documents. The image may be a scan of an HVAC diagram saved in an image format or an image vector format. It includes a number of elementary components represented by graphic HVAC symbols for pumps, valves, heating systems, etc. Some of the elementary components are grouped into functional blocks 110, although not all functional blocks are identified as such on the schematic. The image also includes various types of fluid and / or electrical connections 112 between elementary components of functional blocks. Some text 115 may be included to designate or indicate parameters for some elementary components, some functional blocks, some connections, or other parts or components of the schematic.In addition, document 11B in this figure may be linked to other text and / or image documents that may, for example, contain additional details or specifications for some elementary components, functional blocks and / or the entire HVAC system.

[0065] Fig. Figure 6A shows part of the schematic representation of Fig. 5, which corresponds to a discrepancy-free functional block 110A. This discrepancy-free certified functional block can be stored in the database 80 in the form of an image or, preferably, an equivalent electrical diagram or graphic and / or marked as discrepancy-free and used to train the machine learning system 8. Fig. Figure 6B shows another example of a functional block consisting of a circuit diagram similar to that of Fig. 5 can be extracted, but which contains discrepancies, see the replaced or missing elementary components 1110B, 1111B and 1112B.

[0066] The documents can refer to hardware and / or software components of the HVAC system. For example, a document may contain the program code for controlling a component or function block.

[0067] Some of the documents may include a description of the building where the HVAC system will be installed. Some of the documents may also refer to the system's specifications or technical requirements or calculations.

[0068] In step 102 (optional), the image documents 11B received from the server 4 are preprocessed by the image processing module to improve the quality of the image. As already mentioned, the preprocessing may include, for example, one or more of the following elements: cropping, noise reduction, sharpness enhancement, brightness correction, white balance correction, contrast enhancement, etc.

[0069] In step 103 (optional), the pixel images can be vectorized, i.e., converted into vector images. For this vectorization, an artificial intelligence system trained on vector representations of graphic HVAC symbols can be used. The output can contain a vector image file, i.e., a set of vectors, as well as possibly icons or images of standard HVAC system symbols.

[0070] In step 104 (optional), the glyphs of the text (if any) in documents 11A, 11B are converted to text using OCR (optical character recognition software). OCR may use a neural network trained on HVAC-related text documents to improve the quality of text recognition.

[0071] In step 105, computer vision algorithms are used to isolate the graphical symbols of the HVAC elementary components 113, texts 115, functional blocks 110 (if already identified), and connections 112 between graphical symbols or between functional blocks.

[0072] A graphical symbol 113 may correspond to a portion of a document, e.g., a rectangular portion containing features, vectors, pixels, and / or glyphs that represent a known single elementary component or a known functional unit.

[0073] Text blocks can be isolated from the schematic representations. A text can be linked to an elementary component or an already isolated functional block.

[0074] The text can correspond to the parameters or specifications of a basic component. The text can also correspond to the software used to control the basic component or to control a functional block. The text can also correspond to requirements or calculations associated with an HVAC schematic.

[0075] The text can be extracted from a document 11B containing a schematic representation or from another text or image document 11A, 11B from the set of documents entered by the user. The machine learning system 8 can thus link text and / or images from different documents. For example, the details and / or specifications of a component depicted in one document can be found in another document entered by the user or be known to the system.

[0076] Fig. Figure 7 shows a series of graphic symbols taken from part 110 of the scheme of Fig. can be isolated. Each isolated symbol can be described as a vector. The vector can also specify the position of the graphic symbol on the schematic diagram. The vector can also specify the connections of the graphic symbol to other graphic symbols or external components. The vector can specify the text associated with the graphic symbol.

[0077] Other vectors may be generated to represent texts 115 of the schematic representation or other texts in associated documents along with their position in the documents.

[0078] In step 106, the isolated graphical symbols 113 and texts 115 are input as vectors into the machine learning system 8 to extract functional blocks. The position of these symbols on the circuit diagram can be encoded using the symbols or texts.

[0079] The connections 112 between graphical symbols 113 can be considered as graphical symbols and input into the machine learning system as vectors; alternatively, they can be included in the vector representing the corresponding connected graphical symbols.

[0080] The machine learning system 8 may be a deep neural network using a large document model (LDM) trained on a comprehensive set of documents or automatically generated documents about HVAC systems, including schematic diagrams.

[0081] The machine learning system 8 may comprise a graphic neural network (GNN).

[0082] The machine learning system 8 may comprise a transformer. An example of such a transformer is shown in Fig. 7. It comprises a predictor 200, a norm layer 201, an adder 202, a forward module 203, a second norm layer 204, a second adder 205, introspection heads 207, and a text, image, and position embedding module 207. The invention can use the pure decoder part of the transformer, as used, for example, in GPT-3. Multiple transformer blocks can also be used. The transformer can be used to generate content, such as new diagrams, text, etc. In the inventive method, it is used to detect discrepancies by checking the probability with which it would have generated a verified document or part of a document. If the verified document corresponds to a document that the transformer could have generated, then it is likely that the document does not contain any discrepancies.If the reviewed document does not correspond to a document that could have been created, then it is more likely that it contains discrepancies.

[0083] The machine learning system 8 may include a recurrent neural network (RNN), such as an LSTM network. A recurrent neural network may be used to extract functional blocks of higher hierarchical levels, i.e., functional blocks whose components are themselves functional blocks, as will be described below.

[0084] The machine learning system 8 may include a convolutional neural network (CNN).

[0085] A functional block can contain further smaller functional blocks. Ultimately, a complete HVAC system corresponds to a functional block, which in turn is hierarchically structured from smaller, underlying functional blocks.

[0086] The set of graphical symbols, connections, and text extracted from a document or series of documents relating to the same HVAC system is input into the machine learning system, along with the position of these components on the schematic diagrams. The machine learning system 8 outputs a set of extracted functional blocks.

[0087] Extraction can be viewed as a pattern matching problem in which sets of interconnected graphical symbols and / or texts are compared with known patterns of known functional blocks. The known functional blocks may be stored in a reference database and / or taught to the machine learning system 8 during training.

[0088] The machine learning system 8 thus extracts functional blocks from sets of interconnected graphical symbols and possibly associated text blocks isolated from the schematic representations of an HVAC system.

[0089] The reference database used to train the machine learning system is a collection of functional blocks of existing HVAC systems with hardware and software specifications.

[0090] As already mentioned, a functional block can consist of interconnected elementary components and / or interconnected functional sub-blocks. Each functional sub-block can, in turn, consist of interconnected elementary components and / or interconnected functional sub-blocks. The representation of functional blocks is therefore hierarchical and includes both low-level functional blocks (consisting only of interconnected elementary components) and higher-level functional blocks, i.e., of interconnected sub-blocks. Furthermore, a functional block can consist of any number of elementary components and / or functional sub-blocks. An elementary component or functional sub-block can therefore belong to several higher-level functional blocks, depending on the number of elementary components or functional sub-blocks considered.

[0091] The machine learning system 8 can extract both lower-level and higher-level functional blocks. The extraction of functional blocks at different levels can be performed in a single pass in a neural network trained to extract and classify functional blocks at different hierarchical levels. In one variant, this extraction is performed using an RNN-type network, in which the functional block resulting from an extraction is iteratively fed back to the input of the network to extract a higher-level functional block in a subsequent iteration. In a preferred embodiment, the extraction is performed using an LDM based on transformer blocks that capture a hierarchical internal representation of HVAC systems.

[0092] It is possible that interconnected elementary components do not form a functional block; for example, the component at the output of a central heating component does not form a functional block with the component at the input. In one embodiment, groups of interconnected elementary components or functional sub-blocks at the input of the system 8 that cannot be classified, i.e., that do not correspond to a known functional block, are not considered functional blocks. Alternatively, the system asks the user whether these assemblies should form a new functional block to be stored in the reference database 80 and / or used to train the machine learning system 8.

[0093] Functional blocks can be represented equivalently in different ways. For example, the length and / or direction of the connections between elementary components can differ without changing the functionality of a functional block. The connections can be straight, angled, or curved. Elementary components can be represented by various equivalent graphical symbols. The machine learning system 8 was preferably trained to extract and classify functional blocks even when the connections have different lengths, shapes, or directions, or when the elementary components are represented by different graphical symbols.

[0094] In one variant, the machine learning system 8 classifies the type or function of each functional block, e.g., whether it is a heating system, a duct, a pipe, a furnace, an air conditioner, a boiler, a ventilation system, or another known type of functional block. The classification may depend on the representation of the functional block, the associated text, possible colors, and / or connections between this functional block and other functional blocks in the schematic representation.

[0095] Additionally or alternatively, the machine learning system 8 can be used to classify each functional block and indicate whether or not it contains discrepancies. A functional block at any hierarchy level can be considered discrepancy-free if it is identical or substantially identical to a previously certified functional block. A functional block can be considered deviant if it is identical or substantially identical to a previously certified functional block.

[0096] In step 107, depending on the previous extraction step, a test is performed to verify whether all required documents or information have been provided. The test can be performed using a dedicated software module and / or the machine learning system 8. A checklist of the required documents or information can be used for the test.

[0097] The list of required documents or information depends on the extracted functional blocks. For example, if a specific type of functional block has been extracted, the dedicated software module can check whether a detailed schematic or specifications of all or some of the basic components of that functional block have been provided. The checklist can also include certain types of documents that are always required, such as a computer-implemented schematic of the HVAC system, the building specifications, etc.

[0098] The missing information may also relate to a single elementary component or functional block. For example, a functional block extracted by the machine learning system 8 may require specific information, such as technical specifications in the form of text, calculations, and / or images.

[0099] If test 107 determines that a document or piece of information is missing, the process returns to step 101, and conversational agent 5 prompts the user to provide this additional document or piece of information. The process described above then continues with this new document or piece of information.

[0100] However, if test 107 determines that no additional documents or information are required, the process proceeds to step 108 to determine any discrepancies in the documents submitted.

[0101] In step 108, the discrepancies are identified. As previously mentioned, a functional block may be considered nonconforming if it is identical or substantially identical to a functional block previously identified as nonconforming; alternatively, a functional block may be considered likely nonconforming if it does not conform to a previously certified functional block or if it conforms to a function known to have previously been identified as nonconforming.

[0102] The system can provide an error probability for each functional block and / or for an entire document.

[0103] In one embodiment, discrepancy detection is performed using the machine learning system. In this case, the Large Document Model (LDM) was trained on a comprehensive training dataset of existing HVAC systems, supplemented by synthetic systems and subsystems, to predict every valid connection, component, functional block, and the hierarchical compositions of functional blocks, etc., in a self-supervised learning process. The resulting learned internal hierarchical representation is included in the neural network architecture of the layered transformer blocks. The generative model has learned to generate schematics or other HVAC-relevant documents that are part of the valid training material.Discrepancy detection is then based on measuring the probability that the LDM would have generated an extracted component, connection, functional block, or a combination thereof. If the probability for a particular functional block is low, this means that the extracted blocks are likely not part of a known HVAC system. Detection uses a hierarchical approach that attempts to predict the presence of each elementary component.

[0104] Alternatively or additionally, a separate discrepancy detection software module 9 may be used to analyze the functional blocks extracted by the machine learning system 8 and their interconnections and to detect conforming functional blocks or discrepancies.

[0105] The mismatch detection module 9 or the machine learning system can also analyze the interactions between the functional blocks. For example, the mismatch detection module 9 can determine whether the load transferred between different functional blocks, energy flow rate, etc., is coherent and can be supported by each functional block.

[0106] A functional block is considered compliant, i.e., free of discrepancies, if its behavior is fully controlled from the perspective of both the embedded software and the verified hardware components. Consequently, its behavior is guaranteed to conform with respect to a set of defined parameters and specifications.

[0107] If a description of the building, other specifications, requirements, or calculations related to the HVAC system have been entered, the discrepancy detection module 9 or the machine learning system can determine whether the HVAC system is suitable for the building, requirements, or calculations.

[0108] In step 109, a report is generated by the report generator module 90 and made available to user 2. The report may be generated by the generative machine learning system 6. It may contain text and possibly schematic representations, e.g., a list of extracted functional blocks and associated features. The report also includes a list of inconsistencies identified in the documentation by the inconsistency detection module 9.

[0109] Fig. Figure 3 shows an example of a neural network (in this example, a convolutional neural network, CNN) that can be used in a variant of the machine learning system 8. The neural network receives an input vector 81 representing a set of graphic symbols and text isolated from the user-entered set of documents 11A, 11B, along with their position in the document and / or connections. The network then comprises a convolution layer 82, a ReLU (rectified linear unit) activation function 83, a pooling layer 84, and a fully connected layer 85. It outputs an output vector C1, C2, Ci corresponding to a set of extracted feature blocks and associated text.

[0110] Fig. Figure 4 shows an example of a method using a machine learning system 8 relating to a transformer. A document 11B, which in this example contains an HVAC circuit diagram with associated text, is input by the user. In steps 102-105, the documents are vectorized, preprocessed, and OCR-enhanced; graphic symbols 81B and text 81A are then extracted along with their position within the documents and input to the encoding section 86 of a transformer. The set of vectors 81A, 81B is then processed by the self-attention of the transformer 86, which focuses on neighboring graphic symbols, text, and connections to ensure efficient computation. Block queries are then used by the transformer decoder 87 to recognize entire functional blocks.Finally, the extracted functional blocks 880 are linked according to their connections on the HVAC schematic 11B and a graphical representation 89 of the HVAC system is generated, which is input to the discrepancy detection module 9 to automatically detect design discrepancies against a database of known discrepancies.

[0111] Fig. Figure 6A shows an example of a discrepancy-free functional block derived from the scheme of Fig. 5 can be taken.

[0112] Fig. Figure 6B shows an example of a functional block taken from the circuit diagram of Fig. 5 and which does not correspond to a discrepancy-free function block. Additional features and terminology

[0113] Depending on the embodiment, certain acts, events, or functions of the algorithms described herein may be performed in a different order, added, merged, or omitted entirely (e.g., not all described acts or events are required to perform the methods). Furthermore, in certain embodiments, acts or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors or processor cores, or on other parallel architectures, rather than sequentially. Furthermore, different tasks or processes may be performed by different machines or computing systems that can cooperate with each other.

[0114] The various logical blocks, modules, engines, servers, and algorithm steps described herein may be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability of hardware and software, various components, blocks, modules, computer platforms, engines, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design constraints imposed on the overall system. The described functionality may be implemented in different ways for each particular application, but such implementation choices should not be interpreted as a discrepancy from the scope of the disclosure.

[0115] A server, module, or machine can be based on software and hardware components. The hardware components can be based on any type of computing system, including, but not limited to, a computing system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a physical or virtual server, a cloud-based server, a hybrid system that includes both the processing power of a server and additional processing power in the cloud, or a computing engine in a device, to name a few.

[0116] A software module can consist of one or more software programs or agents executed by a server, a hardware module or a machine to perform various data operations.

[0117] The steps of a method, process, or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module stored in one or more storage devices and executed by one or more processors, or in a combination of both. A software module may be stored in RAM, flash memory, a hard disk, a removable disk, a CD-ROM, or other form of non-transferable, computer-readable storage medium, media, or physical computer memory. An example storage medium may be connected to the processor so that the processor can read and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. The storage medium may be volatile or non-volatile.

[0118] Conditional expressions such as "may," "could," "might," "e.g.," and the like, unless expressly stated otherwise or understood by context, are generally intended to convey that certain embodiments include certain features, entities, or conditions while other embodiments do not include them. Therefore, such conditional language is generally not intended to imply that any features, entities, or conditions are in any way required of one or more embodiments, or that one or more embodiments necessarily include logic for determining, with or without input or prompting from the author, whether to include or implement those features, entities, or conditions in a particular embodiment.The terms "comprising," "including," "having," and the like are synonymous and are used in an open-ended manner that does not exclude additional entities, characteristics, acts, operations, etc. Also, the term "or" is used in its inclusive (rather than exclusive) sense, so that, for example, when used to join a list of entities, the term "or" means one, some, or all of the entities in the list. Moreover, as used herein, the term "each" can, in addition to its ordinary meaning, mean any subset of a set of entities to which the term "each" is applied. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 2017235705 B2

[0012] CN 112464036 A

[0013] CN 116306563 A

[0014] CN 109146419 A

[0015]

Claims

[1] An interactive computer-implemented method for testing an HVAC system comprising: - input of at least one document (11) relating to the HVAC system by a user into a data processing system (4), the documents containing schematic representations of the HVAC system; - in the data processing system (4), extracting a plurality of functional blocks (880) (106) from the schematic representations by a machine learning system (8); - detecting inconsistencies, including missing documents or information, within the data processing system depending on the result of the extraction (108); - a conversation agent (5) in the data processing system interacting with the user to request missing documents or information relating to the HVAC system (107, 101); - Creating a report based on the extraction and detection of discrepancies (109). [2] The method of claim 1, wherein the machine learning system (8) is used to compare the at least one extracted functional block with at least one discrepancy-free functional block. [3] The method of claim 2, wherein the report comprises a list of discrepancy-free functional blocks and a list of functional blocks likely to contain discrepancies. [4] The method according to any one of claims 1 to 3, wherein the extracted functional blocks are compared with a reference database of discrepancy-free functional blocks. [5] The method according to any one of claims 1 to 4, comprising a step of isolating graphic HVAC symbols representing elementary components of the HVAC network and texts from the schematic representations, and inputting said graphic HVAC symbols and texts into the machine learning system (8). [6] The method of claim 5, wherein the position of the graphical HVAC symbols on the circuit diagram is encrypted with the graphical symbol. [7] The method according to any one of claims 1 to 6, wherein the machine learning system (8) comprises a transformer (86-87). [8] The method according to any one of claims 1 to 7, wherein the machine learning system performs an iterative extraction of functional blocks of different hierarchical levels. [9] The method of any one of claims 1 to 8, wherein the machine learning system uses a large document model (LDM), such as a generative pre-trained transformer (GPT). [10] The method according to any one of claims 1 to 9, comprising a step of vectorizing and optical character recognition (OCR) of the schematic representations, wherein vectorized and OCR-captured schematic representations are input into the machine learning system. [11] The method according to any one of claims 1 to 10, wherein the set of documents contains text descriptions and / or calculations relating to the HVAC system, and wherein the method comprises a step of interpreting the text descriptions and / or calculations. [12] The method according to any one of claims 1 to 11, wherein the extracted functional blocks are sent to a discrepancy detection module (9) to detect discrepancies. [13] The method of any one of claims 1 to 12, wherein the documents include a description of a building in which the HVAC system is to be installed, and wherein the data processing system determines whether the HVAC system is suitable for the building. [14] The method of any one of claims 1 to 13, wherein the interactions between the conversational agent and the user are multimodal. [15] The method of any one of claims 1 to 14, wherein the conversational agent is controlled by the LDM understanding of the HVAC system. [16] The method according to any one of claims 1 to 15, wherein the documents relate to hardware and software components of the HVAC system.

Images