HVAC equipment identification method based on multi-information fusion and large model reasoning

Abstract

The present invention discloses a method for identifying HVAC equipment in drawings based on multivariate information fusion and large model reasoning. In order to solve the problem of few equipment training samples and difficult identification in complex cold and heat source drawings, topological relationship analysis and domain knowledge base are introduced to combine shape contours, text annotations, topological associations and domain knowledge bases. Through weighted fusion of multivariate information and iterative confidence update mechanism, accurate identification and classification of HVAC equipment are achieved. The present invention effectively reduces the uncertainty and misjudgment rate caused by excessive reliance on a single feature, significantly improves the accuracy and robustness of automatic identification of HVAC equipment, and is suitable for automatic equipment identification and management of various types of cold and heat source drawings.

Description

Technical Field

[0001] The present invention relates to the field of HVAC digitization, and in particular to a HVAC drawing equipment identification method based on multi-information fusion and large model reasoning. Background Art

[0002] The heat and cold source system diagram is the core drawing of the building HVAC system design. It intuitively presents the equipment type and physical connection relationship, and provides key information for subsequent energy system modeling. However, in the current design process, such drawings are mostly output in raster image format, which is unstructured data and difficult for computers to directly understand and process. The traditional method requires users to manually parse the drawings and convert the equipment configuration and connection information into model input parameters. This is not only time-consuming and labor-intensive, but also highly dependent on professional experience, which is easy to introduce human bias and affect the accuracy and reliability of modeling.

[0003] In the prior art, there are many methods for detecting text or objects from images. For example, Chinese patent document with publication number CN118334634A discloses an image text detection method; Chinese patent document with publication number CN117876711A discloses an image object detection method based on image processing.

[0004] When current automatic recognition technologies identify equipment in drawings, they mostly rely on the target detection network to preliminarily locate the equipment outline, and use text annotation information for classification and confirmation. This strategy that relies solely on visual and text features is insufficient when faced with high-resolution, information-intensive cold and hot source system drawings. In particular, the recognition effect is easily limited for equipment with fewer samples or unclear text annotations. Summary of the invention

[0005] The present invention provides a method and device for HVAC equipment identification based on multivariate information and large model reasoning, which helps to improve the accuracy of identification when labels are missing or information is vague.

[0006] The HVAC equipment identification method based on multivariate information and large model reasoning includes the following steps:

[0007] (1) Perform image preprocessing on the HVAC drawings to be identified, and use the trained object detection model to extract the contour information and location information of various candidate devices from the drawings; for each detected candidate device, output the preliminary classification confidence of the device type, that is, the contour recognition confidence;

[0008] (2) Obtain text information in the drawing through the pre-trained text detection model and text recognition model, and then associate the recognized text information with the candidate device area through the coordinate matching strategy; establish a mapping relationship between mutually close text information and the location information of the corresponding candidate device, and extract the device name, model or marking features from the text information; calculate the text recognition confidence corresponding to each candidate device based on the consistency and correlation between the text information and the expected type of the device;

[0009] (3) According to the known equipment connection relationship in the typical HVAC cycle mode, the upstream and downstream connection relationships of the equipment in the drawing are matched to obtain the first classification confidence of the candidate equipment; by comparing the similarity of the connection relationship between the equipment, the second classification confidence of the candidate equipment is obtained; the first classification confidence and the second classification confidence are weighted and fused to generate a topological classification confidence for each candidate equipment;

[0010] (4) The obtained contour recognition confidence, text recognition confidence, topological classification confidence and information in the domain knowledge base are combined to form the original information base for device identification; the confidence of different information sources is weighted and synthesized to obtain the initial fusion confidence of each candidate device;

[0011] The initial fusion confidence and related information of the candidate device are input into the pre-trained large model, and the device types whose fusion confidence is lower than the set threshold or have uncertainty are iteratively inferred and corrected, the fusion confidence is updated, and the final recognition results and related information of the candidate device are updated to the original information database.

[0012] In step (1), the training data used for pre-training the target detection model is labeled as follows:

[0013] Equipment with standardized contour features (such as water pumps and manifolds) are clearly classified and labeled, and equipment with unclear contour features are clustered and labeled based on contour similarity to form training samples for equipment category groups.

[0014] In step (2), the text detection model and the text recognition model can use the excellent design atlas as training data to fine-tune the pre-trained model.

[0015] In step (3), the formula for weighted fusion of the first classification confidence and the second classification confidence is as follows:

[0016] ;

[0017] in, For equipment The topological classification confidence of is the weighting coefficient, is the domain correction factor, For equipment The typical classification confidence of the connection relationship in the domain knowledge, The overall confidence is the first classification; Indicates the device and The connection weight coefficients are Indicates the device and The similarity of the connection relationship between For equipment The weighted confidence of contour and text classification, ; The overall confidence level is the second classification.

[0018] In step (4), the confidence of different information sources is weighted and synthesized to obtain the initial fusion confidence of each candidate device, and the formula is:

[0019] ;

[0020] in, For equipment The initial fusion weights, Equipment The contour recognition confidence, text recognition confidence, and topological classification confidence of are the fusion weights of the corresponding items respectively.

[0021] In step (4), the reasoning process of the large model is expressed as follows:

[0022] ;

[0023] in, , , is the probability distribution associated with each information source, the large model Combine these probabilities with existing knowledge Perform reasoning integration and output a value for correcting the final confidence .

[0024] In step (4), the formula for updating the fusion confidence is:

[0025] ;

[0026] in, Indicates the device The fusion weight after iteration n-1 times, is the knowledge base formed after n-1 iterations, Indicates that the large model outputs a reasoning confidence correction term based on the knowledge base and current device information. is the fusion weight.

[0027] In step (4), during each iteration, when the fusion confidence of the candidate device reaches the set threshold, the identification information of the device and its related information are updated to the original knowledge base for reference and optimization of the identification of other devices in subsequent drawings.

[0028] Preferably, the threshold is set to 0.85.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] Most existing target recognition methods rely only on equipment contours, which are limited in the recognition of high-resolution, information-intensive cold and heat source system diagrams, especially when equipment samples are lacking and contour features are not obvious. At the same time, traditional methods ignore the comprehensive utilization of potential topological structure logic and domain knowledge in HVAC systems. To this end, the present invention innovatively combines multivariate information (contours, text, topology) with large model knowledge reasoning, and can still achieve high-precision automated equipment recognition under the conditions of unclear contour features and lack of samples, effectively improving the recognition accuracy and robustness of HVAC equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a method for identifying HVAC equipment based on multivariate information and large model reasoning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0033] like Figure 1 As shown in the figure, the HVAC equipment identification method based on multivariate information and large model reasoning mainly includes five steps: model training, equipment identification and confidence calculation based on graphic contours, extraction of text information of the drawings to be identified and calculation of classification confidence, equipment classification confidence calculation based on system topology relationships, and multivariate information fusion and large model knowledge reasoning.

[0034] Step 1: Model training.

[0035] Model training is divided into three parts: equipment detection model, text detection model and text recognition model training. First, the cold and hot source system drawings are collected and standardized, and image data sets of uniform size are generated through grayscale conversion, binarization, cropping and noise reduction.

[0036] Equipment detection model training: Use annotation tools to calibrate the equipment type and location, adopt the YOLO target detection model, and combine the BIFPN and ECA modules for training to improve equipment detection accuracy.

[0037] Text detection model training: Label the text area of ​​the drawing, use the differentiable binary method (DB) to train the text detection model, and use pre-trained parameters for initialization to ensure efficient and accurate text area recognition.

[0038] Text recognition model training: organize common characters, build a special character library for HVAC design, use the SVTR_HGNet model for training, and combine the field-specific character library to optimize text recognition accuracy.

[0039] For more detailed steps of model training, please refer to 1.1-1.7. The specific steps are as follows:

[0040] 1.1. The images of the cold and hot source drawing set are first converted into grayscale and binarized, and then the original image dataset with a size of 2560×2560 is obtained by cropping, rotating, and denoising the images.

[0041] 1.2. Use annotation tools to annotate the equipment in the images of the dataset, and annotate the equipment type and coordinates respectively. The equipment types should include: cold and hot source equipment, water pumps, cooling towers, manifolds / collectors, heat exchangers, auxiliary equipment, etc. The coordinates should be the coordinates of the upper left vertex and lower right vertex of the smallest rectangle containing the equipment.

[0042] 1.3. Build a target detection network based on the bidirectional feature pyramid network (BIFPN) and the efficient channel attention module (ECA) to fuse the YOLO model to enhance the feature extraction capability of small targets.

[0043] 1.4. Randomly divide the data set into training set and validation set according to a certain ratio, use stochastic gradient descent (SGD) to optimize the hyperparameters, complete model training and save the optimal weights.

[0044] 1.5. Refer to step 1.1 to obtain the original image dataset with a size of 1280×1280, and then use the annotation tool to perform text detection and text recognition annotation on the image dataset, including the coordinates of the text content and the upper left vertex and lower right vertex of the smallest rectangle containing the text content.

[0045] 1.6. Organize excellent design documents and design specifications, and use semantic segmentation methods to generate a training corpus; at the same time, count the characters in the design documents to form a special character library suitable for the HVAC design field.

[0046] 1.7. Use the differentiable binary method (DB) and the scene text recognition algorithm based on the variable-length single vision model (SVTR_HGNet) as the training models for text detection and text recognition respectively. Initialize the model with the pre-trained model parameters of the PaddlePaddle platform, and use the character library generated in 1.6 to optimize the recognition effect. Complete the model training and save it according to the method in step 1.4.

[0047] Step 2: Device identification and confidence calculation based on graphic outline.

[0048] The HVAC drawings to be identified are preprocessed, and the contour information and location information of various candidate devices are extracted from the drawings using the trained object detection algorithm. For each detected candidate device, the detection results output by the model are used to assign a preliminary classification confidence of the device type (i.e., contour recognition confidence).

[0049] Step 3: Extract the text information of the drawing to be identified and calculate the classification confidence.

[0050] First, the text information in the drawing is obtained through text detection and text recognition technology, and then the recognized text is associated with the candidate device area through the coordinate matching strategy. In this process, a mapping relationship can be established between the text annotation information that is close to each other and the corresponding device position, from which the device name, model or marking features can be extracted. According to the consistency and relevance of the text content and the expected type of the device, the text recognition confidence is assigned to the device.

[0051] Step 4: Calculate the confidence level of device classification based on system topology.

[0052] 4.1 Cover the devices and text areas extracted in steps 2 and 3 with white rectangles to retain drawings containing only the device connection segments. Then perform dilation and erosion operations on the processed drawings to effectively remove redundant noise points. Use the line extraction algorithm to identify the horizontal and vertical line segments in the drawings, and determine the relationship between the line segments (such as broken line points and three-way nodes) based on predefined rules. Finally, the line segment endpoints are associated with the device locations through the coordinate matching strategy to reconstruct the complete topological structure between the devices.

[0053] 4.2 First, according to the known equipment connection relationship in the typical HVAC cycle mode (such as cooling water cycle), the upstream and downstream connection patterns of the equipment in the drawing are matched. When a candidate device meets the specific connection characteristics in the typical cycle, it can be given a higher topology recognition confidence, that is, the first classification confidence is obtained. Secondly, by comparing the similarity of the connection relationship between devices, if two or more devices are highly similar in the type and number of upstream and downstream devices connected, the topological classification confidence of the device can be further improved, and it can be inferred that it belongs to the same type as the reference device, that is, the second classification confidence is obtained. By weighted fusion of the confidence obtained from typical cycle matching and connection similarity analysis, a topological classification confidence is generated for each device.

[0054] Step 5: Multi-information fusion and large model knowledge reasoning.

[0055] The contour recognition confidence, text information recognition confidence, topological classification confidence and information in the domain knowledge base obtained in the previous steps are combined to form the original information base for device recognition. According to the preset confidence fusion and update formula, the confidence of different information sources is weighted to synthesize the initial fusion confidence of the device.

[0056] After obtaining the initial fusion confidence, the candidate devices and their corresponding information (such as confirmed device types, global topology and industry knowledge) are input into the prompt word instruction generator, and prompt word instructions for the current situation are generated according to different preset templates. The prompt word instructions are then used to guide the pre-trained large model to perform high-level reasoning and decision correction. The large model uses logical analysis and knowledge reasoning capabilities to iteratively evaluate and correct device types with low confidence or uncertainty. When a device reaches the credibility threshold after the large model reasoning update, its final recognition result and related information are updated to the knowledge base for reference and optimization of recognition of other devices in subsequent drawings. Through this iterative and closed-loop enhancement mechanism, the recognition accuracy and robustness of the entire system are gradually improved.

[0057] In order to further demonstrate the advantages of the above-mentioned HVAC drawing equipment identification method based on multi-information fusion and large model reasoning of the present invention, this embodiment will conduct comparative experiments with traditional methods on public data sets to demonstrate its technical effects.

[0058] In this implementation, the HVAC Engineering Excellent Design Atlas (Volumes 4-6) compiled by the HVAC Branch of the Architectural Society of China is used as a public data set. After sorting, a total of 230 pictures are available. After preprocessing such as binarization and rotation, the LABELIMG software is used to generate the label of each drawing, including the equipment category and the coordinates of the upper left vertex and lower right vertex of the corresponding smallest rectangle containing the equipment, so as to obtain the processed training data. The training data is divided into training set, validation set and test set in a ratio of 7:1:2, of which the training set consists of 161 drawings, which meets the experimental conditions of fewer training samples.

[0059] In this embodiment, in order to compare and verify the method, a traditional target detection method based only on contour features is selected as a reference, and the representative one-stage target detection algorithm YOLOV8 and the two-stage target detection algorithm FAST-RCNN are selected. At the same time, in order to examine the impact of topology and domain knowledge on detection performance, an ablation experiment for the classification of contour and text information is also designed. Through the above settings, a total of three groups of control experiments are formed.

[0060] In this embodiment, common target detection and recognition model evaluation criteria are used, including detection precision (Precision, P), classification accuracy (Accuracy, ACC), and mean average precision (MAP). The closer the values ​​of P and ACC are to 100%, the more accurate the results of device detection and device classification are, respectively, and the closer the value of MAP is to 1, the better the overall performance of the model. Table 1 shows the experimental results of P, ACC, and MAP of different methods under this data. It can be seen from the experimental results that under the condition of relatively few training samples, traditional detection methods that only rely on contour features (such as FAST-RCNN and YOLOV8) are difficult to accurately identify non-standardized device contours. Mainly due to the limited number of samples, this type of model cannot fully learn complex contour features, especially for devices with non-fixed device contours, resulting in low detection accuracy (P) and classification accuracy (ACC).

[0061] Table 1

[0062]

[0063] The method of the present invention does not perform overly detailed specific classification of each device outline during training, but determines the device location and category in a more flexible way through multi-information fusion (outline, text, topological association and domain knowledge) and large model reasoning. The results show that the device classification accuracy ACC of this method can be greatly improved from 59.85% of FAST-RCNN and 56.65% of YOLOV8 to 87.35%. At the same time, compared with the ablation experiment that only adds text information, when topological association and domain knowledge reasoning are further introduced, the classification accuracy ACC is increased from 79.39% to 87.35%. This proves that topological information and domain knowledge have a significant reinforcing effect on device classification in complex scenarios.

[0064] Overall, the MAP index of the proposed method reaches 0.81, which is significantly better than the traditional algorithm and ablation scheme, fully demonstrating the effectiveness of the strategy of integrating multivariate information and large model knowledge reasoning in improving the recognition accuracy and robustness of HVAC equipment.

[0065] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying HVAC equipment in drawings based on multivariate information fusion and large model reasoning, characterized in that: The following steps are involved: (1) Perform image preprocessing on the HVAC drawings to be identified, and use the trained object detection model to extract the contour information and location information of various candidate devices from the drawings; for each detected candidate device, output the preliminary classification confidence of the device type, that is, the contour recognition confidence; (2) Obtain text information in the drawing through the pre-trained text detection model and text recognition model, and then associate the recognized text information with the candidate device area through the coordinate matching strategy; Establish a mapping relationship between mutually adjacent text information and location information of corresponding candidate devices, and extract the device name, model or marking feature from the text information; Calculate the text recognition confidence of each candidate device based on the consistency and relevance between the text information and the expected type of the device; (3) According to the known equipment connection relationship in the typical HVAC cycle mode, match the upstream and downstream connection relationship of the equipment in the drawing to obtain the first classification confidence of the candidate equipment; By comparing the similarity of the connection relationship between devices, the second classification confidence of the candidate device is obtained; the first classification confidence and the second classification confidence are weighted and fused to generate a topology classification confidence for each candidate device; (4) The obtained contour recognition confidence, text recognition confidence, topological classification confidence and information in the domain knowledge base are combined to form the original information base for device identification; the confidence of different information sources is weighted and synthesized to obtain the initial fusion confidence of each candidate device; The initial fusion confidence and related information of the candidate device are input into the pre-trained large model, and the device types whose fusion confidence is lower than the set threshold or have uncertainty are iteratively inferred and corrected, the fusion confidence is updated, and the final recognition results and related information of the candidate device are updated to the original information database.

2. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 1 is characterized in that: In step (1), the training data used for pre-training the target detection model is labeled as follows: Devices with standardized profile features are clearly classified and labeled, and devices with unclear profile features are clustered and labeled based on profile similarity to form training samples for device category groups.

3. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 1 is characterized in that: In step (3), the formula for weighted fusion of the first classification confidence and the second classification confidence is as follows: ； in, For equipment The topological classification confidence of is the weighting coefficient, is the domain correction factor, For equipment The typical classification confidence of the connection relationship in the domain knowledge, The overall confidence is the first classification; Indicates the device and The connection weight coefficients are Indicates the device and The similarity of the connection relationship between For equipment The weighted confidence of contour and text classification, ; The overall confidence level is the second classification.

4. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 1 is characterized in that: In step (4), the confidence of different information sources is weighted and synthesized to obtain the initial fusion confidence of each candidate device, and the formula is: ； in, For equipment The initial fusion weights, Equipment The contour recognition confidence, text recognition confidence, and topological classification confidence of are the fusion weights of the corresponding items respectively.

5. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 4 is characterized in that: In step (4), the reasoning process of the large model is expressed as follows: ； in, , , is the probability distribution associated with each information source, the large model Combine these probabilities with existing knowledge Perform reasoning integration and output a value for correcting the final confidence .

6. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 5 is characterized in that: In step (4), the formula for updating the fusion confidence is: ； in, Indicates the device The fusion weight after iteration n-1 times, is the knowledge base formed after n-1 iterations, Indicates that the large model outputs a reasoning confidence correction term based on the knowledge base and current device information. is the fusion weight.

7. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 1 is characterized in that: In step (4), during each iteration, when the fusion confidence of the candidate device reaches the set threshold, the identification information of the device and its related information are updated to the original knowledge base for reference and optimization of the identification of other devices in subsequent drawings.

8. The HVAC equipment identification method based on multivariate information fusion and large model reasoning according to claim 7 is characterized in that: Set the threshold to 0.85.

Images