A method and system for image recognition modeling

Abstract

The application provides a kind of picture recognition modeling method and system, the method comprises: first, construct standard equipment dataset, equipment template library and RAG (retrieval enhancement generation) knowledge base;When receiving the CAD equipment image input by user, the image is coarsely recognized using the classification model pre-trained based on the standard equipment dataset, to determine the corresponding equipment type;According to equipment type, the template subset of corresponding type is called from equipment template library, and the CAD equipment image is finely matched with the subset to determine the specific equipment model;Then, based on the determined equipment model, CAD drawing attribute and equipment block attribute, the matched BIM modeling scheme is retrieved through RAG knowledge base;Finally, the corresponding BIM model is automatically generated according to the retrieved modeling scheme.The application realizes accurate analysis and automatic modeling of equipment information in CAD drawing through three-level collaborative mechanism of classification-matching-retrieval, significantly improves modeling efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of drawing-aided modeling technology, and in particular to a drawing recognition modeling method and system. Background Technology

[0002] In engineering fields such as power and chemical engineering, engineering drawings are core technical data for project lifecycle management, and drawing interpretation and modeling methods are key technologies for realizing the digital transformation of drawings and 3D modeling. As engineering scale and complexity increase, the types of equipment in drawings become increasingly diverse. Drawing interpretation and modeling require accurate identification of various equipment, and the accuracy and efficiency of this identification directly determine the quality of the modeling. Therefore, accurate and efficient equipment identification technology for drawing interpretation and modeling has become an important guarantee for promoting the digital transformation of engineering projects.

[0003] Currently, the equipment identification stage in image recognition modeling generally adopts the traditional approach of full-database matching. Its core logic is: pre-construct a database of all equipment features; during identification, extract equipment features from the drawings; then compare these features one by one with all features in the database; and determine the equipment category through similarity calculation. This method, due to its simplicity and ease of implementation, has seen initial application in early low-complexity image recognition modeling scenarios.

[0004] Furthermore, the drawbacks of traditional full-database matching methods become increasingly apparent under the demands of large-scale, highly complex engineering projects requiring map reading and modeling. On the one hand, the expansion of the database leads to a surge in the number of comparisons, causing a sharp increase in computational costs and making it impossible to meet the needs of rapid modeling. On the other hand, some different types of equipment (such as outgoing GIS and some busbar equipment) have similar appearances, making it difficult to accurately distinguish them using a rough comparison method, which easily leads to mismatches. This can cause distortion of modeling data, affecting subsequent design and maintenance work, and even triggering engineering quality and safety risks. Summary of the Invention

[0005] Based on this, the purpose of the present invention is to provide a method and system for image recognition and modeling, so as to solve the problem that the existing technology of full-database matching is difficult to meet the needs of rapid modeling, and at the same time, it is difficult to achieve accurate device differentiation, which reduces the modeling efficiency.

[0006] The first aspect of the present invention proposes:

[0007] A method for image recognition and modeling, wherein the method includes:

[0008] Construct a standard equipment dataset, an equipment template library, and a RAG knowledge base; wherein, the standard equipment dataset is used to train the equipment classification model, the equipment template library stores standardized model templates organized by equipment type, and the RAG knowledge base stores structured BIM modeling schemes associated with modeling rules;

[0009] In response to receiving a CAD device image input by the user, a classification model pre-trained based on the standard device dataset is used to perform coarse-grained recognition on the CAD device image to determine the device type corresponding to the CAD device image;

[0010] Based on the equipment type, a template subset of the corresponding type is retrieved from the equipment template library, and the CAD equipment image is matched with the template subset in a fine-grained manner to determine the specific equipment model;

[0011] Based on the equipment model, CAD drawing attributes, and equipment block attributes, a matching BIM modeling scheme is retrieved from the RAG knowledge base;

[0012] Based on the BIM modeling scheme, the corresponding BIM model is automatically generated.

[0013] The beneficial effects of this invention are as follows: This technical solution constructs a standard equipment dataset, an equipment template library, and a RAG knowledge base. First, it performs coarse-grained recognition on CAD equipment images based on the dataset to lock the equipment type. Then, it calls similar templates to determine the model. Combined with the RAG knowledge base, it parses the modeling information and generates a BIM model. This effectively avoids redundant calculations in full-database matching, significantly shortens the matching time, improves the accuracy of equipment differentiation, and ensures the rationality of the modeling scheme, thereby significantly improving the generation efficiency and accuracy of the BIM model.

[0014] Furthermore, the steps for constructing the standard device dataset, device template library, and RAG knowledge base include:

[0015] Import CAD floor plan, iterate through all components, label equipment categories, and export equipment images to form an equipment dataset; at the same time, use random flipping, random rotation, random scaling, random grayscale, and random HSV channel adjustment methods to enhance the images and improve training accuracy.

[0016] Standard CAD equipment images are selected from the equipment dataset for all equipment types as templates to form an equipment template set. At the same time, ORB feature extraction and descriptor generation processes are performed on the equipment template set in batches to obtain descriptor subsets and point sets, and then divided into multiple template subsets according to equipment type.

[0017] Collect GIM modeling specifications, model library documents, and GIM models, and embed them into the RAG knowledge base.

[0018] Furthermore, the steps of retrieving a template subset of the corresponding type from the equipment template library and performing fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model include:

[0019] When the CAD device image is detected, an HSV color space adapted to the CAD device image is created;

[0020] Within the HSV color space, the S channel is used as the main correction channel, and gamma nonlinear mapping is introduced simultaneously. The pixel values ​​of the V channel are superimposed on the S′ channel in a weighted manner, and an enhanced image is constructed based on the first preset algorithm.

[0021] The enhanced image is subjected to feature extraction processing to determine the corresponding device model.

[0022] Furthermore, the step of performing feature extraction processing on the enhanced image to determine the corresponding device model includes:

[0023] The enhanced image is subjected to ORB feature extraction and descriptor generation processes to obtain a descriptor subset and a point set;

[0024] The FLAN nearest neighbor matching algorithm is used to match the descriptive subset of the enhanced image with the descriptive subset of the template subset to construct the corresponding original matching point set.

[0025] The original matching point set is filtered using the Lowe ratio algorithm to generate a target matching point set. The optimal affine transformation matrix corresponding to the target matching point set is then solved using a second preset algorithm. Simultaneously, the model corresponding to the template with the highest number of matching points in the optimal affine transformation matrix is ​​taken as the device model.

[0026] Furthermore, the expression for the first preset algorithm is:

[0027]

[0028] Where S is the main correction channel, γ is the correction parameter, S′ is the enhancement saturation channel, α and β are the stacking ratio coefficients, and O is the enhanced image.

[0029] Furthermore, the expression for the second preset algorithm is:

[0030]

[0031] Where, p i q represents the matching point of the input image in the target matching point set. i A represents the matching points of the template image in the target matching point set, where m is the number of matching points in the target matching point set, and A is the number of matching points in the target matching point set. j Let be the affine transformation matrix to be found. , It is a linear transformation matrix (including rotation, scaling, and shearing). It is a translation vector.

[0032] Furthermore, the step of automatically generating the corresponding BIM model based on the BIM modeling scheme includes:

[0033] Analyze the modeling scheme and simultaneously reconstruct the corresponding 3D model;

[0034] The three-dimensional model is projected to generate a corresponding template top view, and the pose transformation matrix between the user-input CAD device drawing and the template top view is calculated simultaneously.

[0035] The pose of the 3D model is dynamically adjusted according to the pose transformation matrix to generate the corresponding BIM model.

[0036] The second aspect of the present invention proposes:

[0037] An image recognition and modeling system, wherein the system comprises:

[0038] The resource construction module is used to build a standard equipment dataset, an equipment template library, and a RAG knowledge base. The standard equipment dataset is used to train the equipment classification model, the equipment template library stores standardized model templates organized by equipment type, and the RAG knowledge base stores structured BIM modeling schemes associated with modeling rules.

[0039] The type recognition module is used to respond to the received CAD equipment image input by the user, and to perform coarse-grained recognition of the CAD equipment image using a classification model pre-trained based on the standard equipment dataset, so as to determine the equipment type corresponding to the CAD equipment image;

[0040] The model matching module is used to retrieve a template subset of the corresponding type from the equipment template library according to the equipment type, and perform fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model;

[0041] The scheme generation module is used to retrieve matching BIM modeling schemes from the RAG knowledge base based on the equipment model, CAD drawing attributes, and equipment block attributes.

[0042] The model building module is used to automatically generate the corresponding BIM model based on the BIM modeling scheme.

[0043] Furthermore, the steps for constructing the standard device dataset, device template library, and RAG knowledge base include:

[0044] Import CAD floor plan, iterate through all components, label equipment categories, and export equipment images to form an equipment dataset; at the same time, use random flipping, random rotation, random scaling, random grayscale, and random HSV channel adjustment methods to enhance the images and improve training accuracy.

[0045] Standard CAD equipment images are selected from the equipment dataset for all equipment types as templates to form an equipment template set. At the same time, ORB feature extraction and descriptor generation processes are performed on the equipment template set in batches to obtain descriptor subsets and point sets, and then divided into multiple template subsets according to equipment type.

[0046] Collect GIM modeling specifications, model library documents, and GIM models, and embed them into the RAG knowledge base.

[0047] Furthermore, the steps of retrieving a template subset of the corresponding type from the equipment template library and performing fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model include:

[0048] When the CAD device image is detected, an HSV color space adapted to the CAD device image is created;

[0049] Within the HSV color space, the S channel is used as the main correction channel, and gamma nonlinear mapping is introduced simultaneously. The pixel values ​​of the V channel are superimposed on the S′ channel in a weighted manner, and an enhanced image is constructed based on the first preset algorithm.

[0050] The enhanced image is subjected to feature extraction processing to determine the corresponding device model.

[0051] Furthermore, the step of performing feature extraction processing on the enhanced image to determine the corresponding device model includes:

[0052] The enhanced image is subjected to ORB feature extraction and descriptor generation processes to obtain a descriptor subset and a point set;

[0053] The FLAN nearest neighbor matching algorithm is used to match the descriptive subset of the enhanced image with the descriptive subset of the template subset to construct the corresponding original matching point set.

[0054] The original matching point set is filtered using the Lowe ratio algorithm to generate a target matching point set. The optimal affine transformation matrix corresponding to the target matching point set is then solved using a second preset algorithm. Simultaneously, the model corresponding to the template with the highest number of matching points in the optimal affine transformation matrix is ​​taken as the device model.

[0055] Furthermore, the expression for the first preset algorithm is:

[0056]

[0057] Where S is the main correction channel, γ is the correction parameter, S′ is the enhancement saturation channel, α and β are the stacking ratio coefficients, and O is the enhanced image.

[0058] Furthermore, the expression for the second preset algorithm is:

[0059]

[0060] Where, p i q represents the matching point of the input image in the target matching point set. i A represents the matching points of the template image in the target matching point set, where m is the number of matching points in the target matching point set, and A is the number of matching points in the target matching point set. j Let be the affine transformation matrix to be found. , It is a linear transformation matrix (including rotation, scaling, and shearing). It is a translation vector.

[0061] Furthermore, the step of automatically generating the corresponding BIM model based on the BIM modeling scheme includes:

[0062] Analyze the modeling scheme and simultaneously reconstruct the corresponding 3D model;

[0063] The three-dimensional model is projected to generate a corresponding template top view, and the pose transformation matrix between the user-input CAD device drawing and the template top view is calculated simultaneously.

[0064] The pose of the 3D model is dynamically adjusted according to the pose transformation matrix to generate the corresponding BIM model.

[0065] The third aspect of the present invention proposes:

[0066] A computer includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the image recognition and modeling method as described above.

[0067] The fourth aspect of the present invention proposes:

[0068] A readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the image recognition and modeling method as described above.

[0069] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0070] Figure 1 A flowchart of the image recognition and modeling method provided in the first embodiment of the present invention;

[0071] Figure 2 This is a schematic diagram of the data preprocessing topology provided in the second embodiment of the present invention;

[0072] Figure 3 This is a flowchart of the image recognition and modeling algorithm provided in the second embodiment of the present invention;

[0073] Figure 4 This is a type identification topology diagram provided in the second embodiment of the present invention;

[0074] Figure 5 This is a model identification topology diagram provided in the second embodiment of the present invention;

[0075] Figure 6 AI-generated topology graph provided in the second embodiment of the present invention;

[0076] Figure 7 This is an image showing the enhanced template image provided in the second embodiment of the present invention.

[0077] Figure 8 This is a structural block diagram of the image recognition and modeling system provided in the third embodiment of the present invention.

[0078] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0079] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0080] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0081] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0082] Please see Figure 1 The image shown is a method for image recognition and modeling provided in the first embodiment of the present invention. The image recognition and modeling method provided in this embodiment can comprehensively analyze the information contained in the input image, thereby accurately generating modeling information and accurately completing the construction of the model.

[0083] Specifically, this embodiment provides:

[0084] A method for image recognition and modeling, wherein the method includes:

[0085] Step S10: Construct a standard equipment dataset, an equipment template library, and a RAG knowledge base;

[0086] Step S20: In response to receiving a CAD device image input by the user, a classification model pre-trained based on the standard device dataset is used to perform coarse-grained recognition on the CAD device image to determine the device type corresponding to the CAD device image;

[0087] Step S30: Based on the equipment type, retrieve a template subset of the corresponding type from the equipment template library, and perform fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model;

[0088] Step S40: Based on the equipment model, CAD drawing attributes, and equipment block attributes, retrieve a matching BIM modeling scheme through the RAG knowledge base;

[0089] Step S50: Based on the BIM modeling scheme, automatically generate the corresponding BIM model.

[0090] Second Embodiment

[0091] Furthermore, the steps for constructing the standard device dataset, device template library, and RAG knowledge base include:

[0092] Import CAD floor plan, iterate through all components, label equipment categories, and export equipment images to form an equipment dataset; at the same time, use random flipping, random rotation, random scaling, random grayscale, and random HSV channel adjustment methods to enhance the images and improve training accuracy.

[0093] Standard CAD equipment images are selected from the equipment dataset for all equipment types as templates to form an equipment template set. At the same time, ORB feature extraction and descriptor generation processes are performed on the equipment template set in batches to obtain descriptor subsets and point sets, and then divided into multiple template subsets according to equipment type.

[0094] Collect GIM modeling specifications, model library documents, and GIM models, and embed them into the RAG knowledge base.

[0095] Furthermore, the steps of retrieving a template subset of the corresponding type from the equipment template library and performing fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model include:

[0096] When the CAD device image is detected, an HSV color space adapted to the CAD device image is created;

[0097] Within the HSV color space, the S channel is used as the main correction channel, and gamma nonlinear mapping is introduced simultaneously. The pixel values ​​of the V channel are superimposed on the S′ channel in a weighted manner, and an enhanced image is constructed based on the first preset algorithm.

[0098] The enhanced image is subjected to feature extraction processing to determine the corresponding device model.

[0099] Furthermore, the step of performing feature extraction processing on the enhanced image to determine the corresponding device model includes:

[0100] The enhanced image is subjected to ORB feature extraction and descriptor generation processes to obtain a descriptor subset and a point set;

[0101] The FLAN nearest neighbor matching algorithm is used to match the descriptive subset of the enhanced image with the descriptive subset of the template subset to construct the corresponding original matching point set.

[0102] The original matching point set is filtered using the Lowe ratio algorithm to generate a target matching point set. The optimal affine transformation matrix corresponding to the target matching point set is then solved using a second preset algorithm. Simultaneously, the model corresponding to the template with the highest number of matching points in the optimal affine transformation matrix is ​​taken as the device model.

[0103] Furthermore, the expression for the first preset algorithm is:

[0104]

[0105] Where S is the main correction channel, γ is the correction parameter, S′ is the enhancement saturation channel, α and β are the stacking ratio coefficients, and O is the enhanced image.

[0106] Furthermore, the expression for the second preset algorithm is:

[0107]

[0108] Where, p i q represents the matching point of the input image in the target matching point set. i A represents the matching points of the template image in the target matching point set, where m is the number of matching points in the target matching point set, and A is the number of matching points in the target matching point set. j Let be the affine transformation matrix to be found. , It is a linear transformation matrix (including rotation, scaling, and shearing). It is a translation vector.

[0109] Furthermore, the step of automatically generating the corresponding BIM model based on the BIM modeling scheme includes:

[0110] Analyze the modeling scheme and simultaneously reconstruct the corresponding 3D model;

[0111] The three-dimensional model is projected to generate a corresponding template top view, and the pose transformation matrix between the user-input CAD device drawing and the template top view is calculated simultaneously.

[0112] The pose of the 3D model is dynamically adjusted according to the pose transformation matrix to generate the corresponding BIM model.

[0113] In addition, it should be noted that the image recognition and modeling method provided in this embodiment is specifically implemented by the following steps:

[0114] S1: Data Preprocessing: Constructing a standard equipment dataset, an equipment template library, and a RAG knowledge base. The process is as follows: Figure 2 As shown.

[0115] S1-1: The steps for building a dataset include dataset creation and data augmentation:

[0116] Dataset creation: Import CAD floor plan, iterate through all components, label equipment categories, and export the equipment image set (IS). I represents the device image, and N represents the total number of devices.

[0117] Dataset augmentation: Augmentation methods include random flipping, random rotation, random scaling, random grayscale, and random HSV channel adjustment.

[0118] S1-2: The steps for building the equipment template library include template selection, template detection, and data import.

[0119] Template filtering: Please refer to Figure 3Standard CAD device images are selected from the image set IS as templates for all classes to form a paired template set TPS. , Let Z be the template image for the k-th category, where K is the total number of categories and Z is the total number of templates. The number of templates for each device category is also shown. ≥1, Category template subset .

[0120] Template detection: The template set TPS is detected using the detection algorithm in S3 to obtain the template keypoint set. and descriptor subsets kps and des are the set of key points and the descriptive subset of a single template, respectively.

[0121] Data import: Construct a JSON dataset containing fields such as model name, key points (KPS), and descriptor (DES). Compile all device-related information from the TPS into a JSON collection. Save to the database.

[0122] S1-3: Building a RAG knowledge base: Collect design files and GIM models such as GIM modeling specifications and model library documents, and embed them into the knowledge base.

[0123] S2: The first step, identification, specifically type identification, is used to perform coarse-grained identification on the input CAD equipment image to determine the equipment type. This includes model building, pre-training, and inference. For details, please refer to [link to relevant documentation]. Figure 4 .

[0124] S2-1: Model Construction: The model is a deep neural network classification model, which can be a convolutional neural network structure such as ResNet or EfficientNet, and the output is the probability distribution of the predefined category to which the device belongs.

[0125] S2-2: Pre-training: The classification model is trained end-to-end using the dataset constructed in S1. The cross-entropy loss function is used to optimize the model parameters, enabling the model to accurately predict the device category of the input image. After completion, the pre-trained model is obtained. If no new categories are added subsequently, there is no need to repeat the training.

[0126] S2-3: Reasoning: The CAD image to be identified... Input the pre-trained classification model to obtain the output class probability distribution:

[0127]

[0128] in The predicted device category is used for subsequent matching range constraints.

[0129] S3: Second-stage identification, specifically, fine-grained matching of similar templates, mainly including image enhancement, key point detection and descriptor generation, and key point matching (see...). Figure 5 Specifically:

[0130] S3-1: Image enhancement is mainly used to improve feature point detection performance. Some images have light colors, resulting in less obvious features after grayscale transformation, such as... Figure 7 As shown. Common binarization algorithms easily destroy connected components and introduce noise. Therefore, this method uses the S channel as the primary correction channel in the HSV color space, introducing gamma nonlinear mapping. Simultaneously, the pixel values ​​of the V channel are weighted and superimposed onto S′ to construct an enhanced saturation channel. The mathematical calculations are as follows:

[0131]

[0132] In the formula This is the correction parameter, with a default value of 0.35. and This is the overlay ratio, which defaults to 0.5. O represents the output image. O preserves the original color information while achieving high fidelity through brightness compensation.

[0133] S3-2: In predicting categories Corresponding template subset In this process, ORB feature extraction and descriptor generation are performed on the image to be recognized and each template to obtain a descriptor set and a point set. .

[0134] S3-3: Key points are matched using the FLANNN nearest neighbor matching algorithm to obtain the original set of matching points. , the point set includes the optimal matching point and suboptimal matching point .

[0135] First, use the lowe ratio to filter and retain those that meet the criteria. The point, among which Here, is the proportional threshold, and Distance is the distance function; Hamming distance is used here.

[0136] Then, the optimal affine transformation matrix is ​​solved using the least squares method. , so that:

[0137]

[0138] In the formula Let m be the key point in the template point set. Number of templates, among which , It is a linear transformation matrix (including rotation, scaling, and shearing). It is a translation vector.

[0139] Select the template with the highest number of matching points in the affine transformation. As the best match, and based on obtaining the corresponding device name .

[0140] S4: AI-generated modeling schemes primarily utilize an AI agent to automatically schedule knowledge base retrieval and related auxiliary tools to complete GIM model information extraction and modeling scheme assembly. Specific steps include prompt word template filling, RAG knowledge base retrieval, and modeling scheme generation. For details, please refer to [link to relevant documentation]. Figure 6 .

[0141] S4-1: Prompt word template filling: Fill in the device name The drawing attributes and equipment block tree structure are used as variables to replace the placeholders in the prompt message. The prompt message template is as follows:

[0142] Find the GIM model for {equipment type}, adjust the model parameters, and output the corresponding parametric modeling scheme.

[0143] The relevant attributes of the device are:

[0144] {Device Block Attributes}

[0145] {Drawing Attributes};

[0146] S4-2: RAG Knowledge Base Retrieval: Obtaining knowledge through a semantic + bm25 hybrid retrieval mode. The corresponding actual modeling information.

[0147] S4-3: Modeling Scheme Generation: Decompose the GIM model into a tree-like JSON structure and assemble it into a modeling scheme. .

[0148] S5: BIM modeling mainly includes three steps: equipment modeling, pose estimation, and equipment layout.

[0149] S5-1: Equipment Modeling: Analytical Modeling Scheme And reconstruct the GIM model .

[0150] S5-2: Pose Estimation: Model Project the template to obtain a top view, and calculate the pose transformation matrix between the template and the top view. .

[0151] S5-3: Equipment Layout: Based on the pose transformation matrix of the top view and the template And the pose transformation matrix between the device image and the template in S3 Adjust the model's pose. The adjusted model. Compared with the original model The following relationship must be satisfied:

[0152]

[0153] Once the adjustments are complete, it can be placed at the original coordinates of the equipment.

[0154] Please see Figure 8 The third embodiment of the present invention provides:

[0155] An image recognition and modeling system, wherein the system comprises:

[0156] The resource construction module is used to build a standard equipment dataset, an equipment template library, and a RAG knowledge base. The standard equipment dataset is used to train the equipment classification model, the equipment template library stores standardized model templates organized by equipment type, and the RAG knowledge base stores structured BIM modeling schemes associated with modeling rules.

[0157] The type recognition module is used to respond to the received CAD equipment image input by the user, and to perform coarse-grained recognition of the CAD equipment image using a classification model pre-trained based on the standard equipment dataset, so as to determine the equipment type corresponding to the CAD equipment image;

[0158] The model matching module is used to retrieve a template subset of the corresponding type from the equipment template library according to the equipment type, and perform fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model;

[0159] The scheme generation module is used to retrieve matching BIM modeling schemes from the RAG knowledge base based on the equipment model, CAD drawing attributes, and equipment block attributes.

[0160] The model building module is used to automatically generate the corresponding BIM model based on the BIM modeling scheme.

[0161] Furthermore, the steps for constructing the standard device dataset, device template library, and RAG knowledge base include:

[0162] Import CAD floor plan, iterate through all components, label equipment categories, and export equipment images to form an equipment dataset; at the same time, use random flipping, random rotation, random scaling, random grayscale, and random HSV channel adjustment methods to enhance the images and improve training accuracy.

[0163] Standard CAD equipment images are selected from the equipment dataset for all equipment types as templates to form an equipment template set. At the same time, ORB feature extraction and descriptor generation processes are performed on the equipment template set in batches to obtain descriptor subsets and point sets, and then divided into multiple template subsets according to equipment type.

[0164] Collect GIM modeling specifications, model library documents, and GIM models, and embed them into the RAG knowledge base.

[0165] Furthermore, the steps of retrieving a template subset of the corresponding type from the equipment template library and performing fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model include:

[0166] When the CAD device image is detected, an HSV color space adapted to the CAD device image is created;

[0167] Within the HSV color space, the S channel is used as the main correction channel, and gamma nonlinear mapping is introduced simultaneously. The pixel values ​​of the V channel are superimposed on the S′ channel in a weighted manner, and an enhanced image is constructed based on the first preset algorithm.

[0168] The enhanced image is subjected to feature extraction processing to determine the corresponding device model.

[0169] Furthermore, the step of performing feature extraction processing on the enhanced image to determine the corresponding device model includes:

[0170] The enhanced image is subjected to ORB feature extraction and descriptor generation processes to obtain a descriptor subset and a point set;

[0171] The FLAN nearest neighbor matching algorithm is used to match the descriptive subset of the enhanced image with the descriptive subset of the template subset to construct the corresponding original matching point set.

[0172] The original matching point set is filtered using the Lowe ratio algorithm to generate a target matching point set. The optimal affine transformation matrix corresponding to the target matching point set is then solved using a second preset algorithm. Simultaneously, the model corresponding to the template with the highest number of matching points in the optimal affine transformation matrix is ​​taken as the device model.

[0173] Furthermore, the expression for the first preset algorithm is:

[0174]

[0175] Where S is the main correction channel, γ is the correction parameter, S′ is the enhancement saturation channel, α and β are the stacking ratio coefficients, and O is the enhanced image.

[0176] Furthermore, the expression for the second preset algorithm is:

[0177]

[0178] Where, p i q represents the matching point of the input image in the target matching point set. i A represents the matching points of the template image in the target matching point set, where m is the number of matching points in the target matching point set, and A is the number of matching points in the target matching point set. j Let be the affine transformation matrix to be found. , It is a linear transformation matrix (including rotation, scaling, and shearing). It is a translation vector.

[0179] Furthermore, the step of automatically generating the corresponding BIM model based on the BIM modeling scheme includes:

[0180] Analyze the modeling scheme and simultaneously reconstruct the corresponding 3D model;

[0181] The three-dimensional model is projected to generate a corresponding template top view, and the pose transformation matrix between the user-input CAD device drawing and the template top view is calculated simultaneously.

[0182] The pose of the 3D model is dynamically adjusted according to the pose transformation matrix to generate the corresponding BIM model.

[0183] The fourth embodiment of the present invention provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image recognition and modeling method as described above.

[0184] The fifth embodiment of the present invention provides a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the image recognition and modeling method as described above.

[0185] In summary, the image recognition and modeling method and system provided in the above embodiments of the present invention can accurately parse the image information contained in the input image, thereby accurately generating the corresponding modeling information and accurately completing the model construction.

[0186] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0187] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0188] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0189] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0190] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0191] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for image recognition and modeling, characterized in that, The method includes: Construct a standard equipment dataset, an equipment template library, and a RAG knowledge base; wherein, the standard equipment dataset is used to train the equipment classification model, the equipment template library stores standardized model templates organized by equipment type, and the RAG knowledge base stores structured BIM modeling schemes associated with modeling rules; In response to receiving a CAD device image input by a user, a device classification model pre-trained based on the standard device dataset is used to perform coarse-grained identification of the CAD device image to determine the device type corresponding to the CAD device image; Based on the equipment type, a template subset of the corresponding type is retrieved from the equipment template library, and the CAD equipment image is matched with the template subset in a fine-grained manner to determine the specific equipment model; Based on the equipment model, CAD drawing attributes, and equipment block attributes, a matching BIM modeling scheme is retrieved from the RAG knowledge base; Based on the BIM modeling scheme, the corresponding BIM model is automatically generated.

2. The image recognition and modeling method according to claim 1, characterized in that, The steps for constructing the standard equipment dataset, equipment template library, and RAG knowledge base include: Import CAD floor plan, iterate through all components, label equipment categories, and export equipment images to form an equipment dataset; at the same time, use random flipping, random rotation, random scaling, random grayscale, and random HSV channel adjustment methods to enhance the images and improve training accuracy. Standard CAD equipment images are selected from the equipment dataset for all equipment types as templates to form an equipment template set. At the same time, ORB feature extraction and descriptor generation processes are performed on the equipment template set in batches to obtain descriptor subsets and point sets, and then divided into multiple template subsets according to equipment type. Collect GIM modeling specifications, model library documents, and GIM models, and embed them into the RAG knowledge base.

3. The image recognition and modeling method according to claim 1, characterized in that, The steps of retrieving a template subset of the corresponding type from the equipment template library and performing fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model include: When the CAD device image is detected, an HSV color space adapted to the CAD device image is created; Within the HSV color space, the S channel is used as the main correction channel, and gamma nonlinear mapping is introduced simultaneously. The pixel values ​​of the V channel are superimposed on the S′ channel in a weighted manner, and an enhanced image is constructed based on the first preset algorithm. The enhanced image is subjected to feature extraction processing to determine the corresponding device model.

4. The image recognition and modeling method according to claim 3, characterized in that, The step of performing feature extraction processing on the enhanced image to determine the corresponding device model includes: The enhanced image is subjected to ORB feature extraction and descriptor generation processes to obtain a descriptor subset and a point set; The FLAN nearest neighbor matching algorithm is used to match the descriptive subset of the enhanced image with the descriptive subset of the template subset to construct the corresponding original matching point set. The original matching point set is filtered using the Lowe ratio algorithm to generate a target matching point set. The optimal affine transformation matrix corresponding to the target matching point set is then solved using a second preset algorithm. Simultaneously, the model corresponding to the template with the highest number of matching points in the optimal affine transformation matrix is ​​taken as the device model.

5. The image recognition and modeling method according to claim 3, characterized in that, The expression for the first preset algorithm is: Where S is the main correction channel, γ is the correction parameter, S′ is the enhancement saturation channel, α and β are the stacking ratio coefficients, and O is the enhanced image.

6. The image recognition and modeling method according to claim 4, characterized in that, The expression for the second preset algorithm is: Where, p i q represents the matching point of the input image in the target matching point set. i A represents the matching points of the template image in the target matching point set, where m is the number of matching points in the target matching point set, and A is the number of matching points in the target matching point set. j Let be the affine transformation matrix to be found. , It is a linear transformation matrix. It is a translation vector.

7. The image recognition and modeling method according to claim 1, characterized in that, The steps for automatically generating the corresponding BIM model based on the BIM modeling scheme include: Analyze the modeling scheme and simultaneously reconstruct the corresponding 3D model; The three-dimensional model is projected to generate a corresponding template top view, and the pose transformation matrix between the user-input CAD device image and the template top view is calculated simultaneously. The pose of the 3D model is dynamically adjusted according to the pose transformation matrix to generate the corresponding BIM model.

8. A map recognition and modeling system, characterized in that, The system includes: The resource construction module is used to build a standard equipment dataset, an equipment template library, and a RAG knowledge base. The standard equipment dataset is used to train the equipment classification model, the equipment template library stores standardized model templates organized by equipment type, and the RAG knowledge base stores structured BIM modeling schemes associated with modeling rules. The type recognition module is used to respond to receiving a CAD equipment image input by the user, and to perform coarse-grained recognition of the CAD equipment image using a device classification model pre-trained based on the standard device dataset, so as to determine the device type corresponding to the CAD equipment image; The model matching module is used to retrieve a template subset of the corresponding type from the equipment template library according to the equipment type, and perform fine-grained template matching between the CAD equipment image and the template subset to determine the specific equipment model; The scheme generation module is used to retrieve matching BIM modeling schemes from the RAG knowledge base based on the equipment model, CAD drawing attributes, and equipment block attributes. The model building module is used to automatically generate the corresponding BIM model based on the BIM modeling scheme.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image recognition and modeling method as described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image recognition and modeling method as described in any one of claims 1 to 7.

Images