Automatic equipment modeling method and system based on artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence and digital modeling, and particularly discloses an automatic equipment modeling method and equipment based on artificial intelligence, and the method comprises the steps: obtaining data to construct a multi-modal fusion data set; extracting target features and integrating the target features into a unified high-dimensional feature vector so as to generate a semantic feature map for modeling equipment; obtaining geometric modeling parameters and assembly constraint conditions of modeling equipment by using a parameter optimization model constructed based on a reinforcement learning framework; based on the fault data, optimization is carried out through a Markov decision principle to generate a preliminary 3D model; and performing key point detection and feature alignment on the planar projection of the model and the BIM drawing vector diagram through a hierarchical matching strategy, correcting model deviation based on vertex information to generate a 3D model, and generating a digital twin 3D model. Thus, by means of multi-source data collaborative acquisition and deep feature fusion, the precision, efficiency and adaptability of equipment modeling are remarkably improved in combination with dynamic optimization and a closed loop verification mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and digital modeling, and in particular to an artificial intelligence-based automatic equipment modeling method and system. Background Art

[0002] Traditional equipment modeling methods mainly rely on manual analysis of CAD / BIM drawings or point cloud data collected by a single sensor (such as lidar), and manually constructing 3D models through parametric modeling tools (such as SolidWorks and Revit). Therefore, only point cloud or drawing data is used for modeling, resulting in one-sided model features. At the same time, modeling parameters are generally based on fixed industry standards or manual experience and cannot be dynamically adjusted according to the real-time status of the equipment. In addition, the model construction and verification stages are separated, requiring multiple manual interventions, which makes it difficult to meet the real-time requirements of industrial sites. Therefore, how to flexibly and efficiently realize automatic equipment modeling based on multimodal data has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] The purpose of the present invention is to provide an artificial intelligence-based automatic equipment modeling method and system, which significantly improves the accuracy, efficiency and adaptability of equipment modeling through collaborative multi-source data collection and deep feature fusion, combined with dynamic optimization algorithms and closed-loop verification mechanisms.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present application provides an artificial intelligence-based automatic device modeling method, comprising:

[0006] Receive modeling requests including design drawing data of modeling equipment, collect point cloud data of modeling equipment through lidar, collect multi-view image data of modeling equipment through RGB-D camera, and obtain historical operation and maintenance logs of modeling equipment;

[0007] Constructing a multimodal fusion dataset based on the information in the request, the image data, and the point cloud data, and extracting and generating structured drawing metadata based on the multimodal fusion dataset;

[0008] Performing feature extraction on the structured drawing metadata to obtain target features, and integrating the target features into a unified high-dimensional feature vector to generate a semantic feature map for the modeled device; wherein the target features include visual features, point cloud features, and text features, wherein the text features include device parameters and function description information;

[0009] The semantic feature map is input into a preset parameter optimization model built based on a reinforcement learning framework to obtain the geometric modeling parameters and assembly constraints of the modeling equipment; fault data is determined based on historical operation and maintenance logs, and the fault tolerance threshold of the modeling parameters is dynamically adjusted through the Markov decision principle to generate adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model;

[0010] The planar projection of the preliminary 3D model is subjected to key point detection and feature alignment with the vector diagram of the BIM drawing through a hierarchical matching strategy, and the model deviation is corrected based on vertex information to generate a 3D model; the 3D model is imported into the digital twin platform, and interacts with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model.

[0011] Furthermore, in some embodiments of the present application, the method further includes: pre-processing the point cloud data;

[0012] The preprocessing includes: distinguishing the modeling device from the environmental background point cloud by an adaptive threshold segmentation algorithm, and removing outlier noise points by DBSCAN clustering, performing denoising by Gaussian filtering, and compressing redundant data by wavelet transform.

[0013] Furthermore, in some embodiments of the present application, constructing a multimodal fusion dataset based on the information in the request, the image data, and the point cloud data, and extracting and generating structured drawing metadata based on the multimodal fusion dataset includes:

[0014] Designing a graph-based fusion framework to convert the point cloud data into a three-dimensional voxel grid, and mapping the image data to voxel units of the voxel grid after SLIC superpixel segmentation;

[0015] For the design drawing data, Mask R-CNN is used to identify the modeling device categories, boundaries and topological relationships, and conditional random fields are used to optimize boundary accuracy.

[0016] Furthermore, in some embodiments of the present application, constructing a multimodal fusion dataset based on the information in the request, the image data, and the point cloud data, and extracting and generating structured drawing metadata based on the multimodal fusion dataset further includes:

[0017] A dual-stream cross-attention network is set up for the visual stream and the text stream. The visual stream is used to extract local texture features from the 3D voxel grid and voxel units, and the text stream is used to extract global semantic text features from the design drawing data and the optimized Mask R-CNN recognition results.

[0018] The similarity matrix between the local texture features and the text features is calculated by a multi-head attention mechanism, a weight distribution map is generated, and the local texture features and the text features are fused based on the weight distribution map.

[0019] Through graph neural networks, the fusion results are converted into a functional-structural joint representation of the modeled device.

[0020] Furthermore, in some embodiments of the present application, the method further includes: verifying the semantic feature graph, wherein the verification includes:

[0021] By training the discriminator network to distinguish the feature distribution of the real modeling device from the generated features, if the discriminator confidence is not lower than the preset threshold, the model is built; if the discriminator confidence is lower than the preset threshold, the feature re-extraction process is triggered;

[0022] For the feature areas where abnormalities are detected, incremental feature completion is performed using a small sample learning method based on a prototype network.

[0023] Furthermore, in some embodiments of the present application, the step of performing key point detection and feature alignment between the planar projection of the preliminary 3D model and the vector diagram of the BIM drawing using a hierarchical matching strategy, and correcting model deviations based on vertex information to generate a 3D model includes:

[0024] In the coarse matching stage, the similarity between the plane projection of the preliminary 3D model and the vector image outline of the BIM drawing is calculated based on the KD-Tree to perform coarse matching; in the fine matching stage, based on semantic guidance, components with high functional importance are prioritized for alignment;

[0025] A vertex-driven deformation model is designed. A sparse-dense deformation field is constructed by calculating the bidirectional projection error between the model surface points and the annotated points on the design drawings. Physical deformation constraints are used to prevent local distortion of the model, and the optimal deformation variable is solved using the conjugate gradient method.

[0026] Furthermore, in some embodiments of the present application, the step of importing the 3D model into the digital twin platform and interacting with preset IoT sensors in real time via the OPC-UA protocol to generate a digital twin 3D model includes:

[0027] A hybrid storage architecture based on a time series database is designed, columnar storage is used to optimize query efficiency for static data, and a streaming computing engine is used to update dynamic data. The OPC-UA proxy server is used to standardize the conversion of cross-vendor device protocols. The static model data includes the classification of data in the digital twin platform, and the static data includes 3D model data, BIM drawing metadata and industry standard knowledge base. The dynamic data includes device status data collected by sensors in real time and user interaction record data.

[0028] Further, in some embodiments of the present application, further comprising: encrypting the device state data by AES-256 algorithm, and performing CRC32 check and blockchain hash comparison to determine abnormal data.

[0029] Further, in some embodiments of the present application, further comprising:

[0030] Based on the digital twin 3D model, the knowledge graph embedded with the time sequence inference module is used to predict the failure of the modeled device; and the uncertainty of the prediction result is quantified, the confidence interval of the failure probability is calculated through the Bayesian neural network, and the early warning is triggered when the confidence exceeds the preset threshold.

[0031] In a second aspect, the present application provides an equipment automatic modeling system based on artificial intelligence, comprising:

[0032] The basic data acquisition module is used to receive a modeling request including design drawing data of the modeled device, collect point cloud data of the modeled device through a laser radar, collect multi-view image data of the modeled device through an RGB-D camera, and obtain historical operation and maintenance logs of the modeled device;

[0033] The multi-modal fusion module is used to construct a multi-modal fusion data set based on the information in the request, the image data and the point cloud data, and extract and generate structured drawing metadata based on the multi-modal fusion data set;

[0034] The graph generation module is used to extract features from the structured drawing metadata to obtain target features, and integrate the target features into a unified high-dimensional feature vector to generate a semantic feature graph for the modeled device; wherein the target features include visual features, point cloud features and text features, and the text features include device parameters and function description information;

[0035] The optimization module is used to obtain geometric modeling parameters and assembly constraint conditions of the modeled device through the semantic feature graph based on a parameter optimization model constructed based on a reinforcement learning framework; determine fault data based on the historical operation and maintenance logs, and dynamically adjust the fault tolerance threshold of the modeling parameters to generate an adversarial network verification parameter based on the Markov decision principle to determine the final geometric modeling parameter and generate a preliminary 3D model;

[0036] The modification module is used to perform key point detection and feature alignment on the plane projection of the preliminary 3D model and the BIM drawing vector diagram through a hierarchical matching strategy, and correct the model deviation based on vertex information to generate a 3D model; and import the 3D model into a digital twin platform, interact with a preset Internet of Things sensor in real time through an OPC-UA protocol, and generate a digital twin 3D model

[0037] The present application provides an artificial intelligence-based automatic modeling method and device for equipment, which includes: receiving a modeling request including design drawing data of a modeling device, collecting point cloud data of the modeling device through a lidar, and collecting multi-view image data of the modeling device through an RGB-D camera, and obtaining a historical operation and maintenance log of the modeling device; constructing a multimodal fusion data set based on the information, image data and point cloud data in the request, and extracting and generating structured drawing metadata based on the multimodal fusion data set; performing feature extraction on the structured drawing metadata to obtain target features, and integrating the target features into a unified high-dimensional feature vector to generate a semantic feature map for the modeling device; inputting the semantic feature map into the data structure; The system then imports the model into a preset parameter optimization model built based on a reinforcement learning framework to obtain the geometric modeling parameters and assembly constraints of the modeled equipment. The system then determines fault data based on historical operation and maintenance logs, dynamically adjusts the fault tolerance threshold of the modeling parameters using the Markov decision principle, and generates adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model. A hierarchical matching strategy is used to perform key point detection and feature alignment between the plane projection of the preliminary 3D model and the vector diagram of the BIM drawing, and model deviations are corrected based on vertex information to generate a 3D model. The 3D model is then imported into the digital twin platform, interacting with preset IoT sensors in real time via the OPC-UA protocol to generate a digital twin 3D model. In this way, through the collaborative collection of multi-source data and deep feature fusion, combined with dynamic optimization and closed-loop verification mechanisms, the accuracy, efficiency, and adaptability of equipment modeling are significantly improved. Furthermore, a digital twin 3D model is constructed based on sensor data, improving the effectiveness of model application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the automatic device modeling method based on artificial intelligence proposed in the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of the artificial intelligence-based automatic equipment modeling system proposed in the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] The implementation of the present invention is described in detail below with reference to specific embodiments.

[0042] The same or similar numbers in the drawings of this embodiment correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "up", "down", "left", "right", etc. indicate directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. This is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0043] Figure 1 This is a flow chart of the method for automatic modeling of equipment based on artificial intelligence proposed by the present invention, with reference to Figure 1 The artificial intelligence-based automatic modeling method for devices provided in this application includes at least the following steps:

[0044] S101. Receive a modeling request including design drawing data of a modeling device, collect point cloud data of the modeling device through a lidar, collect multi-view image data of the modeling device through an RGB-D camera, and obtain a historical operation and maintenance log of the modeling device.

[0045] In some embodiments of the present application, after acquiring the aforementioned basic data, the data may be preprocessed. For example, for point cloud data, preprocessing may include: distinguishing the modeling device from the environmental background point cloud using an adaptive threshold segmentation algorithm, removing outlier noise points using DBSCAN clustering, performing denoising using Gaussian filtering, and compressing redundant data using wavelet transforms. In practical applications, this preprocessing can effectively reduce the amount of redundant data while preserving key geometric features, keeping the point cloud reconstruction error within 0.15mm.

[0046] S102: construct a multimodal fusion dataset based on the design drawing data, image data, and point cloud data, and extract and generate structured drawing metadata based on the multimodal fusion dataset.

[0047] Specifically, the image data and point cloud data can be registered in a three-dimensional coordinate system to form a multimodal fusion data set, and the design drawing data can be semantically segmented to extract the boundaries and topological relationships of modeling equipment to generate structured drawing metadata.

[0048] S103. Extract features from the structured drawing metadata to obtain target features, and integrate the target features into a unified high-dimensional feature vector to generate a semantic feature map for the modeling device.

[0049] Among them, target features include visual features, point cloud features and text features, and text features include device parameters and function description information.

[0050] S104. Input the semantic feature map into the preset parameter optimization model built based on the reinforcement learning framework to obtain the geometric modeling parameters and assembly constraints of the modeling equipment; determine the fault data based on the historical operation and maintenance log, and dynamically adjust the fault tolerance threshold of the modeling parameters through the Markov decision principle to generate adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model.

[0051] S105. Use a hierarchical matching strategy to detect key points and align features between the plane projection of the preliminary 3D model and the vector diagram of the BIM drawing, and correct model deviations based on vertex information to generate a 3D model; and import the 3D model into the digital twin platform, interact with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model.

[0052] The artificial intelligence-based automatic equipment modeling method provided in this application significantly improves the accuracy, efficiency and adaptability of equipment modeling through collaborative collection of multi-source data and deep feature fusion, combined with dynamic optimization and closed-loop verification mechanisms. At the same time, it also constructs a digital twin 3D model based on sensor data to improve the model application effect.

[0053] Furthermore, in some embodiments of the present application, a multimodal fusion dataset is constructed based on design drawing data, image data, and point cloud data, and structured drawing metadata is extracted and generated based on the multimodal fusion dataset. Specifically, this may include: designing a fusion framework based on a graph structure, converting point cloud data into a three-dimensional voxel grid, and mapping the image data to the voxel unit of the voxel grid after SLIC superpixel segmentation; and for the design drawing data, using Mask R-CNN to identify the modeling device category, boundary, and topological relationship, and using conditional random fields to optimize boundary accuracy. In some embodiments of the present application, semantic segmentation of the design drawing data can also be performed using a U-Net network, and component boundary accuracy can be optimized in combination with CRF post-processing. In this way, pixel-level alignment of image and point cloud data is achieved through three-dimensional coordinate system registration, solving the problems of texture loss and geometric distortion in traditional single-modal modeling.

[0054] In some embodiments of the present application, a dual-stream cross-attention network of visual stream and text stream can also be set up, where the visual stream is used to extract local texture features from three-dimensional voxel grids and voxel units, and the text stream is used to extract global semantic text features from design drawing data and optimized Mask R-CNN recognition results; then the similarity matrix of the local texture features and the text features is calculated through a multi-head attention mechanism, a weight distribution map is generated, and the local texture features and text features are fused based on the weight distribution map; and through a graph neural network, the fusion result is converted into a function-structure joint representation of the modeling device, thereby obtaining structured drawing metadata for subsequent use.

[0055] In addition, it should be noted that, in this application, it also includes extracting key event tags from historical operation and maintenance logs through time series analysis, and associating and mapping them with modeling devices for subsequent fault analysis and application.

[0056] On this basis, in the embodiment of the present application, feature extraction is performed on the structured drawing metadata to obtain target features, and the target features are integrated into a unified high-dimensional feature vector to generate a semantic feature map for the modeling device, which may specifically include:

[0057] The ResNet-50 network is used to extract visual features, and natural language processing is performed on the text annotations in the design drawings. Equipment parameters and function descriptions are extracted to obtain text features. Then, the visual features, text features, and point cloud features are integrated into a unified high-dimensional feature vector to generate a semantic feature map of the modeled equipment and verify it.

[0058] The verification of the feature map specifically includes:

[0059] The discriminator network is trained to distinguish the feature distribution of real modeling devices from the generated features. If the discriminator confidence is not lower than the preset threshold, subsequent model construction is performed; if the discriminator confidence is lower than the preset threshold, the feature re-extraction process is triggered; and for feature areas where abnormalities are detected, incremental feature completion is performed using a small sample learning method based on a prototype network.

[0060] Furthermore, in this application, the semantic feature map is input into a preset parameter optimization model built based on a reinforcement learning framework to obtain the geometric modeling parameters and assembly constraints of the modeling equipment. In addition, in an embodiment of this application, based on the semantic feature map, the BREP-NURBS-based hybrid modeling reconstruction integrated in the OpenCASCADE kernel can be utilized, and the instanced modeling technology is adopted for the standard parts library, and the parameterized template is quickly matched through hash coding.

[0061] On this basis, the fault data is determined based on the historical operation and maintenance logs, and the fault tolerance threshold of the modeling parameters is dynamically adjusted through the Markov decision principle to generate adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model.

[0062] On this basis, a hierarchical matching strategy is used to detect key points and align features between the plane projection of the preliminary 3D model and the vector diagram of the BIM drawing, and the model deviation is corrected based on vertex information to generate a 3D model; the 3D model is imported into the digital twin platform, and interacts with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model.

[0063] Specifically, a multi-stage matching strategy can be set up, such as coarse matching stage matching and fine matching stage matching. In the coarse matching stage, the KD-Tree is used to accelerate the nearest neighbor search to calculate the similarity between the model projection and the drawing outline; in the fine matching stage, a semantically guided matching mechanism is introduced to prioritize the alignment of components with high functional importance to achieve sub-pixel alignment.

[0064] And design a vertex-driven deformation model, construct a sparse-dense deformation field by calculating the bidirectional projection error between the model surface points and the drawing annotation points, prevent local distortion of the model based on physical deformation constraints, and solve the optimal deformation variable through the conjugate gradient method; or correct model deviations through the vertex binding algorithm.

[0065] Furthermore, in some embodiments of the present application, the 3D model is imported into the digital twin platform, and interacts with IoT sensors in real time through the OPC-UA protocol to dynamically update device status data, including:

[0066] A hybrid storage architecture based on a time series database is designed. For static data, i.e., static model data, column storage is used to optimize query efficiency. For dynamic data, i.e., data collected by dynamic sensors, a streaming computing engine is used to achieve status updates with millisecond-level delays. The OPC-UA proxy server is used to achieve standardized conversion of cross-vendor device protocols.

[0067] On this basis, the method provided by this application also includes: using a temporal reasoning module embedded in the knowledge graph to quantify the uncertainty of the prediction results when predicting equipment failures, calculating the confidence interval of the failure probability through a Bayesian neural network, and triggering an early warning when the confidence level exceeds a preset threshold; and superimposing a heat map of the internal components of the equipment in real time through the semantic segmentation results, and using blockchain technology to record and annotate modification history to ensure that the data cannot be tampered with.

[0068] It should be noted that, in this embodiment, the hybrid storage architecture based on the time series database designed above specifically includes the classification of data in the digital twin platform. Static data includes equipment 3D model parameters, BIM drawing metadata and industry standard knowledge base, etc., and dynamic data includes real-time sensor data and user interaction records, etc., and storage priority is defined by metadata tags; at the same time, the hybrid storage architecture is implemented through multi-dimensional data partitioning and sharding design, and adaptive batch processing and associative query are designed to complete writing and query optimization; in addition, before the sensor data is written, it can also be encrypted by the AES-256 algorithm, and the TLS1.3 protocol is used at the transport layer to prevent man-in-the-middle attacks, and CRC32 check and blockchain hash comparison are run regularly, and abnormal data automatically triggers the repair process.

[0069] The artificial intelligence-based automatic equipment modeling method provided in this application extracts visual and text features based on ResNet-50 and natural language processing, and combines multimodal fusion algorithms to generate semantic feature maps. This can break through the traditional method's dependence on a single data source, improve the accuracy of equipment function parameter analysis, and significantly reduce semantic ambiguity. It also uses a reinforcement learning framework and a Markov decision process to dynamically adjust the modeling parameter tolerance threshold, combines historical fault data to predict potential risks, and generates an adversarial network verification mechanism to screen out the optimal parameter solution, which can effectively reduce the time spent on manual verification.

[0070] Based on the same inventive concept, this application also provides an artificial intelligence-based automatic equipment modeling system. Figure 2 This is a schematic diagram of the structure of the automatic modeling system for equipment based on artificial intelligence proposed by the present invention. Figure 2 As shown, the device automatic modeling system based on artificial intelligence provided by the embodiment of the present application includes

[0071] The basic data acquisition module 11 is configured to receive a modeling request including design drawing data of a modeling device, collect point cloud data of the modeling device using a lidar, collect multi-view image data of the modeling device using an RGB-D camera, and obtain historical operation and maintenance logs of the modeling device;

[0072] The multimodal fusion module 12 is used to construct a multimodal fusion data set based on the design drawing data, image data and point cloud data, and to extract and generate structured drawing metadata based on the multimodal fusion data set;

[0073] A map generation module 13 is configured to extract features from the structured drawing metadata to obtain target features, and integrate the target features into a unified high-dimensional feature vector to generate a semantic feature map for the modeled device; wherein the target features include visual features, point cloud features, and text features, wherein the text features include device parameters and function description information;

[0074] Optimization module 14 is used to construct a parameter optimization model based on a reinforcement learning framework, obtain the geometric modeling parameters and assembly constraints of the modeling equipment through semantic feature maps, determine fault data based on historical operation and maintenance logs, and dynamically adjust the fault tolerance threshold of the modeling parameters through the Markov decision principle to generate adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model;

[0075] Modification module 15 is used to perform key point detection and feature alignment between the plane projection of the preliminary 3D model and the vector diagram of the BIM drawing through a hierarchical matching strategy, and to correct model deviations based on vertex information to generate a 3D model; and import the 3D model into the digital twin platform, interact with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model.

[0076] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0077] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0078] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" is at least two.

[0079] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0080] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0081] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0082] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0083] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0084] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0085] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An automatic equipment modeling method based on artificial intelligence, characterized in that: include: Receive modeling requests including design drawing data of modeling equipment, collect point cloud data of modeling equipment through lidar, collect multi-view image data of modeling equipment through RGB-D camera, and obtain historical operation and maintenance logs of modeling equipment; Constructing a multimodal fusion data set based on the design drawing data, the image data, and the point cloud data, and extracting and generating structured drawing metadata based on the multimodal fusion data set; Performing feature extraction on the structured drawing metadata to obtain target features, and integrating the target features into a unified high-dimensional feature vector to generate a semantic feature map for the modeled device; wherein the target features include visual features, point cloud features, and text features, wherein the text features include device parameters and function description information; The semantic feature map is input into a preset parameter optimization model built based on a reinforcement learning framework to obtain the geometric modeling parameters and assembly constraints of the modeling equipment; fault data is determined based on historical operation and maintenance logs, and the fault tolerance threshold of the modeling parameters is dynamically adjusted through the Markov decision principle to generate adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model; The planar projection of the preliminary 3D model is subjected to key point detection and feature alignment with the vector diagram of the BIM drawing through a hierarchical matching strategy, and the model deviation is corrected based on vertex information to generate a 3D model; the 3D model is imported into the digital twin platform, and interacts with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model.

2. The method for automatic equipment modeling based on artificial intelligence according to claim 1, characterized in that: Also includes: Preprocessing the point cloud data; The preprocessing includes: distinguishing the modeling device from the environmental background point cloud by an adaptive threshold segmentation algorithm, and removing outlier noise points by DBSCAN clustering, performing denoising by Gaussian filtering, and compressing redundant data by wavelet transform.

3. The method for automatic equipment modeling based on artificial intelligence according to claim 1, characterized in that: The step of constructing a multimodal fusion dataset based on the design drawing data, the image data, and the point cloud data, and extracting and generating structured drawing metadata based on the multimodal fusion dataset, includes: Designing a graph-based fusion framework to convert the point cloud data into a three-dimensional voxel grid, and mapping the image data to voxel units of the voxel grid after SLIC superpixel segmentation; For the design drawing data, Mask R-CNN is used to identify the modeling device categories, boundaries and topological relationships, and conditional random fields are used to optimize boundary accuracy.

4. The method for automatic equipment modeling based on artificial intelligence according to claim 3, characterized in that: The design drawing data, the image data, and the point cloud data are used to construct a multimodal fusion data set, and structured drawing metadata is extracted and generated based on the multimodal fusion data set, further comprising: A dual-stream cross-attention network is set up for the visual stream and the text stream. The visual stream is used to extract local texture features from the 3D voxel grid and voxel units, and the text stream is used to extract global semantic text features from the design drawing data and the optimized Mask R-CNN recognition results. Calculating a similarity matrix between the local texture features and the text features through a multi-head attention mechanism, generating a weight distribution map, and fusing the local texture features and the text features based on the weight distribution map; Through graph neural networks, the fusion results are converted into a functional-structural joint representation of the modeled device.

5. The method for automatic equipment modeling based on artificial intelligence according to claim 1, characterized in that: Also includes: Verifying the semantic feature graph includes: The discriminator network is trained to distinguish the feature distribution of the real modeled device from the generated features. If the discriminator confidence score is not lower than the preset threshold, the model is constructed. If the discriminator confidence is lower than the preset threshold, the feature re-extraction process is triggered; For the feature areas where abnormalities are detected, incremental feature completion is performed using a small sample learning method based on a prototype network.

6. The method for automatic equipment modeling based on artificial intelligence according to claim 1, characterized in that: The method of performing key point detection and feature alignment on the plane projection of the preliminary 3D model and the vector diagram of the BIM drawing using a hierarchical matching strategy, and correcting model deviations based on vertex information to generate a 3D model, includes: In the coarse matching stage, the similarity between the plane projection of the preliminary 3D model and the vector image outline of the BIM drawing is calculated based on the KD-Tree to perform coarse matching; in the fine matching stage, based on semantic guidance, components with high functional importance are prioritized for alignment; A vertex-driven deformation model is designed. A sparse-dense deformation field is constructed by calculating the bidirectional projection error between the model surface points and the annotated points on the design drawings. Physical deformation constraints are used to prevent local distortion of the model, and the optimal deformation variable is solved using the conjugate gradient method.

7. The method for automatic equipment modeling based on artificial intelligence according to claim 1, characterized in that: The 3D model is imported into the digital twin platform, and interacts with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model, including: A hybrid storage architecture based on a time series database is designed, columnar storage is used to optimize query efficiency for static data, and a streaming computing engine is used to update dynamic data. The OPC-UA proxy server is used to standardize the conversion of cross-vendor device protocols. The static model data includes the classification of data in the digital twin platform, and the static data includes 3D model data, BIM drawing metadata and industry standard knowledge base. The dynamic data includes device status data collected by sensors in real time and user interaction record data.

8. The method for automatic equipment modeling based on artificial intelligence according to claim 7, characterized in that: Also includes: The device status data is encrypted using the AES-256 algorithm, and a CRC32 check is run against the blockchain hash to identify abnormal data.

9. The method for automatic equipment modeling based on artificial intelligence according to claim 1, characterized in that: Also includes: Based on the digital twin 3D model, the knowledge graph embedded in the temporal reasoning module is used to predict the failure of the modeled equipment. The uncertainty of the prediction results is quantified, the confidence interval of the failure probability is calculated through the Bayesian neural network, and an early warning is triggered when the confidence exceeds the preset threshold.

10. An automatic equipment modeling system based on artificial intelligence, characterized in that: include: The basic data acquisition module is used to receive modeling requests including design drawing data of modeling equipment, collect point cloud data of modeling equipment through lidar, collect multi-view image data of modeling equipment through RGB-D camera, and obtain historical operation and maintenance logs of modeling equipment; a multimodal fusion module, configured to construct a multimodal fusion dataset based on the design drawing data, the image data, and the point cloud data, and to extract and generate structured drawing metadata based on the multimodal fusion dataset; a map generation module for extracting features from the structured drawing metadata to obtain target features, and integrating the target features into a unified high-dimensional feature vector to generate a semantic feature map for the modeled device; wherein the target features include visual features, point cloud features, and text features, wherein the text features include device parameters and function description information; An optimization module is used to construct a parameter optimization model based on a reinforcement learning framework, obtain geometric modeling parameters and assembly constraints of the modeling equipment through the semantic feature map, determine fault data based on historical operation and maintenance logs, and dynamically adjust the fault tolerance threshold of the modeling parameters through the Markov decision principle to generate adversarial network verification parameters to determine the final geometric modeling parameters and generate a preliminary 3D model; The modification module is used to perform key point detection and feature alignment on the plane projection of the preliminary 3D model and the vector diagram of the BIM drawing through a hierarchical matching strategy, and to correct the model deviation based on vertex information to generate a 3D model; and import the 3D model into the digital twin platform, interact with the preset IoT sensors in real time through the OPC-UA protocol to generate a digital twin 3D model.

Images