A 3D Electrical Intelligent Design System Integrating Image Recognition and Multimodal Knowledge Graph

Abstract

This invention discloses a 3D electrical intelligent design system integrating image recognition and multimodal knowledge graphs. It includes a multimodal perception layer, a multimodal knowledge graph layer, an intelligent design engine layer, a full-process interaction layer, a dual-engine collaborative core unit, and a non-functional support unit. The multimodal perception layer collects multi-source data from drawings, 3D scenes, and operations and maintenance. The multimodal knowledge graph layer enables cross-modal knowledge modeling and reasoning. The intelligent design engine layer completes core design tasks such as automatic scheme generation and conflict detection. The full-process interaction layer enables multimodal human-computer interaction and collaborative design. The dual-engine collaborative core unit ensures efficient collaboration between image recognition and the knowledge graph. The non-functional support unit ensures the safe and reliable operation of the system. This invention solves the problems of shallow multimodal fusion, disconnect between design and operations and maintenance, and poor collaboration in existing electrical design systems, significantly shortening the design cycle, reducing the error rate, and improving the intelligence and accuracy of the design.

Description

Technical Field

[0001] This invention relates to the field of electrical design technology, and in particular to a three-dimensional intelligent electrical design system that integrates image recognition and multimodal knowledge graph. Background Technology

[0002] As the power industry transforms towards intelligence and digitalization, 3D electrical design systems have become a core tool for the design of electrical engineering projects such as substations and distribution networks.

[0003] However, existing 3D electrical design systems have the following technical shortcomings: (1) Low degree of multimodal data fusion: The existing system can only process single type of data (such as drawings or 3D models), and cannot achieve deep fusion of multi-source data such as drawings, 3D point clouds, and operation and maintenance data, resulting in a lack of comprehensive data support for design decisions; (2) Limitations of knowledge graph application: The knowledge graphs of existing systems mostly contain only single-dimensional knowledge such as equipment parameters, and do not integrate three-dimensional spatial relationships, full life cycle operation and maintenance data and industry standards. Furthermore, they cannot achieve cross-modal association with image features, resulting in weak knowledge service capabilities. (3) Design and operation and maintenance are separated: the existing system only focuses on modeling and drawing in the design phase, and does not feed back the defect data and equipment life data in the operation and maintenance phase to the design process, resulting in the lack of full life cycle optimization of the design scheme and a high risk of failure in the equipment operation phase; (4) Insufficient collaboration and practicality: The existing system lacks standardized collaborative interfaces for hardware and software components, resulting in low data interaction efficiency. Furthermore, the component combinations have not been optimized for core requirements such as compliance verification and intelligent selection in electrical design, and the design efficiency and accuracy need to be improved.

[0004] Therefore, those skilled in the art provide a three-dimensional electrical intelligent design system that integrates image recognition and multimodal knowledge graphs to solve the problems mentioned in the background art. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a three-dimensional electrical intelligent design system that integrates image recognition and multimodal knowledge graphs, comprising a multimodal perception unit, a multimodal knowledge graph unit, an intelligent design engine unit, a full-process interaction unit, a dual-engine collaborative core unit, and a non-functional guarantee unit; The multimodal sensing unit is used to collect multi-source data related to electrical design and extract features; The multimodal knowledge graph unit is used to realize the storage, processing and application of multi-dimensional knowledge; The intelligent design engine unit is used to automate the entire electrical design process based on perception data and knowledge graphs. The full-process interaction unit is used to realize multimodal interaction and collaborative design between users and the system; The dual-engine collaborative core unit is used to realize bidirectional data interaction and decision triggering between the image recognition engine and the knowledge graph engine; The non-functional protection unit is used to ensure system security, performance, and reliability; Each layer and unit achieves data communication through preset hardware interfaces and software protocols, forming a closed-loop design system of perception-cognition-decision-interaction.

[0006] Preferably, the multimodal perception unit includes a two-dimensional drawing parsing module, a three-dimensional scene perception module, and an operation and maintenance data perception module.

[0007] Preferably, the multimodal knowledge graph unit includes a data access module, a knowledge processing module, and a knowledge application layer.

[0008] Preferably, the intelligent design engine unit includes an automatic solution generation module, a dynamic conflict detection module, an intelligent component selection module, and a design-operation closed-loop module.

[0009] Preferably, the full-process interaction unit includes a multimodal input module, an AR preview module, and a collaborative design module.

[0010] Preferably, the dual-engine collaborative core unit includes a cross-modal interface module, a data synchronization module, and a decision triggering element.

[0011] Preferably, the non-functional protection unit includes a security protection module, a performance optimization module, and a reliability protection component.

[0012] The technical effects and advantages of this invention are as follows: Compared with the prior art, the beneficial effects of the present invention are: This invention enhances the depth of multimodal fusion in electrical design: by integrating multi-source data such as drawings, 3D point clouds, and operation and maintenance data through a multimodal perception unit, and combining it with the Cross-Modal Transformer cross-modal alignment module to achieve image-text-3D cross-modal association, it solves the problem of shallow data fusion in existing systems and provides comprehensive support for design decisions. This invention enhances the knowledge service capabilities of electrical design: it constructs a four-dimensional knowledge graph encompassing electrical semantics, three-dimensional space, the entire lifecycle, and design specifications, integrating multi-dimensional knowledge and enabling real-time reasoning, significantly improving knowledge retrieval efficiency compared to existing systems; This invention enables a closed-loop linkage between electrical design and operation and maintenance: by using a design-operation and maintenance closed-loop module, operation and maintenance defect data is fed back to the design process, which greatly improves the defect prevention rate and reduces the risk of equipment failure during operation. This invention improves the efficiency and accuracy of electrical design: through the automated solution generation and dynamic conflict detection functions of the intelligent design engine unit, the design cycle is effectively shortened and the drawing error rate is reduced; This invention optimizes the collaboration and practicality of electrical design: by using standardized dual-engine collaborative components and multimodal interactive components, it improves the efficiency of system data interaction and user experience, and is suitable for various application scenarios such as substation forward design, distribution network planning, and renovation of old substations. Attached Figure Description

[0013] Figure 1 This is a structural block diagram of a three-dimensional electrical intelligent design system that integrates image recognition and multimodal knowledge graph, as provided in an embodiment of this application. Detailed Implementation

[0014] The present invention will now be described in further detail with reference to specific embodiments. The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and design various embodiments with various modifications suitable for a particular purpose.

[0015] Example 1 Please see Figure 1 This embodiment provides a three-dimensional electrical intelligent design system that integrates image recognition and multimodal knowledge graph, characterized in that it includes a multimodal perception unit, a multimodal knowledge graph unit, an intelligent design engine unit, a full-process interaction unit, a dual-engine collaborative core unit, and a non-functional guarantee unit; The multimodal sensing unit is used to collect multi-source data related to electrical design and extract features, including a two-dimensional drawing parsing module, a three-dimensional scene perception module, and an operation and maintenance data perception module; The multimodal knowledge graph unit is used to realize the storage, processing and application of multi-dimensional knowledge, including a data access module, a knowledge processing module and a knowledge application layer; The intelligent design engine unit is used to automate the entire electrical design process based on perception data and knowledge graphs, including an automatic solution generation module, a dynamic conflict detection module, an intelligent component selection module, and a design-operation closed-loop module; The full-process interaction unit is used to realize multimodal interaction and collaborative design between users and the system, including a multimodal input module, an AR preview module, and a collaborative design module; The dual-engine collaborative core unit is used to realize bidirectional data interaction and decision triggering between the image recognition engine and the knowledge graph engine, including a cross-modal interface module, a data synchronization module, and a decision triggering element; The non-functional protection unit is used to ensure system security, performance and reliability, including a security protection module, a performance optimization module and a reliability protection component; Each layer and unit achieves data communication through preset hardware interfaces and software protocols, forming a closed-loop design system of perception-cognition-decision-interaction.

[0016] The system collects drawings, 3D scenes, and multi-source operation and maintenance data through a multimodal perception layer, realizes cross-modal knowledge modeling and reasoning through a multimodal knowledge graph layer, completes core design tasks such as automatic solution generation and conflict detection through an intelligent design engine layer, realizes multimodal human-computer interaction and collaborative design through a full-process interaction layer, ensures efficient collaboration between image recognition and knowledge graph through a dual-engine collaborative core unit, and ensures safe, efficient and reliable operation of the system through a non-functional guarantee unit.

[0017] The 2D drawing parsing module includes: an industrial-grade scanner, an improved version of the YOLOv8-CBAM SDK, an i-model format converter plugin, an OpenCV image denoising module, and a Python topology extraction script. The industrial-grade scanner is used to scan paper drawings or read electronic drawings. The improved version of the YOLOv8-CBAM SDK is used to identify equipment entities and wiring relationships in the drawings. The OpenCV image denoising module is used to preprocess image data. The Python topology extraction script is used to extract the topological logic between devices. The i-model format converter plugin is used to be compatible with multiple drawing formats and achieve structured output.

[0018] The 3D scene perception module includes: a Velodyne 16-line LiDAR, a Basler acA2500-14gm4K industrial camera, a DJI M300 drone, an optimized ICP registration algorithm plugin, a CNN-LSTM attitude recognition module, and a GB50059-2011 standard verification library. The LiDAR, industrial camera, and drone are used to acquire 3D point clouds and visible light images. The optimized ICP registration algorithm plugin is used to achieve data registration between point clouds and images. The CNN-LSTM attitude recognition module is used to identify the installation attitude of the equipment. The GB 50059-2011 standard verification library is used to automatically mark potential safety distance hazards.

[0019] The operation and maintenance data perception module includes: an OPC UA gateway, an NVIDIA Jetson Xavier edge computing box, a ResNet50 transfer learning SDK, a Kafka message queue, and a SCADA data interface driver. The OPC UA gateway and edge computing box are used to collect device operating parameters, the ResNet50 transfer learning SDK is used to identify device defects in operation and maintenance images, and the Kafka message queue and SCADA data interface driver are used to synchronize operation and maintenance data to the multimodal knowledge graph unit within seconds.

[0020] The data access module includes: a Dell PowerEdge R750 MySQL database server, a MongoDB cluster, a MinIO distributed storage server, a Talend ETL tool, an i-model data reading plugin, and a KKS encoding mapping tool. The database server and storage server are used to store structured equipment parameters, industry standards, and unstructured drawings and report data. The ETL tool is used for incremental data updates, and the i-model data reading plugin and KKS encoding mapping tool are used to achieve cross-disciplinary data integration.

[0021] The knowledge processing module includes: a BERT-BiLSTM-CRF entity extraction SDK, a Cross-Modal Transformer cross-modal alignment module, a Neo4j graph database, and a custom-developed knowledge triplet editing tool. The BERT-BiLSTM-CRF entity extraction SDK is used to extract entities and relationships from text and images. The Cross-Modal Transformer cross-modal alignment module is used to realize cross-modal mapping of image features, text semantics, and 3D coordinates. The Neo4j graph database and the knowledge triplet editing tool are used to construct a multi-dimensional knowledge structure of triples and attributes.

[0022] The automatic solution generation module includes: Baidu PaddleNLP NLP requirement analysis SDK, genetic algorithm optimization plugin, historical case retrieval engine, and NVIDIA A100 GPU computing node; the NLP requirement analysis SDK is used to extract key parameters of user requirements, the genetic algorithm optimization plugin is used to generate multi-objective optimization solutions considering cost, space, and operation and maintenance, the historical case retrieval engine is used to reuse historical design cases, and the GPU computing node is used to improve the solution generation speed.

[0023] The dynamic collision detection module includes: a BVH collision detection algorithm plugin, an electrical clearance verification tool integrating GB 50059-2011, an A rectification path planning module, and an Intel Xeon Platinum 8480C real-time computing chip. The BVH collision detection algorithm plugin is used to verify mechanical collisions, the electrical clearance verification tool is used to verify the compliance of equipment electrical clearances, the A rectification path planning module is used to automatically calculate the optimal rectification plan, and the real-time computing chip is used to ensure the real-time performance of verification and planning.

[0024] The cross-modal interface module includes a gRPC cross-modal data transmission component and a Protobuf data serialization tool, which are used to enable high-speed data interaction of feature vectors and association results between the image recognition engine and the knowledge graph engine.

[0025] The data synchronization module includes a Kafka message queue and a Redis cache server. The Kafka message queue is used to synchronize operation and maintenance data to the knowledge graph in seconds, and the Redis cache server is used to cache frequently accessed device parameters and industry standard data.

[0026] The security protection module includes an AES-256 encryption module, a KeyCloak RBAC access control system, and a Palo Alto Networks firewall. The AES-256 encryption module is used to encrypt design and maintenance data, the RBAC access control system is used to assign permissions to different roles such as designers and reviewers, and the firewall is used to defend against network attacks.

[0027] The performance optimization module includes a Unity HDRP instantiation rendering plugin, a Redis cache server, and an Nginx Plus load balancer. The instantiation rendering plugin is used to quickly load 3D models with millions of polygons, the Redis cache server is used to improve the response speed of high-frequency data, and the load balancer is used to balance the system's computing pressure.

[0028] The AR preview module includes HoloLens 2 AR glasses, a Logitech Brio HD camera, an ORB-SLAM3 virtual-real registration SDK, and an operation and maintenance data overlay rendering plugin. The AR glasses and HD camera are used to achieve virtual-real fusion display, the ORB-SLAM3 virtual-real registration SDK is used to ensure the accuracy of virtual-real registration, and the operation and maintenance data overlay rendering plugin is used to overlay and display device defect records and lifespan prediction data.

[0029] The design-operation closed-loop module includes a defect association mapping tool, an LSTM lifetime prediction SDK, an operation and maintenance data feedback interface, and a HUAWEI AR650 data synchronization gateway. The defect association mapping tool is used to mark design risk points of similar equipment, the LSTM lifetime prediction SDK is used to predict the remaining lifetime of the equipment, and the operation and maintenance data feedback interface and data synchronization gateway are used to connect the data link between design and operation and maintenance.

[0030] The collaborative design module includes an IBM Power Systems E1080 collaborative server, a ProjectWise customized collaborative engine, a version control plugin, and a three-level review and approval workflow tool. The collaborative server is used to support real-time editing by multiple users, the version control plugin is used to automatically generate design version logs, and the three-level review and approval workflow tool is used to integrate a closed-loop process of design, review, and annotation.

[0031] Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art and related fields based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described and explained in the present invention, unless otherwise specified or limited, shall be implemented according to conventional means in the art.

Claims

1. A three-dimensional electrical intelligent design system integrating image recognition and multimodal knowledge graph, characterized in that, It includes a multimodal perception unit, a multimodal knowledge graph unit, an intelligent design engine unit, a full-process interaction unit, a dual-engine collaborative core unit, and a non-functional guarantee unit; The multimodal sensing unit is used to collect multi-source data related to electrical design and extract features; The multimodal knowledge graph unit is used to realize the storage, processing and application of multi-dimensional knowledge; The intelligent design engine unit is used to automate the entire electrical design process based on perception data and knowledge graphs. The full-process interaction unit is used to realize multimodal interaction and collaborative design between users and the system; The dual-engine collaborative core unit is used to realize bidirectional data interaction and decision triggering between the image recognition engine and the knowledge graph engine; The non-functional protection unit is used to ensure system security, performance, and reliability; Each layer and unit communicates with each other through preset hardware interfaces and software protocols, forming a closed-loop design system of perception-cognition-decision-interaction.

2. The three-dimensional electrical intelligent design system integrating image recognition and multimodal knowledge graph as described in claim 1, characterized in that, The multimodal perception unit includes a two-dimensional drawing parsing module, a three-dimensional scene perception module, and an operation and maintenance data perception module.

3. The system according to claim 1, characterized in that, The multimodal knowledge graph unit includes a data access module, a knowledge processing module, and a knowledge application layer.

4. The system according to claim 1, characterized in that, The intelligent design engine unit includes an automatic solution generation module, a dynamic conflict detection module, an intelligent component selection module, and a design-operation closed-loop module.

5. The system according to claim 1, characterized in that, The full-process interactive unit includes a multimodal input module, an AR preview module, and a collaborative design module.

6. The system according to claim 1, characterized in that, The dual-engine collaborative core unit includes a cross-modal interface module, a data synchronization module, and a decision triggering element.

7. The system according to claim 1, characterized in that, The non-functional protection unit includes a security protection module, a performance optimization module, and a reliability protection component.

Images