Graph modal design knowledge mining method based on product part three-dimensional model

Through the knowledge mining method of graphic modal design, combined with 3D format conversion, point cloud processing and feature enhancement modules, a text description of three-dimensional objects is generated and a knowledge graph is constructed, which solves the problem of neglecting local structural information and correlation in three-dimensional model retrieval. It realizes efficient analysis and knowledge expression of complex three-dimensional data, and improves the design and manufacturing efficiency of industrial products.

CN118917392AActive Publication Date: 2024-11-08CHONGQING UNIV +1

Patent Information

Application Number
CN202410958366.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

The existing three-dimensional model search method mainly focuses on global structural information, ignores the local structural information and correlation of the model, which makes it impossible to fully express the internal characteristics and detailed information of the model when processing complex industrial products, and it is difficult to effectively analyze complex heterogeneous three-dimensional Graphic data.

Method used

Using the graphic modal design knowledge mining method based on product components, the text description of three-dimensional objects is generated through 3D format conversion, PCL point cloud library processing, object detection module, feature enhancement module and lightweight prediction network. Knowledge graph generation algorithm to build graphical modal knowledge graphs.

Benefits of technology

Improve the accuracy of 3D object recognition and association mining, enhance the semantic analysis ability of 3D graphical data, realize efficient analysis and knowledge expression of complex heterogeneous data, and improve the knowledge management and design and manufacturing efficiency in the entire life cycle of industrial products. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917392A_ABST
    Figure CN118917392A_ABST
Patent Text Reader

Abstract

The invention discloses a graph modal design knowledge mining method based on a product part three-dimensional model, and the method comprises the following steps: 1, preprocessing a three-dimensional graph: 11) converting three-dimensional graph data into an Obj format through a 3D format conversion tool; 12) performing uniform sampling by adopting a PCL point cloud library to obtain a three-dimensional point cloud set corresponding to each three-dimensional graph; 2, constructing a three-dimensional target detection module to extract initial potential features of the candidate objects; 3, using a feature enhancement module combining attributes and relation perception to enhance the initial potential feature expression; 4, constructing a lightweight prediction network, and generating text description of the three-dimensional object; and 5, generating a graph modal knowledge graph in combination with a text modal knowledge graph generation algorithm. According to the graph modal design knowledge mining method based on the product part three-dimensional model, the accuracy of three-dimensional object recognition and association mining can be effectively improved, and the semantic analysis capability of three-dimensional graph data is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial knowledge representation, and specifically is a method for mining graphic modal design knowledge based on three-dimensional models of product parts. Background Art

[0002] With the rapid development of digital technology and computer vision technology, 3D models have been widely used in computer-aided design, medical diagnosis, bioinformatics, 3D printing, medical imaging, digital entertainment and other fields. 3D models not only improve the accuracy of design and manufacturing, but also provide more intuitive and rich visual information. In the entire life cycle of industrial products, from design, manufacturing to maintenance, the application of 3D graphic data has become particularly important. They are not only the basis of product design, but also play a key role in product manufacturing and maintenance.

[0003] In the industrial field, designers can make more intuitive product designs by building three-dimensional models, and MBD (Model-Based Definition) technology has emerged. The goal of MBD is to integrate the information in the two-dimensional engineering drawings into the three-dimensional model, detailing the manufacturing method and characteristics of the product. However, in the traditional MBD method, the design basis is usually not integrated into the three-dimensional model. This basis includes explicit knowledge in the design manual and implicit knowledge in the designer's mind. If this knowledge is integrated into the three-dimensional model, it will help designers better understand the product and improve design quality and efficiency.

[0004] With the popularity of 3D model applications, how to understand and process 3D model retrieval on large-scale data has become a hot topic. In recent years, the development of deep learning has brought many methods for 3D model retrieval. For example, MVCNN generates compact 3D shape descriptors by extracting information from 2D images of multiple views; PointNet and PointNet++ use dense occupancy grids to represent input data and recursively put them into a hierarchical neural network to obtain a representation of the 3D shape.

[0005] Although existing methods have achieved certain results in 3D model retrieval, they mainly focus on the description of global structural information and ignore the local structural information and correlation of the model. This makes it impossible to fully express the intrinsic characteristics and detailed information of the model when processing complex 3D models of industrial products. In the industrial knowledge system of the product's entire life cycle, knowledge mining and extraction of graphic modalities combined with knowledge graph technology can greatly improve the consistency of knowledge representation and the accuracy of mining. Knowledge graphs not only provide a guiding framework for label generation, but also ensure the unified expression and consistency of knowledge in each modality. In addition, the introduction of the cross-modal learning framework enables knowledge under different modalities to be uniformly expressed and integrated in the text space, and finally generates a multimodal industrial knowledge graph with high consistency, coherence and integrity. Through this method, complex industrial knowledge can be effectively parsed, the efficiency of knowledge sharing and utilization can be improved, and the development of intelligent manufacturing can be promoted.

[0006] In the current widespread application of intelligent manufacturing systems, a large amount of graphic modal data is involved in the entire life cycle of industrial products, including 3D assemblies in the design stage and 3D scenes in the manufacturing twin system. These data are usually composed of multiple graphic objects, and there are complex relationships between the objects. How to effectively perform semantic analysis on these complex and heterogeneous 3D graphic data is an important challenge in the intelligent manufacturing process. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a graphic modal design knowledge mining method based on three-dimensional models of product parts, which can effectively improve the accuracy of three-dimensional object recognition and association mining, and enhance the semantic parsing ability of three-dimensional graphic data.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for mining knowledge of graphical modal design based on three-dimensional models of product parts includes the following steps:

[0010] Step 1: 3D graphics preprocessing

[0011] 11) Convert the 3D graphics data into Obj format using the 3D format conversion tool;

[0012] 12) Use the PCL point cloud library for uniform sampling to obtain the 3D point cloud set corresponding to each 3D figure Where: N is the number of coordinate points, K is the auxiliary feature dimension;

[0013] Step 2: Build a 3D object detection module to extract the initial potential features of the candidate objects;

[0014] Step 3: Use a feature enhancement module that combines attribute and relationship perception to enhance the original latent feature expression;

[0015] Step 4: Build a lightweight prediction network to generate a text description of the 3D object;

[0016] Step 5: Combine the text modal knowledge graph generation algorithm to generate a graphic modal knowledge graph.

[0017] Furthermore, in step 2, the method steps of using the three-dimensional object detection module to extract the initial potential features of the candidate object are as follows:

[0018] 21) Construct a target detection module based on VoteNet to achieve the initial encoding of the potential features of the candidate objects;

[0019] 22) Use PointNet++ to adaptively extract point cloud features and obtain seed point sets Each seed point consists of spatial coordinates and hidden layer features; where M is the number of seed points and C is the dimension of hidden layer features; any seed point s in S i Expressed as: s i =[x i ;f i ], where x i represents the spatial coordinates, f i Represents the hidden layer features;

[0020] 23) Adopt the MLP-based point cloud voting module to transform the seed point in all dimensions, learn the feature offset of the seed point to the candidate target center point, and output the voting point And every point v in the voting point V i Represented as v i =[y i ; g i ], and y i =x i +Δx i , g i =f i +Δf, where Δx i and Δf i Respectively represent the predicted coordinates and hidden feature offsets;

[0021] 24) Cluster the voting points based on spatial location similarity and farthest point sampling to obtain several sample clusters and the voting points wi in each sample cluster = [z i ;h i ], where: z i represents the coordinates of the cluster center point, h i Represents the cluster radius; coordinate normalization is performed inside each sample cluster to make full use of the local geometric information inside the cluster;

[0022] 25) A Proposal extraction module consisting of Max Pooling and multiple MLP layers is used to generate the Proposal of each sample cluster, and the initial potential features of the candidate graphic objects covering the spatial boundary information are obtained.

[0023] Furthermore, in step 3, the method steps of using the feature enhancement module to enhance the initial latent feature expression are as follows:

[0024] 31) Use a fully connected layer to map the multi-view RGB features and local position features that reflect the attributes to the same dimension as the initial potential features of the candidate object, and add the two together to embed fine-grained object attribute features;

[0025] 32) Encode the pairwise Euclidean distances between the center points of different candidate objects and the component distances in the three coordinate axis directions of the Cartesian coordinate system, and construct a spatial adjacency matrix group to express the positional relationship between the candidate objects;

[0026] 33) Encode through the fully connected layer to obtain the relational embedding features Where: H is the number of attention heads of the feature enhancement module;

[0027] Furthermore, in step 4, the method steps for generating a text description of the three-dimensional object are:

[0028] 41) A 3D description text generation network is constructed based on a single-layer multi-head Transformer module to achieve information interaction between the enhanced potential features of the target candidate object and the initial potential features of other candidate objects;

[0029] 42) Select the target candidate object, perform linear transformation on its potential features, and generate the Query matrix, Key and Value matrix;

[0030] 43) Using the KNN strategy to select the initial potential features of the K candidate objects closest to the center of the target candidate object;

[0031] 44) A post-processing module consisting of a fully connected layer and a word prediction layer is used to iteratively generate each word of the description text to obtain a text description of the three-dimensional object.

[0032] Furthermore, in step 5, the steps of the method for generating the graphical modality knowledge graph are as follows:

[0033] 51) Obtain text descriptions of scenes and objects of graphic data with the help of a three-dimensional description model;

[0034] 52) Use the text modal entity relationship extraction algorithm to conduct deep mining of text knowledge, extract the text into triples, and generate a graphical modal knowledge graph that conforms to the ontology system framework.

[0035] The beneficial effects of the present invention are:

[0036] The present invention is based on the graphic modal design knowledge mining method of the three-dimensional model of product parts. First, the three-dimensional graphic data is unified into the Obj format through a 3D format conversion tool, and the PCL point cloud library is used to perform point cloud processing to obtain a three-dimensional point cloud set; then, a target detection module is constructed to extract the initial potential features of the candidate objects, and a feature enhancement module combining attribute and relationship perception is used to strengthen these feature expressions; then, a lightweight prediction network is trained to generate a text description of the three-dimensional object; finally, a graphic modal knowledge graph is generated in combination with a text modal knowledge graph generation algorithm; in summary, the present invention is based on the graphic modal design knowledge mining method of the three-dimensional model of product parts, which can effectively improve the accuracy of three-dimensional object recognition and association mining, enhance the semantic parsing ability of three-dimensional graphic data, and realize efficient parsing and knowledge expression of complex heterogeneous three-dimensional data; it not only contributes to knowledge management and sharing throughout the life cycle of industrial products, but also can significantly improve the efficiency and quality of the design and manufacturing process.

[0037] The present invention also has the following advantages:

[0038] (1) Unified data format and point cloud processing. Preprocess the massive and heterogeneous 3D graphics data, unify their expression forms, and convert them into the standard Obj format to ensure the standardization and consistency of the data. Subsequently, the 3D graphics are uniformly sampled through the PCL point cloud library to generate a 3D point cloud set, which not only captures the geometric structure of the graphics, but also reflects the core intrinsic properties of the 3D object through auxiliary feature encoding (such as multi-view appearance information, global coordinate information, normal vector information, etc.), laying the foundation for subsequent object recognition and association mining.

[0039] (2) Efficient 3D target detection. The present invention constructs a 3D target detection module based on VoteNet, adaptively extracts point cloud features through PointNet++, generates a seed point set, and learns feature offsets through the MLP point cloud voting module to output voting points. The voting points are clustered and coordinate normalized to generate sample clusters, and then the initial potential features of the candidate graphic objects are obtained through the Proposal extraction module. This process efficiently captures the features of the target object in the 3D space, providing a solid foundation for subsequent feature enhancement.

[0040] (3) Feature enhancement of attribute and relationship perception. In the feature enhancement module, the present invention introduces attribute encoding and relationship encoding. The attribute encoding module maps attribute features such as color and material to the same dimension as the initial potential features of the candidate object through a fully connected layer, and performs feature fusion, thereby enhancing the fine-grained attribute expression of the object. The relationship encoding module encodes the spatial relationship between objects, constructs a spatial adjacency matrix group, and uses a Transformer encoder to fuse attribute and relationship features, thereby enhancing the expression of association characteristics between objects. In this way, the potential features of the candidate objects are expressed more accurately and richly.

[0041] (4) Training of lightweight prediction network. The present invention designs a prediction network based on a lightweight Transformer architecture to realize the generation of text descriptions of three-dimensional objects. The description text generation network generates description text that fully reflects the attributes and associations of three-dimensional objects through the interaction of the features of the target object and other objects.

[0042] (5) Comprehensive knowledge expression of text modality and three-dimensional modality. When generating a graphic modality knowledge graph, the present invention uses a three-dimensional description model to achieve a comprehensive text description of graphic data scenes and objects. Through the text modality entity relationship extraction algorithm, a graphic modality knowledge graph that conforms to the ontology system framework is generated.

[0043] In general, the present invention not only improves the ability to parse complex three-dimensional graphic data, but also achieves higher accuracy and comprehensiveness in knowledge expression by fully mining and expressing the attribute information and associated information of three-dimensional objects. This method has important application value in intelligent manufacturing, three-dimensional design and industrial product life cycle management, and can greatly facilitate subsequent knowledge graph generation and knowledge mining tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration:

[0045] Figure 1 It is a flow chart of the method for mining the knowledge of graphic modal design based on the three-dimensional model of product parts of the present invention;

[0046] Figure 2 It is a flow chart of the three-dimensional graphics preprocessing method;

[0047] Figure 3 Schematic diagram for combining attribute and relation-aware feature embedding and enhancement;

[0048] Figure 4 Schematic diagram for generating graphical modal knowledge graph;

[0049] Figure 5 Flowchart for designing and building the training framework;

[0050] Figure 6 Schematic diagram of the knowledge mining method. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0052] 1. Graphic modal design knowledge mining method based on 3D models of product parts

[0053] like Figure 1 As shown, the present embodiment provides a method for mining knowledge of graphical modal design based on three-dimensional models of product parts, comprising the following steps.

[0054] Step 1: 3D graphics preprocessing

[0055] like Figure 2 As shown, in this embodiment, the method steps for preprocessing the three-dimensional graphics are:

[0056] 11) Convert the 3D graphics data into Obj format using the 3D format conversion tool.

[0057] 12) Use the PCL point cloud library for uniform sampling to obtain the 3D point cloud set corresponding to each 3D figure Among them: N is the number of coordinate points, K is the auxiliary feature dimension. Specifically, the auxiliary features are encoded based on the multi-view appearance information, global coordinate information, normal vector information, etc. of the coordinate points, which can reflect the core intrinsic properties of the three-dimensional graphic objects and facilitate the subsequent tasks such as object recognition and association mining.

[0058] Step 2: Build a 3D object detection module to extract the initial potential features of candidate objects.

[0059] In this embodiment, the method steps of using the three-dimensional object detection module to extract the initial potential features of the candidate object are as follows:

[0060] 21) Construct a target detection module based on VoteNet to achieve the initial encoding of the potential features of candidate objects.

[0061] 22) Use PointNet++ to adaptively extract point cloud features and obtain seed point sets Each seed point consists of spatial coordinates and hidden layer features; where M is the number of seed points and C is the hidden layer feature dimension; any seed point si in S is expressed as: s i =[x i ;f i ], where x i represents the spatial coordinates, fi Represents the hidden layer features.

[0062] 23) Adopt the MLP-based point cloud voting module to transform the seed point in all dimensions, learn the feature offset of the seed point to the candidate target center point, and output the voting point And every point v in the voting point V i Represented as v i =[y i ; g i ], and y i =x i +Δx i , g i =f i +Δf, where Δx i and Δf i They represent the predicted coordinates and hidden feature offsets respectively.

[0063] 24) Cluster the voting points based on spatial location similarity and farthest point sampling to obtain several sample clusters and the voting points w in each sample cluster i =[z i ;h i ], where: z i represents the coordinates of the cluster center point, h i Represents the cluster radius; coordinate normalization is performed inside each sample cluster to make full use of the local geometric information inside the cluster.

[0064] 25) A Proposal extraction module consisting of Max Pooling and multiple MLP layers is used to generate the Proposal of each sample cluster, and the initial potential features of the candidate graphic objects covering the spatial boundary information are obtained.

[0065] Step 3: Use a feature enhancement module that combines attribute and relationship awareness to enhance the original latent feature expression.

[0066] Specifically, in this embodiment, the main part of the feature enhancement module adopts the Transformer encoder architecture, which integrates the attribute encoding module and the relationship encoding module to achieve enhanced expression of the potential features of the candidate objects, output more accurate spatial positioning results and feature representations that integrate internal attribute information and multi-object association information.

[0067] In this embodiment, the method steps of using the feature enhancement module to enhance the initial latent feature expression are as follows:

[0068] 31) Use a fully connected layer to map the multi-view RGB features and local position features reflecting attributes to the same dimension as the initial latent features of the candidate object. After adding the two, embed the fine-grained object attribute features.

[0069] 32) Encode the pairwise Euclidean distances between the center points of different candidate objects and the component distances in the three coordinate axes of the Cartesian coordinate system, and construct the spatial adjacency matrix group CD x , D y , D z , Dist) to express the positional relationship between candidate objects. x , D y and D z They are the component distances in the three coordinate axis directions of the Cartesian coordinate system; Dist is the pairwise Euclidean distance between the center points of the candidate objects.

[0070] 33) Encode through the fully connected layer to obtain the relational embedding features Where: H is the number of attention heads of the feature enhancement module.

[0071] Step 4: Build a lightweight prediction network to generate a text description of the three-dimensional object.

[0072] like Figure 3 As shown, in this embodiment, the method steps for generating a text description of a three-dimensional object are:

[0073] 41) A 3D description text generation network is constructed based on a single-layer multi-head Transformer module to achieve information interaction between the enhanced latent features of the target candidate object and the initial latent features of other candidate objects.

[0074] 42) The target candidate objects are selected through a specific strategy, and their potential features are linearly transformed to generate the Query matrix, Key and Value matrix.

[0075] 43) The initial potential features of the K candidate objects that are closest to the center of the target candidate object are screened using the KNN strategy.

[0076] 44) A post-processing module consisting of a fully connected layer and a word prediction layer is used to iteratively generate each word of the description text to obtain a text description of the three-dimensional object.

[0077] Step 5: Combine the text modal knowledge graph generation algorithm to generate a graphic modal knowledge graph.

[0078] like Figure 4 As shown, in this embodiment, the steps of the method for generating a graphical modality knowledge graph are as follows:

[0079] 51) With the help of the three-dimensional description model, obtain the text description of the scene and objects of the graphic data.

[0080] 52) Use the text modal entity relationship extraction algorithm to conduct deep mining of text knowledge, extract the text into triples, and generate a graphical modal knowledge graph that conforms to the ontology system framework.

[0081] 2. Knowledge graph generation technology based on three-dimensional description

[0082] In this embodiment, the generation of the graphic modal knowledge graph refers to the automatic extraction of semantically related triples from massive graphic data through computer algorithms. These triples can effectively describe key information such as the geometric shape, functional characteristics, and assembly relationship of the product. For example, in the product design stage, the key structures and functional components in the design drawings can be automatically extracted through the triple generation technology, thereby realizing the structured storage and intelligent retrieval of design information.

[0083] 3D description technology refers to the intuitive display of the geometric shape and internal structure of a product through a 3D model. This technology can be used not only for visual analysis in the product design and manufacturing process, but also for fault diagnosis and performance optimization in the product use and maintenance process. For example, in the product manufacturing stage, the 3D description technology can intuitively display the product assembly process and the relationship between parts, thereby improving assembly efficiency and accuracy.

[0084] By analyzing and processing graphic modal data, it is possible to identify and describe various graphic objects throughout the life cycle of industrial products, providing basic support for intelligent manufacturing. Graphic modal knowledge graph generation technology can automatically extract triple knowledge, including entities, attributes, and relationships, by analyzing and mining graphic data, thereby achieving semantic description and understanding of graphic objects. The three-dimensional description technology is committed to providing a more detailed and accurate description of graphic objects, including geometric shapes, material properties, functional characteristics, etc., to provide richer information support for the intelligent manufacturing process.

[0085] 3. Training framework design and construction

[0086] The training framework aims to perform semantic analysis on complex, heterogeneous and diverse 3D graphic data in industrial scenarios, and to generate text descriptions of 3D objects through 3D description training. The framework includes five main modules: preprocessing, 3D object detection, feature enhancement, description text generation and knowledge graph generation.

[0087] The loss function of the entire training framework is obtained through the description text loss. Thanks to the full attention paid to the attribute information and association information of three-dimensional objects in the training framework, the obtained description text can fully reflect the entity and relationship information in the three-dimensional graphic data, and the relationship between the text and the three-dimensional object is clearly expressed and reliable, which greatly facilitates the subsequent knowledge graph generation task; in addition, with the help of the large language model, the description information of each object in the same three-dimensional graphic can be integrated to generate the global description information of the entire graphic, further enriching the generated text features.

[0088] The design and construction of the training framework aims to solve the semantic parsing problem of complex, heterogeneous and diverse 3D graphic data in industrial scenarios, and to generate text descriptions of 3D objects through 3D description training methods. The specific process of training framework design and construction is as follows: Figure 5 As shown. The process of designing and constructing a training framework includes the following key steps. First, clarify the needs and goals, and determine the functional and performance requirements of the training framework. Next, design the system architecture and select appropriate technology stacks and tools, such as deep learning frameworks (such as TensorFlow, PyTorch) and hardware resources (such as GPU, TPU). Then, modularly design each component, including data processing modules, model building modules, training management modules, and evaluation modules. Next, perform code implementation and integration testing to ensure that the modules work together. Finally, perform performance optimization and iterative improvements, and adjust parameters and algorithms based on actual training results to achieve the best results.

[0089] 4. Knowledge mining methods

[0090] Knowledge mining refers to obtaining information such as entities, new entity links, and new association rules from given data, which is of great significance for the automated construction of large-scale knowledge graphs. A knowledge mining system is a comprehensive tool or platform used to extract valuable information and knowledge from large amounts of data to support decision-making and action. It integrates multiple functions such as data preprocessing, data mining, model evaluation and selection, knowledge expression, and knowledge application. Knowledge mining methods and systems such as Figure 6 shown.

[0091] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for mining knowledge of graphical modal design based on three-dimensional models of product parts, characterized by: The steps include: Step 1: 3D graphics preprocessing 11) Convert the 3D graphics data into Obj format using the 3D format conversion tool; 12) Use the PCL point cloud library for uniform sampling to obtain the 3D point cloud set corresponding to each 3D figure Where: N is the number of coordinate points, K is the auxiliary feature dimension; Step 2: Build a 3D object detection module to extract the initial potential features of the candidate objects; Step 3: Use a feature enhancement module that combines attribute and relationship perception to enhance the original latent feature expression; Step 4: Build a lightweight prediction network to generate a text description of the 3D object; Step 5: Combine the text modal knowledge graph generation algorithm to generate a graphic modal knowledge graph.

2. The method for mining graphical modal design knowledge based on a three-dimensional model of a product component according to claim 1, characterized in that: In step 2, the method steps of using the three-dimensional object detection module to extract the initial potential features of the candidate object are as follows: 21) Construct a target detection module based on VoteNet to achieve the initial encoding of the potential features of the candidate objects; 22) Use PointNet++ to adaptively extract point cloud features and obtain seed point sets Each seed point consists of spatial coordinates and hidden layer features; where M is the number of seed points and C is the dimension of hidden layer features; any seed point s in S i Expressed as: s i =[x i ;f i ], where x i represents the spatial coordinates, f i Represents the hidden layer features; 23) Adopt the MLP-based point cloud voting module to transform the seed point in all dimensions, learn the feature offset of the seed point to the candidate target center point, and output the voting point And every point v in the voting point V i Represented as v i =[y i ; g i ], and y i =x i +Δx i ,g i =f i +Δf, where Δx i and Δf i Respectively represent the predicted coordinates and hidden feature offsets; 24) Cluster the voting points based on spatial location similarity and farthest point sampling to obtain several sample clusters and the voting points w in each sample cluster i =[z i ;h i ], where: z i represents the coordinates of the cluster center point, h i Represents the cluster radius; coordinate normalization is performed inside each sample cluster to make full use of the local geometric information inside the cluster; 25) A Proposal extraction module consisting of Max Pooling and multiple MLP layers is used to generate the Proposal of each sample cluster, and the initial potential features of the candidate graphic objects covering the spatial boundary information are obtained.

3. The method for mining graphical modal design knowledge based on a three-dimensional model of a product component according to claim 1, characterized in that: In step 3, the method steps of using the feature enhancement module to enhance the initial latent feature expression are as follows: 31) Use a fully connected layer to map the multi-view RGB features and local position features reflecting the attributes to the same dimension as the initial potential features of the candidate object, and add the two together to embed the fine-grained object attribute features; 32) Encode the pairwise Euclidean distances between the center points of different candidate objects and the component distances in the three coordinate axis directions of the Cartesian coordinate system, and construct a spatial adjacency matrix group to express the positional relationship between the candidate objects; 33) Encode through the fully connected layer to obtain the relational embedding features Where: H is the number of attention heads of the feature enhancement module.

4. The method for mining graphical modal design knowledge based on a three-dimensional model of a product component according to claim 1, characterized in that: In the step 4, the method steps for generating a text description of the three-dimensional object are: 41) A 3D description text generation network is constructed based on a single-layer multi-head Transformer module to achieve information interaction between the enhanced potential features of the target candidate object and the initial potential features of other candidate objects; 42) Select the target candidate object, perform linear transformation on its potential features, and generate the Query matrix, Key and Value matrix; 43) Using the KNN strategy to select the initial potential features of the K candidate objects closest to the center of the target candidate object; 44) A post-processing module consisting of a fully connected layer and a word prediction layer is used to iteratively generate each word of the description text to obtain a text description of the three-dimensional object.

5. The method for mining graphical modal design knowledge based on three-dimensional models of product parts according to claim 1, characterized in that: In step 5, the steps of the method for generating the graphical modality knowledge graph are as follows: 51) Obtain text descriptions of scenes and objects of graphic data with the help of a three-dimensional description model; 52) Use the text modal entity relationship extraction algorithm to conduct deep mining of text knowledge, extract the text into triples, and generate a graphical modal knowledge graph that conforms to the ontology system framework.

Citation Information

Patent Citations

  • Three-dimensional model automatic modeling method based on text information guidance

    CN115994990A

  • Neutral geometric model assembly feature semantic construction method based on knowledge graph

    CN116541906A

  • BIM three-dimensional reconstruction method for steel structure factory building

    CN116597108A

  • Knowledge graph construction method for three-dimensional digital-analog design data of mechanical product

    CN117708342A

  • Method, apparatus and device for processing three-dimensional point cloud, and storage medium

    WO2022166400A1

Cited By

  • Space target identification method based on bimodal fusion

    CN120747482A

  • A spatial target recognition method based on bimodal fusion

    CN120747482B