Multi-modal dynamic mechanical knowledge graph construction method and system based on mechanical scene

Abstract

The invention discloses a multi-modal dynamic mechanical knowledge graph construction method and system based on a mechanical scene, and the method comprises the steps: collecting mass data: collecting unstructured text data, semi-structured drawing data and structured bill-of-material data which need to be referenced in the research and design process of a mechanical arm research and development model machine; preprocessing the mechanical data; constructing a knowledge graph, including entity classification and recognition, performing entity recognition by using an LSTM model to obtain entity classification, and putting entities into an entity data set; training an entity classification model; the method comprises the following steps: extracting entity-relationship-attribute sequences of the mechanical field in unstructured and semi-structured text data through a ChGPT large model data extraction algorithm, and directly analyzing structured bill of materials data to obtain the entity-attribute sequences; and fusing multi-modal data. The dynamic mechanical knowledge graph constructed by the method has very high practical value and provides very high reference significance in mechanical design, production, development and manufacturing processes.

Description

Technical Field

[0001] The present invention relates to a method for constructing a multimodal dynamic knowledge graph in a mechanical scenario, and specifically to a method for dynamically constructing a multimodal knowledge graph for a mechanical scenario based on an intelligent algorithm. Background Art

[0002] 1. Knowledge Graph

[0003] Knowledge Graph is an important branch technology of artificial intelligence. It was proposed by Google in 2012. It is a structured semantic knowledge base used to describe concepts and their relationships in the physical world in symbolic form. It is a graph-based knowledge representation and reasoning framework. Its basic components are "entity-relationship-entity" triples, as well as entities and their related attribute-value pairs. Entities are connected to each other through relationships to form a network of knowledge structures.

[0004] With the development of knowledge graphs, general knowledge graphs have been widely used. However, for mechanical scenarios, especially the professional fields of mechanical product research and development and prototype development, universal knowledge graphs cannot meet application needs due to their own high professional requirements.

[0005] Due to the professionalism of prototype development, there is a lack of data extraction sources. The types of data are complex, including structured data, semi-structured data, and unstructured data. The data types are weakly structured and contain many design drawings, which are difficult to parse. For data extraction in the field of prototype development, it is necessary to configure professional semantic rules and define extraction logic algorithms to ensure that the screened data (entities, attributes, relationships) have the professional characteristics corresponding to the prototype and achieve the purpose of high accuracy of data mining. In addition, it is necessary to process the data to improve availability and reduce redundancy and errors.

[0006] On the other hand, with the development of new generation information technology, manufacturing enterprises are accelerating the upgrading and replacement of production technology from mechanization, automation, informatization to intelligence, and industrial manufacturing is developing towards digital manufacturing, network manufacturing, and intelligent manufacturing. Especially in the process of mechanical design, research and development, and manufacturing, designers need to consider many factors such as the principle, structure, movement mode, and component materials of the machine. In order to solve the problem of knowledge waste caused by the ineffective management of a large number of heterogeneous two-dimensional and three-dimensional CAD models in the design and development of mechanical products, a knowledge graph with semantic associations between three-dimensional geometric information and non-geometric information is formed around key capabilities such as knowledge graph semantic model construction and knowledge graph reasoning application.

[0007] Faced with the high performance, high reliability and high efficiency requirements of mechanical R&D prototype development and design, the parts used in different mechanical products are naturally related during the design and development of mechanical products. Based on this, we build a knowledge graph semantic model to improve the application ability of knowledge graph reasoning, break through key technologies such as multi-semantic feature extraction of semantic models, path reasoning based on reinforcement learning, and semantic network language rule mining, build a knowledge graph with the ability to quickly reason about professional design information, improve the intelligent auxiliary design capability of prototypes, and provide technical support for shortening the prototype development cycle.

[0008] Knowledge graphs and large language models are both means of representing and processing knowledge. Large models complement the ability to understand language, while knowledge graphs enrich the way to represent knowledge. The deep combination of the two will surely provide artificial intelligence with a more comprehensive, reliable, and controllable knowledge processing method. In the context of mechanical R&D and manufacturing, there are multiple heterogeneous data sources, and we need to find a way to process different data sources in a targeted manner, find the relationship between data, and build a mechanical knowledge graph.

[0009] In addition, in the process of processing data from various heterogeneous data sources, we used artificial intelligence algorithms and large models, and through continuous iteration, we processed and analyzed the source data more accurately. Based on this, the mechanical knowledge graph we built is dynamically adjusted. When we have a more accurate understanding of the data, we will promptly feed it back to the graph database for data updates, so that the data quality of the knowledge graph can be maintained at a high level. This also makes it possible to build a dynamic mechanical knowledge graph in the face of the continuous influx of mechanical data.

[0010] 2. Recognition algorithm

[0011] In the process of building a dynamic mechanical knowledge graph, the LSTM model is mainly used to realize data entity recognition; in order to automatically extract features from massive data for model training and improve the accuracy of entity recognition, we use the LSTM model. LSTM (Long short-term memory) is a special RNN that is mainly used to solve the gradient vanishing and gradient exploding problems in the long sequence training process. LSTM can perform better in longer sequences.

[0012] 3. Extraction and fusion algorithms

[0013] The TechGPT big model is used to extract and merge the entity-relationship-attribute of data. Compared with traditional deep learning, the TechGPT big model has stronger natural language processing capabilities such as knowledge graph construction, logical reasoning and text understanding. The big model can extract entity and entity relationship triples from a large amount of text data. By automatically learning knowledge, the knowledge graph is intelligently constructed. On the other hand, massive data also helps the big model better understand the relationship and attributes between entities, improving the accuracy and efficiency of the big model's natural language understanding. The big model and the knowledge graph are combined to achieve more complex and precise tasks.

[0014] 4. Inference Algorithm

[0015] Constructing a large-scale knowledge graph requires constantly updating the relationships between entities. One way to fill in minimal facts in a knowledge base is to infer unknown facts based on existing triples, which is called link prediction.

[0016] Due to the noise of the data source and the inaccuracy of the extraction process, there are also noisy knowledge and knowledge contradictions in the knowledge graph. With continuous construction, the accuracy of knowledge analysis gradually decreases. The root cause is that the extraction pattern is not completely reliable, so sometimes wrong instances are extracted. For example, when doing entity recognition or entity relationship extraction, when the training set is small, the entities or relationships obtained are not necessarily reliable. False instances will be used to extract more and more unreliable extraction patterns and wrong instances or contradictory information, and if it depends entirely on manual secondary correction detection, the efficiency is low and the cost is extremely high. Therefore, it is necessary to use knowledge reasoning methods to update and automatically clean up noisy knowledge bases.

[0017] 5. Domain Graph Construction Technology

[0018] Knowledge graphs are divided into general knowledge graphs and domain knowledge graphs. Knowledge graphs based on mechanical scenarios belong to the category of domain graphs. The construction process of knowledge graphs mainly includes six links: knowledge modeling, knowledge storage, attribute relationship extraction, knowledge fusion, knowledge calculation, and knowledge application. The basic unit of knowledge graphs is: entity-relationship-attribute. The triple composed of entity-relationship-attribute is also the core of knowledge graphs.

[0019] If there is a relationship between two entity nodes, they will be connected by an undirected edge. Usually, the knowledge graph is divided into a data layer and a pattern layer. The data layer stores real data, and the pattern stores entity-relationship-entity or entity-attribute-attribute value. Knowledge modeling needs to be based on the application attributes, knowledge characteristics, and actual needs of the industry, and business abstraction and business modeling should be carried out according to the knowledge graph pattern, mainly entity definition, relationship definition, and attribute definition. In order to ensure the quality of the domain knowledge graph, the rationality of entity classification, attribute definition method, and knowledge expansion capability should be considered during modeling. The current prototype development process involves a large amount of standard specifications, design plans, drawings, processes and other information, which are usually scattered in various documents, databases and internal knowledge bases. At the same time, the types of data are complex, including structured data (fixed format such as database files), semi-structured data (with relevant tags such as web pages) and unstructured data (no pre-organized method such as videos), lacking a unified data storage form and organizational management method. This leads to the inability to effectively store knowledge, and the objectively valuable information between knowledge cannot be deeply mined and utilized. Summary of the invention

[0020] The technical problem to be solved by the present invention is a method for automatically constructing a unified structured mechanical knowledge graph based on structured, semi-structured and unstructured multimodal data sources.

[0021] The present invention solves the above technical problems through the following technical means: a method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenes, comprising the following steps:

[0022] S1. Massive data collection

[0023] Collect unstructured text data, semi-structured drawing data, and structured bill of materials data that are needed for reference in the process of developing and designing the robot arm prototype;

[0024] S2, mechanical data preprocessing, preprocessing the mechanical data collected in step S1;

[0025] S3. Constructing a knowledge graph includes the following steps:

[0026] S31, entity classification and recognition, use the LSTM model to perform entity recognition, obtain entity classification, and put the entity into the entity data set for continuous iterative training of the LSTM model;

[0027] S32, entity classification model training, based on the entity classification, the pre-processed data of various types are put into the entity classification training set as the input of the iterative training of the LSTM model;

[0028] S33, entity-relationship triples acquisition, using the TechGPT large model data extraction algorithm to extract entity-relationship-attribute triples in the mechanical field from unstructured and semi-structured text data, and directly parse the structured bill of materials data to obtain entity-relationship-attribute triples;

[0029] S34. Multimodal data fusion, based on the data fusion algorithm, further disambiguates and inserts the disambiguated sequence, that is, the key element entity-relationship-attribute triples in the knowledge graph, into the knowledge graph in an orderly manner to construct the relationship links between the knowledge graph nodes.

[0030] Furthermore, in step S1, unstructured text data is collected: product design plans, design manuals, design specifications, and design standard documents that need to be referenced during the development and design of the robotic arm prototype;

[0031] Collect semi-structured data: design drawings and component drawings of various robotic arm prototypes during their development and design process;

[0032] Collect structured data: Collect the bill of materials files generated during the development and design process of various robotic arm prototypes.

[0033] Furthermore, step S2, mechanical data preprocessing specifically includes:

[0034] S21. For unstructured text data, firstly, the text data should be preprocessed by vectorization; then useless Chinese stop words should be removed; useless punctuation marks and special symbols should be filtered out; word segmentation should be performed, and finally the word segmentation representation of the text should be obtained;

[0035] S22. For semi-structured drawing data, the algorithm is used to perform noise reduction on the drawing files, and then the key information on the drawings is obtained through the OCR recognition algorithm. The key information is reflected in the form of text segmentation vectors;

[0036] S23. For structured bill of materials data, the mechanical entities obtained through parsing are used to construct entity nodes of the robotic arm product knowledge graph, which are inserted into the knowledge graph. Then, the structured data of the bill of materials is parsed through the standardized BOM table format to obtain the specific structural parent-child relationship of the mechanical product.

[0037] Furthermore, in step S31, the entity classification categories defined mainly include: mechanical product design, mechanical product solution, mechanical product quality control, mechanical product electrical system, mechanical product working condition, mechanical product mechanical structure, mechanical product control system, mechanical product process, and mechanical product use.

[0038] Furthermore, in step S31, if the entity category identified by the LSTM model is not in the defined classification type, similarity matching is performed in combination with the inference algorithm to obtain the potential entity category to which the node belongs and insert the entity category into the knowledge graph;

[0039] For the case where there is no specific classification output when the training set is small, the auxiliary information obtained during the entity recognition process is used, and the matching threshold is set using the inference algorithm to obtain potential entity classification.

[0040] Furthermore, the above-mentioned method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios also includes S4, knowledge graph optimization, training the knowledge graph model, and in the process of iterative training of the model, iteratively processing the data that has been included in the knowledge graph, and updating the new entity classifications and relationships obtained into the knowledge graph.

[0041] The present invention also provides a system for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios, comprising:

[0042] Massive data collection module, used to collect unstructured text data, semi-structured drawing data, and structured bill of materials data required for reference in the process of developing and designing robot arm prototypes;

[0043] A mechanical data preprocessing module, used for preprocessing the collected mechanical data;

[0044] Construct a knowledge graph module, including the following units

[0045] The entity classification and recognition unit is used to use the LSTM model to perform entity recognition, obtain entity classification, and put the entity into the entity data set for continuous iterative training of the LSTM model;

[0046] The entity classification model training unit is used to put various pre-processed data into the entity classification training set based on the entity classification, as the input of the iterative training of the LSTM model;

[0047] The entity-relationship triple acquisition unit is used to extract entity-relationship-attribute triples in the mechanical field from unstructured and semi-structured text data through the TechGPT large model data extraction algorithm. For structured bill of materials data, the entity-relationship-attribute triples are directly parsed;

[0048] The multimodal data fusion unit is used to further disambiguate based on the data fusion algorithm, and insert the disambiguated sequence, that is, the key element entity-relationship-attribute triples in the knowledge graph, into the knowledge graph in an orderly manner to construct the relationship links between the knowledge graph nodes.

[0049] Furthermore, the mechanical data preprocessing module specifically includes:

[0050] The preprocessing unit for unstructured text data is used to first preprocess the text data into vectors; then remove useless Chinese stop words; then filter out useless punctuation marks and special symbols; perform word segmentation, and finally obtain the word segmentation representation of the text;

[0051] The pre-processing unit for semi-structured drawing data is used to perform noise reduction on drawing files through algorithms, and then obtain key information on the drawings through OCR recognition algorithms. The key information is reflected in the form of text segmentation vectors;

[0052] The preprocessing unit for structured bill of materials data is used to construct entity nodes of the robotic arm product knowledge graph through the mechanical entities obtained through analysis, insert them into the knowledge graph, and then parse the structured data of the bill of materials through the standardized BOM table format to obtain the specific structural parent-child relationship of the mechanical product.

[0053] Furthermore, in the entity classification recognition unit, if the entity category identified by the LSTM model is not in the defined classification type, similarity matching is performed in combination with the inference algorithm to obtain the potential entity category to which the node belongs and insert the entity category into the knowledge graph;

[0054] For the case where there is no specific classification output when the training set is small, the auxiliary information obtained during the entity recognition process is used, and the matching threshold is set using the inference algorithm to obtain potential entity classification.

[0055] Furthermore, the multimodal dynamic mechanical knowledge graph construction system based on mechanical scenarios also includes a knowledge graph optimization module, which is used to train the knowledge graph model. During the iterative training of the model, the data that has been included in the knowledge graph is iteratively processed, and the new entity classifications and relationships obtained are updated in the knowledge graph.

[0056] Compared with the prior art, the present invention has the following positive effects:

[0057] The dynamic mechanical knowledge graph constructed by the present invention has a strong practical value and provides a high reference significance in the process of mechanical design, production, research and development, and manufacturing. Its construction process and key information acquisition use the latest algorithm models and technologies, providing a good foundation and framework for the construction of domain dynamic graphs. Based on this, various forms of construction schemes can be derived and applied to specific mechanical design, research and development, production and manufacturing and other sub-fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of a large model combined with a knowledge graph.

[0059] Figure 2 Schematic diagram of constructing a knowledge graph from multimodal data sources.

[0060] Figure 3 Schematic diagram of the knowledge graph visualization interface.

[0061] Figure 4 Schematic diagram of knowledge extraction entity-relationship-attribute display. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 and Figure 2 The present invention provides a method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios, comprising the following steps:

[0064] S1. Massive mechanical data collection

[0065] Collect unstructured text data: product design plans, design manuals, design specifications, and design standard documents that need to be referenced during the development and design of robotic arm prototypes;

[0066] Collect semi-structured data: 2D and 3D drawing data such as design drawings and component drawings during the development and design of various robotic arm prototypes;

[0067] Collect structured data: Collect the bill of materials files (BOM) generated during the development and design process of various robotic arm prototypes.

[0068] In this way, a structured, semi-structured, and unstructured multimodal benchmark data source is formed for the construction of the mechanical knowledge graph.

[0069] S2. Mechanical data preprocessing

[0070] Preprocess the mechanical data collected in step S1. The files of the above-mentioned different knowledge sources are important references in the process of prototype development. The preprocessing process includes:

[0071] S21. For unstructured text data, in order to more accurately identify the corresponding classification type of the document, first, preprocess the text data by vectorization; then remove useless Chinese stop words, such as some interjections like "ba", "ma", "ne", etc.; further, filter out some useless punctuation marks and special symbols; based on this series of operations, perform word segmentation, and finally obtain the word segmentation representation of the text.

[0072] S22. For semi-structured drawing data, perform noise reduction processing on the drawing files through an algorithm, and then obtain the key information on the drawing through the OCR recognition algorithm. The key information is embodied in the form of a text word segmentation vector.

[0073] S23. For structured bill of materials data, construct the entity nodes of the robotic arm product knowledge graph by parsing the obtained mechanical entities and insert them into the knowledge graph. Further, through the parsing of the standardized BOM table format, parse the structured data of the bill of materials to obtain the specific structural parent-child relationships of the mechanical product, including raw materials, parts, components, sub-components, and the association relationships between sub-components, etc.

[0074] The above processing steps are not in a sequential order.

[0075] S3. Construct a knowledge graph, including the following steps:

[0076] S31. Entity classification and recognition

[0077] Whether it is unstructured, semi-structured, or structured data, through the preprocessing in step S2, the key information word segmentation vectors have been extracted. Use the LSTM model for entity recognition to obtain entity classifications. And put the entities into the entity dataset (for continuous iterative training of the LSTM model). The defined classification categories mainly include: mechanical product design, mechanical product solution, mechanical product quality control, mechanical product electrical system, mechanical product working conditions, mechanical product mechanical structure, mechanical product control system, mechanical product process, mechanical product use.

[0078] If the entity category recognized by the LSTM model is not in the defined classification types, perform similarity matching in combination with the inference algorithm to obtain the potential possible entity category to which the node belongs and insert the entity category into the knowledge graph.

[0079] In addition, for the situation where there may be no specific classification output in the case of a small training set, use the auxiliary information obtained during the entity recognition process and try to set a matching threshold using the inference algorithm to obtain the potential possible entity classification.

[0080] S32. Entity classification model training

[0081] On the basis of obtaining entity classification, the preprocessed data of various types are put into the entity classification training set as the input of the iterative training of the LSTM model. Through continuous training, the accuracy and precision of classification recognition are improved.

[0082] S33, entity relationship triples acquisition

[0083] By adding multi-domain and multi-task data, the TechGPT large model demonstrates the ability to extract nested entities, answer hallucinations, answer unanswerable questions, and answer long text questions. Relationship extraction is to determine whether two entities in a sentence contain a relationship and to determine the category of the relationship. For unstructured and semi-structured text data, with the help of the powerful text understanding and generation capabilities of the TechGPT large model, entity-relationship-attribute triples in the text are extracted and included in the mechanical entity relationship data set (used as extraction training data and put into the extraction training set). For structured bill of materials data, which already naturally contains parent-child relationships, the entity-relationship-attribute triples are directly parsed.

[0084] S34. Multimodal data fusion

[0085] Under the premise of the above multimodal knowledge sources, different types of data and features are fused together to improve the intelligence and accuracy of the data processing system. This not only includes adding new information to the existing structure, but also involves solving conflicts and inconsistencies between data. Knowledge disambiguation is the step of identifying and solving ambiguity problems in this process, ensuring that the same name or term is accurately distinguished in different contexts. The automatically constructed knowledge graph relies on the automatic extraction of information from text, databases and other data sources. In this process, information from different sources may contradict each other, or the description of the same entity may be different. Effective knowledge fusion is achieved to solve the information conflicts and information duplication that exist in the process of automatically building knowledge graphs. Information conflicts occur when the information about the same entity provided by different data sources contradicts each other; information duplication refers to the multiple recording of the same information for the same entity. In the process of knowledge fusion, these problems need to be solved strategically to determine which information is accurate and which is outdated or wrong. Knowledge fusion and disambiguation are quality assurance to ensure its quality as the basis for decision support and intelligent analysis. In the construction process, invalid or low-quality data is eliminated through context analysis and entity linking (entity linking technology is to associate entities identified in text with unique identifiers in the knowledge base). Through the above, the final entity classification and entity-relationship-attribute triples are inserted into the knowledge graph.

[0086] S4. Knowledge Graph Optimization

[0087] In the above process, the entity classification and entity relationship triples obtained have accuracy problems. In particular, the information inferred by inference algorithms and other methods has deviations. Based on this, the model needs to be trained. In addition, during the iterative training of the model, the data that has been included in the knowledge graph is iteratively processed, and the new entity classifications and relationships obtained are updated to the knowledge graph. After repeated iterations, it becomes possible to build a high-quality knowledge graph. In the process of model iteration, the accuracy of recognition will be improved and influence each other.

[0088] The knowledge graph is a knowledge base based on a graph structure. The visualization technology we built uses human-computer interaction to help people effectively observe the knowledge graph.

[0089] The present invention realizes a knowledge graph visualization system that supports dynamic query. The system interface is as follows: Figure 3 As shown, the extracted entity-relationship-attribute value is displayed as Figure 4 shown.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios, characterized in that: The steps include: S1. Massive data collection Collect unstructured text data, semi-structured drawing data, and structured bill of materials data that are needed for reference in the process of developing and designing the robot arm prototype; S2, mechanical data preprocessing, preprocessing the mechanical data collected in step S1; S3. Constructing a knowledge graph includes the following steps: S31, entity classification and recognition, use the LSTM model to perform entity recognition, obtain entity classification, and put the entity into the entity data set for continuous iterative training of the LSTM model; S32, entity classification model training, based on the entity classification, the pre-processed data of various types are put into the entity classification training set as the input of the iterative training of the LSTM model; S33, entity-relationship triples acquisition, using the TechGPT large model data extraction algorithm to extract entity-relationship-attribute triples in the mechanical field from unstructured and semi-structured text data, and directly parse the structured bill of materials data to obtain entity-relationship-attribute triples; S34. Multimodal data fusion, based on the data fusion algorithm, further disambiguates and inserts the disambiguated sequence, that is, the key element entity-relationship-attribute triples in the knowledge graph, into the knowledge graph in an orderly manner to construct the relationship links between the knowledge graph nodes.

2. The method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios according to claim 1, characterized in that: In step S1, unstructured text data is collected: product design plans, design manuals, design specifications, and design standard documents that need to be referenced during the development and design of the robotic arm prototype; Collect semi-structured data: design drawings and component drawings of various robotic arm prototypes during their development and design process; Collect structured data: Collect the bill of materials files generated during the development and design process of various robotic arm prototypes.

3. The method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios according to claim 1, characterized in that: Step S2, mechanical data preprocessing specifically includes: S21. For unstructured text data, firstly, the text data should be preprocessed by vectorization; then useless Chinese stop words should be removed; useless punctuation marks and special symbols should be filtered out; word segmentation should be performed, and finally the word segmentation representation of the text should be obtained; S22. For semi-structured drawing data, the algorithm is used to perform noise reduction on the drawing files, and then the key information on the drawings is obtained through the OCR recognition algorithm. The key information is reflected in the form of text segmentation vectors; S23. For structured bill of materials data, the mechanical entities obtained through parsing are used to construct entity nodes of the robotic arm product knowledge graph, which are inserted into the knowledge graph. Then, the structured data of the bill of materials is parsed through the standardized BOM table format to obtain the specific structural parent-child relationship of the mechanical product.

4. The method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios according to claim 1, characterized in that: In step S31, the defined entity classification categories mainly include: mechanical product design, mechanical product solution, mechanical product quality control, mechanical product electrical system, mechanical product working condition, mechanical product mechanical structure, mechanical product control system, mechanical product process, and mechanical product use.

5. The method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios according to claim 1, characterized in that: In step S31, if the entity category identified by the LSTM model is not in the defined classification type, similarity matching is performed in combination with the inference algorithm to obtain the potential entity category to which the node belongs and insert the entity category into the knowledge graph; For the case where there is no specific classification output when the training set is small, the auxiliary information obtained during the entity recognition process is used, and the matching threshold is set using the inference algorithm to obtain potential entity classification.

6. The method for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios according to claim 1, characterized in that: It also includes S4, knowledge graph optimization, which trains the knowledge graph model. During the iterative training of the model, the data that has been included in the knowledge graph is iteratively processed, and the new entity classifications and relationships obtained are updated to the knowledge graph.

7. A system for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios, characterized in that: include: Massive data collection module, used to collect unstructured text data, semi-structured drawing data, and structured bill of materials data required for reference in the process of developing and designing robot arm prototypes; A mechanical data preprocessing module, used for preprocessing the collected mechanical data; Construct a knowledge graph module, including the following units The entity classification and recognition unit is used to use the LSTM model to perform entity recognition, obtain entity classification, and put the entity into the entity data set for continuous iterative training of the LSTM model; The entity classification model training unit is used to put various pre-processed data into the entity classification training set based on the entity classification, as the input of the iterative training of the LSTM model; The entity-relationship triple acquisition unit is used to extract entity-relationship-attribute triples in the mechanical field from unstructured and semi-structured text data through the TechGPT large model data extraction algorithm. For structured bill of materials data, the entity-relationship-attribute triples are directly parsed; The multimodal data fusion unit is used to further disambiguate based on the data fusion algorithm, and insert the disambiguated sequence, that is, the key element entity-relationship-attribute triples in the knowledge graph, into the knowledge graph in an orderly manner to construct the relationship links between the knowledge graph nodes.

8. The system for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios as claimed in claim 7, characterized in that: The mechanical data preprocessing module specifically includes: The preprocessing unit for unstructured text data is used to first preprocess the text data into vectors; then remove useless Chinese stop words; then filter out useless punctuation marks and special symbols; perform word segmentation, and finally obtain the word segmentation representation of the text; The pre-processing unit for semi-structured drawing data is used to perform noise reduction on drawing files through algorithms, and then obtain key information on the drawings through OCR recognition algorithms. The key information is reflected in the form of text segmentation vectors; The preprocessing unit for structured bill of materials data is used to construct entity nodes of the robotic arm product knowledge graph through the mechanical entities obtained through analysis, insert them into the knowledge graph, and then parse the structured data of the bill of materials through the standardized BOM table format to obtain the specific structural parent-child relationship of the mechanical product.

9. The system for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios as claimed in claim 7, characterized in that: In the entity classification recognition unit, if the entity category identified by the LSTM model is not in the defined classification type, similarity matching is performed in combination with the inference algorithm to obtain the potential entity category to which the node belongs and insert the entity category into the knowledge graph; For the case where there is no specific classification output when the training set is small, the auxiliary information obtained during the entity recognition process is used, and the matching threshold is set using the inference algorithm to obtain potential entity classification.

10. The system for constructing a multimodal dynamic mechanical knowledge graph based on mechanical scenarios according to claim 7, characterized in that: It also includes a knowledge graph optimization module, which is used to train the knowledge graph model. During the iterative training of the model, the data that has been included in the knowledge graph is iteratively processed, and the new entity classifications and relationships obtained are updated to the knowledge graph.

Images