Process knowledge graph construction method and system for ar assembly guidance

Abstract

This invention provides a method and system for constructing a process knowledge graph for AR assembly guidance, comprising: Step S1: establishing a process knowledge graph model for AR assembly guidance by sorting out the knowledge required for assembly processes; Step S2: converting traditional process information into knowledge based on natural language processing and semantic recognition technologies, realizing the transformation from traditional process cards to a process knowledge graph; Step S3: the augmented reality engine accesses the process knowledge graph, maps process knowledge to corresponding procedures / process nodes, and establishes a mapping relationship between procedure nodes and the visualization model. Developers define the assembly visualization guidance content based on the mapping relationship. This invention reduces the burden on developers in understanding and transforming assembly processes in the customized development of AR assembly guidance content, effectively improving the construction efficiency of augmented reality programs.

Description

Technical Field

[0001] This invention relates to the technical fields of augmented reality assembly, knowledge graphs, and augmented reality program construction. Specifically, it relates to a method and system for constructing a process knowledge graph for AR assembly guidance. Background Technology

[0002] Augmented reality assembly comprehensively utilizes technologies such as stereoscopic display, intelligent interaction, scene positioning, and real-time registration and tracking to overlay and blend virtual images used for assembly guidance onto the real assembly scene. This provides assembly operators with a virtual-real fusion assembly guidance visual environment, which can effectively improve the work efficiency of assembly personnel and reduce risks.

[0003] Currently, augmented reality assembly guidance programs for complex processes still employ a customized build process. For different assembly processes, developers need to read and understand the contents of process cards in order to develop corresponding assembly guidance programs. This increases the cognitive burden on developers and reduces the efficiency of building augmented reality programs.

[0004] In summary, the problem of low efficiency in building augmented reality (AR) applications stems from the fact that the information on the process cards is not intuitive and requires developers to understand and translate it. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for constructing a process knowledge graph for AR assembly guidance.

[0006] According to the present invention, a method and system for constructing a process knowledge graph for AR assembly guidance are provided, the scheme of which is as follows:

[0007] Firstly, a method for constructing a process knowledge graph for AR assembly guidance is provided, the method comprising:

[0008] Step S1: By sorting out the knowledge required for assembly process, establish a process knowledge graph model for AR assembly guidance;

[0009] Step S2: Based on natural language processing and semantic recognition technologies, traditional process information is transformed into knowledge, realizing the conversion of traditional process cards into process knowledge graphs;

[0010] Step S3: The augmented reality engine accesses the process knowledge graph, maps process knowledge to corresponding processes / process nodes, and establishes a mapping relationship between process nodes and the visualization model. Developers define the assembly visualization guidance content based on the mapping relationship.

[0011] Preferably, the process knowledge graph model includes three layers: assembly model AM layer, assembly process AP layer, and assembly step AS layer. The assembly process knowledge ontology O is defined, then O = {AM, AP, AS}.

[0012] Preferably, the assembly model AM layer is instantiated in the augmented reality engine as a visualization model resource for a certain assembly process, used to realize static model display and assembly demonstration animation, and to perform AR visualization presentation of the assembly under the corresponding process based on the mapping relationship with the AP layer.

[0013] Preferably, the assembly model AM layer describes the assembly hierarchy of the assembly target design model. Depending on the different model levels, AM includes the final assembly level fa, sub-assemblies sa, and parts pa.

[0014] Among them, part pa is the most basic element that constitutes the 3D model, representing the basic part unit of the solid model. Subassemblies at different levels can be composed of multiple parts pa and subassemblies sa, that is, sa′={sa t pa t};

[0015] The final assembly level fa is the highest level of the assembly model, consisting of multiple sa and pa, i.e., fa = {sa} t pa t The model hierarchy is associated with each other through parent and child, which define the reference relationships between entities at each model hierarchy. Each hierarchy entity contains references to its parent and all its children. According to the above definition, AM = {pa} t ,sa t ,fa,parent t child t}

[0016] Preferably, the assembly process AP layer describes the process flow information of product assembly, and the knowledge entities of the assembly process AP layer are instantiated as process nodes in the augmented reality building engine, reflecting the specific process steps of product assembly.

[0017] Preferably, the assembly process AP layer consists of a series of assembly processes ap. The connection relationship between ap elements is defined by next and prev, which represent the sequential relationship between assembly processes ap. The process chain composed of next and prev describes the sequential process of product assembly. The process entity marked by the cycle attribute is a process that needs to be repeated.

[0018] The assembly process ap establishes a connection with AM layer elements through assemble, which represents the component model information associated with the current process;

[0019] Assembly process AP consists of a series of assembly steps AS. AP is connected to AS layer elements through contain. According to the above definition, AP = {ap} t , next t ,prev t assemble t ,contain t}

[0020] Preferably, the assembly step AS layer is instantiated as an assembly step node in the augmented reality engine, describing the specific steps to achieve a certain process, including tools, auxiliary materials, quality parameters and inspection parameter information, and presented in AR visualization in the form of text, images or 3D demonstration animation.

[0021] Preferably, the process information knowledge transformation includes: the transformed process information content is semi-structured data, that is, the processes and steps in the process card are arranged and numbered according to hierarchical relationships; natural language processing technology is applied to extract the information of the process and steps in the original process card, and finally the entity information, attribute information and association information in the process card are transformed into a process knowledge network of assembly process AP and assembly step AS, specifically including the following steps:

[0022] 1) Identify the current process according to the table hierarchy and sequence number in the process card, extract the attribute information of the current process, and create a process knowledge entity;

[0023] 2) Extract process step information according to the table hierarchy and serial number in the process card, segment the extracted process step items into words, construct the word segmentation set content, and create the corresponding process step knowledge entity;

[0024] 3) Group the verbs in the word segmentation set content, with each group array containing one verb and several nouns;

[0025] 4) Match the verbs and nouns in the grouped array with the semantic template of the work steps to extract information on tooling, equipment, auxiliary materials, and operating procedures;

[0026] 5) Match the verbs and nouns in the grouped array with the quality semantic template to extract quality requirements, quality parameters, and inspection parameter information;

[0027] 6) Based on the matching results of the process semantic template and the quality semantic template, establish knowledge entities for tools, fixtures, auxiliary materials, and quality requirements, and associate them with the process entities through use and constraint;

[0028] 7) Match nouns with part name templates, extract part name information, and store it in the part_content collection;

[0029] 8) Determine whether the information in the word segmentation set content corresponding to the current step has been processed. If not, repeat steps 4) to 7).

[0030] 9) Establish the association between the step entity and the process entity through contain, and establish the order relationship between the steps through prev and next;

[0031] 10) Determine whether the process steps included in this process have been completed. If not, proceed to step 2.

[0032] 11) Based on part_content, establish the association between the process entity of the assembly process AP layer and the part model entity of the assembly model AM layer through assemble;

[0033] 12) Establish connections between processes using `prev` and `next`;

[0034] 13) Determine if all process items have been processed. If not, proceed to step 1.

[0035] Secondly, a process knowledge graph construction system for AR assembly guidance is provided, the system comprising:

[0036] Module M1: By sorting out the knowledge required for assembly processes, a process knowledge graph model for AR assembly guidance is established;

[0037] Module M2: Based on natural language processing and semantic recognition technologies, traditional process information is transformed into knowledge, realizing the conversion of traditional process cards into process knowledge graphs;

[0038] Module M3: The augmented reality engine maps process knowledge to corresponding processes / process nodes through the process knowledge graph, and establishes a mapping relationship between process nodes and the visualization model. Developers define the assembly visualization guidance content based on the mapping relationship.

[0039] Preferably, the process knowledge graph model includes three layers: assembly model AM layer, assembly process AP layer, and assembly step AS layer. The assembly process knowledge ontology O is defined, then O = {AM, AP, AS}.

[0040] The assembly model AM layer is instantiated in the augmented reality engine as a visual model resource for a certain assembly process, used to realize static model display and assembly demonstration animation, and based on the mapping relationship with the AP layer, the assembly is presented in AR visualization under the corresponding process.

[0041] The assembly model AM layer describes the assembly hierarchy of the assembly target design model. Depending on the model level, AM includes the final assembly level fa, sub-assemblies sa, and parts pa.

[0042] Among them, part pa is the most basic element that constitutes the 3D model, representing the basic part unit of the solid model. Subassemblies at different levels can be composed of multiple parts pa and subassemblies sa, that is, sa′={sa t pa t};

[0043] The final assembly level fa is the highest level of the assembly model, consisting of multiple sa and pa, i.e., fa = {sa} t pa t The model hierarchy is associated with each other through parent and child, which define the reference relationships between entities at each model hierarchy. Each hierarchy entity contains references to its parent and all its children. According to the above definition, AM = {pa} t ,sa t ,fa,parent t child t};

[0044] The assembly process AP layer describes the process flow information of product assembly. The knowledge entities of the assembly process AP layer are instantiated as process nodes in the augmented reality building engine, reflecting the specific process steps of product assembly.

[0045] The assembly process AP layer consists of a series of assembly processes ap. The connection relationship between ap elements is defined by next and prev, which represent the sequential relationship between assembly processes ap. The process chain composed of next and prev describes the sequential process of product assembly. The process entity marked by the cycle attribute is a process that needs to be repeated.

[0046] The assembly process ap establishes a connection with AM layer elements through assemble, which represents the component model information associated with the current process;

[0047] Assembly process AP consists of a series of assembly steps AS. AP is connected to AS layer elements through contain. According to the above definition, AP = {ap} t , next t ,prev t assemble t ,contain t};

[0048] The assembly process AS layer is instantiated as assembly process node in the augmented reality engine, describing the specific steps to achieve a certain process, including tools, auxiliary materials, quality parameters and inspection parameter information, and is presented in AR visualization in the form of text, images or 3D demonstration animation.

[0049] The process information knowledge transformation includes: the transformed process information content is semi-structured data, that is, the processes and steps in the process card are arranged and numbered according to hierarchical relationships; natural language processing technology is used to extract the process and step information in the original process card, and finally the entity information, attribute information and association information in the process card are transformed into a process knowledge network of assembly process AP and assembly step AS, which specifically includes the following steps:

[0050] 1) Identify the current process according to the table hierarchy and sequence number in the process card, extract the attribute information of the current process, and create a process knowledge entity;

[0051] 2) Extract process step information according to the table hierarchy and serial number in the process card, segment the extracted process step items into words, construct the word segmentation set content, and create the corresponding process step knowledge entity;

[0052] 3) Group the verbs in the word segmentation set content, with each group array containing one verb and several nouns;

[0053] 4) Match the verbs and nouns in the grouped array with the semantic template of the work steps to extract information on tooling, equipment, auxiliary materials, and operating procedures;

[0054] 5) Match the verbs and nouns in the grouped array with the quality semantic template to extract quality requirements, quality parameters, and inspection parameter information;

[0055] 6) Based on the matching results of the process semantic template and the quality semantic template, establish knowledge entities for tools, fixtures, auxiliary materials, and quality requirements, and associate them with the process entities through use and constraint;

[0056] 7) Match nouns with part name templates, extract part name information, and store it in the part_content collection;

[0057] 8) Determine whether the information in the word segmentation set content corresponding to the current step has been processed. If not, repeat steps 4) to 7).

[0058] 9) Establish the association between the step entity and the process entity through contain, and establish the order relationship between the steps through prev and next;

[0059] 10) Determine whether the process steps included in this process have been completed. If not, proceed to step 2.

[0060] 11) Based on part_content, establish the association between the process entity of the assembly process AP layer and the part model entity of the assembly model AM layer through assemble;

[0061] 12) Establish connections between processes using `prev` and `next`;

[0062] 13) Determine if all process items have been processed. If not, proceed to step 1.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] 1. This invention establishes a process knowledge graph model for augmented reality-assisted assembly by sorting out the knowledge required for assembly processes, and maps it into process node information in the augmented reality engine, reducing the burden on developers to understand process knowledge;

[0065] 2. This invention is based on knowledge graph technology and uses the "class-relationship-class" and "class-attribute-value" expression methods to structurally express traditional process information, which can improve the organization and management level of the process;

[0066] 3. This invention uses natural language processing and semantic recognition technology to segment, recognize, and extract information from traditional process information, thereby achieving rapid and automatic conversion of traditional process card information into a process knowledge graph and improving the construction efficiency of the process knowledge graph model.

[0067] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0068] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0069] Figure 1 A flowchart for the conversion of process information;

[0070] Figure 2 This is a structural diagram of the assembly process knowledge model;

[0071] Figure 3 A flowchart for knowledge-based process information. Detailed Implementation

[0072] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0073] This invention provides a method for constructing a process knowledge graph for AR assembly guidance. By organizing the knowledge required for assembly processes, a process knowledge graph model for AR assembly guidance is established based on the graph database Neo4j (in this embodiment, Neo4j is a high-performance, NoSQL graph database that stores structured data on the network instead of in tables). The process content is expressed in a structured manner according to "class-relationship-class" and "class-attribute-value" patterns. Traditional process information is transformed into knowledge using natural language processing and semantic recognition technologies, realizing the conversion from traditional process cards to a knowledge graph. The augmented reality engine accesses the process knowledge graph, maps process knowledge to corresponding procedures / process nodes, and establishes a mapping relationship between procedure nodes and the visualization model, thereby accelerating the augmented reality construction process. Specifically, this invention includes the following:

[0074] A knowledge graph model is instantiated based on the Neo4j graph database. The process information is then converted into knowledge using the NLTK natural language processing tool, enabling rapid transformation from traditional process cards to a knowledge graph. The augmented reality engine accesses the process knowledge graph, mapping process knowledge to corresponding procedures / process nodes and establishing a mapping relationship between these nodes and the visualization model. This accelerates the augmented reality construction process, as follows: Figure 1 As shown.

[0075] Among them, such as Figure 2 As shown, the process knowledge graph model includes three layers: assembly model (AM), assembly process (AP), and assembly steps (AS). The assembly process knowledge ontology O is defined as O = {AM, AP, AS}.

[0076] The following section provides a detailed explanation of each level of the process knowledge model:

[0077] (1) Assembly Model AM Layer

[0078] The Assembly Model (AM) layer describes the hierarchical relationships of the assembly target design model. Depending on the model level, AM includes the final assembly level (fa), sub-assemblies (sa), and parts (pa). Parts (pa) are the most basic elements constituting the 3D model, representing the fundamental part units of the solid model. Sub-assemblies at different levels can be composed of multiple parts (pa) and sub-assemblies (sa), i.e., sa′={sa t pa tThe final assembly level fa is the highest level of the assembly model, consisting of multiple sa and pa, i.e., fa = {sa}. t pa t The model hierarchy is associated with each other through parent and child, which define the reference relationships between entities at each model hierarchy. Each hierarchy entity contains references to its parent and all its children. According to the above definition, AM = {pa} t ,sa t ,fa,parent t child t}

[0079] (2) Assembly process AP layer

[0080] The assembly process AP layer consists of a series of assembly processes ap. The connection relationship between ap elements is defined by next and prev, which represent the sequential relationship between assembly processes ap. The process chain formed by next and prev describes the sequential process of product assembly. The process entity marked with the cycle attribute is a process that needs to be repeated. The assembly process ap is associated with AM layer elements through assemble, which represents the component model information associated with the current process. The assembly process ap consists of a series of assembly steps as. The ap is associated with AS layer elements through contain. According to the above definition, AP = {ap...} t , next t ,prev t assemble t ,contain t}

[0081] (3) Assembly step AS layer

[0082] The Assembly Steps (AS) layer, composed of a series of assembly steps (AS), describes the specific operational steps of a particular assembly process. AS contains elements such as tooling (FT), tools (OT), gauges (MT), auxiliary materials (AC), and quality requirements (QR). AS is connected to FT, OT, MT, and AC through the `use` relationship, and to QR through the `constraint` relationship. AS entities establish sequential relationships through `next` and `prev`. According to the above definition, AS = {AS} t ft t ot t ,mt t ac t ,qr t use t ,constraint t}

[0083] Specifically, the knowledge-based transformation of process information includes: the transformed process information content is semi-structured data, that is, the processes and steps in the process card are arranged and numbered according to hierarchical relationships; based on NLTK, natural language processing technology is used to extract the process and step information from the original process card, and finally the entity information, attribute information, and association information in the process card are transformed into a process knowledge network of assembly process AP and assembly step AS, such as... Figure 3 As shown, the specific steps include:

[0084] 1) Identify the current process according to the table hierarchy and sequence number in the process card, extract the attribute information of the current process, and create a process knowledge entity;

[0085] 2) Extract process step information according to the table hierarchy and serial number in the process card, segment the extracted process step items into words, construct the word segmentation set content, and create the corresponding process step knowledge entity;

[0086] 3) Group the verbs in the word segmentation set content, with each group array containing one verb and several nouns;

[0087] 4) Match the verbs and nouns in the grouped array with the semantic template of the work steps to extract information on tooling, equipment, auxiliary materials, and operating procedures;

[0088] 5) Match the verbs and nouns in the grouped array with the quality semantic template to extract quality requirements, quality parameters, and inspection parameter information;

[0089] 6) Based on the matching results of the process semantic template and the quality semantic template, establish knowledge entities for tools, fixtures, auxiliary materials, and quality requirements, and associate them with the process entities through use and constraint;

[0090] 7) Match nouns with part name templates, extract part name information, and store it in the part_content collection;

[0091] 8) Determine whether the information in the word segmentation set content corresponding to the current step has been processed. If not, repeat steps 4) to 7).

[0092] 9) Establish the association between the step entity and the process entity through contain, and establish the order relationship between the steps through prev and next;

[0093] 10) Determine whether the process steps included in this process have been completed. If not, proceed to step 2.

[0094] 11) Based on part_content, establish the association between the process entity of the assembly process AP layer and the part model entity of the assembly model AM layer through assemble;

[0095] 12) Establish connections between processes using `prev` and `next`;

[0096] 13) Determine if all process items have been processed. If not, proceed to step 1.

[0097] The augmented reality building engine acquires assembly process knowledge by accessing the Neo4i graph database. Based on the knowledge entity mapping relationships between the AM, AP, and AS layers, it quickly transforms assembly process knowledge into process flow nodes. Developers then rapidly define the assembly visualization guidance content based on the mapping relationship between the process flow and the model.

[0098] The assembly process AP layer data describes the process flow information of product assembly. The knowledge entities of the AP layer are instantiated as process nodes in the augmented reality building engine, reflecting the specific process steps of product assembly.

[0099] The assembly model AM layer data is instantiated in the augmented reality engine as a visualization model resource for a certain assembly process, used to realize static model display and assembly demonstration animation. Based on the mapping relationship with the AP layer, the assembly is presented in AR visualization under the corresponding process.

[0100] Assembly step AS layer data is instantiated as assembly step nodes in the augmented reality engine. It describes the specific steps to achieve a certain process, including tools, auxiliary materials, quality parameters and inspection parameter information, and is presented in AR visualization in the form of text, images or 3D demonstration animation.

[0101] This invention also provides a process knowledge graph construction system for AR assembly guidance. This system can be implemented by executing the process steps of the AR assembly guidance process knowledge graph construction method. That is, those skilled in the art can understand the AR assembly guidance process knowledge graph construction method as a preferred embodiment of the AR assembly guidance process knowledge graph construction system. Specifically, the system includes the following:

[0102] Module M1: By sorting out the knowledge required for assembly processes, a process knowledge graph model for AR assembly guidance is established;

[0103] Module M2: Based on natural language processing and semantic recognition technologies, traditional process information is transformed into knowledge, realizing the conversion of traditional process cards into process knowledge graphs;

[0104] Module M3: The augmented reality engine maps process knowledge to corresponding processes / process nodes through the process knowledge graph, and establishes a mapping relationship between process nodes and the visualization model. Developers define the assembly visualization guidance content based on the mapping relationship.

[0105] The process knowledge graph model includes three layers: assembly model (AM), assembly process (AP), and assembly steps (AS). The assembly process knowledge ontology O is defined as O = {AM, AP, AS}.

[0106] The Assembly Model (AM) layer describes the hierarchical relationships of the assembly target design model. Depending on the model level, AM includes the final assembly level (fa), sub-assemblies (sa), and parts (pa). Parts (pa) are the most basic elements constituting the 3D model, representing the fundamental part units of the solid model. Sub-assemblies at different levels can be composed of multiple parts (pa) and sub-assemblies (sa), i.e., sa′={sa t pa t The final assembly level fa is the highest level of the assembly model, consisting of multiple sa and pa, i.e., fa = {sa}. t pa t The model hierarchy is associated with each other through parent and child, which define the reference relationships between entities at each model hierarchy. Each hierarchy entity contains references to its parent and all its children. According to the above definition, AM = {pa} t ,sa t ,fa,parent t child t}

[0107] The assembly process AP layer consists of a series of assembly processes ap. The connection relationship between ap elements is defined by next and prev, which represent the sequential relationship between assembly processes ap. The process chain formed by next and prev describes the sequential process of product assembly. The process entity marked with the cycle attribute is a process that needs to be repeated. The assembly process ap is associated with AM layer elements through assemble, which represents the component model information associated with the current process. The assembly process ap consists of a series of assembly steps as. The ap is associated with AS layer elements through contain. According to the above definition, AP = {ap...} t , next t ,prev t assemble t ,contain t}

[0108] Specifically, the knowledge-based transformation of process information includes: the transformed process information content is semi-structured data, that is, the processes and steps in the process card are arranged and numbered according to hierarchical relationships; natural language processing technology is used to extract the information of the process and steps in the original process card, and finally the entity information, attribute information and association information in the process card are transformed into a process knowledge network of assembly process AP and assembly step AS, which specifically includes the following steps:

[0109] 1) Identify the current process according to the table hierarchy and sequence number in the process card, extract the attribute information of the current process, and create a process knowledge entity;

[0110] 2) Extract process step information according to the table hierarchy and serial number in the process card, segment the extracted process step items into words, construct the word segmentation set content, and create the corresponding process step knowledge entity;

[0111] 3) Group the verbs in the word segmentation set content, with each group array containing one verb and several nouns;

[0112] 4) Match the verbs and nouns in the grouped array with the semantic template of the work steps to extract information on tooling, equipment, auxiliary materials, and operating procedures;

[0113] 5) Match the verbs and nouns in the grouped array with the quality semantic template to extract quality requirements, quality parameters, and inspection parameter information;

[0114] 6) Based on the matching results of the process semantic template and the quality semantic template, establish knowledge entities for tools, fixtures, auxiliary materials, and quality requirements, and associate them with the process entities through use and constraint;

[0115] 7) Match nouns with part name templates, extract part name information, and store it in the part_content collection;

[0116] 8) Determine whether the information in the word segmentation set content corresponding to the current step has been processed. If not, repeat steps 4) to 7).

[0117] 9) Establish the association between the step entity and the process entity through contain, and establish the order relationship between the steps through prev and next;

[0118] 10) Determine whether the process steps included in this process have been completed. If not, proceed to step 2.

[0119] 11) Based on the part content, establish the association between the process entity of the assembly process AP layer and the part model entity of the assembly model AM layer through the assemble method;

[0120] 12) Establish connections between processes using `prev` and `next`;

[0121] 13) Determine if all process items have been processed. If not, proceed to step 1.

[0122] This invention provides a method and system for constructing a process knowledge graph for AR assembly guidance. It utilizes knowledge graph technology for process modeling, including an assembly model (AM) layer, an assembly process (AP) layer, and an assembly step (AS) layer. Natural language processing tools are used to digitize process information, enabling rapid conversion from traditional process cards to a knowledge graph. An augmented reality (AR) building engine accesses the knowledge graph to instantiate AP and AS layer knowledge into process node information, and constructs AR visualization guidance elements based on the mapping relationship between AM and AP layer knowledge. This invention reduces the burden on developers in understanding and converting assembly processes during customized development of AR assembly guidance content, effectively improving the efficiency of AR program construction.

[0123] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0124] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for constructing a process knowledge graph for AR assembly guidance, characterized in that, include: Step S1: By sorting out the knowledge required for assembly process, establish a process knowledge graph model for AR assembly guidance; Step S2: Based on natural language processing and semantic recognition technologies, traditional process information is transformed into knowledge, realizing the conversion of traditional process cards into process knowledge graphs; Step S3: The augmented reality engine accesses the process knowledge graph, maps process knowledge to corresponding processes / process nodes, and establishes a mapping relationship between process nodes and the visualization model. Developers define the assembly visualization guidance content based on the mapping relationship. The process knowledge graph model includes: assembly model Layers, assembly processes Layers, assembly steps Three levels define the assembly process knowledge ontology. ,but ; The assembly model The layers describe the assembly hierarchy of the assembly target design model. Depending on the different model levels, Including the final assembly level Subassemblies and parts ; Among them, parts It is the most basic element that constitutes a 3D model, representing the basic part unit of a solid model. Subassemblies at different levels can be composed of multiple parts. and subassemblies Composition, that is ; General Assembly Level It is the highest level of the assembly model, consisting of multiple and Composition, that is Inter-model levels are connected via , Association, which defines the reference relationships between entities at different model levels, means that each level of entity contains references to its parent and all its children. According to the above definition, ; The assembly process AP layer consists of a series of assembly processes. composition, The connection between elements is through and Definition, which represents the assembly process. The order relationship between them is determined by and The process chain describes the sequential process of product assembly, consisting of... The process entity annotated with attributes is a process that requires repeated operations; Assembly process pass and Layer elements establish associations, representing the component model information associated with the current process; Assembly process Consists of a series of assembly steps composition, and Layer elements through To establish a connection, according to the above definition, ; The process information knowledge transformation includes: the transformed process information content is semi-structured data, that is, the processes and steps in the process card are arranged and numbered according to hierarchical relationships; natural language processing technology is used to extract the process and step information in the original process card, and finally the entity information, attribute information and association information in the process card are transformed into a process knowledge network of assembly process AP and assembly step AS, which specifically includes the following steps: 1) Identify the current process according to the table hierarchy and sequence number in the process card, extract the attribute information of the current process, and create a process knowledge entity; 2) Extract process step information according to the table hierarchy and serial number in the process card, segment the extracted process step items into words, construct the word segmentation set content, and create the corresponding process step knowledge entity; 3) Group the verbs in the word segmentation set content, with each group array containing one verb and several nouns; 4) Match the verbs and nouns in the grouped array with the semantic template of the work steps to extract information on tooling, equipment, auxiliary materials, and operating procedures; 5) Match the verbs and nouns in the grouped array with the quality semantic template to extract quality requirements, quality parameters, and inspection parameter information; 6) Based on the matching results of the process step semantic template and the quality semantic template, establish knowledge entities for tools, fixtures, auxiliary materials, and quality requirements, and associate them with the process step entities through use and constraint; 7) Match nouns with part name templates, extract part name information, and store it in the part_content collection; 8) Determine whether the information in the word segmentation set "content" corresponding to the current step has been processed. If not, repeat steps 4) to 7). 9) Establish the association between the step entity and the process entity through contain, and establish the order relationship between the steps through prev and next; 10) Determine whether all the steps in this process have been processed. If not, proceed to step 2. 11) Based on part_content, establish the association between the process entity of the assembly process AP layer and the part model entity of the assembly model AM layer through assemble; 12) Establish connections between processes using `prev` and `next`; 13) Determine if all process items have been processed. If not, proceed to step 1.

2. The method for constructing a process knowledge graph for AR assembly guidance according to claim 1, characterized in that, The assembly model In the augmented reality engine, the layer is instantiated as a visual model resource for a certain assembly process. It is used to realize static model display and assembly demonstration animation. Based on the mapping relationship with the AP layer, the assembly is presented in AR visualization under the corresponding process.

3. The method for constructing a process knowledge graph for AR assembly guidance according to claim 1, characterized in that, The assembly process AP layer describes the process flow information of product assembly. The knowledge entities of the assembly process AP layer are instantiated as process nodes in the augmented reality building engine, reflecting the specific process steps of product assembly.

4. The method for constructing a process knowledge graph for AR assembly guidance according to claim 1, characterized in that, The assembly steps In the augmented reality engine, layers are instantiated as assembly process nodes, describing the specific steps to achieve a certain process, including tools, auxiliary materials, quality parameters, and inspection parameter information, which are presented in AR visualization in the form of text, images, or 3D demonstration animations.

5. A process knowledge graph construction system for AR assembly guidance, characterized in that, The method for constructing a process knowledge graph for AR assembly guidance according to any one of claims 1 to 4 includes: Module M1: By sorting out the knowledge required for assembly processes, a process knowledge graph model for AR assembly guidance is established; Module M2: Based on natural language processing and semantic recognition technologies, traditional process information is transformed into knowledge, realizing the conversion of traditional process cards into process knowledge graphs; Module M3: The augmented reality engine accesses the process knowledge graph, maps process knowledge to corresponding processes / process nodes, and establishes a mapping relationship between process nodes and the visualization model. Developers define the assembly visualization guidance content based on the mapping relationship.

Images