Generation and application method of augmented reality data package for spacecraft assembly

Abstract

This application discloses a method for generating and applying augmented reality data packages for spacecraft assembly. The method includes the following steps: based on an AR prototype template and a structured process template, establishing an AR process prototype and obtaining corresponding AR structured process data; defining virtual-reality initial registration data; storing the AR process prototype, AR structured process data, and virtual-reality initial registration data in a single data package to obtain an AR data package; constructing an AR interaction and visualization platform, updating the process data of the AR data package to obtain an updated version data package, and storing it in the AR interaction and visualization platform; issuing the process parameters of the updated version data package to the site through the AR interaction and visualization platform to guide actual production. Through AR prototype templates and structured process templates, rapid and structured process data generation is achieved, improving the efficiency and standardization of process view generation; constructing an interaction and visualization platform according to set requirements to improve the management capabilities of augmented reality application maintenance, updates, and expansion.

Description

Technical Field

[0001] The present disclosure generally relates to the technical field of spacecraft assembly, and in particular to a method for generating and applying an augmented reality data package for spacecraft assembly. Background Art

[0002] Spacecraft assembly is a high-variety, small-batch, and differentiated manufacturing process that relies heavily on manual labor. Given the current short development cycles, numerous product types, complex technical requirements, and frequent changes in production status, operators are required to possess proficient technical expertise and a thorough understanding of all assembly process requirements. Traditionally, assembly technicians prepare process documentation based on the technical requirements of the spacecraft assembly phase. For each product type, design and process requirements, such as thermal control requirements, electrical performance requirements, and assembly relationships, are incorporated into the process instructions and illustrated with atlases or models. These various requirement documents or attachments are distributed to the assembly site via paper documents or electronic display boards, resulting in fragmented and difficult-to-access information on aerospace assembly processes, significantly impacting spacecraft assembly efficiency and production enthusiasm.

[0003] Augmented reality (AR) technology is used to realize the virtual-real fusion display of assembly conditions and virtual process information, and the integrated distribution of product models and process requirements is realized based on a unified data source. The model, attribute information and process technology requirements of each product object are retrieved through the human-computer interface interaction on the augmented reality terminal, which greatly improves the guidance of assembly process data.

[0004] However, current augmented reality applications for spacecraft assembly remain time-consuming and labor-intensive in creating and editing virtual-reality fusion process content. For each assembly object of each spacecraft model, a lightweight process model must be generated, with product attributes, process requirements, and other information annotated in three-dimensional space. The entire engineering file, organized by product model, is then deployed to an augmented reality mobile terminal. This customized augmented reality process design model severely impacts the efficiency and quality of process data distribution. Therefore, we propose a method for generating and applying augmented reality data packages for spacecraft assembly to address these issues. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the existing technology, it is desired to provide a method for generating and applying augmented reality data packages for spacecraft assembly that supports the rapid generation and distribution of structured process data, improves technical management capabilities and is easy to implement.

[0006] In a first aspect, the present application provides a method for generating an augmented reality data package for spacecraft assembly, comprising the following steps:

[0007] Obtain AR prototype templates and structured process templates;

[0008] Based on the AR prototype template and structured process template, establish the AR process prototype and obtain the corresponding AR structured process data;

[0009] Register the AR process prototype with the real scene and define the initial registration data of virtual and real;

[0010] The AR process prototype, AR structured process data and virtual-reality initial registration data are stored in a single data package to obtain an AR data package;

[0011] Build an AR interaction and visualization platform, update the process data of the AR data package, and obtain an updated version of the data package;

[0012] Store the updated version data package in the AR interaction and visualization platform;

[0013] Through the AR interaction and visualization platform, the process parameters of the updated version of the data package are distributed to the site to guide actual production.

[0014] According to the technical solution provided in the embodiment of this application, an AR process prototype is established according to the following method:

[0015] Obtain product CAD design model;

[0016] Rebuild the product model structure tree of the product CAD design model according to the preset structural model specifications;

[0017] Lightweight processing of product model structure tree, storage and output of process data in preset format;

[0018] Create a prototype interface table based on the preset format process data;

[0019] Create AR process prototype template based on the prototype interface table;

[0020] Generate AR process prototype based on AR process prototype template.

[0021] According to the technical solution provided in the embodiment of the present application, AR structured process data is obtained according to the following method:

[0022] According to the process of AR process prototype, a structured process interface table is established;

[0023] Establish a structured process template based on the structured process interface table;

[0024] Generate AR structured process data based on structured process templates.

[0025] According to the technical solution provided in the embodiment of the present application, the virtual and real initial registration data are defined according to the following method:

[0026] Predefine QR code logos and place them at the assembly site;

[0027] Measure the relative position relationship between the QR code mark and the on-site product structure to obtain the initial virtual and real registration data.

[0028] According to the technical solution provided in the embodiment of the present application, the AR data packet is saved to the storage path of the AR interaction and visualization platform through a wired or local area network.

[0029] On the second aspect, the present application provides an application method based on the above-mentioned method for generating an augmented reality data package for spacecraft assembly, including: issuing process parameters of an updated version of the data package to the site through an AR interaction and visualization platform to guide actual production.

[0030] In summary, this technical solution specifically discloses a specific process of a method for generating and applying augmented reality data packages for spacecraft assembly. This application specifically establishes an AR process prototype based on an AR prototype template and a structured process template, and obtains corresponding AR structured process data to improve the efficiency and standardization of process view generation; the AR prototype and the structured process data have a data link relationship to improve the process data management capabilities and the interactivity of on-site visualization process data; further, the AR process prototype is aligned with the real scene, and the virtual and real initial registration data is defined; the AR process prototype, AR structured process data, and virtual and real initial registration data are stored in the form of a single data package to obtain an AR data package; an AR interaction and visualization platform is constructed, and the process data of the AR data package is updated to obtain an updated version data package; the updated version data package is stored in the AR interaction and visualization platform; the process parameters of the updated version data package are distributed to the site through the AR interaction and visualization platform to guide actual production; an interaction and visualization platform is constructed according to set requirements to improve the technical management capabilities of augmented reality application maintenance, update, and expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0032] Figure 1 Schematic diagram of the process of generating and applying augmented reality data packages for spacecraft assembly.

[0033] Figure 2 Schematic diagram of the process of generating and applying augmented reality data packages for spacecraft assembly.

[0034] Figure 3 Schematic diagram of the structural reconstruction of the spacecraft design model.

[0035] Figure 4 Schematic diagram of the AR process prototype.

[0036] Figure 5 Schematic diagram of AR structured process data.

[0037] Figure 6 Schematic diagram of the AR interaction and visualization platform. DETAILED DESCRIPTION

[0038] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0039] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] Example 1

[0041] Please refer to Figure 1 and Figure 2 The flowchart of a method for generating an augmented reality data package for spacecraft assembly provided by the present application includes the following steps:

[0042] Obtain AR prototype templates and structured process templates;

[0043] Based on the AR prototype template and structured process template, establish the AR process prototype and obtain the corresponding AR structured process data;

[0044] Combine the AR process prototype with the real scene and define the initial registration data of virtual and real;

[0045] The AR process prototype, AR structured process data and virtual-real initial registration data are stored in a single data package to obtain an AR data package;

[0046] Build an AR interaction and visualization platform, update the process data of the AR data package, and obtain an updated version of the data package;

[0047] Store the updated version data package in the AR interaction and visualization platform;

[0048] Through the AR interaction and visualization platform, the process parameters of the updated version of the data package are distributed to the site to guide actual production.

[0049] In this embodiment, an AR prototype template and a structured process template are obtained; based on the AR prototype template and the structured process template, an AR process prototype is established, and corresponding AR structured process data is obtained; thereby improving the efficiency and standardization of process view generation;

[0050] Specifically,

[0051] Build an AR process prototype according to the following method:

[0052] Obtaining a product CAD design model; where the type of the product CAD design model is, optionally, Pro / E format or Catia format, for example;

[0053] Rebuild the product model structure tree of the product CAD design model according to the preset structural model specifications;

[0054] According to the principles of deletion, merging, compression, and simplification, the product model structure tree is lightweighted, and process data in a preset format is stored and output;

[0055] Furthermore, a prototype interface table is established based on the process data in a preset format; here, the prototype interface table includes the prototype attribute data and the process data that meets the assembly requirements;

[0056] Based on the prototype interface table, an AR process prototype template is created to meet the structured generation and storage requirements of prototype attributes. At the same time, the link relationship between the product object and the structure tree object in the prototype attributes is realized, which improves the process data management capability and the interactivity of on-site visual process data.

[0057] Generate AR process prototypes based on AR process prototype templates to achieve automatic and batch generation of AR process prototypes.

[0058] Furthermore, AR structured process data is obtained according to the following method:

[0059] According to the process of the AR process prototype, a structured process interface table is established; here, the structured process interface table includes structured process data;

[0060] Based on the structured process interface table, a structured process template is established to meet the structured generation and storage requirements of the assembly process. At the same time, the link relationship between the product object and the prototype object in the structured process is realized, which improves the process data management capability and the interactivity of on-site visual process data.

[0061] Generate AR structured process data based on structured process templates, and realize automatic and batch generation of AR structured process data.

[0062] Register the AR process prototype with the real scene and define the initial registration data of virtual and real;

[0063] Specifically,

[0064] Define the virtual and real initial registration data according to the following method:

[0065] Realize the 3D registration between the virtual model and the real scene based on the recognition of predefined QR code logos;

[0066] Predefine QR code logos and place them at the assembly site;

[0067] Measure the relative position relationship between the QR code mark and the on-site product structure to obtain the initial virtual and real registration data.

[0068] The AR process prototype, AR structured process data and virtual-reality initial registration data are stored in the form of a single data packet to obtain an AR data packet.

[0069] Build an AR interaction and visualization platform, save the AR data package to the storage path of the AR interaction and visualization platform through a wired or local area network; use the AR interaction and visualization platform to update the process data of the AR data package to obtain an updated version of the data package.

[0070] Store the updated version data package in the AR interaction and visualization platform.

[0071] The process parameters of the updated version of the data package are distributed to the site through the AR interaction and visualization platform. Operators switch the process views of the working procedures or assembly objects through gestures, voice and other interactive methods to guide actual production and realize real-time interactive guidance of spacecraft assembly operations.

[0072] Among them, the assembly object BOM and process catalog are displayed in the three-dimensional interface of the AR immersive scene in a predefined structure, supporting multiple query methods such as process object selection according to the structure tree and keyword search.

[0073] The specific process is as follows:

[0074] The application background is the overall assembly of spacecraft;

[0075] like Figure 3 As shown, the CAD design release models of various types of spacecraft are rebuilt into a product model structure tree according to the predefined model data specifications;

[0076] Furthermore, when rebuilding the product model structure tree, if physical elements of the design model are missing, such as the electrical connector model or thermistor model at the cable end, these elements need to be modeled physically according to the design attributes and added to specific nodes. For example, the electrical connector models missing from a certain cable can be generated in batches at the end of the cable branch, named with the electrical connector code, and added to the cable node.

[0077] Alternatively, when model names are repeated, such as when similar objects such as cable brackets and nylon bases display the same model name in the model, differentiated naming (name + number format, etc.) is required to distinguish them. For example, for a group of nylon base models with the same code "XX-NLDZ", each model object can be differentiated by naming them "XX-NLDZ-01, ..., XX-NLDZ-03, ..." to distinguish instances of different model objects in different locations, ensuring data association and accurate positioning of each model instance in a specific process.

[0078] Alternatively, when the node relationship is chaotic, the "parent-child" relationship of the model generated according to the design logic does not meet the requirements of AR interaction for efficient object retrieval and visualization of the model grouped by nodes. Nodes need to be rebuilt according to product assembly relationships, subordinate relationships, etc.

[0079] Lightweight models are made according to the principles of deletion, merging, compression, and simplification. For example, the internal structure model of the equipment that is not involved in the assembly process is deleted, and the number of triangular facets representing the cylindrical surface is simplified through interpolation calculation, and then saved and output in a unified data format that takes up little storage space.

[0080] like Figure 4 As shown, a standardized prototype interface table is established for various product objects assembled in spacecraft;

[0081] The prototype attributes in the prototype interface table include but are not limited to attribute data such as product name / code, model name / code, type, and subordinate relationship, as well as process data for assembly requirements. For example, the process prototype attributes of a spacecraft cable include: cable code, cable main bundle type, cable binding requirements, model name, branch list, associated electrical connector code, associated equipment code, associated cable bracket code, associated nylon base code, associated penetration hole, etc.

[0082] Create AR process prototype templates based on the prototype interface table; realize the structured generation and storage of prototype attributes, and realize the link relationship between product objects in the prototype attributes and structure tree objects based on the product unique code;

[0083] Based on the prototype interface table and AR prototype template, AR process digital prototypes are automatically and batch-generated, keeping the structure of the original lightweight process model unchanged, and displaying the prototype properties through the AR prototype parameter interface of the corresponding object node.

[0084] like Figure 5 As shown, a standardized structured process interface table is established; wherein, the structured process data includes but is not limited to the process volume name / code, process name / code, product main object name / code, associated product name / code, process step requirements, model visibility configuration, etc.

[0085] For the cable laying process, the associated structures include the structural panels and penetrations through which the cable path passes. The associated direct components mainly include the cable brackets and nylon bases corresponding to the cable binding points. In addition, the visibility status of the associated models is defined according to the process and step requirements. For example, step 01 defines that the binding path and direct components of the binding point of cable branch 01 are visible, while the other models are invisible.

[0086] Create a structured process template based on the structured process interface table. According to the association and correspondence between the process specification and the process prototype in the process interface table, it can meet the needs of quickly querying the properties of the associated object prototype from the process specification, such as querying the height property of the associated cable bracket 01 in the cable laying branch 01 step; based on the structured process interface table and the structured process template, the structured process data is automatically and batch generated in the structure of "XX model structured process-process volume-process". The structured process data includes process text information and parameterized process prototype objects. The information of each process node can be viewed through the AR structured process parameter interface.

[0087] Based on the recognition of predefined QR code marks, the three-dimensional spatial alignment of the virtual model and the real scene is realized, the QR code mark is arranged at the assembly site, and the relative position relationship between the QR code mark and the product structure is measured; and, for specific spacecraft products, the relative translation, rotation parameters and scale relationship between the QR code mark and the product structure are mapped to the transformation relationship between the mark and the product model in the virtual space, that is, the initial virtual and real registration parameters are transferred.

[0088] The AR process prototype, AR structured process data and virtual-reality initial registration data are integrated in a single data package to obtain an AR data package. The name and label of the AR data package are specified according to the spacecraft product model code or assembly stage and data package version to facilitate the management and update of the data package.

[0089] Develop AR terminal applications and deploy AR data packages to AR mobile terminal devices;

[0090] Here, the type of AR mobile terminal device is, for example, a tablet computer or see-through AR glasses;

[0091] Furthermore, the initial virtual-reality registration is achieved based on the preset QR code mark, the scene model is built in real time based on the surrounding spatial environment characteristics, and the initial positioning data is taken over to achieve continuous tracking and registration in a larger assembly scene range.

[0092] According to the AR data package structure and application interaction logic, an interactive control framework is customized. At the same time, an AR-based three-dimensional interactive interface is designed to obtain an AR interaction and visualization platform to realize the update and visualization of process data such as work step requirements and information identification in the data package.

[0093] like Figure 6 As shown in the figure, the specific interaction logic examples of the AR interaction and visualization platform are as follows:

[0094] 1. Read all local AR data packages and list them on the AR data package switching interface, showing all AR data packages of different models, different process stages and versions for users to select and call based on the overall assembly task of the current model;

[0095] 2. Read and confirm the initial pose registration data of the corresponding AR data package on the virtual-reality tracking registration interface. The virtual model is displayed on the AR terminal and matched and integrated with the real scene in a predefined pose relationship. Then, the consistency of the virtual and real geometry in space is evaluated through simple visual methods. The position and pose of the virtual model can be interactively adjusted to further improve the accuracy of virtual-reality registration. The general interactive methods include "adjustment by model bounding box control" and "parametric adjustment of the model's six degrees of freedom".

[0096] 3. To meet different process guidance needs, there are two different interactive visualization methods for process data: 1) Process objects can be directly selected in the process object selection interface, and the prototype directly drives the visualization of process parameters such as visual identification, animation, and product technical status. The process object selection interface inherits the model structure of the AR prototype, which facilitates manual rapid query and object selection;

[0097] 2) By reading the structured parameters of each process volume, process and step in the data package, the process / step switching, process / step status push and process / step content release are realized. On the one hand, the execution status of each process / step is displayed, and on the other hand, the process requirements such as the basis documents, quality and safety requirements, and step operation content of the process / step are visualized. In addition, the parametric configuration of visibility identification, model display and hidden status, and animation is modeled, annotated and animated according to the structured process.

[0098] 4. Based on the interactive control of structured process procedures and process models, a real-time updated virtual-reality fusion visualization view is used to provide dynamic simulation and real-time guidance of the process. The main forms include key feature annotation, text description display of installation requirements, and animated simulation display of installation paths.

[0099] 5. Dispatchers can schedule tasks based on the structured process of the corresponding model, assign the corresponding process to designated teams and specify the task start time node. The relevant data is written into the scheduling attributes of the structured process. The teams then decompose and implement the tasks according to the task requirements displayed on the visual terminal.

[0100] 6. After the process / step is completed, it is signed and confirmed through interaction. The execution status of this process or step, photos of the implementation results, and other data are then written into the AR data package and become part of its structured parameters;

[0101] 7. Supports accounts within the specified permission range to modify visualization parameters, write the modification results into the data package and update the data package version.

[0102] The generated AR data packet is copied to the specific storage path of the AR terminal interaction and visualization platform via a wired or local area network for the interaction and visualization control module to call or write.

[0103] Relying on AR terminals, virtual-reality fusion process parameters are issued to the site. Operators switch the process views of the working procedures or assembly objects through interactive methods such as gestures and voice, and operate according to the intuitive guidance of the virtual-reality fusion view. After the operation is completed, inspection, signing and status confirmation are carried out to realize real-time interactive guidance of spacecraft assembly operations.

[0104] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in this application.

Claims

1. A method for generating an augmented reality data package for spacecraft assembly, characterized in that: The following steps are involved: Obtain AR prototype templates and structured process templates; Based on the AR prototype template and structured process template, establish the AR process prototype and obtain the corresponding AR structured process data; Register the AR process prototype with the real scene and define the initial registration data of virtual and real; The AR process prototype, AR structured process data, and virtual-reality initial registration data are stored in a single data package to obtain an AR data package; Build an AR interaction and visualization platform, update the process data of the AR data package, and obtain an updated version of the data package; Store the updated version data package in the AR interaction and visualization platform; Build an AR process prototype according to the following method: Obtain product CAD design model; Rebuild the product model structure tree of the product CAD design model according to the preset structural model specifications; Lightweight processing of product model structure tree, storage and output of process data in preset format; Create a prototype interface table based on the preset format process data; Create AR process prototype template based on the prototype interface table; Generate AR process prototype based on AR process prototype template; Obtain AR structured process data using the following methods: According to the process of AR process prototype, a structured process interface table is established; Establish a structured process template based on the structured process interface table; Generate AR structured process data based on structured process templates.

2. The method for generating an augmented reality data package for spacecraft assembly according to claim 1, characterized in that: Define the virtual and real initial registration data according to the following method: Predefine QR code logos and place them at the assembly site; Measure the relative position relationship between the QR code mark and the on-site product structure to obtain the initial virtual and real registration data.

3. The method for generating an augmented reality data package for spacecraft assembly according to claim 1, characterized in that: Save the AR data packet to the storage path of the AR interaction and visualization platform through a wired or local area network.

4. An application method based on the method for generating an augmented reality data package for spacecraft assembly according to any one of claims 1 to 3, characterized in that: include: Through the AR interaction and visualization platform, the process parameters of the updated version of the data package are distributed to the site to guide actual production.

Images