Augmented reality visualization method, apparatus, and electronic device for aircraft panel

Abstract

Embodiments of the present application disclose an augmented reality visualization method and device for an aircraft panel and an electronic device. The method comprises: obtaining point cloud data of the aircraft panel, and establishing a measured model of the aircraft panel according to the point cloud data; obtaining actual load data of the aircraft panel; performing finite element simulation analysis on the aircraft panel according to the measured model and the actual load data to obtain a processing result; rendering the processing result to generate a model for augmented reality projection, fusing the model with the aircraft panel, and displaying the fused result in augmented reality. The present application effectively reduces the gap between the finite element simulation analysis result and the actual aircraft panel, and more intuitively displays the simulation prediction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of finite element simulation and augmented reality, and particularly relates to an augmented reality visualization method and device for an aircraft panel and an electronic device. BACKGROUND

[0002] The aircraft panel, as a main part of the aircraft structure, not only determines the aerodynamic shape of the aircraft, but also has the function of structural load bearing. The aircraft panels used on the aircraft mainly include wing panels and fuselage panels. Such parts are not only large in size, but also complex in shape, and are key components in the design and manufacture of the aircraft structure. However, due to the poor rigidity of the aircraft panel, it is easy to deform, and the parts entering the assembly site will finally have a certain shape deviation from the theoretical three-dimensional model, and will also be deformed by the assembly force in the assembly process. At present, the simulation calculation of the aircraft assembly deformation basically relies on professional finite element analysis software such as Ansys, Abaqus, etc. The simulation environment is completely virtual and offline. Virtual means that the model used for simulation is a grid model generated based on the theoretical three-dimensional model, which cannot reflect some characteristics of the parts and tooling in the physical environment, such as the error in the real tooling manufacturing and installation process, the influence of the environment temperature on the tooling, etc. Offline means that the pre-set load and boundary conditions used in the finite element analysis cannot reflect the real load received by the parts in the actual assembly process, and cannot completely restore the state of the parts in the assembly site, so the simulation is a simulation under theoretical conditions. Therefore, there is still a certain gap between the results obtained by the traditional finite element simulation and the real deformation of the workpiece.

[0003] No effective solution has been proposed for the above problems. SUMMARY

[0004] The embodiments of the present application provide an augmented reality visualization method, device and electronic equipment for an aircraft panel, to at least solve the technical problem that the simulation environment of the traditional finite element deformation simulation is completely virtual and offline, and the simulation prediction results are not intuitive enough to watch.

[0005] According to an aspect of the embodiments of the present application, an augmented reality visualization method for an aircraft panel is provided, including: acquiring point cloud data of the aircraft panel, and establishing a measured model of the aircraft panel according to the point cloud data of the aircraft panel; acquiring actual load data of the aircraft panel; performing finite element simulation analysis on the aircraft panel according to the measured model of the aircraft panel and the actual load data, to obtain a processing result; rendering the processing result to generate a model for augmented reality projection, fusing the model with the aircraft panel, and displaying the fused result by augmented reality.

[0006] Optionally, acquiring point cloud data of the aircraft panel includes: performing block scanning on the surface of the aircraft panel to obtain point cloud data of multiple target blocks; preprocessing the point cloud data of the multiple target blocks to obtain preprocessed point cloud data of multiple target blocks; and stitching the point cloud data of the multiple preprocessed target blocks together to obtain the point cloud data of the aircraft panel.

[0007] Optionally, the point cloud data of the multiple target blocks are preprocessed to obtain preprocessed point cloud data of the multiple target blocks, including: using the moving least squares method to filter and denoise the point cloud data of the multiple target blocks to obtain the preprocessed point cloud data of the multiple target blocks.

[0008] Optionally, the point cloud data of multiple preprocessed target blocks are stitched together to obtain the point cloud data of the aircraft panel, including: stitching together the point cloud data of multiple preprocessed target blocks using the Iterative Closest Point (ICP) algorithm to obtain the point cloud data of the aircraft panel.

[0009] Optionally, establishing a measured model of the aircraft panel based on the point cloud data of the aircraft panel includes: using a greedy projection triangulation algorithm to perform three-dimensional reconstruction of the point cloud data of the aircraft panel to obtain the measured model of the aircraft panel.

[0010] Optionally, different datasets in the processing results are recorded in a first file and a second file, respectively. The first file and the second file are files of different types. Rendering the processing results to generate a model for augmented reality projection includes: obtaining node and cell data from the first file; obtaining scalar field data from the second file; and using the Open Graphics Library (OpenGL) to draw scalar cloud maps of the nodes, the cell data, and the scalar field data to obtain a model for augmented reality projection.

[0011] Optionally, the model and the aircraft panel are fused together using virtual and real methods, and the result of the virtual and real fusion is displayed using augmented reality. This includes: using a camera to identify a label pasted on the surface of the aircraft panel, wherein the label is a pre-made marker for virtual and real fusion; and overlaying the model onto the surface of the aircraft panel according to the label, and displaying mesh model partitioning cloud map, deformation cloud map and stress cloud map respectively.

[0012] Optionally, before using a camera to identify the label pasted on the surface of the aircraft panel, the method further includes: registering the label on the template used, and pasting the registered label onto the surface of the aircraft panel.

[0013] According to another aspect of the embodiments of the present application, there is also provided an augmented reality visualization device for an aircraft panel, comprising: a three-dimensional reconstruction module configured to acquire point cloud data of the aircraft panel and to establish a measured model of the aircraft panel according to the point cloud data of the aircraft panel; a load data acquisition module configured to acquire actual load data of the aircraft panel; a finite element simulation analysis module configured to perform finite element simulation analysis on the aircraft panel according to the measured model of the aircraft panel and the actual load data, to obtain a processing result; and an augmented reality display module configured to render the processing result to generate a model for augmented reality projection, to perform virtual-real fusion of the model and the aircraft panel, and to perform augmented reality display on the result after the virtual-real fusion.

[0014] According to another aspect of the embodiments of the present application, there is also provided an electronic device, comprising: a processor; a memory configured to store processor-executable instructions; and wherein the processor is configured to perform the method steps of any one of the above.

[0015] In the embodiments of the present application, the point cloud data of the aircraft panel is acquired, and the measured model of the aircraft panel is established according to the point cloud data of the aircraft panel; the actual load data of the aircraft panel is acquired; the finite element simulation analysis is performed on the aircraft panel according to the measured model of the aircraft panel and the actual load data, to obtain a processing result; the processing result is rendered to generate a model for augmented reality projection, the model is virtually fused with the aircraft panel, and the result after the virtual-real fusion is displayed in augmented reality. That is, the embodiments of the present application can perform the finite element simulation analysis on the aircraft panel according to the measured model of the aircraft panel and the actual load data of the aircraft panel, render the processing result obtained by the finite element simulation analysis to generate a model for augmented reality projection, then virtually fuse the model with the aircraft panel, and display the result after the virtual-real fusion in augmented reality, thereby effectively reducing the gap between the finite element simulation analysis result of the aircraft panel and the actual aircraft panel, and more intuitively displaying the simulation prediction result. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0017] Figure 1 A flowchart of the augmented reality visualization method for an aircraft panel provided by the embodiments of the present application;

[0018] Figure 2A schematic diagram of a processing flow of the aircraft panel augmented reality rapid finite element structure analysis system provided by the embodiment of the present application is shown in the figure;

[0019] Figure 3 A schematic diagram of a point cloud data processing flow of the aircraft panel provided by the embodiment of the present application is shown in the figure;

[0020] Figure 4 A schematic diagram of the three-point cloud data scanning provided by the embodiment of the present application is shown in the figure;

[0021] Figure 5 A flowchart of the scalar cloud chart drawing provided by the embodiment of the present application is shown in the figure;

[0022] Figure 6 A schematic diagram of the interface of the aircraft panel augmented reality rapid finite element structure analysis system provided by the embodiment of the present application is shown in the figure;

[0023] Figure 7 A schematic diagram of the augmented reality visualization device of the aircraft panel provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] In order to enable the person skilled in the art to better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, not to limit a specific order.

[0026] According to one aspect of the embodiment of the present application, an augmented reality visualization method of an aircraft panel is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0027] Figure 1 A flowchart of the augmented reality visualization method of the aircraft panel provided by the embodiment of the present application is shown in the figure, Figure 1 The method comprises the following steps:

[0028] Step S102, acquiring point cloud data of the aircraft panel, and establishing a measured model of the aircraft panel according to the point cloud data of the aircraft panel;

[0029] It should be noted that the actual measurement model of the aircraft panel is a three-dimensional grid model.

[0030] In step S104, actual load data of the aircraft panel is acquired.

[0031] The actual load data includes, but is not limited to, tension force, etc. Optionally, the aircraft panel to be assembled can be installed on a positioning tooling jig, and then a certain tension force is applied to the aircraft panel to simulate the stress process of assembly, and then the tension force borne by the aircraft panel is obtained through a mechanical sensor.

[0032] In step S106, finite element simulation analysis is performed on the aircraft panel according to the actual measurement model of the aircraft panel and the actual load data, and a processing result is obtained.

[0033] Optionally, the actual measurement model of the aircraft panel can be imported into a finite element simulation analysis software, and corresponding constraints and actual load data are automatically applied, and finite element simulation analysis is performed on the aircraft panel, and a processing result is output. It should be noted that the constraints can be set according to the needs of the application scenario.

[0034] In step S108, a model for augmented reality projection is generated by rendering the processing result, the model is fused with the aircraft panel, and the result after the fusion is displayed in augmented reality.

[0035] In the embodiment of the application, point cloud data of the aircraft panel is acquired, and an actual measurement model of the aircraft panel is established according to the point cloud data of the aircraft panel; actual load data of the aircraft panel is acquired; finite element simulation analysis is performed on the aircraft panel according to the actual measurement model of the aircraft panel and the actual load data, and a processing result is obtained; a model for augmented reality projection is generated by rendering the processing result, the model is fused with the aircraft panel, and the result after the fusion is displayed in augmented reality. That is, the embodiment of the application can perform finite element simulation analysis on the aircraft panel according to the actual measurement model of the aircraft panel and the actual load data of the aircraft panel, generate a model for augmented reality projection by rendering the processing result obtained by the finite element simulation analysis, then fuse the model with the aircraft panel, and display the result after the fusion in augmented reality, thereby solving the technical problem that the simulation environment of the traditional finite element deformation simulation is completely virtual and offline, and the simulation prediction result is not intuitive enough, and achieving the technical effects of effectively reducing the gap between the finite element simulation analysis result of the aircraft panel and the actual aircraft panel, and more intuitively displaying the simulation prediction result.

[0036] It should be noted that the application scenarios of the above method include, but are not limited to, aircraft panels, and the method can also be applied to finite element simulation analysis and augmented reality visualization of other components. When the processing object is not an aircraft panel, the implementation steps are consistent with those of the above method.

[0037] In one optional implementation, acquiring point cloud data of an aircraft panel includes: performing block scanning on the surface of the aircraft panel to obtain point cloud data of multiple target blocks; preprocessing the point cloud data of the multiple target blocks to obtain preprocessed point cloud data of multiple target blocks; and stitching the point cloud data of the multiple preprocessed target blocks together to obtain point cloud data of the aircraft panel.

[0038] In the specific implementation process, the first step is to initiate point cloud scanning. The laser contour sensor will scan the surface of the aircraft panel in blocks. Due to the large area of ​​the aircraft panel, it needs to be divided into multiple scans, each scanning one target block at a time, thus obtaining point cloud data for multiple target blocks. Then, the point cloud data of multiple target blocks undergoes preprocessing such as random noise removal, data filling, and data downsampling to obtain preprocessed point cloud data for multiple target blocks. In addition, to ensure the accuracy of the point cloud data, secondary filtering can be performed on the point cloud data of multiple target blocks. This secondary filtering includes, but is not limited to, using the moving least squares method for data denoising, data smoothing, and curvature calculation. Finally, the point cloud data of multiple preprocessed target blocks are stitched together to obtain the complete point cloud data of the aircraft panel.

[0039] In the above embodiments of the present invention, complete point cloud data of the aircraft panel can be obtained by methods such as block scanning, preprocessing and stitching.

[0040] In one optional implementation, the point cloud data of multiple target blocks are preprocessed to obtain preprocessed point cloud data of multiple target blocks, including: using the moving least squares method to filter and denoise the point cloud data of multiple target blocks to obtain preprocessed point cloud data of multiple target blocks.

[0041] Since the point cloud data obtained from scanning will be affected by noise, it is necessary to denoise the point cloud data of multiple target blocks. The least squares method can be used to filter and denoise the point cloud data of multiple target blocks, and the filtered and denoised point cloud data of multiple target blocks can be used as the preprocessed point cloud data of multiple target blocks.

[0042] In the above embodiments of the present invention, the moving least squares method is used to filter and denoise the point cloud data of multiple target blocks respectively, which can effectively reduce the adverse effects of noise on the point cloud data.

[0043] In one optional implementation, the point cloud data of multiple preprocessed target blocks are stitched together to obtain the point cloud data of the aircraft panel, including: using an iterative nearest-point algorithm to stitch together the point cloud data of multiple preprocessed target blocks to obtain the point cloud data of the aircraft panel.

[0044] Since the point cloud data of the plurality of pre-processed target sub-blocks are in different coordinate systems, the point cloud data needs to be spliced, for example, the corresponding point cloud data is spliced by using an iterative closest point algorithm. It should be noted that the above-mentioned splicing processing includes but is not limited to coarse splicing and fine splicing, etc.

[0045] In the above-mentioned embodiments of the present application, the point cloud data of the plurality of target sub-blocks is spliced by using the iterative closest point algorithm to obtain the point cloud data of the complete aircraft panel.

[0046] In an alternative embodiment, the measured model of the aircraft panel is established according to the point cloud data of the aircraft panel, comprising: three-dimensional reconstruction of the point cloud data of the aircraft panel by using a greedy projection triangulation algorithm to obtain the measured model of the aircraft panel.

[0047] In the above-mentioned embodiments of the present application, the three-dimensional reconstruction of the point cloud data of the spliced aircraft panel is realized by using the greedy triangulation method, thereby obtaining the measured model of the aircraft panel.

[0048] In an alternative embodiment, the different data sets in the above-mentioned processing results are recorded in a first file and a second file, the first file and the second file are different types of files, the rendering of the processing results generates a model for augmented reality projection, comprising: obtaining node and element data in the first file; obtaining scalar field data in the second file; using an open graphics library (OpenGL) to perform scalar cloud mapping on the node, element data and scalar field data to obtain the model for augmented reality projection.

[0049] It should be noted that the above-mentioned first file can be an inp file, and the second file can be an rpt file; further, the node and element data in the inp file are obtained, the scalar field data in the rpt file is obtained, then the OpenGL is used to perform scalar cloud mapping on the node, element data and scalar field data to realize the reconstruction of the model and the coloring of the model, and further to obtain the model for augmented reality projection.

[0050] In the above-mentioned embodiments of the present application, the model for augmented reality projection can be constructed by scalar cloud mapping.

[0051] In an alternative embodiment, the model is virtually fused with the aircraft panel, and the result after the virtual-real fusion is displayed in augmented reality, comprising: using a camera to recognize a label pasted on the surface of the aircraft panel, wherein the label is a pre-prepared marker for virtual-real fusion; superimposing the model on the surface of the aircraft panel according to the label to display a grid model division cloud map, a deformation cloud map and a stress cloud map, respectively.

[0052] In the implementation process, the camera can be used to recognize the label pasted on the surface of the aircraft panel, the model can be superimposed on the aircraft panel through the recognition of the label, so that the grid model drawing, the deformation cloud drawing and the stress cloud drawing can be realized, and the grid model, the deformation cloud and the stress cloud can be displayed.

[0053] In the above embodiment of the application, the model is superimposed on the surface of the aircraft panel through the virtual-real fusion, and the augmented reality display visualization is realized.

[0054] In an optional embodiment, before the camera is used to recognize the label pasted on the surface of the aircraft panel, the method further comprises: label registration of the used template, and pasting the registered label on the surface of the aircraft panel.

[0055] In the implementation process, the label and the template are made by using the augmented reality tool ARToolkit, then the label registration of the used template is performed, and the registered label is pasted on the surface of the aircraft panel, so that the label can be easily recognized in the subsequent process, the model is superimposed on the aircraft panel through the virtual-real fusion, and the result of the virtual-real fusion is displayed through the augmented reality display.

[0056] An optional embodiment of the application is described in detail below.

[0057] The optional embodiment of the application provides an aircraft panel real-time finite element simulation and augmented reality visualization system. First, an aircraft panel augmented reality rapid finite element structure analysis system is built, which at least comprises a three-dimensional reconstruction module, a load data acquisition module, a finite element simulation analysis module and an augmented reality display module. Figure 2 As shown in the schematic diagram of the process flow of the aircraft panel augmented reality rapid finite element structure analysis system provided by the embodiment of the application, Figure 2 The three-dimensional reconstruction module scans and collects the geometric information of the surface of the aircraft panel in the assembly site, and the point cloud data processing process of the three-dimensional reconstruction module obtains the measured model of the aircraft panel. The measured model is input into the finite element simulation analysis module as the basis for subsequent finite element processing. Then, the load data acquisition module uses appropriate sensor equipment to obtain the actual load data required in the assembly site, and inputs the converted data into the finite element simulation analysis module. Then, the finite element simulation analysis module divides the complete finite element analysis process into offline calculation and online simulation according to the obtained measured model and actual load data, and performs finite element analysis and calculation on the deformation of the aircraft panel through the finite element rapid analysis module. After the calculation is completed, the augmented reality display module is used to render the post-processing result of the finite element analysis, generate a model for augmented reality projection, and then perform virtual-real fusion of the model and the real object in the assembly site, and display the result through the computer display.

[0058] Further, the system further comprises a user interaction module for finite element simulation result analysis and tooling adjustment.

[0059] Further, the three-dimensional reconstruction module at least comprises a laser profile sensor, a Z1 -direction adjustable laser profile sensor support, a Y1 -direction linear guide rail sliding table, a base sliding rail, a Y1 -direction linear guide rail sliding table motion controller, a computer for laser profile sensor control and point cloud processing, and the like.

[0060] Optionally, the point cloud data of the surface of the aircraft panel is scanned by the laser profile sensor, and then the point cloud data is subjected to operations such as denoising, filtering, splicing and three-dimensional reconstruction. Figure 3 A schematic diagram of a point cloud data processing flow of an aircraft panel provided by an embodiment of the present application is shown in Figure 3 As shown, for a real aircraft panel on site, point clouds are collected, stored and subjected to filtering preprocessing, then the point cloud data is read in, subjected to secondary filtering of point clouds, splicing of point cloud data and generation of a measured model, and finally a measured model of the aircraft panel on site is obtained, wherein the filtering preprocessing includes random noise removal, data filling and data downsampling, the reading in of the point cloud data includes GenTL driving, 16-bit RGB images and Object Model 3D, the secondary filtering of point clouds includes a moving least squares method, data smoothing and curvature calculation, the splicing of point cloud data includes coarse splicing and fine splicing, and the generation of the measured model includes a greedy projection triangulation algorithm.

[0061] Optionally, the point cloud data of the aircraft panel is acquired by using the laser profile sensor.

[0062] Optionally, the point cloud data is subjected to smoothing processing and random noise removal by using a moving least squares method.

[0063] Since the area of the aircraft panel is large, the aircraft panel cannot be completely scanned at one time, so the aircraft panel is scanned three times, and then the point cloud data of the three times of scanning is subjected to splicing processing, Figure 4 A schematic diagram of three times of point cloud data scanning provided by an embodiment of the present application is shown in Figure 4 As shown, the target blocks are local point cloud 1, local point cloud 2 and local point cloud 3.

[0064] The splicing of the three times of scanning point cloud data is implemented by using an iterative closest point (ICP) algorithm, and the splicing of the point cloud data is to unify the point cloud data in different coordinate systems to one coordinate. Therefore, the point cloud data in different coordinate systems needs to be subjected to rigid body transformation, and the principle of the rigid body transformation is as follows: assuming that the coordinates of point P in coordinate system O-XYZ are P[x, y, z], and the coordinates of point Q in coordinate system O-UVW are Q[u, v, w], then the rigid body transformation of coordinate system O-XYZ to coordinate system O-UVW is as follows:

[0065]

[0066] It should be noted that R represents the rotation matrix to be solved, and T represents the translation matrix to be solved.

[0067] During data acquisition, a point cloud initial pose with little difference from the real position is obtained by setting a proper coordinate system offset inside the sensor, and then the ICP algorithm is used to realize point cloud splicing.

[0068] The generated aircraft panel measured model is obtained by splicing the point cloud data, and the greedy projection triangulation algorithm is used for three-dimensional reconstruction of the 3D point cloud data.

[0069] The load data acquisition module can obtain the force of the aircraft panel at the corresponding position through the mechanical sensor;

[0070] The finite element simulation analysis module realizes the finite element simulation analysis of the aircraft panel in the assembly field by starting Abaqus, reads the inp file and rpt file output after Abaqus simulation, obtains the node and element data in the inp file, obtains the scalar field data in the rpt file, and then uses OpenGL to reconstruct the model and colorize the model. Figure 5 The flow chart of the scalar cloud drawing provided by the embodiment of the application is shown in Figure 5 The node and element data are read based on the inp file, the node and element data are stored in the txt file, and the node and element data are read by OpenGL; in addition, the scalar field data are read based on the rpt file, the scalar field data are stored in the txt file, the scalar field data are read by OpenGL, the maximum and minimum values of the field quantity are calculated by OpenGL, the color lookup table is established in OpenGL, and the color is mapped with the scalar value of each node in OpenGL; then, the scalar cloud drawing is performed according to the node and element data read by OpenGL and the mapping of the color with the scalar value of each node in OpenGL; if the node number i in the loop is greater than the number of elements at this time, the process is ended, otherwise the scalar cloud drawing is continuously performed.

[0071] The augmented reality display module mainly superimposes the simulated color cloud on the surface of the actual aircraft panel, mainly uses a virtual-real fusion method based on a marker, uses the augmented reality tool ARToolkit to make the label, calibrates the camera to obtain the internal and external parameters of the camera, and finally performs augmented reality display on the cloud. The augmented reality rapid finite element structure analysis system for the aircraft panel is designed by using MATLAB, Figure 6A schematic diagram of an interface of an aircraft panel augmented reality rapid finite element structure analysis system provided by an embodiment of the present application is shown in FIG. 1. Figure 6 As shown in FIG. 1, the system interface includes three-dimensional reconstruction, load data acquisition, finite element simulation analysis, and augmented reality display, wherein the three-dimensional reconstruction includes point cloud scanning, point cloud format conversion, three-dimensional reconstruction, and model format conversion operations, the load data acquisition includes sensor force value and conversion operations, the finite element simulation analysis includes Abaqus starting operation, the augmented reality display includes platform initialization, parameter setting, and real-time display three parts, wherein the platform initialization includes camera calibration and template registration operations, the parameter setting includes grid model parameter reading and scalar cloud map parameter reading operations, and the real-time display includes grid model drawing, deformation cloud map drawing, and stress cloud map drawing operations.

[0072] According to another aspect of the embodiment of the present application, an augmented reality visualization device for an aircraft panel is also provided, Figure 7 A schematic diagram of the augmented reality visualization device for an aircraft panel provided by an embodiment of the present application is shown in FIG. 2. Figure 7 As shown in FIG. 2, the augmented reality visualization device for an aircraft panel includes a three-dimensional reconstruction module 702, a load data acquisition module 704, a finite element simulation analysis module 706, and an augmented reality display module 708. The augmented reality visualization device for an aircraft panel is described in detail as follows.

[0073] The three-dimensional reconstruction module 702 is configured to acquire point cloud data of an aircraft panel and establish a measured model of the aircraft panel according to the point cloud data of the aircraft panel.

[0074] The load data acquisition module 704 is configured to acquire actual load data of the aircraft panel.

[0075] The finite element simulation analysis module 706 is connected to the three-dimensional reconstruction module 702 and the load data acquisition module 704, respectively, and is configured to perform finite element simulation analysis on the aircraft panel according to the measured model of the aircraft panel and the actual load data to obtain a processing result.

[0076] The augmented reality display module 708 is connected to the finite element simulation analysis module 706 and is configured to render the processing result to generate a model for augmented reality projection, perform virtual-real fusion of the model and the aircraft panel, and perform augmented reality display on the result after the virtual-real fusion.

[0077] In the embodiment of the present application, the point cloud data of the aircraft panel is acquired, and the measured model of the aircraft panel is established according to the point cloud data of the aircraft panel; the actual load data of the aircraft panel is acquired; the finite element simulation analysis of the aircraft panel is performed according to the measured model of the aircraft panel and the actual load data, and the processing result is obtained; the model for augmented reality projection is generated by rendering the processing result; the model is fused with the aircraft panel; and the augmented reality display is performed on the fused result. That is, the finite element simulation analysis of the aircraft panel can be performed according to the measured model of the aircraft panel and the actual load data of the aircraft panel, the model for augmented reality projection is generated by rendering the processing result obtained by the finite element simulation analysis, the model is fused with the aircraft panel, and the augmented reality display is performed on the fused result, thereby solving the technical problem that the simulation environment of the traditional finite element deformation simulation is completely virtual and offline, and the simulation prediction result is not intuitive enough, and achieving the technical effects of effectively reducing the gap between the finite element simulation analysis result of the aircraft panel and the actual aircraft panel, and more intuitively displaying the simulation prediction result.

[0078] It should be noted that the three-dimensional reconstruction module 702, the load data acquisition module 704, the finite element simulation analysis module 706, and the augmented reality display module 708 correspond to steps S102 to S108 in the method embodiment, and the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above method embodiment.

[0079] In an optional implementation, the three-dimensional reconstruction module 702 includes: a scanning unit configured to perform block scanning on the surface of the aircraft panel to obtain point cloud data of a plurality of target blocks; a preprocessing unit configured to respectively preprocess the point cloud data of the plurality of target blocks to obtain preprocessed point cloud data of the plurality of target blocks; and a splicing unit configured to perform splicing processing on the preprocessed point cloud data of the plurality of target blocks to obtain the point cloud data of the aircraft panel.

[0080] In an optional implementation, the preprocessing unit includes a denoising subunit configured to respectively filter and denoise the point cloud data of the plurality of target blocks using a moving least square method to obtain the preprocessed point cloud data of the plurality of target blocks.

[0081] In an optional implementation, the splicing unit includes a splicing subunit configured to perform splicing on the preprocessed point cloud data of the plurality of target blocks using an iterative closest point algorithm to obtain the point cloud data of the aircraft panel.

[0082] In an optional implementation, the three-dimensional reconstruction module 702 comprises a three-dimensional reconstruction unit configured to perform three-dimensional reconstruction on the point cloud data of the aircraft panel by using a greedy projection triangulation algorithm to obtain the measured model of the aircraft panel.

[0083] In an optional implementation, the different data sets in the processing result are recorded in a first file and a second file respectively, the first file and the second file are different types of files, and the augmented reality display module 708 comprises a first acquisition unit configured to acquire the node and element data in the first file, a second acquisition unit configured to acquire the scalar field data in the second file, and a drawing unit configured to perform scalar cloud drawing on the node, element data and scalar field data by using an open graphics library (OpenGL) to obtain a model for augmented reality projection.

[0084] In an optional implementation, the augmented reality display module 708 comprises an identification unit configured to identify a label pasted on the surface of the aircraft panel by using a camera, wherein the label is a pre-prepared marker for virtual-real fusion, and a superimposed display unit configured to superimpose the model on the surface of the aircraft panel according to the label to display the mesh model division cloud, the deformation cloud and the stress cloud respectively.

[0085] In an optional implementation, the apparatus further comprises a registration and pasting module configured to perform label registration on a template to be used before identifying the label pasted on the surface of the aircraft panel by using the camera, and paste the registered label on the surface of the aircraft panel.

[0086] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising a processor, a memory for storing processor-executable instructions, wherein the processor is configured to perform the method steps of any one of the above.

[0087] The above merely describes the preferred embodiments of the present application, but not for limiting the protection scope of the present application.

Claims

1. An augmented reality visualization method for aircraft panel, characterized in that, include: Acquiring point cloud data of an aircraft panel includes: performing block scanning on the surface of the aircraft panel to obtain point cloud data of multiple target blocks; preprocessing the point cloud data of the multiple target blocks to obtain preprocessed point cloud data of multiple target blocks; stitching together the point cloud data of the multiple preprocessed target blocks to obtain point cloud data of the aircraft panel; and establishing a measured model of the aircraft panel based on the point cloud data of the aircraft panel, including: using a greedy projection triangulation algorithm to perform three-dimensional reconstruction of the point cloud data of the aircraft panel to obtain the measured model of the aircraft panel. Obtain the actual load data of the aircraft panel; the actual load data includes the tension force received by the aircraft panel; Based on the measured model of the aircraft panel and the actual load data, a finite element simulation analysis was performed on the aircraft panel to obtain the processing results. Different datasets in the processing results were recorded in a first file and a second file, which are of different types. The first file includes node and element data, and the second file includes scalar field data. Rendering the processing results to generate a model for augmented reality projection includes: obtaining node and cell data from the first file; obtaining scalar field data from the second file; and using the OpenGL library to draw scalar cloud maps of the nodes, cell data, and scalar field data to obtain a model for augmented reality projection. The different datasets in the processing results are recorded in the first file and the second file, respectively, and the first file and the second file are files of different types. The model is fused with the aircraft panel, and the result of the fusion is displayed in augmented reality. This includes: using a camera to identify a label pasted on the surface of the aircraft panel, wherein the label is a pre-made marker for fusion; and overlaying the model onto the surface of the aircraft panel based on the label, thereby displaying mesh model partitioning cloud map, deformation cloud map, and stress cloud map respectively.

2. The method according to claim 1, characterized in that, The point cloud data of the multiple target blocks are preprocessed to obtain preprocessed point cloud data of the multiple target blocks, including: The point cloud data of the multiple target blocks are filtered and denoised using the moving least squares method to obtain the preprocessed point cloud data of the multiple target blocks.

3. The method according to claim 1, characterized in that, The point cloud data of the preprocessed target blocks are stitched together to obtain the point cloud data of the aircraft panel, including: The point cloud data of the aircraft panel is obtained by stitching together the point cloud data of multiple preprocessed target blocks using the iterative nearest point algorithm.

4. The method according to claim 1, characterized in that, Before using a camera to identify labels affixed to the surface of the aircraft panel, the method further includes: Register the templates used with tags and then paste the registered tags onto the surface of the aircraft panel.

5. An augmented reality visualization device for aircraft panel, characterized in that, include: The three-dimensional reconstruction module is used to acquire point cloud data of the aircraft panel and establish a measured model of the aircraft panel based on the point cloud data of the aircraft panel. The 3D reconstruction module includes: a scanning unit for scanning the surface of the aircraft panel in blocks to obtain point cloud data of multiple target blocks; a preprocessing unit for preprocessing the point cloud data of multiple target blocks to obtain preprocessed point cloud data of multiple target blocks; a stitching unit for stitching the point cloud data of multiple preprocessed target blocks to obtain point cloud data of the aircraft panel; and a 3D reconstruction unit for using a greedy projection triangulation algorithm to perform 3D reconstruction of the point cloud data of the aircraft panel to obtain a measured model of the aircraft panel. The load data acquisition module is used to acquire the actual load data of the aircraft panel; The finite element simulation analysis module is used to perform finite element simulation analysis on the aircraft panel based on the measured model of the aircraft panel and the actual load data, and obtain the processing results. An augmented reality display module is used to render the processing results to generate a model for augmented reality projection, blend the model with the aircraft panel, and display the blended result in augmented reality. The augmented reality display module includes: a recognition unit for using a camera to recognize labels pasted on the surface of the aircraft panel, wherein the labels are pre-made markers for virtual-real fusion; an overlay display unit for overlaying the model onto the surface of the aircraft panel according to the labels, displaying mesh model partitioning cloud maps, deformation cloud maps, and stress cloud maps respectively; a first acquisition unit for acquiring node and element data from a first file; a second acquisition unit for acquiring scalar field data from a second file; and a rendering unit for using the OpenGL library to render scalar cloud maps of the node, element, and scalar field data to obtain a model for augmented reality projection. The different datasets in the processing results are recorded in the first and second files respectively, and the first and second files are files of different types.

6. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method steps of any one of claims 1 to 4.

Images