Aerospace engine deep-blind narrow inner cavity intelligent assembling method based on mixed reality

Abstract

The invention provides an aerospace engine deep-blind narrow inner cavity intelligent assembly method based on mixed reality, and the method is based on the mixed reality technology, through constructing a novel human-centered multi-dimensional data fusion intelligent assembly mechanism, and through combining virtual-real matching and image homogenization technologies, the deep-blind narrow inner cavity of an aerospace engine is assembled. Novel'human-centered 'deep-blind narrow-inner-cavity intelligent assembly of the aerospace engine is realized, and a referential scheme is provided for assembly of the aerospace engine.

Description

Technical Field

[0001] The present invention relates to the technical field of deep-blind narrow inner cavity assembly of aerospace engines, and in particular to an intelligent assembly method of deep-blind narrow inner cavity of aerospace engines based on mixed reality. Background Art

[0002] The deep, blind, and narrow internal cavity of the turbine duct rotor and stator structure of aerospace engines complicates assembly and measurement, making it difficult to ensure overall assembly accuracy. The narrowest point of the rotor and stator cavity in an aerospace engine turbine duct is only 50mm, while the overall height can reach 1000mm, with a depth-to-diameter ratio of up to 20:1. Due to these deep, blind, and narrow internal characteristics, direct observation of the rotor and stator's position during assembly is difficult. Currently, assembly positions are primarily determined by the assembly worker's experience, severely limiting assembly accuracy and efficiency.

[0003] Intelligent guided assembly based on mixed reality is expected to be an effective way to improve the assembly accuracy of deep, blind, and narrow cavities. Mixed reality technology can combine virtual models of deep, blind, and narrow engine cavities with the actual object, providing precise visual guidance and accurate prediction of assembly conditions. Currently, mixed reality technology has not been effectively applied in the assembly process of deep, blind, and narrow cavities in aerospace engines. Therefore, the development of an intelligent assembly system for deep, blind, and narrow cavities in aerospace engines is urgently needed. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of mixed reality imaging degradation, virtual-reality matching mismatching, difficulty in decoupling multi-dimensional data of the inner cavity, and excessive assembly errors caused by complex inner cavity lighting. A mixed reality-based intelligent assembly method for deep-blind narrow inner cavities of aerospace engines is proposed.

[0005] The present invention is achieved through the following technical solutions. The present invention proposes a method for intelligent assembly of aerospace engines with deep blind and narrow inner cavities based on mixed reality. The method is specifically as follows:

[0006] Construct a deep-blind narrow cavity visualization system based on virtual-reality matching. After completing the virtual-reality matching, the images taken by the cavity camera need to be processed for illumination uniformity, and a real-time interaction mechanism is established with the mixed reality system. Based on the fusion of high-definition images of deep-blind cavities and real-time assembly data based on mixed reality, the monitoring capability of cavity assembly characteristics is optimized, and a new "human-centered" intelligent assembly system for deep-blind narrow cavities of aerospace engines is constructed.

[0007] Furthermore, virtual-real matching is the basis for realizing the intelligent visualization guidance assembly of deep blind narrow cavity mixed reality. It reflects the state of the actual object through the virtual model; uses the registration of the environment point cloud and the model point cloud to realize the transfer of entity and model coordinates and ensure the accuracy of virtual-real matching; from the point cloud registration form, a 3D corresponding set Rigid transformation Point Cloud Perform rotation and translation operations:

[0008]

[0009] Where R is the rotation matrix and t is the translation matrix; the rigid transformation T* that makes point cloud x closest to point cloud y is considered the optimal transformation for point cloud matching:

[0010]

[0011] Furthermore, based on the expectation maximization EM algorithm, the best high-frequency detail image is selected iteratively; for the initialization of the basis μ, Gaussian initialization is adopted to satisfy Let μ∈R K×C , k is 1, C is the number of images, X∈R N×C is a set of Rx series images, where N is the number of all pixels in an image. After T iterations, the weight μ is obtained, and then the weight is used to adjust each image:

[0012]

[0013] Represents the defogging pseudo image after fusion, and the resize() function represents the adjustment of the image size so that Change to the original image size; the transmittance is estimated as follows:

[0014]

[0015] Where t1 and t2 represent the upper and lower limits of transmittance. The initially obtained transmittance is optimized using the guided filter to achieve the final transmittance value. The final clear illumination uniformity map is expressed as:

[0016]

[0017] Furthermore, t1 and t2 are set to 0 and 0.97; ω and θ are both set to 0.95.

[0018] Furthermore, the communication is based on the socket asynchronous communication mechanism and the 5G / WiFi 6 module. Operators can monitor real-time data by wearing mixed reality devices and realize intelligent assembly guidance through gesture recognition, voice recognition or iris recognition methods.

[0019] The present invention has the following beneficial effects:

[0020] This paper proposes a mixed reality-based intelligent assembly method for aerospace engines with deep blind spots and narrow cavities. Based on mixed reality technology, this method constructs a novel "human-centric" multidimensional data fusion intelligent assembly mechanism, combined with virtual-reality matching and image homogenization technology. This method achieves a novel "human-centric" intelligent assembly of aerospace engines with deep blind spots and narrow cavities, providing a reference solution for aerospace engine assembly. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The mixed reality virtual-real matching results are shown in Figure 1. (a) Before the mixed reality virtual-real matching, (b) After the mixed reality virtual-real matching.

[0022] Figure 2 This is a new type of "human-centered" multi-dimensional data fusion intelligent assembly mechanism and image illumination uniformity processing effect diagram.

[0023] Figure 3 A framework diagram for the implementation of a new type of "human-centered" intelligent guided assembly. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] The present invention proposes a method for intelligent assembly of deep blind narrow cavities of aerospace engines based on mixed reality. This method uses mixed reality technology to monitor the cavity environment and provide assembly guidance for the special structure of deep blind narrow cavities of aerospace engines. The main work content includes analyzing the structural characteristics of deep blind narrow cavities, combining mixed reality virtual-reality matching technology, and studying real-time visualization technology for virtual models to track the assembly status of the cavity; studying a cavity visualization method based on image enhancement and illumination homogenization for the complex assembly environment of deep blind cavities, and establishing a high-frequency detail processing model based on expectation maximization; based on mixed reality fusion of multi-source data such as deep blind cavity high-definition images and real-time assembly data, optimizing the monitoring capability of cavity assembly characteristics, exploring mixed reality intelligent assembly mechanisms with multi-dimensional data fusion in complex assembly environments, and constructing a new "human-centered" intelligent assembly system for deep blind narrow cavities of aerospace engines.

[0026] Specifically, see Figure 1-Figure 3 The present invention proposes a method for intelligent assembly of deep blind narrow cavity of aerospace engine based on mixed reality, and the method is specifically as follows:

[0027] Construct a deep-blind narrow cavity visualization system based on virtual-reality matching. After completing the virtual-reality matching, the images taken by the cavity camera need to be processed for illumination uniformity, and a real-time interaction mechanism is established with the mixed reality system. Based on the fusion of high-definition images of deep-blind cavities and real-time assembly data based on mixed reality, the monitoring capability of cavity assembly characteristics is optimized, and a new "human-centered" intelligent assembly system for deep-blind narrow cavities of aerospace engines is constructed.

[0028] Virtual-real matching is the basis for realizing the intelligent visualization guidance assembly of deep blind narrow cavity mixed reality. It reflects the state of the actual object through the virtual model; uses the registration of the environment point cloud and the model point cloud to realize the transfer of entity and model coordinates and ensure the accuracy of virtual-real matching; from the point cloud registration form, a 3D corresponding set Rigid transformation Point Cloud Perform rotation and translation operations:

[0029]

[0030] Where R is the rotation matrix and t is the translation matrix; so that the point cloud Closest point cloud The rigid transformation T* is considered to be the optimal transformation for point cloud matching:

[0031]

[0032] After completing the virtual-real matching, the image captured by the intracavity camera needs to be processed for illumination uniformity. Based on the expectation maximization EM algorithm, the optimal high-frequency detail image is selected iteratively; for the initialization of the basis μ, Gaussian initialization is adopted to satisfy Since the number of bases can be chosen freely, let μ∈R K×C , k is 1, C is the number of images, X∈R N ×C is a set of Rx series images, where N is the number of all pixels in an image. After T iterations, the weight μ is obtained, and then the weight is used to adjust each image:

[0033]

[0034] Represents the defogging pseudo image after fusion, and the resize() function represents the adjustment of the image size so that Change to the original image size; the transmittance is estimated as follows:

[0035]

[0036] Where t1 and t2 represent the upper and lower limits of transmittance; t1 and t2 are set to 0 and 0.97; ω and θ are both set to 0.95; the initial transmittance obtained will have some pixel discontinuities, and the guide filter can be used to optimize the transmittance to achieve the final transmittance value; the final clear illumination uniformity map is expressed as:

[0037]

[0038] After image processing, a mixed reality device is used to monitor internal cavity assembly characteristics, leveraging multi-source data, including high-definition images of deep-blind internal cavities and real-time assembly data. Communication is primarily based on asynchronous socket communication and 5G / WiFi 6 modules. Operators wearing mixed reality devices monitor real-time data and receive intelligent assembly guidance through gesture recognition, voice recognition, and iris recognition. This creates a mixed reality intelligent assembly mechanism that integrates multidimensional data in complex assembly environments, integrating operators into the interactive engine assembly system and embodying a new "human-centered" assembly approach.

[0039] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0040] The virtual-reality matching process uses the mixed reality device's depth sensor, RGB sensor, and IMU sensor to acquire depth and RGB image data streams, obtain their orientation information, and calculate an environmental point cloud. Using virtual-reality matching, the environmental point cloud containing the actual engine is registered with the virtual engine point cloud. Figure 1 The new "human-centered" multi-dimensional data fusion intelligent assembly mechanism and image illumination uniformity processing are shown. Figure 2 The new “human-centered” intelligent guided assembly implementation framework is shown in Figure 3 Its core components include a mixed reality device, an assembly model, and an assembly assistance display. The mixed reality device combines voice recognition, gesture recognition, and iris recognition to control and interact with the virtual model, ensuring operational safety.

[0041] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the claims.

Claims

1. A mixed reality-based intelligent assembly method for deep blind narrow cavity aerospace engines, characterized by: The method is specifically as follows: Construct a deep blind narrow cavity visualization system based on virtual-reality matching. After completing the virtual-reality matching, the images taken by the cavity camera need to be processed for illumination uniformity, and a real-time interaction mechanism is established with the mixed reality system. Based on the fusion of high-definition images of deep blind cavities and real-time assembly data based on mixed reality, the monitoring capability of cavity assembly characteristics is optimized, and a new "human-centered" intelligent assembly system for deep blind narrow cavities of aerospace engines is constructed.

2. The method according to claim 1, characterized in that Virtual-real matching is the basis for realizing the intelligent visualization guidance assembly of deep blind narrow cavity mixed reality. It reflects the state of the actual object through the virtual model; uses the registration of the environment point cloud and the model point cloud to realize the transfer of entity and model coordinates and ensure the accuracy of virtual-real matching; from the point cloud registration form, a 3D corresponding set Rigid transformation Point Cloud Perform rotation and translation operations: Where R is the rotation matrix and t is the translation matrix; so that the point cloud Closest point cloud The rigid transformation T* is considered to be the optimal transformation for point cloud matching:

3. The method according to claim 2, characterized in that Based on the expectation maximization EM algorithm, the best high-frequency detail image is selected iteratively; for the initialization of the basis μ, Gaussian initialization is adopted to satisfy Let μ∈R K×C , k is 1, C is the number of images, X∈R N×C is a set of Rx series images, where N is the number of all pixels in an image. After T iterations, the weight μ is obtained, and then the weight is used to adjust each image: Represents the defogging pseudo image after fusion, and the resize() function represents the adjustment of the image size so that Change to the original image size; the transmittance is estimated as follows: Where t1 and t2 represent the upper and lower limits of transmittance. The initially obtained transmittance is optimized using the guided filter to achieve the final transmittance value. The final clear illumination uniformity map is expressed as:

4. The method according to claim 3, characterized in that t1 and t2 are set to 0 and 0.97; ω and θ are both set to 0.

95.

5. The method according to claim 4, characterized in that Communication is based on the socket asynchronous communication mechanism and 5G / WiFi6 module. Operators can monitor real-time data by wearing mixed reality devices and realize intelligent assembly guidance through gesture recognition, voice recognition or iris recognition methods.

Images