System and method for "Augmented Reality" inspection of a gas turbine or steam turbine and data visualization
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- GENERAL ELECTRIC TECH GMBH
- Filing Date
- 2008-12-15
- Publication Date
- 2026-07-30
AI Technical Summary
Existing inspection methods for complex structures, such as aircraft engines and gas turbines, rely heavily on operator skill and are subjective, lacking precise tracking of the endoscope tip's location and position, leading to inaccuracies and inefficiencies in measurement and damage detection.
A 3D tracking device with acoustic emission sensors and sensors for height and azimuth detection, combined with an augmented reality system to overlay precise graphical information onto real-time camera images, providing accurate probe location and orientation data.
Enhances inspection accuracy by enabling precise tracking and automated data overlay, reducing human error and improving damage detection and reporting efficiency.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates generally to non-destructive inspection techniques and in particular to inspection techniques that utilize augmented reality. (augmented reality: AR).
[0002] Inspection or viewing techniques are commonly used in a variety of applications, including those in the aircraft industry. Applications range from the medical industry to security systems. The inspection of complex parts and structures generally requires... Extensive inspection skills and experience. Inspection with an industrial endoscope is one of the most frequently used sources of information. for monitoring industrial infrastructure, due to easy access to the parts requiring maintenance while simultaneously reducing downtime. Condition-based maintenance strategies for gas turbines and similar systems depend heavily on the data obtained through a Such an inspection will be obtained. Generally, measuring probes with long cables and a display device on a display or Use the display swivel arm, which is used for inspection with an industrial endoscope. However, once the measuring probe is in the inspection hole for The industrial endoscope has been introduced, but little information is available to the operator regarding the location and position of the endoscope's tip. available. Tracking the location and position reduces measurement errors and is crucial for accurately identifying defects and Locating damage is difficult. Furthermore, it is nearly impossible to keep the tip in the same position while the tip's behavior changes.
[0003] Consequently, much of the inspection depends on the operator's skill and is subjective. More detailed information about the tip of the The industrial endoscope and its position also enable the automation and control of an entire inspection process, starting with the Inspection planning, from execution to damage reporting.
[0004] There is therefore a need for an improved inspection system that solves the problems mentioned above. SHORT DESCRIPTION
[0005] According to one embodiment of the invention, a 3D tracking device is created. The 3D tracking device It contains at least two acoustic emission sensors arranged around an object. The acoustic emission sensors are configured to locate a A measuring probe inserted into the object is used to identify it based on the arrival of an acoustic signal emanating from a location on or near it. emitted by the measuring probe. The 3D tracking device also includes a first measuring probe, which is set up to determine the height of the The 3D tracking device also includes a second measuring probe configured to detect an azimuth or To detect the direction angle of the measuring probe.
[0006] According to another embodiment of the invention, an “augmented reality system” (AR system) is used for the inspection of the created inside an object. The "augmented reality system" includes a tracking device designed to to identify the 3D location of a measuring probe inserted into the object. The 3D tracking device includes at least one An acoustic emission sensor arranged around an object. The acoustic emission sensors are set up to determine the location of a measuring probe that is inserted into the object is introduced, to identify it based on the arrival time of a sequence of sound signals or sound signature emanating from a location on or near emitted by the measuring probe. The 3D tracking device also includes a first sensor, which is set up to determine the height of the measuring probe. to detect probe. The 3D tracking device also includes a second sensor that is set up to detect a direction angle or to detect the azimuth of the measuring probe. The "augmented reality system" also contains a microprocessor configured to... to generate graphics and overlay the graphics onto the image captured by the camera based on the 3D location defined by the tracking device has been identified. The “augmented reality system” also includes a display or representation unit, which is located in the The invention is referred to as a display unit which is set up to display an augmented reality image.
[0007] According to a further embodiment of the invention, a method for 3D tracking within an object is created. The procedure involves inserting a measuring probe into the object. The procedure also includes positioning at least one The method involves placing an acoustic emission sensor around the object. It further includes attaching a first sensor and a second sensor to the measuring probe.
[0008] According to a further embodiment of the invention, a method for creating an augmented reality image for inspection inside an object is provided. The method includes capturing an image using a camera. The method also includes the Identification of the measuring probe's location within the object using multiple acoustic emission sensors. The method further includes The method involves determining the height of the measuring probe using a first sensor. It also includes determining the azimuth of the measuring probe. Probe using a second sensor. The method also includes the generation of a graphic of the object. The method further includes the Registration of the graphic on the image, which was taken based on the location, altitude and azimuth, to create an image of the extended To create reality. DRAWING
[0009] These and other features, aspects and advantages of the present invention will be better understood when the following more detailed The description is read in relation to the following drawing, in which the same reference numerals in all figures denote the same parts, wherein:
[0010] Fig. 1 is a block diagram representation of an “augmented reality system” which is a tracking device according to a embodiment of the invention includes;
[0011] Fig. 2 is a block diagram representation of elements within the tracking device of Fig. 1;
[0012] Fig. 3 is a diagram representation of an exemplary industrial endoscope according to an embodiment of the invention;
[0013] Fig. 4 is a schematic representation of an “augmented reality image” used to inspect a gas turbine using of the industrial endoscope in Fig. 3;
[0014] Fig. 5 is a schematic representation of an exemplary display unit according to an embodiment of the invention;
[0015] Fig. 6 is a flowchart that illustrates the steps in an exemplary method for 3D tracing within an object; and
[0016] Fig. 7 is a flowchart showing the steps in an exemplary method for creating an “augmented reality image” for Inspection within an object. DETAILED DESCRIPTION
[0017] As will be explained in more detail below, embodiments of the invention include a system and a method for not Destructive or non-destructive inspection of an object. The system and method disclosed below generate a “Augmented reality image” or image of enhanced reality using an improved tracking device for inspection. As used herein, an "augmented reality image" refers to an image that contains real-world data superimposed with computer-generated Data are superimposed. Non-restrictive examples of the object include aircraft engines, gas turbines, steam turbines, diesel engines, and living organisms.
[0018] Referring to the drawing below, Fig. 1 shows a block diagram comprising several levels or steps, which represents an “augmented reality system” 10 or system of extended reality 10 for inspecting an object 12. A Tracking device 14 is used to identify a 3D location and 3D position of a measuring probe 13 inserted into the object 12. is introduced. In a particular embodiment, the measuring probe 13 includes an industrial endoscope or an endoscope. The camera 18 takes a view of object 12 as a real image. In a particular embodiment, the camera 18 captures a monocular image. In a In another embodiment, the camera 18 records a stereo image. Non-limiting examples of the camera 18 include a webcam, a A video camera or a CCD camera. In another embodiment, more than one camera can be used; for example, Two cameras are arranged in such a way as to capture a stereo image. Non-restrictive examples of the real image include a Video image or a single image.
[0019] The recorded real image is used as a reference by a microprocessor 20, which is configured to generate a graphic, that belong to the real image. In one example, the graphic contains computer-aided drawings of object 12. The microprocessor 20 further overlays the graphics onto the real image based on the 3D location determined by the tracking device. 14 is identified as being used to create an “augmented reality image”. Consequently, the stereo view obtained from camera 18 is augmented with Additional information is added and provided to the user in real time. This additional information can include, for example: text, Sound, image sequences or videos, and individual images. For example, in a medical or surgical setting, a doctor is shown a view of a Patients provided, including, among other things, the patient's view and an overlap generated by the microprocessor 20. The overlay can include a view of the patient's internal anatomical structures, such as those seen during an axial tracheostomy. Computer Tomo was obtained through graphy scans or magnetic resonance imaging (MR imaging).
[0020] In another embodiment, the overlap includes a text view of the patient's medical and family history. The overlay can be displayed in real time. The "augmented reality image" contains the real image captured by the camera. The recorded image is overlaid with an additional virtual view. The virtual view is generated by microprocessor 20 and the stored data. Information, such as images, can be obtained. The "augmented reality image" also enables the detection of defects or cracks. in object 12. In a particular embodiment, the microprocessor 20 contains a portable computer. The microprocessor 20 represents a The “augmented reality image” is displayed on the display unit 22, but is not limited to use on a digital personal assistant (personal). digital assistant (PDA), an external computer, or a semi-transparent screen on a swivel arm.
[0021] It should be noted that embodiment of the invention is not limited to a specific microprocessor for carrying out the The processing tasks of the invention are limited. The term "microprocessor", as used herein, is intended to encompass any machine. to designate who is able to perform the calculations or computations that are necessary and the expenses of the invention to carry out. The term "microprocessor" is intended to refer to any machine capable of processing structured input. to accept and process the input according to the previous rules in order to generate an output.
[0022] Fig. 2 is a block diagram illustrating elements 30 within the tracking device 14 in Fig. 1. The Tracking device 14 includes at least one acoustic emission sensor 32, which is arranged around the object 12 (Fig. 1). The acoustic emission sensor 32 is set up to identify the location of the measuring probe 13 (Fig. 1) which is inserted into the object 12 for inspection. In one particular embodiment, the sound emission sensor has a diameter in a range between approximately 6 mm and approximately 12 mm. The location is calculated based on the arrival time of an acoustic signal emitted by object 12. In one embodiment, the acoustic signature is emitted via a single loudspeaker or loudspeakers located at the tip of the measuring probe 13. A first sensor 34 detects the height of the measuring probe. In one embodiment, the first sensor includes a gravity sensor integrated into a micro-electrical mechanical system (MEMS) and configured to detect the height based on acceleration due to gravity.In another embodiment, the first sensor 34 includes a gyroscope implemented in an electrical micromechanical system (MEMS) and configured to detect the height based on the conservation of angular momentum of the motion. In another embodiment, the output of the first sensor 34 is integrated over time to determine the position of the tip of the measuring probe 13. In a non-restrictive example, the acceleration is integrated twice over time to determine the position.
[0023] The tracking device 14 further includes a second sensor 36 configured to detect the azimuth of the measuring probe 13. In one exemplary embodiment, the second sensor 36 includes a magnetic sensor, for example, but not limited to, a magnetic compass configured to detect the azimuth in the presence of a magnetic field. In one example, a solenoid is used to apply a magnetic field to the magnetic sensor. In another embodiment, the sensor 36 is a gyroscope that detects the angular acceleration about three orthogonal axes. In another embodiment, the output of the second sensor 36 is integrated over time to determine the position of the tip of the measuring probe 13. In a further non-limiting example, the acceleration is integrated twice over time to determine the position.
[0024] Fig. 3 is a schematic representation of an exemplary measuring probe 13 (Fig. 1), such as an industrial endoscope 50, which is used for inspecting an object, such as, but not limited to, a gas turbine. The industrial endoscope 50 includes a first sensor 52 and a second sensor 54, which are arranged near a tip 56 of the industrial endoscope 50. Several small loudspeakers 58 are also arranged near the tip 56. It has become clear that although several loudspeakers 58 are shown in Fig. 3, only a single loudspeaker can also be used. The loudspeakers 58 emit sound signals at different locations within the object, which are detected by the sound emission sensor 32 (Fig. 2). In the illustrated embodiment, the first sensor 52 is a MEMS gravity sensor.The MEMS gravity sensor 52 measures the acceleration / gravity ratio along a sensitive axis pointing in the direction 60°. In one version... The form of the rung shows that as soon as the industrial endoscope 50 is vertically downwards towards the ground in a direction 62, the sensitive Axis 60 points vertically downwards, resulting in an acceleration / gravity value of "1". In another embodiment, As soon as the industrial endoscope 50 is tilted relative to the vertical, the sensitive axis of the MEMS gravity sensor 52 is aligned along the The acceleration / gravity is measured, tilted relative to the vertical, and the MEMS gravity sensor 52 measures a value of Acceleration / gravity less than "1". Therefore, the output of the MEMS gravity sensor detects a height of 50 for the industrial endoscope. In one particular embodiment, the MEMS gravity sensor has a diameter in the range of approximately one to approximately four millimeters. mm. The second sensor 54 can be an angular acceleration gyroscope implemented in the integrated MEMS technology. The gyroscope detects a change in the rotation of the industrial endoscope 50 and consequently detects an azimuth of the industrial endoscope 50. In a In this embodiment, the second sensor 54 has a diameter in the range of approximately 2 to approximately 8 mm.
[0025] Fig. 4 is a schematic representation of an “augmented reality” image 70, which combines real-world data, such as that found in a gas turbine 72, and contains computer-generated data, such as the turbine blades 74 inside the gas turbine 72. The “augmented reality” Figure 70 is obtained by superimposing graphics obtained from the turbine blades 74 onto an image of the gas turbine, which has a casing. contains the image captured by camera 18 (Fig. one). In a particular embodiment, the image contains a 2D image. The microprocessor 20 (Fig. 1) stores and records information taken from the image of the gas turbine and generates graphical representations on the basis of the 3D position obtained by the tracking device 14 (Fig. 1). The microprocessor 20 contains the necessary software to a graphic representation of an “augmented reality” image based on the image from camera 18 and the generated graphic representation to generate. Furthermore, the microprocessor 20 contains a storage medium to save previously stored information and to reuse it. save.
[0026] In order to overlay an image, the position and orientation of the camera 18 relative to the gas turbine 72 and the orientation must be the gas turbine. As a result, it is desirable to determine the relationship between two coordinate systems, a Camera coordinate system (not shown) belonging to camera 18, and a coordinate system 78 belonging to gas turbine 72, to know. Tracking refers to the process of monitoring the relationship between coordinate systems. The microprocessor 20 (Fig. 1) registers 3D locations obtained by the tracking device 14 with respect to the reference frame, which defines the coordinate system 78 the gas turbine 72 has.
[0027] Fig. 5 is a schematic representation of an exemplary display unit 100. The display unit 100 contains a Handheld display, commercially available from General Electric Inspection Technologies under the name: Everest XLG3® is known. The display unit 100 presents an image of turbine blades 72 (Fig. 4) inside the gas turbine and is superimposed. with information 104, which is generated by the microprocessor 20. Some examples of the information include a serial number of the Turbine blade, the operating time, and the identification of the crack. The display unit 100 also contains navigation buttons 106 to control the To select and edit the display.
[0028] As shown, a real view and a virtual view are mixed or blended. For example, the virtual The view is displayed as a transparent overlay on the real view of gas turbine 72. The registration between the real view and the virtual view is shown. View aligns the real-world view and the virtual view with each other. The virtual view registration includes, among other things, the position, Orientation, scaling, perspective, and internal camera parameters for each camera. Internal camera parameters are preferred, such as... For example, but not limited to, the magnification is determined before the camera calibration procedure. The registered virtual view is aligned with the real-time image of the gas turbine 72. In one particular embodiment, the operator wears the display unit 100, which provides him / her with an augmented reality view of gas turbine 72. In an exemplary embodiment, the display unit 100 is of of the type of a see-through or transparent video.
[0029] The “transparent video” creates and presents a world of augmented reality via a handheld display device, such as For example, the display unit 100. The camera integrated into the display unit is used to record a live videotape of the real thing. to capture the world. The camera 18 (Fig. 1) is arranged in relation to the display unit 100 such that it creates the same view as the, which a user would get if they looked through the display setup. The live videotape combined with the computer's The generated graphic is displayed in real time on the display unit 100. An additional feature is camera zoom with an output. the current focal length of the camera. This will allow for an accurate representation of the computer-generated graphics during zooming.
[0030] Fig. 6 is a flowchart illustrating the steps in an exemplary method 120 for 3D tracking within an object. In step 122, the method 120 includes inserting a measuring probe into the object. In step 124, one or more acoustic emission sensors are arranged around the object. Furthermore, in step 126, a first sensor and a second sensor are attached to the measuring probe. In one embodiment, the first and second sensors are arranged at the tip of the measuring probe. In another embodiment, a single loudspeaker or several loudspeakers are arranged at the tip of the measuring probe.
[0031] Fig. 7 is a flowchart illustrating the steps of an exemplary method 140 for generating an augmented reality image for inspections inside an object. In step 142, the method 140 includes capturing an image using a camera. In step 144, the location of a measuring probe within the object is identified using one or more acoustic emission sensors. In one embodiment, the location is determined by calculating the time it takes for the sound signal emitted from the tip of the measuring probe to reach the acoustic sensor. The height of the measuring probe is determined by a first sensor in step 146. The azimuth of the measuring probe is further determined by a second sensor in step 148. In one exemplary embodiment, the azimuth of the measuring probe is determined by applying a magnetic field to the second sensor. The graphic of the object is generated in step 150.The graphic is registered on the image, which is captured based on the location, height, and azimuth determined to form an "augmented reality" image in step 152.
[0032] The various embodiments of an augmented reality system and methods described above thus provide a convenient and efficient means of inspection. The system and methods also enable improved in-situ inspection, repair, and removal of foreign wear particles. Furthermore, they create a lower risk of forced downtime due to improved damage reporting.
[0033] It should be understood that not all such objects or advantages described above can necessarily be achieved according to a particular embodiment. Consequently, the person skilled in the art will recognize, for example, that the devices and techniques such as those described above can be implemented or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein, without necessarily achieving other objects or advantages than those taught or suggested herein.
[0034] Furthermore, the person skilled in the art will clearly recognize that various features of the different embodiments are interchangeable. For example, the use of a webcam, as described in one embodiment, can be adapted for use with a swivel arm as a display unit, as described in relation to another embodiment. Similarly, the various features described herein, as well as other known equivalents for each feature, can be combined and adapted by the person skilled in the art to construct additional systems and techniques in accordance with the principles of this disclosure.
[0035] While the invention has been described in more detail with respect to a limited number of embodiments, it should be understood that the invention is not limited to such embodiments. On the contrary, the invention can be modified to incorporate any number of variation alternatives and substitutions or equivalent arrangements not described above, provided they are within the spirit of the invention. Furthermore, it should be understood that aspects of the invention may include only some of the described embodiments, whereas various embodiments of the invention have been described. Accordingly, the invention is not to be seen as limited to the preceding description, but merely to the scope of the following claims.
[0036] A 3D tracking device 14 is provided. The 3D tracking device 14 includes at least one acoustic sensor 32 arranged around an object 12. The acoustic emission sensor 32 is configured to identify the location of a measuring probe 13 inserted into an object 12 based on the arrival time of the acoustic signal emitted from a location on or near the measuring probe 13. The 3D tracking device 14 also includes a first sensor 34 configured to detect the height of the measuring probe 13. The 3D tracking device 14 further includes a second sensor 36 configured to detect the azimuth of the measuring probe 13.
Claims
[1] 3D tracking device (14) which includes: at least one sound emission sensor (32) arranged around an object (12), wherein the at least one sound emission sensor (32) is set up to determine the location of a measuring probe (13) inserted into an object (12) based on the time of arrival of a sound signal. identify the emission from the location on or near the measuring probe (13 ); a first sensor (34) which is configured to detect a height of the measuring probe (13); and a second sensor (36) which is configured to detect an azimuth of the measuring probe (13). [2] 3D tracking device (14) according to claim 1, wherein the sound signal has an acoustic signature that is transmitted via a single or Several loudspeakers are emitted, which are arranged at the tip of the measuring probe (13 ). [3] 3D tracking device (14) according to claim 1, wherein the first sensor (34) comprises at least one of the following: a electrical micromechanical gravity sensor, which is set up to determine altitude based on acceleration due to gravity detect, or a gyroscope. [4] 3-D tracking device (14 ) according to claim 1, wherein the second sensor (36 ) is an electrical micromechanical magnetic sensor features a device that is set up to detect the azimuth in the presence of a magnetic field. [5] "Augmented reality" system (10 ) for inspecting the interior of an object (12 ) which features: a tracking device (14) which is set up to identify a 3-D location of a measuring probe (13) which is inserted into an object (12) is introduced, based on the time of arrival of an acoustic signal emitted from a location on or near the measuring probe (13 ); a first sensor (36) which is set up to detect an azimuth of the measuring probe (13); a camera (18) which is set up to take an image of the object (12); a microprocessor (20) configured to generate a graphic and to overlay the graphic onto the image captured by the camera (18) based on the 3D location recorded, which was identified by the tracking device (14); and a display unit (22) which is set up to display an "augmented reality" image. [6] "Augmented Reality" system (10 ) according to claim 5, wherein the display unit (22 ) is a handheld display. [7] "Augmented reality" system (10 ) according to claim 5, wherein the recorded image is a 2-D or a 3-D image. [8] Method (120 ) of 3-D tracking within an object that exhibits: Inserting (122) a measuring probe into an object; Arrange (124) at least one acoustic emission sensor around the object; and Arrange (126) a first sensor and a second sensor on the measuring probe. [9] Method (140 ) for forming an "augmented reality" image for inspecting the interior of an object which has: Taking (142) a picture using a camera; Identifying (144) a location of the measuring probe within the object using multiple acoustic emission sensors; Determining (146) the height of the measuring probe using a first sensor; Determine (148 ) the azimuth of the measuring probe using a second sensor; Generating (150) a graphic of the object; and Register (152 ) the graphic on the image that was taken, on the basis of the location, height and azimuth that were determined to form an "augmented reality" image. [10] Method (140 ) according to claim 9, wherein the identification comprises the calculation of a time required for a sound signal emitted from the tip of the measuring probe to reach the sound sensors.