Method for the non-invasive inspection of a mechanical device comprising at least one object to be characterized in order to determine the presence of impacts or erosion

Abstract

The invention relates to a method for inspecting an object to be characterized, comprising: - acquiring (11) an image of the object; - registering a 3D model of the object on the acquired image; - superimposing (13) a projection of the 3D model of the object onto the acquired image; - determining (14) whether the object is eroded when there is at least one deviation between contours of the acquired image and contours of the projected 3D model; - estimating the measure of the deviation based on a deviation in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance as a unit of length.

Description

[0001] DESCRIPTION

[0002] TITLE: Non-invasive inspection method for a mechanical device comprising at least one object to be characterized in order to determine the presence of impacts or erosion

[0003] technical field

[0004] The invention relates to the technical field of the inspection of mechanical devices, and more particularly to the non-invasive inspection of such devices.

[0005] Previous techniques

[0006] Gas turbine maintenance requires regular inspections. Among these tasks, measuring erosion levels and identifying impacts on airflow components is mandatory and frequent for airworthiness reasons. To avoid disrupting equipment availability, these inspections are performed non-invasively, without dismantling.

[0007] A commonly used inspection technique relies on erosion or impact testing using a gauge. However, this method is limited by the need for direct manual access to the component being inspected.

[0008] Other state-of-the-art methods consist of erosion or impact control by straight endoscopic rod with reticle or impact control only (no erosion measurement) with 3D measuring endoscope.

[0009] Each of these techniques has disadvantages in terms of dismantling, accessibility, cost, or limits of investigation.

[0010] Therefore, there is a need, without removal or dismantling, to quantify the level of erosion or to characterize the impacts present on an internal component of a gas turbine.

[0011] Description of the invention The objective of the invention is therefore to improve and simplify existing methods of inspecting air stream components, such as compressor or turbine blades.

[0012] The invention relates to a non-invasive method for inspecting a mechanical device comprising at least one object to be characterized, comprising the following steps: a. an image of the object to be characterized is acquired, b. an initial pose is determined, a 3D model of the object to be characterized is registered on the acquired image, and a corrected pose is determined from the initial pose of the acquisition device based on the registration performed, c. a projection of the 3D model of the object to be characterized is superimposed on the acquired image, based on the corrected pose, d. erosion of the object to be characterized is determined when there is at least one position or distance difference between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances, e.If this is the case, the measurement of the position or distance difference is estimated as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

[0013] To determine the initial pose, perform the registration of the 3D model and determine the corrected pose, the following sub-steps can be carried out: a. identify on the acquired image at least four known or characteristic points of the 3D model and determine the initial pose of the image acquisition device, in the coordinate system of the 3D model as a function of the position in the acquired image of the at least four points of the 3D model, b. project the contours of the 3D model onto the acquired image, then register said contours so that they correspond to the contours of the object to be characterized present on the acquired image, and c. use the registered contours to determine a corrected pose of the acquisition device.

[0014] The presence of an impact can be determined when at least one additional contour is detected on the image without a corresponding contour in the 3D model.

[0015] The acquisition device can be an endoscope.

[0016] The mechanical device could be an aeronautical device.

[0017] The invention also relates to a data processing unit comprising:

[0018] - means to acquire an image of the object and to characterize it;

[0019] - means to determine an initial pose, perform a registration of a 3D model of the object to be characterized on the acquired image, and determine a corrected pose from the initial pose of the acquisition device according to the registration performed;

[0020] - means to superimpose a projection of the 3D model of the object to be characterized onto the acquired image, depending on the corrected pose;

[0021] - means to determine whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances; and

[0022] - means to, if such is the case, estimate the measure of the position or distance difference as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

[0023] The invention also relates to a computer program product comprising code instructions which, when the program is executed by a computer, lead the computer to implement the control method as defined above.

[0024] Brief description of the drawings. Other objects, features and advantages of the invention will become apparent from the following description, given solely by way of non-limiting example and made with reference to the accompanying drawings in which:

[0025] - Figure [Fig 1] illustrates the main steps of a non-invasive control procedure according to a first approach,

[0026] - Figures [Fig 2] [Fig 3] [Fig 4] [Fig 5] [Fig 6] illustrate several images acquired from the trailing edge of a compressor impeller blade in different positions,

[0027] - Figure [Fig 7] illustrates a composite image resulting from the superposition of images acquired from the trailing edge,

[0028] - Figure [Fig 8] illustrates the characteristic elements identified on the composite image,

[0029] - Figures [Fig 9] [Fig 10] [Fig 11] [Fig 12] [Fig 13] illustrate pairs of images included in the composite image showing the characteristic elements allowing the determination of an essential matrix,

[0030] - Figure [Fig 14] illustrates the main steps of the non-invasive control process according to a second approach,

[0031] - Figure [Fig 15] illustrates a 3D model of the object to be characterized on which four characteristic points are identified,

[0032] - Figure [Fig 16] illustrates an acquired image of the object to be characterized on which the same four points of the 3D model are identified,

[0033] - Figure [Fig 17] illustrates the projection of the contours of the 3D model onto the acquired image of the object to be characterized,

[0034] - Figure [Fig 18] illustrates the realignment of the contours of the 3D model so that they correspond to the contours of the object to be characterized,

[0035] - Figure [Fig 19] illustrates a discrepancy in the object to be characterized identified by the process,

[0036] - Figure [Fig 20] illustrates the measurement of the deviation at the level of the object to be characterized identified by the process, and

[0037] - Figure [Fig 21] illustrates a processing unit and a computer program product. Detailed description

[0038] To overcome the inherent disadvantages of state-of-the-art inspection techniques, a non-invasive system and method based on processing endoscopic images obtained with a standard endoscope are proposed to enable erosion and impact measurements. A "standard" endoscope is defined as any equipment allowing remote visual inspection of hard-to-reach areas (e.g., fiberscopes, videoscopes, endoscopic rods, etc.), whose optics, lighting, and acquisition systems do not include any specific features enabling three-dimensional measurement of the observed scene. In contrast, equipment known as "3D" endoscopes, equipped with specific systems (e.g., stereoscopic vision lenses, structured light projectors, laser beam projectors, etc.), allows for this type of measurement.The present invention relies preferentially on the use of a "standard" endoscope.

[0039] Two complementary approaches are provided for in the non-invasive control system and method. The first approach is not included in the scope and claims of this patent application.

[0040] The first approach uses images that correspond to lateral views of the object to be characterized in order to measure erosion and impacts in thickness.

[0041] The second approach uses images that correspond to direct views of the object to be characterized in order to measure surface erosion and impacts.

[0042] The first approach to these erosion and impact measurements is based on an uncalibrated stereoscopic technique to perform detailed characterization of dimensional parameters on a component location. Figure [Fig 1] illustrates the main steps of a non-invasive inspection procedure according to this first approach. As an example, we consider the measurement of the thickness e of a trailing edge (BE) of a compressor impeller blade at a height h.

[0043] In the first step 1, at least two images of the object to be characterized are acquired. In figures [Fig 2] to [Fig 6], the object is the trailing edge of a compressor impeller blade.

[0044] Image acquisition is specific in that each image is taken while the object to be characterized is in different positions relative to the viewpoint. In the example illustrated in Figures [Fig 2] to [Fig 6], the turbine blade can be rotated. Each image is then acquired while the blade is in a different angular position, labeled A, B, C, D, and E respectively in Figures [Fig 2] to [Fig 6], while the viewpoint remains the same. By "same viewpoint," we mean that the image acquisition device is held in an unchanged position from one image to the next. In the example illustrated in Figures [Fig 2] to [Fig 6], the acquisition device is an endoscope, positioned in the air stream of the gas turbine (GT), and an accessory, such as a locking tool, is used to hold the endoscope stationary while the blades are moved.

[0045] In a second step 2, the fixed parts of the acquired images are determined, the different images are aligned according to these fixed parts, and then they are superimposed to obtain a composite image. Figure [Fig 7] illustrates such a composite image, formed from figures [Fig 2] to [Fig 6]. The vanishing edge is represented simultaneously in positions A, B, C, D, E, also illustrated in figures [Fig 2] to [Fig 6]. It should also be noted that the vanishing edge and the fixed parts of the image are accentuated in figure [Fig 7].

[0046] In a third step 3, at least two characteristic points of the object are identified and characterized by means of edge detection algorithms (for example; canny edge detector, deepEdge algorithm based on CNNs according to the English acronym "Convolutional Neural Network", SED algorithm according to the English acronym "Structure Edge Detection" ...) and point detection algorithms (for example: Harris corner detection, FAST corner detector, corner image registration by correlation, extremum search...).

[0047] In the example shown in Figures [Fig 2] to [Fig 7], the characteristic points of the blades are the blade roots, the blade tips, and the contour edge of each blade. From these, characteristic geometric elements are derived, such as a baseline BA and a midline M for each blade position. Figure [Fig 8] illustrates the characteristic elements identified on the composite image.

[0048] Next, the blade edge contours are determined for each blade in the region where we wish to measure erosion. These contours can be obtained using image edge detection techniques (e.g., canny contour detector, deepEdge, or SED), image segmentation techniques (Mask R-CNN, SegNet, or Segment Anything Model), or manual or semi-manual methods such as image edge registration techniques (e.g., geodetic active contours, deformable models, or contour line methods). Figure [Fig. 8] illustrates these contours, labeled CA1 to CE1 and CA2 to CE2.

[0049] In a fourth step 4, the essential matrix is ​​estimated for several pairs of images of the obj and for each determined characteristic element.

[0050] Recall that an essential matrix is ​​a square matrix of size 3 describing the geometric relationship between two views of the same 3D scene captured by one or two different intrinsically calibrated cameras. An essential matrix is ​​generally represented as the matrix product of a translation matrix and a rotation matrix. Figures [Fig 9] to [Fig 13] illustrate pairs of images included in the composite image of Figure [Fig 8], showing the characteristic elements that allow the determination of an essential matrix. These elements are, for each image of the blade's trailing edge, the baseline BA and the previously identified median lines M, as well as the epipolar line DE for each position A, B, C, D, E.

[0051] The essential matrix for each image pair is obtained using at least 5 pairs of points from the image. The points used can be the leading and trailing points of each blade, but also points calculated at the intersections of lines parallel to the baseline Ba (lines BAI and BA2, for example) and contour lines at the edges of the blades (CAI, CA2, ... CE2, for example). The calculation is based on David Nistér's algorithm using 5 points, combined with a robust RANSAC-type estimator when the number of points is greater than 5. David Nistér's algorithm is defined in the paper David Nistér et al., An efficient solution to the five-point relative pose problem, Pattern Analysis and Machine Intelligence, IEEE Transactions, 26(6):756-770, 2004. The RANSAC-type estimator is defined in the paper Martin A. Fischler and Robert C.Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, 24(6):381 -395, June 1981.

[0052] The essential matrix is ​​decomposed into rotation and translation to reconstruct the blade root and tip in 3D using triangulation. The known blade height is used to obtain the scale factor (Fech) for the reconstruction.

[0053] In a fifth step, 5, a measurement of at least one of the characteristic elements is estimated from each essential matrix. For example, the characteristic element could be the blade width at mid-blade. To achieve this, the measurement is taken at a predetermined relative distance, such that it is located between the tip and the root of the blade (for example, here at mid-blade, i.e., 50% of the way between the tip and the root). Points EA and EB are fixed at this distance along the midlines MA and MB, as illustrated in Figure [Fig. 9]. The essential matrix is ​​used to determine the epipolar lines DEA and DEB passing through each of these points EA and EB, respectively.

[0054] The thickness measurement is then obtained by reconstructing, as before (by triangulation and application of the scaling factor Fech), the 3D points at the intersections of the blade edges and the epipolar lines. In a particular embodiment, for each characteristic element, the set of estimated measurements is averaged over at least two pairs of images to increase the reliability of the estimate and account for the displacement of the object being characterized.

[0055] This first approach allows, for example, the determination of the erosion of the trailing edge of a compressor blade, by determining the thickness of the trailing edge at a height h relative to the base of the blade.

[0056] The second approach is based on a technique of registering 3D data onto 2D contours identified on the acquired images, allowing then a comparison with the expected.

[0057] 3D data consists primarily of 3D points and contours derived from a model or prototype created using CAD (computer-aided design) software. The object may be a single piece or composed of a fixed part and a moving part.

[0058] As in the first approach, the acquisition device used is an endoscope.

[0059] The comparison is then either dimensional or geometric between the observed object to be characterized and the object to be characterized in its initial or nominal state (as it appears, for example, from 3D data). This comparison indirectly allows the determination of contour erosion or the presence of impacts.

[0060] Figure [Fig 14] illustrates the main steps of the non-invasive control process according to this second approach.

[0061] In a first step 11, an image of the object to be characterized is acquired with a fixed position of the objective.

[0062] For example, an endoscope is inserted into the air stream of the gas turbine (TAG) and is held in place by means of an accessory (locking tool).

[0063] In a second step 12, the 3D model is realigned with the image obtained. This is done in three sub-steps 12a, 12b, 12c.

[0064] In a first sub-step 12a, at least four known or characteristic points of the 3D model are identified on the acquired image. These points can be obtained automatically (for example with a CNN-based object detection algorithm like Yolo if several images of each point are available for training, by template matching if only one image of this point is available), by a manual or semi-manual method such as point registration (for example with template matching or extrema point search).

[0065] Figure [Fig 15] illustrates a 3D model on which four characteristic points referenced 21, 22, 23, 24 are located.

[0066] Figure [Fig 16] illustrates an acquired image on which the same four points illustrated in Figure [Fig 15] are identified.

[0067] This first sub-step allows us to determine the initial position of the image acquisition device, in the coordinate system of the 3D model. By position of the image acquisition device, we mean the position and orientation of the image acquisition device.

[0068] In a second substep 12b, the contours of the 3D model of the object are projected onto the acquired image ([Fig 17]), and then these contours are registered so that they correspond to the contours of the object to be characterized present in the acquired image ([Fig 18]). In Figures [Fig 17] and [Fig 18], contour 25 of the stator in a radially proximal position relative to the axis, contour 26 of the stator in a radially distal position relative to the axis, and contours 27 and 28 of the stator are identified. It should be noted that the contours illustrated in this example are linked to the stator. However, contours linked to the rotor or contours linked to both the stator and the rotor could be used. This registration can be achieved manually or semi-manually with contour registration algorithms (e.g., geodetic active contours, deformable models or contour line methods).

[0069] In a third substep 12c, the registered contours are used to determine a corrected pose for the acquisition device. This corrected pose is preferably obtained through optimization; we seek the camera pose that minimizes the discrepancies between 2D curves corresponding to the projections of the 3D model and the 2D curves registered in step 12b. In a third step 13, a projection of the 3D model of the object to be characterized is superimposed onto the acquired image, based on the corrected pose. The operator can verify that the corrected pose accurately registers the 3D model.In the case of an object composed of a fixed part and a moving part, the projected 3D model of the moving part of the object can be repositioned manually or automatically (by contour optimization, as in the calculation of the corrected pose) by adjusting the parameters related to the movement of the moving part (translations and rotations). These adjustments also allow the 3D model of the object to be registered to other images acquired of the same part or to other areas of the moving part with comparable geometry.

[0070] In a fifth step 14, it is determined whether there is at least one positional or distance discrepancy between the contours of the acquired image and the contours of a projected 3D model of the object, taking into account the erosion tolerance zone. This may involve manufacturing tolerances in the 3D model for determining erosion in the area under consideration, and / or tolerances related to uncertainty in the erosion measurement, such that the measured discrepancy must exceed the erosion tolerance before erosion is considered to be present.

[0071] Figure [Fig 19] illustrates a difference at the trailing edge between position 29 of the trailing edge on the model and position 30 of the trailing edge on the trailing edge to be characterized.

[0072] If this is the case, the deviation is estimated based on the pixel difference between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance in the projected 3D model expressed in pixels and the same distance in units of length. Figure [Fig 20] illustrates a measurement associated with the deviation at the trailing edge.

[0073] The presence of impact can also be determined as additional contours 31 compared to the contours of the 3D model or contours not corresponding to those of the 3D model. A non-invasive control system includes, with reference to Figure [Fig 21], at least one processing means 32, at least one memory 33 and at least one display means (not shown) such as a screen, the whole being connected to or integrated into an endoscope.

[0074] The processing means 32 is configured so as to be able to carry out at least one of the non-invasive control processes described above.

[0075] The processing means 32, for example a data processing unit, preferably comprises:

[0076] - means 34 to acquire an image of the object to be characterized;

[0077] - means 35 to determine an initial pose, perform a registration of a 3D model of the object to be characterized on the acquired image, and determine a corrected pose from the initial pose of the acquisition device according to the registration carried out;

[0078] - means 36 to superimpose a projection of the 3D model of the object and to characterize on the acquired image, according to the corrected pose;

[0079] - means 37 for determining whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances; and

[0080] - means 38 for, if such is the case, estimating the measure of the position or distance difference as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

[0081] At least one memory 33 includes reference data such as a 3D model and / or at least one of the non-invasive control methods.

[0082] The memory 33 preferably includes a computer program product 39 comprising code instructions which, when the program is executed by a computer, cause the computer to implement at least one of the non-invasive control methods described above.

[0083] The system can be equipped with wired or wireless communication means, allowing access to reference data or at least one of the non-invasive control methods.

[0084] The present invention has been described and illustrated in relation to the trailing edge of a compressor blade. However, the present invention is also applicable to other technical fields requiring a dimensional assessment of damage characterized by dimensional criteria. Examples include coating spalling area, impact depth, crack length, or surface density of corrosion points.

[0085] On the other hand, the examples presented above relate to the field of aeronautical gas turbine maintenance. Transposing the teaching disclosed here to all mechanical systems requiring non-invasive inspections based on dimensional damage criteria, within the context of their maintenance (steam turbines in power plants, wind turbine gearboxes, etc.), does not depart from the scope of the invention.

Claims

DEMANDS 1. A non-invasive method for testing a mechanical device comprising at least one object to be characterized, comprising the following steps: a. (1) an image of the object to be characterized is acquired by an acquisition device, b. (12) an initial pose of the acquisition device is determined, a 3D model of the object to be characterized is registered on the acquired image, and a corrected pose of the acquisition device is determined from the initial pose of the acquisition device as a function of the registration performed, c. (13) a projection of the 3D model of the object to be characterized is superimposed on the acquired image, as a function of the corrected pose of the acquisition device, d. we determine (14) if there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model taking into account erosion tolerances, e.If this is the case, we estimate the measurement of the position or distance difference as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length, and to determine the initial pose of the acquisition device, to perform the registration of the 3D model and to determine the corrected pose of the acquisition device, we carry out the following sub-steps:. a. We identify (12a) on the acquired image at least four known or characteristic points of the 3D model and we determine the initial pose of the image acquisition device, in the frame of the 3D model as a function of the position in the acquired image of the at least four points of the 3D model, b. we project (12b) the contours of the 3D model onto the acquired image, then we re-register said contours so that they correspond to the contours of the object to be characterized present on the acquired image, and c. we use the re-registered contours to determine a corrected pose of the acquisition device.

2. A non-invasive method for testing a mechanical device according to claim 1, wherein the presence of an impact is determined when at least one additional contour is detected on the contourless image corresponding to said additional contour in the 3D model.

3. Non-invasive method for checking a mechanical device according to any one of claims 1 to 2, wherein the acquisition device is an endoscope.

4. A method for non-invasive control of a mechanical device according to any one of claims 1 to 3, wherein the mechanical device is an aeronautical device.

5. Data processing unit (32) comprising: - means (34) to acquire an image of the object to be characterized; - means (35) to determine an initial pose of the acquisition device, to perform a registration of a 3D model of the object to be characterized on the acquired image, and to determine a corrected pose of the acquisition device from the initial pose of the acquisition device according to the registration carried out; - means (36) to superimpose a projection of the 3D model of the object to be characterized onto the acquired image, according to the corrected pose of the acquisition device; - means (37) for determining whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances; and - means (38) for, if so, estimating the measure of the position or distance deviation as a function of a pixel difference between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length, means (35) for determining the initial pose of the acquisition device, performing the registration of the 3D model and determining the corrected pose of the acquisition device, being configured to perform the following sub-steps: a. at least four known or characteristic points of the 3D model are identified on the acquired image and the initial pose of the image acquisition device is determined, in the coordinate system of the 3D model as a function of the position in the acquired image of the at least four points of the 3D model, b.we project the contours of the 3D model onto the acquired image, then we realign said contours so that they correspond to the contours of the object to be characterized present on the acquired image, and c. we use the realigned contours to determine a corrected pose of the acquisition device.

6. Product computer program (39) comprising code instructions which, when the program is executed by a computer, cause the computer to implement the control method according to any one of claims 1 to 4.

Images