Non-invasive method for inspecting a mechanical device comprising at least one object to be characterized in order to determine the presence of impacts or erosion

Abstract

A method for inspecting an object to be characterized, in which: - an image of the object is acquired (11); - a 3D model of the object is registered to the acquired image; - a projection of the 3D model of the object is superimposed (13) onto the acquired image; - erosion of the object is determined (14) when there is at least one discrepancy between contours of the acquired image and contours of the projected 3D model; - the measure of the discrepancy is estimated as a function of the difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance in the projected 3D model expressed in pixels and the same distance in units of length. Figure for the abstract: [Fig 14]

Description

Title of the invention: Non-invasive method for inspecting a mechanical device comprising at least one object to be characterized in order to determine the presence of impacts or erosion. Technical field

[0001] The invention has as its technical field the inspection of mechanical devices, and more particularly the non-invasive inspection of such devices. Previous techniques

[0002] Gas turbine maintenance requires the regular implementation of inspection tasks. Among these tasks, measuring the level of erosion and the presence of impacts on the airflow components is mandatory and frequent for airworthiness reasons. In order to avoid impacting equipment availability, these inspections are carried out non-invasively, without dismantling.

[0003] A commonly used inspection technique relies on erosion or impact testing using a gauge. However, this method is limited by the direct manual access required to the component being inspected.

[0004] Other prior art methods consist of erosion or impact control by straight endoscopic rod with reticle or impact control only (no erosion measurement) with 3D measuring endoscope.

[0005] Each of these techniques has disadvantages in terms of dismantling, accessibility, cost price or investigation limits.

[0006] There is therefore a need, without removal or dismantling, to quantify the level of erosion or to characterize the impacts present on an internal component of a gas turbine. Description of the invention

[0007] The objective of the invention is therefore to improve and simplify existing methods of inspecting air stream components, such as compressor or turbine blades.

[0008] The invention relates to a non-invasive method for testing a mechanical device comprising at least one object to be characterized, comprising the following steps:

[0009] a. an image of the object to be characterized is acquired,

[0010] b. an initial pose is determined, a registration of a 3D model of the object is performed to be characterized on the acquired image and a corrected exposure is determined from the initial exposure of the acquisition device based on the registration performed,

[0011] c. a projection of the 3D model of the object to be characterized is superimposed on the acquired image, according to the corrected pose,

[0012] d. It is determined whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances,

[0013] e. if this is the case, the measure of the position or distance difference is estimated as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

[0014] To determine the initial pose, realign the 3D model and determine the corrected pose, the following sub-steps can be performed:

[0015] a. at least four known or characteristic points of the 3D model are identified on the acquired image and the initial pose of the image acquisition device is determined, in the frame of the 3D model, as a function of the position in the acquired image of the at least four points of the 3D model,

[0016] b. the contours of the 3D model are projected onto the acquired image, then these contours are realigned so that they correspond to the contours of the object to be characterized present on the acquired image, and

[0017] c. the registered contours are used to determine a corrected pose of the acquisition device.

[0018] The presence of an impact can be determined when at least one additional contour is detected on the image without a corresponding contour in the 3D model.

[0019] The acquisition device may be an endoscope.

[0020] The mechanical device may be an aeronautical device.

[0021] The invention also relates to a data processing unit comprising:

[0022] - means for acquiring an image of the object to be characterized;

[0023] - means for determining an initial pose, performing a registration of a 3D model of the object to be characterized on the acquired image, and determine a corrected pose from the initial pose of the acquisition device according to the registration carried out;

[0024] - means for superimposing a projection of the 3D model of the object to be characterized on the acquired image, depending on the corrected exposure;

[0025] - means for determining whether there is erosion of the object to be characterized when it at least one positional or distance difference exists between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances; and

[0026] - means for, if so, estimating the measure of the position gap or of distance as a function of a pixel difference between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

[0027] The invention also relates to a computer program product comprising code instructions which, when the program is executed by a computer, lead the latter to implement the control method as defined above. Brief description of the drawings

[0028] Other objects, features and advantages of the invention will become apparent from the following description, given solely by way of non-limiting example and made with reference to the accompanying drawings in which:

[0029] - Figure [Fig. 1] illustrates the main steps of a non-invasive control procedure according to a first approach,

[0030] - Figures [Fig.2] [Fig.3] [Fig.4] [Fig.5] [Fig.6] illustrate several acquired images of the trailing edge of a compressor impeller blade in different positions,

[0031] - Figure [Fig.7] illustrates a composite image resulting from the superposition of images acquired from the trailing edge,

[0032] - Figure [Fig.8] illustrates the characteristic elements identified in the image composite,

[0033] - Figures [Fig.9] [Fig.10] [Fig.11] [Fig.12] [Fig.13] illustrate pairs of images included in the composite image showing the characteristic elements that allow for the determination of an essential matrix,

[0034] - Figure [Fig. 14] illustrates the main steps of the non-invasive control process according to a second approach,

[0035] - Figure [Fig. 15] illustrates a 3D model of the object to be characterized on which four characteristic points are identified,

[0036] - Figure [Fig. 16] illustrates an image acquired of the object to be characterized on which the four identical points on the 3D model are identified,

[0037] - Figure [Fig. 17] illustrates the projection of the contours of the 3D model onto the image acquired from the object to be characterized,

[0038] - Figure [Fig. 18] illustrates the registration of the contours of the 3D model so that they correspond to the contours of the object to be characterized,

[0039] - Figure [Fig. 19] illustrates a deviation at the level of the object to be characterized identified by the process,

[0040] - Figure [Fig.20] illustrates the measurement of the deviation at the level of the object to be characterized identified by the process, and

[0041] - Figure [Fig. 21] illustrates a processing unit and a program product computer. Detailed description

[0042] In order to overcome the disadvantages inherent in prior art inspection techniques, a non-invasive inspection system and method based on the processing of endoscopic images obtained with a standard endoscope are proposed to enable erosion and impact measurements. A "standard" endoscope is defined as any equipment allowing remote visual inspection of hard-to-reach areas (e.g., fiberscopes, videoscopes, endoscopic rods, etc.), whose optics, lighting, and acquisition systems do not include any specific features enabling three-dimensional measurement of the observed scene. In contrast, equipment known as "3D" endoscopes, equipped with specific systems (e.g., stereoscopic vision lenses, structured light projectors, laser beam projectors, etc.), enables this type of measurement.The present invention relies preferentially on the use of a "standard" endoscope.

[0043] Two complementary approaches are provided in the system and the non-invasive control method.

[0044] The first approach uses images that correspond to lateral views of the object to be characterized in order to measure erosions and impacts in thickness.

[0045] The second approach uses images that correspond to direct views of the object to be characterized in order to measure surface erosion and impacts.

[0046] The first approach to these erosion and impact measurements is based on an uncalibrated stereoscopic technique to perform a detailed characterization of dimensional parameters on a component location. Figure [Fig. 1] illustrates the main steps of a non-invasive inspection process according to this first approach.

[0047] As an example, we consider the measurement of the thickness e of a trailing edge (BF) of a compressor wheel blade at a height h.

[0048] In a first step 1, at least two images of the object to be characterized are acquired. In figures [Fig.2] to [Fig.6], the object is the trailing edge of a compressor impeller blade.

[0049] Image acquisition is specific in that each image is taken while the object to be characterized is in different positions relative to the viewpoint. In the example illustrated in Figures [Fig. 2] to [Fig. 6], the turbine blade can be rotated. Each image is then acquired while the blade is in a different angular position, labeled A, B, C, D, E respectively in Figures [Fig. 2] to [Fig. 6], while the viewpoint remains the same. By "same viewpoint," it is understood that the image acquisition device is maintained in an unchanged position from one image to the next. In the example illustrated in Figures [Fig. 2] to [Fig. 6], The acquisition device is an endoscope, positioned in the air stream of the gas turbine (TAG), and an accessory, such as a locking tool, is used to hold the endoscope stationary while the blades are moved.

[0050] In a second step 2, the fixed parts of the acquired images are determined, the different images are aligned according to these fixed parts, and then they are superimposed to obtain a composite image. Figure [Fig. 7] illustrates such a composite image, formed from figures [Fig. 2] to [Fig. 6]. The trailing edge is represented simultaneously in positions A, B, C, D, E, as shown elsewhere in figures [Fig. 2] to [Fig. 6]. It should also be noted that the trailing edge and the fixed parts of the image are emphasized in figure [Fig. 7].

[0051] In a third step 3, at least two characteristic points of the object to be characterized are identified by means of edge detection algorithms (for example; canny edge detector, deepEdge algorithm based on CNN according to the English acronym "Convolutional Neural Network", SED algorithm according to the English acronym "Structure Edge Detection"...) and point detection algorithms (for example: Harris corner detection, FAST corner detector, corner image registration by correlation, extremum search...).

[0052] In the example shown in Figures [Fig. 2] to [Fig. 7], the characteristic points of the blades are the blade roots, the blade tips, and the contour edge of each blade. From these, characteristic geometric elements are derived, such as a baseline BA and a median line M for each blade position. Figure [Fig. 8] illustrates the characteristic elements identified on the composite image.

[0053] The contours of the blade edges are then determined for each blade in the region where erosion is to be measured. These contours can be obtained using image edge detection techniques (e.g., canny contour detector, deepEdge, or SED), image segmentation techniques (Mask R-CNN, SegNet, or Segment Anything Model), or manual or semi-manual methods such as image edge registration techniques (e.g., geodetic active contours, deformable models, or contour line methods). Figure [Fig. 8] illustrates these contours, labeled CAI to CEI and CA2 to CE2.

[0054] In a fourth step 4, the essential matrix is ​​estimated for several pairs of images of the object and for each determined characteristic element.

[0055] It is recalled that an essential matrix is ​​a square matrix of size 3 describing the geometric relationship between two views of the same 3D scene captured by one or two different intrinsically calibrated cameras. An essential matrix is ​​generally presented as the matrix product of a translation matrix and a rotation matrix. Figures [Fig. 9] to [Fig. 13] illustrate pairs of images included in the composite image of Figure [Fig. 8] showing the characteristic elements allowing the determination of an essential matrix. This consists, for each image of the trailing edge of the blade, of the baseline BA and the median lines M previously identified as well as the epipolar line DE for each position A, B, C, D, E.

[0056] The essential matrix for each pair of images is obtained using at least 5 pairs of points from the image. The points used can be the top and bottom points of the view of each blade, but also points calculated at the intersections of lines parallel to the baseline Ba (lines BAI and BA2, for example) and contour lines at the edges of the blades (CAI, CA2,... CE2, for example). The calculation is based on David Nistér's algorithm using 5 points, combined with a robust RANSAC-type estimator when the number of points is greater than 5. David Nistér's algorithm is defined in the paper David Nistér et al., An efficient solution to the five-point relative pose problem, Pattern Analysis and Machine Intelligence, IEEE Transactions, 26(6):756-770, 2004. The RANSAC-type estimator is defined in the paper Martin A. Fischler and Robert C.Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, 24(6):381-395, June 1981.

[0057] The essential matrix is ​​decomposed into rotation and translation to reconstruct the blade root and tip in 3D by triangulation. The known blade height is used to obtain the scale factor of the reconstruction Fech.

[0058] In a fifth step 5, a measurement of at least one of the characteristic elements is estimated from each essential matrix. For example, the characteristic element could be the blade width at mid-blade. To achieve this, the measurement is taken at a predetermined relative distance, such as to be located between the tip and the root of the blade (for example, here at mid-blade, i.e., 50% of the way between the tip and the root). Points EA and EB are fixed at this distance along the mid-lines MA and MB as illustrated in Figure [Fig. 9]. The essential matrix is ​​used to determine the epipolar line DEA and DEB passing through each of these points EA and EB, respectively.

[0059] The thickness measurement is then obtained by reconstructing, as before (by triangulation and application of the scaling factor Fech), the 3D points at the intersections of the blade edges and the epipolar lines. In a particular embodiment, for each characteristic element, the set of estimated measurements is averaged over at least two pairs of images in order to increase the reliability of the estimate to take into account the displacement of the object to be characterized.

[0060] This first approach makes it possible, for example, to determine the erosion of the trailing edge of a compressor blade, by determining the thickness of the trailing edge at a height h relative to the base of the blade.

[0061] The second approach is based on a 3D data registration technique on 2D contours identified on the acquired images, allowing then a comparison with the expected.

[0062] 3D data includes, in particular, 3D points and contours from a model or prototype of an object generated by CAD (computer-aided design) software. The object may be a single piece or composed of a fixed part and a moving part.

[0063] As in the first approach, the acquisition device used is an endoscope.

[0064] The comparison is then either dimensional or geometric between the observed object to be characterized and the object to be characterized in its initial or nominal state (as it appears, for example, from 3D data). This comparison indirectly allows the determination of contour erosion or the presence of impacts.

[0065] Figure [Fig. 14] illustrates the main steps of the non-invasive control process according to this second approach.

[0066] During a first step 11, an image of the object to be characterized is acquired with a fixed position of the objective.

[0067] By example, an endoscope is inserted into the air stream of the gas turbine (TAG) and is held in place by means of an accessory (locking tool).

[0068] In a second step 12, the 3D model is registered to the image obtained. This is carried out in three sub-steps 12a, 12b, 12c.

[0069] In a first sub-step 12a, at least four known or characteristic points of the 3D model are identified on the acquired image. These points can be obtained automatically (for example with a CNN-based object detection algorithm such as Yolo if several images of each point are available for training, by template matching if only one image of this point is available), by a manual or semi-manual method such as point registration (for example with template matching or the search for extrema points).

[0070] Figure [Fig. 15] illustrates a 3D model on which four characteristic points referenced 21, 22, 23, 24 are identified.

[0071] Figure [Fig. 16] illustrates an acquired image on which the same four points illustrated in Figure [Fig. 15] are identified.

[0072] This first sub-step allows the initial pose of the image acquisition device to be determined, in the coordinate system of the 3D model. By pose of the image acquisition device, we mean the position and orientation of the image acquisition device.

[0073] In a second substep 12b, the contours of the 3D model of the object are projected onto the acquired image ([Fig. 17]), and then said contours are registered so as that they correspond to the contours of the object to be characterized present in the acquired image ([Fig. 18]). In Figures [Fig. 17] and [Fig. 18], contour 25 of the stator in a radially proximal position relative to the axis, contour 26 of the stator in a radially distal position relative to the axis, and contours 27 and 28 of the stator are identified. It should be noted that the contours illustrated in this example are linked to the stator. Nevertheless, contours linked to the rotor or contours linked to both the stator and the rotor could be used. This registration can be achieved manually or semi-manually using contour registration algorithms (e.g., geodetic active contours, deformable models, or contour line methods).

[0074] In a third substep 12c, the registered contours are used to determine a corrected pose for the acquisition device. This corrected pose is preferably obtained by optimization; we seek the camera pose that minimizes the differences between 2D curves corresponding to the projections of the 3D model and the 2D curves registered in step 12b.

[0075] In a third step 13, a projection of the 3D model of the object to be characterized is superimposed on the acquired image, according to the corrected pose. The operator can verify that the corrected pose allows for the accurate registration of the 3D model. In the case of an object composed of a fixed part and a moving part, the projected 3D model of the moving part of the object can then be repositioned manually or automatically (by contour optimization as in the calculation of the corrected pose) by adjusting the parameters related to the movement of the moving part of the object (translations and rotations). These adjustments also allow the registration of the 3D model of the object with other images acquired of the same part or with other areas of the moving part having a comparable geometry.

[0076] In a fifth step 14, it is determined whether there is at least one positional or distance difference between the contours of the acquired image and the contours of a projected 3D model of the object, taking into account the erosion tolerance zone. This may involve manufacturing tolerances in the 3D model for determining erosion in the area under consideration, and / or tolerances related to uncertainty in the erosion measurement, such that the measured difference must exceed said erosion tolerance before erosion is considered to be present.

[0077] Figure [Fig. 19] illustrates a difference at the trailing edge between position 29 of the trailing edge on the model and position 30 of the trailing edge on the trailing edge to be characterized.

[0078] If this is the case, the measure of the gap is estimated as a function of the gap in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length. Figure [Fig.20] illustrates a measurement associated with the gap at the trailing edge.

[0079] The presence of impact can also be determined as additional contours 31 compared to the contours of the 3D model or contours not corresponding to those of the 3D model.

[0080] A non-invasive control system includes, with reference to Figure [Fig.21], at least one processing means 32, at least one memory 33 and at least one display means (not shown) such as a screen, the whole being connected to or integrated into an endoscope.

[0081] The processing means 32 is configured so as to be able to carry out at least one of the non-invasive control processes described above.

[0082] The processing means 32, for example a data processing unit, preferably comprises:

[0083] - means 34 for acquiring an image of the object to be characterized;

[0084] - means 35 for determining an initial pose, performing a registration of a model 3D of the object to be characterized on the acquired image, and determine a corrected pose from the initial pose of the acquisition device according to the registration carried out;

[0085] - means 36 for superimposing a projection of the 3D model of the object to characterize on the acquired image, according to the corrected exposure;

[0086] - means 37 for determining whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances; and

[0087] - means 38 for, if so, estimating the measure of the position gap or of distance as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

[0088] The at least one memory 33 includes reference data such as a 3D model and / or at least one of the non-invasive control methods.

[0089] The memory 33 preferably includes a computer program product 39 comprising code instructions which, when the program is executed by a computer, cause the computer to implement at least one of the non-invasive control methods described above.

[0090] The system may be equipped with wired or wireless communication means allowing access to reference data or to at least one of the non-invasive control methods.

[0091] The present invention has been described and illustrated in relation to the trailing edge of a compressor blade. Nevertheless, the present invention is no less applicable to other technical fields requiring a dimensional assessment of damage characterizable by dimensional criteria. Examples include coating spalling area, impact depth, crack length, or surface density of corrosion points.

[0092] On the other hand, the examples presented above relate to the field of maintenance of aeronautical gas turbines. Transposing the teaching disclosed here to all mechanical systems requiring non-invasive inspections based on dimensional damage criteria, within the framework of their maintenance (steam turbines in power plants, wind turbine gearboxes, etc.), does not depart from the scope of the invention.

Claims

Demands

1. A non-invasive method for testing a mechanical device comprising at least one object to be characterized, comprising the following steps: a. We perform (11) the acquisition of an image of the object to be characterized, b. an initial pose is determined (12), a registration of a 3D model of the object to be characterized is performed on the acquired image, and a corrected pose is determined from the initial pose of the acquisition device as a function of the registration performed, c. a projection of the 3D model of the object to be characterized is superimposed (13) onto the acquired image, according to the corrected pose, d. We determine (14) whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances, e. if this is the case, the measure of the position or distance difference is estimated as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length.

2. A non-invasive method for testing a mechanical device according to claim 1, wherein, in order to determine (12) the initial pose, perform the registration of the 3D model and determine the corrected pose, the following sub-steps are carried out: a. (12a) at least four known or characteristic points of the 3D model are identified on the acquired image, and the initial pose of the image acquisition device is determined, in the coordinate system of the 3D model, as a function of the position in the acquired image of the at least four points of the 3D model. b. we project (12b) the contours of the 3D model onto the acquired image, then we realign said contours so that they correspond to the contours of the object to be characterized present on the acquired image, and c. we use the realigned contours to determine a corrected pose of the acquisition device.

3. A non-invasive method for testing a mechanical device according to any one of claims 1 or 2, wherein the presence of an impact is determined when at least one additional contour is detected on the contourless image corresponding to said additional contour in the 3D model.

4. A non-invasive method for inspecting a mechanical device according to any one of claims 1 to 3, wherein the acquisition device is an endoscope.

5. A method for non-invasive testing of a mechanical device according to any one of claims 1 to 4, wherein the mechanical device is an aeronautical device.

6. Data processing unit (32) comprising: - means (34) for acquiring an image of the object to be characterized; - means (35) for determining an initial pose, registering a 3D model of the object to be characterized on the acquired image, and determining a corrected pose from the initial pose of the acquisition device as a function of the registration performed; - means (36) for superimposing a projection of the 3D model of the object to be characterized onto the acquired image, as a function of the corrected pose; - means (37) for determining whether there is erosion of the object to be characterized when there is at least one difference in position or distance between contours of the acquired image and contours of the projected 3D model, taking into account erosion tolerances;and - means (38) for, if so, estimating the measure of the position or distance difference as a function of a difference in pixels between the contours of the acquired image and the contours of the projected 3D model, and the correspondence between a distance of the projected 3D model expressed in pixels and the same distance in units of length. 13

7. Product computer program (39) comprising code instructions which, when the program is executed by a computer, cause the computer to implement the control method according to any one of claims 1 to 5.

Images