IMPROVED METHOD FOR AUTOMATED INSPECTION OF A SURFACE OF A SOLID OBJECT AND SYSTEM CONFIGURED TO EXECUTE THE METHOD.

Abstract

The invention relates to a method for the automated inspection of a surface (AS) of a solid object such as, for example, an aircraft surface (AS) (100); the method employing a step of blurring said surface, represented as a depth-map image, followed by three successive derivation operations applied to said image. The invention further relates to a system configured to perform the method. Advantageously, it is thus possible to automatically detect flatness defects in an aircraft surface (AS) (100) with a level of accuracy not yet achieved. Fig. 1

Description

Title of the invention: IMPROVED METHOD FOR AUTOMATED INSPECTION OF A SURFACE OF A SOLID OBJECT AND SYSTEM CONFIGURED TO EXECUTE THE METHOD. technical field

[0001] The present invention relates to an improved method for automated inspection of a surface of an object, such as, for example, a surface of an aircraft component, from information obtained from a surface inspection device, as well as a system configured to perform such a method. PREVIOUS STATE OF THE ART

[0002] Automated inspection and quality control methods used in industry often lead to significant time savings in product manufacturing stages. This is particularly advantageous for equipment whose manufacture requires a large number of components and manufacturing steps, where increased production rate is often desired. Surface inspection of aircraft during manufacturing or maintenance operations is common, and there is a need for automated means of inspecting and detecting potential surface defects, enabling a level of perception and detection at least as high as, or even higher than, that of the human eye, for detecting very small surface defects during manufacturing or maintenance operations. Surface inspection is also used during scheduled, recurring maintenance operations.

[0003] The situation can be improved. Description of the invention

[0004] An object of the present invention is obtaining means for automated inspection of object surfaces, for example aircraft surfaces, capable of detecting a surface defect of a depth which may, depending on the case, be up to one-third of the resolution of the surface inspection device (or acquisition system) used.

[0005] To this end, an automated inspection method for the condition of a solid surface is proposed, said method comprising:

[0006] - to obtain initial information, representative of the shape of a surface of a solid object, in the form of a distance map between said surface and a sensor of a distance measuring device,

[0007] - to obtain second information, by applying a blurring filter based on this initial information,

[0008] - obtain third pieces of information by a first derivation operation of said second pieces of information, using a first derivation operator,

[0009] - obtain fourth pieces of information by a second derivation operation of the third pieces of information, using a second derivation operator,

[0010] and the method further comprising:

[0011] - obtain fifth pieces of information by a third derivation operation of the fourth piece of information, using a third derivation operator, then,

[0012] - detect a defect in said surface, using a defect detection module, at based on the fifth pieces of information obtained.

[0013] The method according to the invention may further comprise the following optional features, considered alone or in combination: - The automated inspection process also includes a step of notifying the presence of a defect detected in a surface. - The automated inspection process is included in a manufacturing process for an aircraft component, which manufacturing process further includes a step of modifying a surface based on at least one piece of information representative of a defect detected by the automated inspection process. - The blurring filter used by the process is determined from at least one characteristic of the distance measuring device (sensor) used to obtain the first information.

[0014] - The bypass operators used by the inspection process are such that:

[0015] said first derivation operator is a gradient type operator, the second derivation operator is a Hessian matrix type operator, and the third derivation operator is a non-planarity gradient type operator.

[0016] Another object of the invention is an automated system for inspecting the condition of a surface of a solid object, the system comprising electronic circuitry configured to:

[0017] - to obtain initial information, representative of the shape of a surface of a object of a solid object, in the form of a distance map between said surface and a sensor of a distance measuring device,

[0018] - to obtain second information, by applying a blurring filter based on this initial information,

[0019] - obtain third pieces of information by a derivation operation of said second pieces of information, using a first derivation operator,

[0020] - obtain fourth pieces of information by a second derivation operation of the third pieces of information, using a second derivation operator,

[0021] and the method further comprising:

[0022] - obtain fifth pieces of information by a third derivation operation of the fourth piece of information, using a third derivation operator, then,

[0023] - detect a defect in said surface, using a defect detection module, at based on the fifth pieces of information obtained.

[0024] The system according to the invention may further have the following optional characteristics considered alone or in combination: - The automated surface condition inspection system further includes electronic circuitry configured to operate a notification step for the presence of a detected defect in a surface. - The automated inspection system is included in a manufacturing system for an aircraft component which also includes means for performing a surface modification step based on at least one piece of information representative of a defect in that surface. - At least one characteristic of the blurring filter used by the automated inspection system is determined from at least one characteristic of the distance measuring device used to obtain the initial information. - The bypass operators used by the inspection system are as follows:

[0025] said first derivation operator is a gradient type operator, the second derivation operator is a Hessian matrix type operator, and the third derivation operator is a non-planarity gradient type operator.

[0026] Another object of the invention is a computer program product comprising program code instructions to execute steps of a process as previously described when said instructions are executed by a processor of an automated inspection system of a surface of a solid object.

[0027] Finally, the invention relates to a storage device comprising a computer program product as mentioned above. Brief description of the drawings

[0028] [Fig.1] schematically illustrates an automated inspection system for the condition of a surface of a solid object according to one embodiment;

[0029] [Fig.2] schematically illustrates an aircraft with surfaces that can be inspected using the system shown in [Fig.1];

[0030] [Fig.3] is a schematic representation of a set of information representative of the surface of a solid object and taking the form of a type image of depth map, (or distances) used as input data of the automated inspection system already represented on [Fig.1];

[0031] [Fig.4] is a flowchart illustrating steps of an automated inspection process of the condition of a surface of a solid object, carried out in the automated inspection system already shown in [Fig.1], according to one embodiment;

[0032] [Fig. 5] illustrates details of an overall derivation step of the process described in relation to [Fig. 4]; and,

[0033] [Fig.6] is a diagram illustrating an example of the architecture of a data processing unit comprising electronic circuitry configured to perform the process described in relation to [Fig.4], according to one embodiment.

[0034] DETAILED STATEMENT OF IMPROVEMENTS

[0035] Figure 1 represents an automated inspection system 1 for the condition of a surface AS of a solid object according to one embodiment. The automated inspection system 1 comprises a sensor device 10 connected to a processing unit 12 via a communication link 11. In one embodiment, the communication link 11 is wired. In another embodiment, the communication link 11 is configured to operate wirelessly. In one embodiment, the sensor device 10 is a camera or scanner operating as a distance sensor configured to deliver a set of data of the type "3D point cloud," this data being representative of the inspected surface AS, or more precisely, of the surface condition of the inspected surface AS. The distance sensor device 10 performs distance measurements between a reference point that it includes and a plurality of points on a surface of a solid object.In one embodiment, each point in the 3D point cloud is instantiated as a pair of x and y coordinates considered in combination with a distance value S(x,y) relative to the camera positioned with reference to the normal to the tangent plane to the inspected surface AS within the measurement field of the sensor 10 (e.g., camera or scanner). For each inspected area of ​​the surface AS, a 3D point cloud S(x,y) can be transmitted by the sensor device 10 to the processing unit 12 via the communication link 11. This 3D point cloud S(x,y) constitutes initial information representing the shape of the inspected surface AS (also referred to here as the "surface state" of the inspected surface AS). In other words, this 3D point cloud S(x,y) constitutes a depth map defining a set of distances for points on the surface AS observed and inspected within the measurement field of the sensor device 10.The processing unit 12 is configured to perform successive processing operations from this initial information, in order to detect and possibly characterize surface defects present in the inspected surface AS, such as dents and reliefs of various shapes and sizes (impacts, scratches, embossments, hollows, etc.).

[0036] [Fig.2] illustrates an example of an inspected surface AS of an aircraft 100. According to the example described in relation to [Fig.2], the surface AS is a surface of a fuselage element of the aircraft 100. Obviously, this example is not limiting and the automated inspection system 1 can be useful for inspecting many surfaces of an aircraft, including in particular the fuselage and the wing.

[0037] Figure 3 schematically illustrates a data structure comprising the aforementioned information, stored in the form of a matrix S of values, each element of which S(X, Y) is determined and stored with reference to coordinates X and Y of a plane. For example, an element S(XI, YI) represents the distance between the sensor device 10 and a point on the surface AS whose position in space is determined by the coordinates XI and YI. According to one embodiment, the matrix of values ​​S further constitutes an image of the distances respectively measured for the different points of the matrix S. For example, the brighter a point in the image is, the greater the distance between the corresponding point on the inspected surface AS and the distance measurement sensor of the sensor device 10, or vice versa.The term "corresponding point of the inspected surface" here refers to a point on the inspected surface AS that is representative of a given point in the image (or matrix) S. Thus, the structuring of the initial information obtained constitutes a 3D to 2D transformation of the inspected surface AS into an image S, which can then be processed by the processing unit 12 to detect the presence of any defects in the inspected surface AS.

[0038] Figure 4 describes steps of an automated inspection process performed by and within the automated inspection system 1, according to one embodiment of the invention. A step S0 is an initial (or initialization) step at the end of which all the circuits and elements of the automated inspection system 1 are energized, normally powered, configured, and operational. In particular, the sensor device 10 is positioned on a support or by an operator facing an area of ​​the inspected surface AS, and initial information representative of the surface condition of the inspected surface AS is transmitted by the sensor device 10 to the processing unit 12. During a step SI, a blurring operation is performed on the matrix (or image) S, which is instantiated by the initial information obtained by the processing unit 12.According to one embodiment, the blurring operation creates a blur of sufficient size to smooth the surface of the solid object and reduce noise from the measuring tool, but without eliminating the desired defects (for example, a Gaussian blur with a sigma of 2 mm). The resulting matrix or image from this blurring constitutes new information, representative of the surface AS, in the form of a Boutée image matrix. Then, a step S2 consists of cleverly performing a triple derivation of the Boutée image matrix applied to the... Information derived from blurring. The term "triple derivation" here refers to three successive derivation operations applied to the image matrix representing the inspected surface AS (or a zone or portion of this surface), cleverly using three different derivation operators. The details of these operators are described later in this description, in relation to [Fig. 5]. Finally, the detection of one or more potential defects present in the inspected surface AS is performed during step S3, using a defect detection module. In one embodiment, the defect detection module is implemented by the processing unit 12.In one embodiment, the fault detection module relies on a classical segmentation approach that may use a blob detection algorithm, such as a Gaussian difference filter, or a partitioning method such as the k-means algorithm, or even a watershed algorithm. In another embodiment, the fault detection module is a neural network trained to perform segmentation and / or classification tasks. For example, it could be the "YOLO" (You Only Look Once) algorithm or an algorithm from the ResNet (Residual Networks) family, capable of simultaneously performing detection and classification, or the U-Net segmentation algorithm.

[0039] The selected detection algorithm operates using the image matrix resulting from the three successive derivations following the Routings and provides information enabling the identification and localization of any surface defects. In one embodiment, this detection is achieved by instantiating bounding boxes, each with a probability of the presence of a defect in the area delimited by the bounding box of the image matrix resulting from the three derivations. It is then possible, during a notification step S4 (not shown in [Fig. 4]), subsequent to step S3, to provide, if necessary, information on the location of one or more potential defects detected in the inspected AS surface, in order to monitor or organize modifications to the manufacturing process and / or maintenance operations.

[0040] Figure 5 provides details of the S2 derivation step according to a non-limiting embodiment.

[0041] According to this embodiment, step S2 comprises three successive derivation operations using three derivation operators respectively.

[0042] A first step S20 (first sub-step of S2), subsequent to step SI, performs a gradient calculation of the inspected surface, which is expressed in the form:

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060] where S (x, y) is the image matrix after blurring obtained at the end of the SI blurring step. Indeed, the first-order differential operator for a surface is the gradient. It allows us to determine, at each point of the surface, a vector quantity defining the direction and magnitude of the steepest slope at that point. A two-dimensional representation of the surface is thus obtained, in the form of a scalar field corresponding to the magnitude of the gradient. This representation advantageously provides information on the slope's inclination at each point of the analyzed surface. A second step S21 (second sub-step of S2), subsequent to step S20, performs a Hessian matrix calculation, which is expressed in the form: Sxx Sxy1 Or d~S(xy); d“S(x,y); ù2S(x,j'); and d~S(x,y), SXX ~ SyX - Syy = 3^ allowing the determination of local variations in flatness: (7(X, y) = Sly The Hessian matrix, also called the second derivative matrix, is the equivalent of the gradient in second-order differential geometry. It is a matrix quantity that provides information about the curvature of each point on the analyzed surface at that point. Regarding the eigenvalues ​​of the Hessian matrix: The eigenvalues ​​of H are called principal curvatures and correspond to the strength of the curvature along the principal directions. Invariant under rotation, they are real values, denoted Uct K2 such that K\ > K^. These eigenvalues ​​of H are calculated as follows: K = ~-------2------“ where K is Ki or K2 depending on the + or - operator Regarding the principal directions or second-order gauge coordinates: The eigenvectors of H are called principal directions and correspond to the directions of the principal curvatures, which directions are always orthogonal. These principal directions are named gauge coordinates p and ¢, such that p corresponds to the direction of greatest curvature and the direction of least curvature. These main directions are calculated as follows:

[0061] p- u I ||w|| and q - ||v||

[0062] where ( 1 2^ 1 and where f 1 1

[0063] Partial derivatives with respect to second-order Gauge coordinates can then be determined and used.

[0064] According to alternative embodiments, several two-dimensional projections of the Hessian matrix can be obtained and used in the form of a scalar field. Note that, in this context, by definition,

[0065] _ 3¾ = Kiet, _ ÿs = K2 SPP ~ dtP SW ~ ^2

[0066] According to one embodiment, a Gaussian curvature type projection G is determined from the Hessian matrix, corresponding to the determinant of the Hessian matrix, and providing an intrinsic measure of the curvature at each point of the surface:

[0067] G = SppSqq = Kl.K2 = de{H)

[0068] According to one embodiment, a projection of the average curvature type HM is determined from the Hessian matrix, corresponding to the average of the principal curvatures or eigenvalues ​​of the Hessian matrix, expressing at each point the average curvature of the surface at that point:

[0069] _ ^^2

[0070] According to one embodiment, a Laplacian-type projection is determined from the Hessian matrix, corresponding to the trace of the Hessian matrix, which can also be calculated directly as the divergence of the gradient, and also provides information on the local curvature at each point: [0071 ] ^(x, y) = V ^xy) = div(v S(x, j)) = tract^H) = SXJC + Syy

[0072] According to one embodiment, a projection of the flatness deviation type is determined from the Hessian matrix, corresponding to the sum of the squares of the principal curvatures or eigenvalues ​​of the Hessian matrix, expressing at each point of the surface its local “rate of non-flatness”:

[0073] (j ~SpP + Sqq-K2{+Kl

[0074] According to one embodiment, a projection of the degree of curvature type C, corresponding to the square root of the flatness deviation, is determined from the Hessian matrix:

[0075] C = ^S2pp+Sù=^ + ^

[0076] According to one embodiment, an angle-type projection of shape F, corresponding to the arctangent of the ratio of the principal curvatures, and providing information on the local shape of the surface at each point:

[0077] p, 4(¾) / -1CM F = tan - tan I 77 )

[0078] Finally, a third step S22 (third sub-step of S2), subsequent to the step S21 operates a so-called "third-order" derivation, using a new operator defining a previously determined local flatness variation rate, capable of highlighting defects, including very small ones, by inserting the resulting information into a module for detecting potential defects during

[0079] Vcr(x, y) = 2- Step S3, subsequent to step S22: ■^xx ' $xxx + Sy? ' Syjx SXX ' S*xy + Syy ■ Syyj

[0080] There is no third-order differential operator that can be expressed in a trivial form. According to one embodiment, it is proposed to apply the first-order differentiation operator (gradient) to one of the scalar fields obtained by second-order differentiation (e.g., calculation of the gradient of flatness deviations).

[0081] A two-dimensional representation is then obtained in the form of a scalar field corresponding to the norm of said gradient. This representation provides information on the rate of change of the curvature or flatness of the surface at each of its points.

[0082] According to various embodiments, it is possible to perform numerous other combinations to obtain third-order differential information (e.g., applying the gradient operator and then calculating the associated scalar field three times successively, calculating the Hessian of the scalar field obtained after the first differentiation, etc.). These variations all advantageously allow for highlighting information related to a rate of change in the surface curvature.

[0083] Fig. 6 schematically illustrates an example of the internal architecture of the processing unit 12 of the automated inspection system 1, configured to perform the process described above.

[0084] According to the hardware architecture example shown in [Fig. 6], the processing unit 12 then comprises, connected by a communication bus 129: a processor or CPU (Central Processing Unit) 121; a RAM (Random Access Memory) 122; a ROM (Read Only Memory) 123; a storage unit such as a hard drive (or a storage media reader, such as an SD (Secure Digital) card reader) 124; a communication interface module 125 enabling the processing unit 12 to communicate with remote devices, such as the configured sensor device 10 to perform surface analyses by measuring distances or other remote equipment, for example equipment at an aircraft production site or an aircraft maintenance site.

[0085] The processor 121 of the processing unit 12 is capable of executing instructions loaded into the RAM 122 from the ROM 123, external memory (not shown), a storage medium (such as an SD card), or a communication network. When the processing unit 12 is powered on, the processor 121 is capable of reading instructions from the RAM 122 and executing them. These instructions form a computer program causing the processor 121 of the processing unit 12 to implement all or part of an automated inspection process as previously described.

[0086] All or part of such an automated surface inspection method, for example for aircraft surfaces, can then be implemented in software form by executing a set of instructions by a programmable machine, for example a DSP (Digital Signal Processor) or a microcontroller, or be implemented in hardware form by a dedicated machine or component, for example a FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit). In general, the processing unit 12 comprises electronic circuitry configured to implement an automated inspection method for the surfaces of solid objects.Obviously, the processing unit 12 also includes all the elements usually present in a system comprising a control unit and its peripherals, such as a power supply circuit, a power supply monitoring circuit, one or more clock circuits, a reset circuit, input / output ports, interrupt inputs, bus drivers, this list being non-exhaustive.

Claims

Demands

1. A method for automated inspection of the condition of a surface (AS) of a solid object (100), said method comprising: - obtaining (S0) first information, representative of the shape of the surface (AS) of the solid object, in the form of a distance map between said surface (AS) and a sensor of a measuring device (10), - obtaining (S1) second information, by applying a blurring filter to said first information, - obtaining (S2, S20) third information by a first derivation operation of said second information, using a first derivation operator, - obtaining (S2, S21) fourth information by a second derivation operation of the third information, using a second derivation operator, and said method being characterized in that it further comprises: - obtaining (S2, S22) fifth information by a third derivation operation of the fourth information,using a third derivation operator, then, - detect a defect in said surface (AS), using a defect detection module, from the fifth information obtained.

2. Automated inspection method according to claim 1, further comprising a notification step (S4) of the presence of said defect detected in said surface (AS).

3. Method of manufacturing an aircraft component (100) comprising an automated inspection method according to claim 2 and a step of modifying said surface (AS) from at least one information representative of said defect.

4. A method according to any one of claims 1 to 3, wherein said blurring filter is determined from at least one feature of said distance measuring device (10).

5. A method according to any one of claims 1 to 4, wherein: said first derivation operator is a gradient-type operator, said second derivation operator is a Hessian matrix-type operator, and, said third derivation operator is a non-planarity gradient type operator.

6. Automated inspection system (1) for the condition of a surface (AS) of a solid object, (100) said system comprising electronic circuitry configured to: - obtain (S0) first information, representative of the shape of a surface of a solid object, in the form of a distance map between said surface and a sensor of a measuring device, - obtain (S1) second information, by applying a blurring filter to said first information, - obtain (S2, S20) third information by a derivation operation of said second information, using a first derivation operator, - obtain (S2, S21) fourth information by a second derivation operation of the third information, using a second derivation operator, and said system being characterized in that it further comprises electronic circuitry configured to: - obtain (S2,S22) of the fifth information by a third derivation operation of the fourth information, using a third derivation operator, then, - detect (S3) a defect in said surface (AS), using a defect detection module, from the fifth information obtained.

7. Automated inspection system (1) of the condition of a surface (AS) according to claim 6 further comprising electronic circuitry configured to operate a notification step (S4) of the presence of said defect detected in said surface (AS).

8. A manufacturing system for an aircraft component (100) comprising an automated inspection system (1) according to claim 7 and means for performing a modification step of said surface (AS) from at least one representative piece of information of said defect.

9. System according to any one of claims 6 to 8, wherein said blurring filter is determined from at least one feature of said distance measuring device (10).

10. Product computer program comprising program code instructions for executing steps of a process according to any one of claims 1 to 5 when said instructions are 13 executed by a processor of an automated inspection system of a surface of a solid object.

11. Storage device comprising a computer program product according to claim 10.