Method for detecting a defect on a metal part

Abstract

One aspect of the invention relates to a method for detecting a surface defect on a metal part, i.e. a part of which the outer surface is reflective, with a potentially complex geometry. This method is based on the use of a neural network, referred to as a detection network, specifically trained to detect such a defect, either directly by analysing images of the part in question, which images are acquired using a camera and with illumination by artificial lighting, or by analysing properties of the part and the acquisition conditions, which are determined from the images of the part. The proposed approach has the advantage of not requiring a priori information on the acquisition or on the part thanks to the combined use of two self-supervised training neural networks to form the training base necessary for the supervised training of the detection network.

Description

DESCRIPTION TITLE: Method for detecting a defect on a metal part TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of non-destructive testing.

[0002] The present invention relates to a method for detecting a defect on a metal part with a complex geometry. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] In non-destructive testing, optical inspection methods are commonly used to detect surface defects in mechanical parts. Most of these methods rely on acquiring an image of the part using a camera, illuminated by a controlled artificial light source. This enhances image contrast and reveals the part's texture, allowing for the detection of potential defects such as cracks. Such approaches automate part inspections, thereby improving their robustness, reliability, and speed.

[0004] The drawback is that conventional optical inspection methods cannot reliably inspect reflective parts, such as metallic parts, and those with complex external geometries, which are often critical in various industries, such as aircraft turbine blades. Indeed, comparing an image of such a part to a reference image (i.e., an image of a part without defects or with a known defect) to detect a defect is difficult due to the natural variability of the part's grain (for example, caused by variations in alloys and / or colorimetry), even when the parts are defect-free, resulting in significant texture differences. This natural grain variability then masks the variability in the image associated with the presence of a defect.In other words, it is difficult to obtain images of metal parts with reference defects (i.e., known defects) that can be reliably used, especially since the occurrence of a defect on such a part is rare due to the particularly robust and optimized manufacturing processes. It is also noted that the images obtained by these conventional methods are subject to masking effects, whether by... shading or light saturation, due to the reflectivity of the pieces and their geometry.

[0005] In this context, methods have been developed specifically adapted to metallic parts, which are highly reflective and may have complex shapes. These methods rely on the acquisition, by a camera, of images of the part illuminated from different angles of incidence by different artificial light sources, typically using stereophotometry.

[0006] Existing methods based on this acquisition principle require knowledge of certain acquisition parameters, such as the direction and / or intensity of light from each source relative to the part within the camera's frame of reference, or certain part properties, such as its reflectance. This information serves as a reference for detecting the presence of a part defect. These methods therefore require precise calibration of the acquisition and the use of additional equipment to measure this data, which is time-consuming, complicates the inspection process, and introduces further errors due to measurement uncertainties.

[0007] There is therefore a need for a defect detection tool for a complex shaped metal part that is reliable, robust, quick to implement, and does not require prior information on the acquisition or the part in question. SUMMARY OF THE INVENTION

[0008] The invention offers a solution to the problems mentioned above, by enabling the detection of a defect in a part via a neural network trained on a training database built through the joint use of two self-supervised training neural networks adapted to automatically determine the properties necessary for the implementation of defect detection.

[0009] A first aspect of the invention relates to a computer-implemented method for detecting a defect on a metal part, the method comprising: Obtaining at least three images of the part, the images of the part being acquired via a camera and an artificial light source; Detecting the defect in the part by analyzing images of the part via a neural network, called a detection network, the detection network being a supervised training artificial neural network configured to detect the defect from the images of the part, the detection network being trained on a training database built through a first neural network and a second neural network, the first neural network being an artificial neural network configured to determine, for each reference image of a first plurality of reference images, a reference reflectance map including reference reflectance values ​​and a reference normal direction map including reference vectors of normal directions, and the second neural network being an artificial neural network configured to determine,For each reference image in a first plurality of reference images, a reference vector of illumination direction and / or a luminous intensity of the artificial lighting source, each reference image in the first plurality of reference images being an image of a reference part among a plurality of reference parts, the first and second neural networks being self-supervised and jointly optimized via minimization of a distance between the first plurality of reference images and a plurality of intermediate images, the plurality of intermediate images being constructed from the reference reflectance maps, the reference normal direction maps and the reference vectors of illumination directions.

[0010] The term "defect" refers to any type of defect detectable on the surface of the part, such as a crack or deformation due to shock or impact.

[0011] "Artificial lighting" refers to a light source that is not a natural light source but is specifically added to the scene to acquire the image.

[0012] A "mapping" refers to a representation of the distribution of a specific property within an image, typically reflectance or the normal direction relative to the surface of the part. A mapping is therefore an image whose The value of each pixel (or a vector assigned to each pixel) corresponds to the value of that property (or, respectively, a vector of values ​​for that property) for the corresponding pixel in the image from which the mapping is derived. Thus, for each reference image, the image corresponding to the reference reflectance map represents how the part scatters light in the reference image, while the image corresponding to the reference normal direction map indicates the orientation of the surfaces of the reference part in the associated reference image.

[0013] Reflectance refers to the rate at which light is reflected by a surface at a given wavelength. Typically, it is the reflection rate of the surface of a part under artificial lighting. Reflectance is sensitive to the presence of a defect, particularly because such a defect significantly alters the intensity and / or direction of the light reflected locally by the part.

[0014] The term "normal direction" refers to the direction normal to a point on the surface of the part. It is therefore a purely geometric property of the part that can be determined from image analysis. The normal direction is also sensitive to the presence of a defect, since this induces a local deformation, in the form of depressions and bumps, on the surface of the part.

[0015] The term "self-supervised" means that the optimization, or learning, of the first and second neural networks is performed unsupervised by their own optimization mechanism directly by comparing the result of their joint prediction—i.e., the intermediate images—with the reference images. In other words, the first and second neural networks are optimized by iteratively comparing the intermediate images, formed from the reference reflectance value and the reference vectors of normal and illumination directions, with the reference images.

[0016] The term "illumination direction" refers to the principal direction of the light beam emitted by the artificial light source towards the surface of the room in the camera's frame of reference. It is therefore an intrinsic property of the artificial light source. Preferably, this property is given in the camera's frame of reference. Luminous intensity then corresponds to the intensity of the light emitted by the artificial light source in the illumination direction.

[0017] Thanks to the invention, it is possible to detect a defect on a metal part, particularly one with a complex geometry, without contact, with improved reliability and robustness compared to known state-of-the-art methods. Indeed, the approach underlying the method of the invention makes it possible to effectively exploit all the information produced through the use of directional lighting.

[0018] Thus, unlike other defect detection methods that use all acquired images or select a specific image based on certain criteria, the proposed approach provides the detection network with two specific pieces of information (for example, in the form of images): the reflectance of the part and the direction of the normals to the part's surface. This information is estimated by the first and second neural networks dedicated to analyzing the acquired images, typically obtained by stereophotometry. Furthermore, by using a self-supervised training method, for example, based on a differentiable rendering function to construct the intermediate images, the first neural network learns to estimate the reflectance and normal directions accurately and without prior information about the acquisition and / or the part being inspected.The network's performance is then evaluated by comparing the reconstructed images to the original images (i.e., the reference images) using a cost function based on the distance between these images. This function enables backpropagation of the gradient, thus facilitating the updating of the weights of the first and second neural networks.

[0019] It is well known that image synthesis—that is, its reconstruction from reflectance and normal directions, typically using a rendering function—requires knowledge of the illumination direction and intensity of the light source. Traditionally, these directions and intensity values ​​are obtained using conventional calibration methods, which have the drawback of relying on the assumption that the light source is a perfectly point source, which is not the case in reality. Furthermore, the calibration phase is tedious to perform, particularly in an industrial production context where acquisition parameters and conditions can vary from one acquisition to another, necessitating frequent calibrations to ensure the accuracy of the results.

[0020] To overcome these limitations, the present approach proposes to simultaneously estimate, in each reference image, using first and second neural networks, not only the reflectance and normal directions, but also the illumination direction and / or the light intensity of the light source (the illumination direction and light intensity being one or both unknowns at the time of implementation). The proposed method thus eliminates the assumptions and simplifications associated with traditional models. In other words, the first and second neural networks are capable of finding the implicit model that best explains the observations and inferring the illumination direction and / or light intensity more accurately and adaptively than current methods.

[0021] Therefore, incorporating the estimation of the illumination direction and / or light intensity (which are no longer measured) improves the robustness and accuracy of the results. This approach reduces the impact of errors related to traditional calibration and provides a more reliable estimate of the illumination direction and / or light intensity, resulting in a better representation of the physical properties of the inspected parts.

[0022] Finally, the use of reflectance and normal direction maps offers several advantages over other traditional approaches, notably by enabling more precise defect detection, typically in the presence of an impact or scratch. Indeed, such defects significantly alter the reflectance and normal direction values ​​contained in these maps. Furthermore, these reflectance and normal direction maps encapsulate physical information relevant to defect detection, providing more complete information about the part and the scene (light, lighting conditions) than state-of-the-art methods for analyzing anomalous features in images of the part being inspected.

[0023] It is noted that the architecture of the first neural network is designed to simultaneously estimate the reflectance and normal direction maps for each reference image

[0024] In addition to the characteristics just mentioned, the process according to the first aspect of the invention may have one or more other characteristics complementary from the following, considered individually or according to all technically possible combinations.

[0025] In one embodiment, the metal part has a complex geometry.

[0026] The term "complex geometry" refers to a part whose external shape exhibits significant variations, such as a curvature of its surface, resulting in light not being reflected uniformly or unidirectionally. This typically applies to aircraft blades or aircraft engine transmission components, such as gears.

[0027] In one embodiment, the training database is augmented by generating a plurality of pairs of new reference reflectance maps and new reference normal direction maps, the new reference reflectance and reference normal direction maps of each pair of new maps being generated by selecting reference reflectance values ​​and reference normal direction vectors corresponding to a predetermined defect and taken respectively from a reference reflectance map and a reference normal direction map corresponding to a reference image of the first plurality of reference images including the predetermined defect, the augmentation being implemented by replacement, with the selected reference values ​​and reference vectors,of reflectance reference values ​​and normal direction reference vectors included in:, A portion not corresponding to a predetermined defect in the respective reference reflectance and reference normal direction maps from which the reference values ​​and reference vectors for replacement are selected; or A reference reflectance map and a reference normal direction map, respectively, corresponding to a reference image of the first plurality of reference images not containing any defects.

[0028] A "predetermined defect" is defined as a known defect on the reference part and identified in one or more corresponding reference images. said reference part. The predetermined term therefore refers, in this context, to the fact that the defect is detected and identified before the implementation of the process according to the invention.

[0029] Most state-of-the-art methods for detecting defects in metal parts rely on deep learning. Therefore, a large amount of data is required for these algorithms to produce sufficiently reliable predictions. However, identified defects in such parts that can serve as references are rare, making it impossible to base learning solely on these few known defects.

[0030] It is well known to use data augmentation techniques to improve machine learning. For example, generating new images using neural networks is a known method. One drawback is that these techniques do not account for the interaction between light and matter, resulting in unrealistic and overly synthetic data that limits the reliability and repeatability of the tests. Other data augmentation techniques rely on simulation tools. However, these techniques are time-consuming and only allow for limited variability in the augmented data, further limiting the reliability and repeatability of the tests.

[0031] The invention therefore proposes to remedy this drawback by augmenting the training database with data constructed realistically based on existing and identified defects in the reference images, and not on simulations or on data obtained simply via mathematical transformations (rotations, deformations, enlargement, etc.), as is commonly done in known data augmentation approaches.

[0032] In one embodiment, the replacement by the selected reflectance reference values ​​and normal direction reference vectors is implemented by applying a geometric transformation of their distribution.

[0033] It is therefore possible to generate other realistic defects from existing defects identified in the reference images.

[0034] In one embodiment, the detection network is configured to detect the fault from a reflectance map comprising values of reflectance and a normal direction map including directions normal to the part, the reflectance and normal direction maps being determined in the part images, the training database including the reference reflectance maps and the reference normal direction maps determined by the first neural network.

[0035] It is thus possible to detect a defect directly in the maps determined from the images of the part to be inspected. This embodiment is particularly advantageously compatible with a multi-source artificial lighting acquisition technique, typically using stereophotometry.

[0036] In one embodiment, reflectance mapping and normal directions to the part are determined in the part images via the first neural network.

[0037] It is therefore possible to use the first and second neural networks to determine the image maps of the part to be inspected necessary for defect detection. In other words, the first and second neural networks are also configured to determine the reflectance and normal direction maps for the images of the part to be inspected.

[0038] In one embodiment, the detection network is configured to detect the defect directly from the part images, the training database comprising a second plurality of reference images, the second plurality of reference images being constructed from the reference reflectance maps and the reference normal direction maps.

[0039] In this alternative implementation, the defect is detected by the detection network directly in the images of the part to be inspected. The images of the third plurality of images are, for example, constructed using the differentiated rendering function employed for optimizing the first and second neural networks.

[0040] In one embodiment, the training database is augmented by generating a third plurality of reference images, each reference image of the third plurality of reference images being generated from the reference reflectance maps and normal directions of reference of one of the reference images of the first plurality of reference images comprising a predetermined defect and a modified illumination direction reference vector, the illumination direction reference vector comprising at least two reference values, the modified illumination direction reference vector being obtained by modifying at least one of the reference values ​​of said illumination direction reference vector, said at least one modified reference value being selected from a predefined range of values ​​pertaining to a dimension corresponding to said reference value to be modified and comprising the reference value to be modified.

[0041] In one embodiment, the training database is augmented by generating a fourth plurality of reference images, each reference image of the fourth plurality of reference images being generated from the new reference reflectance maps and reference normal directions of a pair of new maps from the plurality of pairs of new maps, and a modified illumination direction reference vector, the illumination direction reference vector comprising at least two reference values, the modified illumination direction reference vector being obtained by modifying at least one of the reference values ​​of said illumination direction reference vector,said at least one modified reference value being selected from a predefined range of values ​​relating to a dimension corresponding to said reference value to be modified and including the reference value to be modified of the illumination direction reference vector.

[0042] In both embodiments, the database is further augmented by generating new reference images. The idea here is to generate images from existing maps by modifying the illumination direction associated with those maps. In other words, this approach allows for the generation of new "views" of the room—that is, new images as if the room were illuminated by the light source from a different direction. These embodiments are particularly advantageous when the number of artificial light sources is small (typically between 1 and 10). Indeed, this data augmentation mechanism allows for a realistic expansion of the database without having to reacquire images with new artificial light sources.

[0043] In one embodiment, each reference image comprises a plurality of pixels, the first neural network accepting as input a predefined position of each pixel of the plurality of pixels relative to an artificial light source illuminating the reference room corresponding to the reference image during the acquisition of said reference image, and wherein the second neural network accepts as input said reference image.

[0044] In one embodiment, for each of the reference parts in the plurality of reference parts, the first and second neural networks are jointly optimized from each of the reference images of said reference part among the first plurality of reference images.

[0045] Thus, the first and second neural networks are both simultaneously and jointly optimized for each of the reference parts, independently of the other reference parts. Therefore, there is joint optimization of the first and second neural networks for each of the reference parts.

[0046] Furthermore, all images acquired for the same reference part are used for training. The goal is therefore not to generalize training from one reference part to another, but rather to encapsulate the geometric and photometric information specific to the image acquisition for a particular reference part.

[0047] Thus, the first and second neural networks are intentionally trained to overlearn the details of reference images of a given reference part, by exploiting the set of reference images available for that reference part.

[0048] A second aspect of the invention relates to a device for detecting a defect on a metal part with complex geometry, the device comprising a processor adapted to implement the process according to the first aspect.

[0049] A third aspect of the invention relates to a computer program product comprising instructions which, when the program is executed on a computer, lead the computer to implement the steps of the process according to the first aspect.

[0050] A fourth aspect of the invention relates to a computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the process according to the first aspect.

[0051] The invention and its various applications will be better understood by reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0052] The figures are presented for illustrative purposes only and are in no way limiting to the invention. Figure 1 is a block diagram illustrating a method for detecting defects on a metal part according to one embodiment of the invention. Figure 2 is a block diagram illustrating the optimization of neural networks according to another embodiment of the invention. Figure 3 is a synoptic diagram illustrating the augmentation of a training database according to one embodiment of the invention. Figure 4 is a synoptic diagram illustrating a variant implementation of the process according to Figure 1. Figure 5 is a block diagram illustrating the increase in the training database according to another embodiment of the invention. Figure 6 is a synoptic diagram illustrating the increase in the training database according to another embodiment of the invention, following the data increase in Figure 3. Figure 7 is a schematic representation of a defect detection device on a metal part according to one embodiment of the invention. Figure 8 is a synoptic diagram illustrating a method for augmenting a database. DETAILED DESCRIPTION

[0053] Unless otherwise specified, the same element appearing on different figures has a unique reference.

[0054] The present invention proposes a method for detecting a surface defect on a metal part, that is to say, a part whose external surface is reflective and potentially has a complex geometry. This method relies on the use of a neural network, called a detection network, specifically trained to detect such a defect, either directly by analyzing images of the part in question acquired via a camera and under artificial lighting, or by analyzing the properties of the part and the acquisition conditions, determined from the images of the part.

[0055] The proposed approach has the advantage of not requiring prior information on the acquisition or on the part thanks to the joint use of two self-supervised training neural networks to form the training basis necessary for the supervised training of the detection network.

[0056] In this respect, and with reference to Figure 1, one aspect of the invention relates to a method 100 for detecting a defect on a metal part. This method 100 can be implemented by computer. The defect can be any type of defect observable on the surface of the metal part. The metal part can be any part made of any reflective metallic material.

[0057] In particular, the metal part may have a complex geometry, meaning that its external shape may have significant variations in surface shape, resulting in light not being reflected uniformly or unidirectionally. This is typically an aircraft blade or an aircraft engine transmission component, such as a gear.

[0058] Process 100 includes a step 110 of obtaining at least three images of the part. Hereafter, for the sake of brevity, the phrase "images of the part" refers to "at least three images of the part".

[0059] The images of the room are or have been acquired using a camera and an artificial light source. The camera and the artificial light source are, for example, part of an acquisition system.

[0060] Depending on the application, several artificial lighting sources can be used, for example when the acquisition system relies on a stereo-photometry technique, aiming to acquire several images of the room to be controlled under various illuminations, generated via the activation and / or deactivation of these different artificial lighting sources.

[0061] The camera can be any device capable of capturing one or more images of the room, typically via a lens. The acquired images can be in any format and can be color or grayscale images. Preferably, the acquired images are grayscale or are color images converted to grayscale before being obtained in step 110. The camera is sensitive to the wavelength(s) emitted by the artificial light source.

[0062] The artificial light source can be any type of artificial light source, such as a polychromatic or monochromatic light source. The light source is, for example, a point source of artificial light. In the case of stereophotometric acquisition, the artificial light sources are preferably point sources of artificial light.

[0063] Process 100 then includes a step 120 of defect detection in the images of the part to be inspected. This defect detection is performed by analyzing the part images using an artificial neural network, called a "detection network." The detection network is therefore configured to detect the defect from the part images. The detection network is a supervised training artificial neural network.

[0064] The detection network is built by learning on a database, called the "training database". Since the training of the detection network is supervised, the training database is a database where the data it contains is annotated, that is to say, it is annotated with a label indicating the characteristic that we seek to predict via the learning network, in this case the presence or absence of a defect.

[0065] The unique feature of this database is that it is built using two neural networks in conjunction. More specifically, this database is constructed using a first neural network and a second neural network. These neural networks are used to generate the Data from the training database or data to be generated, which is then transformed to form the training database. More specifically, the database is formed through the joint optimization of the first and second neural networks.

[0066] The first and second neural networks are trained (or optimized) in a self-supervised manner to extract information from an initial plurality of reference images. These reference images are images of reference parts, where each reference image represents one of a plurality of reference parts. It is noted that the reference parts may or may not be of the same type (blade, bearing, piston, fin, etc.); that is, they may be parts of the same type or parts of different types, for all or some of the reference parts.

[0067] Multiple reference images can be images of the same reference part. This is the case, for example, when these different reference images of the same reference part are acquired with different artificial lighting sources.

[0068] A "reference part" is defined as a metal part for which it is already known whether or not a defect exists on its surface. This defect must have been previously detected and identified; in other words, such a defect is predetermined. Thus, for each reference image, it is already known whether that reference image corresponds to a part with a defect and whether that defect is present in the reference image. In other words, each reference image is labeled (i.e., annotated) with a label (or annotation) indicating whether the predetermined defect of the corresponding reference part is present in the reference image or whether the reference part itself is free of defects. These labels, or annotations, are predetermined by an operator.

[0069] Within the framework of the invention, the first and second neural networks are used to determine in the reference images the reflectance of the reference parts and the directions normal to the surface of the reference parts, as well as the direction of illumination and / or the light intensity of the source used to acquire each of the reference images.

[0070] In particular, the first neural network is an artificial neural network that is configured to determine, for each reference image of the first plurality of reference images, a reference reflectance map and a reference normal direction map.

[0071] The reference reflectance map includes reference reflectance values. This map reflects the distribution of the reflectance property of the reference part within the reference image from which it is derived. For example, this map is an image of the same size that includes the same number of pixels as the corresponding reference image, such that each pixel in the reference reflectance map contains a reference reflectance value for the reference part for the corresponding pixel in the reference image. In other words, the reflectance is calculated for each pixel in the reference image, thus forming an image of the distribution of this reflectance, called the reference reflectance map.

[0072] The reference normal direction map comprises reference vectors of normal directions. This map reflects the distribution of the normal direction at different positions on the surface of the reference part, within the reference image from which it is determined. For example, this map is an image of the same size that includes the same number of pixels as the corresponding reference image, such that each pixel of the reference normal direction map includes a vector, called a reference vector, corresponding to the normal direction at that pixel—that is, a vector indicating the normal direction of the reference part for the corresponding pixel in the reference image. In other words, the normal direction is calculated for each pixel of the reference image, thus forming an image of the distribution of this normal direction, called the reference normal direction map.The normal direction reference vector comprises at least two reference values, each corresponding to a dimension in space. Typically, the normal direction reference vector includes one value corresponding to an azimuth of the normal direction and one value corresponding to an inclination of the normal direction relative to the part. Alternatively, the normal direction reference vector can include three reference values ​​corresponding to the three dimensions of a Euclidean space. For a given pixel, the normal direction is therefore defined by the normal direction reference vector corresponding to that pixel.

[0073] The second neural network, on the other hand, is an artificial neural network configured to determine, for each reference image in the first plurality of reference images, the direction of illumination of the source relative to the reference part corresponding to that reference image, for example, in the camera's coordinate system. As detailed in one embodiment of the invention, the second neural network can also be configured to determine the light intensity of the source in the direction of illumination. However, for the sake of illustration, the light intensity is considered, in this example of implementation, to be known at the time of inspection of the part; that is, it is known before the implementation of method 100, unlike the direction of illumination, which is not known and must be determined.

[0074] In particular, the second neural network determines a reference vector corresponding to (i.e., indicating) the direction of illumination. The reference vector for the direction of illumination comprises at least two reference values, each corresponding to a dimension in space. Typically, the reference vector for the direction of illumination includes one value corresponding to the azimuth of the direction of illumination and one value corresponding to the inclination of the direction of illumination relative to the room. Alternatively, the reference vector for the direction of illumination can include three reference values ​​corresponding to the three dimensions of a Euclidean space. The direction of illumination is thus defined by this reference vector for the direction of illumination.

[0075] Thus, for each reference image of the first plurality of reference images, the first and second neural networks determine a reference reflectance map, a reference normal direction map, and the illumination direction.

[0076] The training, also called "optimization," of the first and second neural networks is jointly implemented in a self-supervised manner; that is, these neural networks learn on their own to accurately and reliably produce the maps and illumination direction, and not from labeled (i.e., annotated) data. This training is performed by constructing an image, called an intermediate image, from the reference reflectance map, the reference normal direction map, and... the direction of illumination which are determined from one of the reference images.

[0077] Once constructed, this intermediate image is compared to the reference image corresponding to the mapping and illumination direction used to create the intermediate image. This comparison is performed, in particular, by calculating a distance between this intermediate image and the relevant reference image.

[0078] The distance is, for example, equal to the difference between the pixel values ​​of the intermediate image and the corresponding reference image. Typically, the distance can be a distance determined to the L1 or L2 norm between the reference image and the intermediate image. Indeed, if we take, for example, the mean absolute difference, the distance is equal to where M corresponds to the reference image and M corresponds to the intermediate image, with p and n the number of pixels of the reference and intermediate images according to each of the dimensions of these images, respectively (these two images being of the same size).

[0079] The distance calculated between the two images serves as a cost function for training the first and second neural networks. These first and second neural networks are therefore trained through iterative optimization aimed at minimizing this distance.

[0080] More precisely, this optimization is performed by comparing a plurality of intermediate images, each generated from a reference reflectance map, a reference normal direction map, and an illumination direction map, all determined from one of the reference images in the first set of reference images. In other words, an intermediate image is constructed for each of the reference images. Thus, the optimization of the first and second neural networks is jointly performed by minimizing the calculated distance between each intermediate image and its corresponding reference image.

[0081] In other words, the first and second neural networks are self-supervised training and are jointly optimized via minimizing a distance between the first plurality of reference images and the plurality of intermediate images, the plurality of intermediate images being constructed from the reference reflectance maps, the reference normal direction maps and the reference illumination direction vectors.

[0082] The parameters of the first and second neural networks can be initialized by any known method, for example randomly.

[0083] These parameters can also be optimized using any cost-function-based minimization method, in this case, the calculated distance between the reference and intermediate images. Typically, these parameters are iteratively optimized until the calculated distance is less than or equal to a predefined threshold. This predefined threshold is set prior to training the first and second neural networks using any known approach, for example, automatically or manually, such as by an operator based on their knowledge and / or experience.

[0084] One aspect of the invention therefore also relates, in connection with figure 2, to a mechanism for training the first and second neural networks.

[0085] Training 200 includes a step 210 of determination, by the first neural network, of the reference reflectance maps and reference normal directions for each of the reference images of the first plurality of reference images.

[0086] Training 200 also includes a step 220 of determination, by the second neural network, of the direction of illumination, typically the reference vector of direction of illumination, of the artificial lighting source for each reference image.

[0087] Training 200 also includes a step 230 of constructing the intermediate image from the reference reflectance mapping, the reference normal direction mapping and the illumination direction (typically the illumination direction reference vector) for each of the reference images of the first plurality of reference images.

[0088] Training 200 also includes a step 240 of determining the distance between intermediate images and reference images.

[0089] When the distance is strictly greater than the predefined threshold, the parameters of the first and second neural networks are modified, and the training steps 200 are implemented again. For this purpose, training 200 can also include a step modifying the parameters of the first and second neural networks.

[0090] Conversely, when the distance is less than or equal to the predefined threshold, the training of the first and second neural networks is considered complete and the training steps are not implemented again.

[0091] Training 200 can be implemented during the implementation of process 100, before the implementation of step 120 for defect detection, or even before the implementation of step 110 for obtaining images of the part to be inspected. Alternatively, training 200 can be implemented prior to process 100, and therefore independently of it.

[0092] Training 200 may also include an initiation step of the parameters of the first and second neural networks before step 210 of determining the maps.

[0093] One advantage of using this approach to optimize the first neural network is that the reflectance of the part can be determined without relying on strong assumptions about the assumed shape of the part's reflectance function, unlike established approaches. This approach is therefore particularly well-suited for reflective parts with complex geometries, whereas established methods are not, and are better suited to Lambertian surfaces.

[0094] In one embodiment, the training database can be augmented by generating new reference reflectance and normal direction maps. These new maps are generated based on maps or portions of maps in which there are no defects, and by replacing certain values ​​in these maps with reference reflectance values ​​and / or normal direction reference vectors corresponding to defects in the reference parts in the reference images. This replacement of reference reflectance values ​​and / or normal direction reference vectors in certain maps with other reference reflectance values ​​and / or normal direction reference vectors is also called "overlaying."

[0095] In other words, these new maps are generated by embedding portions of maps corresponding to actual defects in reference parts into maps where there are no defects.

[0096] More specifically, it is possible to augment the training database by generating multiple pairs of new reference reflectance maps and new reference normal direction maps. Each pair of new maps comprises a new reference reflectance map and a new reference normal direction map.

[0097] The new reference reflectance and normal direction maps for each pair of new maps are then generated by selecting reference reflectance values ​​and normal direction vectors that correspond to a predetermined defect and are taken from a reference reflectance map and a reference normal direction map, respectively. These maps, from which these selected reference values ​​and vectors are derived, correspond to a reference image of the first plurality of reference images containing the predetermined defect. That is, an image of the predetermined defect is included within these reference images, for example, in a portion of each or a part of these reference images.

[0098] The training database is augmented by replacing, with selected reference values ​​and reference vectors, reflectance reference values ​​and normal direction reference vectors located within a portion not corresponding to the predetermined defect in the respective reference reflectance and normal direction maps from which the reference values ​​and reference vectors for replacement are selected. In other words, the overlay is performed in the same maps from which the selected reference values ​​and reference vectors originate.

[0099] Alternatively, the training database is augmented by replacing, with selected reference values ​​and reference vectors, the reflectance reference values ​​and normal direction reference vectors contained in a reference reflectance map and a reference normal direction map, respectively, corresponding to a reference image from the first plurality of reference images free of defects. That is, the overlay is performed in maps other than those from which the selected reference values ​​and reference vectors are derived.

[0100] In this regard, in connection with figure 3, another aspect of the invention relates to a mechanism for increasing the database by inset.

[0101] The augmentation 300 includes a step 310 of selecting, for each pair of new maps, the reference reflectance values ​​and reference normal direction vectors corresponding to the predefined defect in a reference reflectance map and in a reference normal direction map, respectively. These two maps are determined from the same reference image of the reference part containing the predefined defect.

[0102] In particular, since for each reference part containing a defect, this defect is predetermined, its position within the part and on the reference images is known. Consequently, the position of this predetermined defect is also known in the maps corresponding to each of the reference images of that reference part. It is therefore possible to precisely identify the pixels that correspond to the predetermined defect in the reference reflectance maps and associated reference normal direction maps, and to select the reference reflectance values ​​and reference normal direction vectors that correspond to this predetermined defect in these maps.

[0103] The 300 increase also includes a step 320 of generating pairs of new reference maps by replacing, with the selected reference values ​​and reference vectors, the reflectance reference values ​​and normal direction reference vectors included in the same reference maps in a portion without defects or in other reference maps that do not include defects, as detailed above.

[0104] The overlay of the selected reference values ​​and reference vectors is performed while preserving the distribution pattern of these values ​​in their respective maps. That is, the distribution pattern of these values, which is induced by the shape of the predetermined defect, is reproduced in the overlay of these selected reference values ​​and reference vectors. In this way, the data augmentation is performed more realistically and representative of the physics of defects encountered in actual production, than the methods classically used.

[0105] In one embodiment, a mathematical transformation can be applied to this distribution pattern during overlay. In other words, replacement with selected reference values ​​and reference vectors can be achieved by applying a geometric transformation to their distribution. This increases the diversity of the new maps while maintaining the realistic nature of the predetermined defect. Furthermore, several different mathematical transformations can be applied to the same selected reference values ​​and reference vectors to create different pairs of new maps.

[0106] The mathematical transformation can be any type of transformation, for example, enlargement, rotation, Gaussian noise, interpolation, etc. Preferably, such a mathematical transformation is a non-rigid spatial deformation applying a displacement field to the selected reference values ​​and reference vectors to be embedded. As an example, the mathematical transformation can be defined based on a statistical estimate, obtained empirically and / or through simulations, of the variations in the predetermined defects of the reference parts.

[0107] Augmentation 300 can be implemented during the execution of process 100 after training 200 and before the execution of step 120 (defect detection), or even before the execution of step 110 (obtaining images of the part to be inspected). Alternatively, augmentation 300 can be implemented prior to process 100, and therefore independently of it, typically after the execution of training 200 prior to process 100.

[0108] In one embodiment variant, it is possible to apply smoothing to the selected reference values ​​and reference vectors after their inset in order to homogenize the selected reference values ​​and reference vectors with the reference values ​​and reference vectors already present in the map where the selected reference values ​​and reference vectors are inset.

[0109] In an example implementation of process 100, the detection network is configured to detect the fault from reflectance maps and light directions. In this case, the detection network detects the defect in a reflectance map and a normal direction map, which are determined from images of the part being inspected. Similar to reference reflectance and normal direction maps, the reflectance map includes reflectance values, and the normal direction map includes directions normal to the part.

[0110] Process 100 thus includes a step 115 of determining the reflectance mapping and the normal direction mapping.

[0111] The reflectance mapping and normal direction mapping corresponding to the images of the part to be inspected can be determined using any known technique. For example, in one embodiment, these two maps are determined by the first neural network, applying the learning described above (i.e., the joint learning of the first and second neural networks), but applying this learning to the images of the part to be inspected instead of the reference images.

[0112] In this embodiment, the training database comprises the reference reflectance maps and the reference normal direction maps determined by the first neural network. Since the training database is annotated, the reference maps it contains are labeled to indicate whether these maps contain a predetermined defect, i.e., whether these maps originate from an image of a reference part containing the predetermined defect and whether the defect is present in said reference image.

[0113] In other words, in the training database, each reference reflectance map is labeled according to whether the reference image from which it is derived includes the predetermined defect of the associated reference part, or not. Furthermore, each reference normal direction map is labeled according to whether the reference image from which it is derived includes the predetermined defect of the associated reference part, or not.

[0114] The label assigned to each reference map is, for example, a binary indication of whether it contains a defect or not. The detection network then indicates whether or not there is a defect in the images of the part being inspected.

[0115] Alternatively, the label assigned to each reference map is, for example, a segmentation of the defect within that map. The segmentation of a predetermined defect in each reference reflectance map and in each reference normal direction map is determined by any known method. The detection network then segments the defect's position in the images of the part to be inspected, when such a defect exists. Defect detection is then performed by segmenting it.

[0116] The first neural network is trained on all images of the plurality of images of the part to be controlled.

[0117] In another example of the implementation of process 100, related to Figure 1, the detection network is configured to detect the defect directly from the images of the part to be inspected. The training database then includes a second plurality of reference images, which is constructed from the reference reflectance maps and the reference normal direction maps.

[0118] In particular, each image in the second plurality of reference images is constructed from the reference reflectance map and the reference normal direction map from one of the reference images in the first plurality of reference images.

[0119] Each reference image in the second plurality of reference images is labeled with the corresponding label from the reference image in the first plurality of reference images from which the maps used to construct it originate.

[0120] In one variant of this embodiment, the database is augmented via the augmentation 200 described above or by another augmentation method, such as one of those described below.

[0121] In one embodiment, the training database used in the second implementation example of process 100 (i.e., when the training database comprises the second plurality of images) can be augmented by generating a third plurality of reference images. In this embodiment, the reference images of the third plurality of images are constructed from the reference maps derived from the first plurality of images by varying the illumination direction.

[0122] Specifically, each reference image in the third plurality of reference images is generated from the reference reflectance maps and reference normal directions of one of the reference images in the first plurality of reference images, which includes a predetermined defect. The construction of each reference image in the third plurality of reference images is further based on a modified illumination direction reference vector. This modified illumination direction reference vector is obtained by modifying at least one of its reference values, with said modified reference value selected from a predefined range of values ​​associated with the dimension corresponding to the reference value to be modified. This predefined range of values ​​is defined to include the reference value of the illumination direction reference vector to be modified.For example, this predefined value range extends from 80%, or even 85% or 90%, of the reference value to be modified, up to 110%, or even 115%, or even 120%, of the reference value to be modified.

[0123] In other words, each reference image in the third plurality of reference images is constructed using the reference reflectance map and the reference normal direction map of a reference image from the first plurality of reference images. This reference image includes a predetermined defect in the relevant reference part, but applies a different illumination direction than that calculated by the second neural network for the reference image in the first plurality of reference images. This different illumination direction is chosen, using any known method, from a predefined range of values. This predefined range of values ​​is defined using any known technique.Preferably, this predefined range of values ​​is defined so as to augment the training database in a realistic way, that is, by preserving the realism of the reference images of the first plurality of reference images.

[0124] Thus, in this embodiment, the training database includes the second plurality of reference images and the third plurality of reference images.

[0125] Each reference image in the third plurality of reference images is labeled with the label indicating that this image includes a defect. Tl

[0126] In connection with figure 5, one aspect of the invention relates to a mechanism for the generation 400 of the third plurality of reference images.

[0127] This generation 400 of the third plurality of reference images includes a step 410 of selecting the modified reference value of the modified illumination direction reference vector for each reference image in the third plurality of reference images. The predefined range of values ​​from which the modified reference value of the modified illumination direction reference vector is selected may be the same or different for all or some of the reference images in the third plurality of reference images to be formed. The modified reference value of the modified illumination direction reference vector is, for example, chosen randomly from the predefined range of values.

[0128] The generation 400 of the third plurality of reference images also includes a step 420 of generating the reference images of the third plurality of reference images, each of these reference images being generated from the reference reflectance mapping and the reference normal direction mapping from the same reference image of the first plurality of reference images, and the modified reference value of the modified illumination direction reference vector corresponding to the reference image in question of the third plurality of reference images.

[0129] The generation of the third plurality of reference images (400) can be implemented during the execution of process 100 after training 200, or even after augmentation 300, and before the execution of defect detection step 120, or even before the execution of step 110 for obtaining images of the part to be inspected. Alternatively, this generation of 400 can be implemented prior to process 100, and therefore independently of it, typically after the execution of training 200, or even after the execution of augmentation 300, prior to process 100.

[0130] In one embodiment, the training database used in the second implementation example of process 100 (i.e., when the training database comprises the second plurality of images) can be augmented by generating a fourth plurality of reference images. In this embodiment, the reference images of the fourth plurality of images are constructed from the new reference maps resulting from the 300 augmentation, and by varying the direction of illumination.

[0131] Specifically, each reference image in the fourth plurality of reference images is generated from the new reference reflectance maps and reference normal directions of a pair of new maps from the plurality of pairs of new maps. The construction of each reference image in the fourth plurality of reference images is further based on a modified illumination direction reference vector. This modified illumination direction reference vector is obtained by modifying at least one of the reference values ​​it comprises, said modified reference value being selected from a predefined range of values ​​pertaining to the dimension corresponding to said reference value to be modified.The said predefined value range is defined as including the reference value to be modified of the illumination direction reference vector, determined in the reference image of the first plurality of reference images from which the new reference reflectance maps and reference normal directions of said pair are determined (i.e., the reference image not containing a predetermined defect and in the reference maps of which the reference values ​​and vectors selected during the 300 augmentation are embedded). For example, this predefined value range extends from 80%, or even 85%, or even 90%, of the reference value in question, up to 110%, or even 115%, or even 120%, of said reference value.

[0132] In other words, each reference image of the fourth plurality of reference images is constructed using the new reference reflectance map and the new reference normal direction map from a pair of new maps from the plurality of pairs of new maps (i.e., maps into which selected reference values ​​and reference vectors corresponding to a predetermined defect have been embedded), but applying a different illumination direction than that calculated by the second neural network for said reference image of the first plurality of reference images from which these new maps are formed (in this case, the reference image of the first plurality of reference images into whose maps the selected reference values ​​and reference vectors are embedded). This direction A different illumination is chosen, using any known method, from a predefined range of values. This predefined range of values ​​is defined using any known technique. Preferably, this predefined range of values ​​is defined so as to realistically augment the training database, that is, while preserving the realism of the reference images in the first plurality of reference images.

[0133] Thus, in this embodiment, the training database includes the second plurality of reference images and the fourth plurality of reference images. By enriching the diversity of defect examples, this mechanism significantly improves the performance, accuracy, and efficiency of defect detection by the detection network, making it more robust in the face of different visualization conditions and variations in defects.

[0134] Each reference image in the fourth plurality of reference images is labeled with the label indicating that this image includes a defect.

[0135] In connection with figure 6, one aspect of the invention relates to a mechanism for the generation 500 of the fourth plurality of reference images.

[0136] This generation 500 of the third plurality of reference images includes a step 510 of selecting the modified reference value of the modified illumination direction reference vector for each reference image in the fourth plurality of reference images. The predefined range of values ​​from which the modified reference value of the modified illumination direction reference vector is selected may be the same or different for all or some of the reference images in the fourth plurality of reference images to be formed. The modified reference value of the modified illumination direction reference vector is, for example, chosen randomly from the predefined range of values.

[0137] The generation 500 of the fourth plurality of reference images also includes a step 520 of generating the reference images of the fourth plurality of reference images, each of these reference images being generated from the new reference reflectance mapping and the new reference normal direction mapping from the same pair of new mappings, as well as the modified reference value of the modified illumination direction reference vector corresponding to the reference image in question of the fourth plurality of reference images.

[0138] The 500 generation of the fourth plurality of reference images can be implemented during the execution of process 100 after training 200, or even after augmentation 300, and before the execution of defect detection step 120, or even before the execution of step 110 for obtaining images of the part to be inspected. Alternatively, this 500 generation can be implemented prior to process 100, and therefore independently of it, typically after the execution of training 200, or even after the execution of augmentation 300, prior to process 100.

[0139] In one embodiment, the training database comprises the second plurality of reference images, the third plurality of reference images, and the fourth plurality of reference images. That is, generation 400 of the third plurality of reference images and generation 500 of the fourth plurality of reference images are both implemented, either during the implementation of process 100 or prior to process 100, as detailed above.

[0140] It is noted that the order in which the 400th generation of the third plurality of reference images and the 500th generation of the fourth plurality of reference images are performed is indifferent.

[0141] Similarly, it is noted that the fourth plurality of images can alternatively be generated by applying the 300 augmentation by overlay to the third plurality of data. That is to say, the overlay is, in this case, performed after the generation of the third plurality of images by modifying the illumination direction reference vector.

[0142] In one embodiment, each reference image comprises a plurality of pixels and the first neural network accepts as input a predefined position of each pixel of the plurality of pixels relative to an artificial light source illuminating the reference room corresponding to the reference image during the acquisition of said reference image.

[0143] In other words, in this embodiment, the first neural network is trained on the coordinates of the position of each pixel of the reference image relative to the artificial light source in the camera frame.

[0144] In one embodiment, the second neural network accepts a reference image as input. The second neural network is thus directly trained on reference images.

[0145] In one embodiment, the training of the first and second neural networks is performed jointly for each of the reference parts, independently of the other reference parts. In this embodiment, for each reference part, the first and second neural networks are jointly optimized from each of the reference images in the first plurality of reference images corresponding to said reference part. In other words, for each of the reference parts in the plurality of reference parts, the first and second neural networks are jointly optimized from each of the reference images of said reference part among the first plurality of reference images.

[0146] In one embodiment, related to Figures 1 and 4, the process 100 also includes a step 130 of issuing an alert regarding the detected defect. The alert indicates, for example, that a defect has been detected and may optionally include additional information such as an identifier corresponding to the inspected part and / or the location of the defect on the part.

[0147] The alert can be established and issued via any known method and can be of any type and format.

[0148] In one embodiment, each intermediate image is constructed by applying a rendering function, for example a differentiable rendering function. This rendering function takes as arguments the reference reflectance map and the reference normal direction map, as well as the reference illumination direction vector, determined for one of the reference images of the first plurality of reference images.

[0149] The rendering function is, for example, such that i = e. p (n, l). max(n T . l, 0), where T] is the amplitude of one of the pixels in the intermediate image to be constructed, e is the light intensity emitted by the artificial lighting source, l is the vector of reference direction of illumination, n is the reference vector of the normal direction to the surface of the part from the pixel of the reference normal direction map corresponding to the pixel of the intermediate image considered and p(.) is the reference reflectance value from the pixel of the reference reflectance map corresponding to the pixel of the intermediate image considered.

[0150] The reference images of the second plurality of reference images, the third plurality of reference images and / or the fourth plurality of reference images can also be determined by applying said rendering function, from the related maps.

[0151] In one embodiment, the reflectance determined by the first neural network is a bidirectional reflectance, from the English "Bidirectional Reflectance Distribution Function" or "BRDF". The first neural network therefore has an architecture adapted to allow the determination of this bidirectional reflectance.

[0152] In one example implementation, the first neural network comprises shared and specialized fully connected layers. For instance, the first neural network includes four shared fully connected layers with 256 neurons, and then a branching pattern is used to specialize, on the one hand, in determining the reference reflectance map and, on the other hand, in mapping reference normal directions. Typically, the first neural network includes four more specialized fully connected layers with 256 neurons to focus on estimating the reflectance map and four more specialized fully connected layers with 256 neurons to focus on estimating the normal direction map.

[0153] The input to the first neural network receives the coordinates of each pixel in the image, encoded using a positional encoding technique. This technique transforms each 2D input coordinate into a higher-dimensional space (typically by applying a ten-level Fourier positional encoding model). The network output is twofold: an estimate of the reference reflectance map and an estimate of the reference normal direction map, with respective sizes Hxwx = 1 and Hxwx = 3, where the afferent reference image has a size of Hxwx = 1, H being the number of pixels of the reference image according to a first dimension and W the number of pixels of the reference image according to a second dimension.

[0154] In one example implementation, the second neural network has seven convolutional layers, each followed by a ReLU-type activation function, then three fully connected layers with 64 neurons.

[0155] In one embodiment, the second neural network is further adapted to produce, for each reference image in the first plurality of reference images, the light intensity of the artificial light source used to acquire said reference image. This is particularly advantageous in cases where the light intensity is not calibrated or differs from one acquisition and / or light source to another. In other words, when the light intensity is unknown, it is determined by the second neural network.

[0156] In one embodiment, the second neural network is adapted to produce, for each reference image in the first plurality of reference images, the light intensity of the artificial lighting source used to acquire said reference image, without producing the direction of illumination. Indeed, it is quite possible that the direction of illumination is known at the time of inspection, that is, it is known before the implementation of method 100, unlike the light intensity, which is not known and must be determined using the second neural network.

[0157] In one embodiment, the third and / or fourth plurality of reference images can also be generated by artificially varying the light intensity. Modifying the light intensity can be implemented similarly to modifying one of the reference values ​​of the illumination direction reference vector, i.e., by selecting a modified light intensity value from a predefined range. For example, this predefined range extends from 80%, 85%, or even 90% of the reference light intensity value up to 110%, 115%, or even 120% of the reference light intensity value. The reference value luminous intensity is the luminous intensity value emitted by the artificial lighting source used during the acquisition of the corresponding reference image of the first plurality of reference images.

[0158] In one embodiment, the BRDF function consists of two elements. The first is a Lambertian component p d , which depends neither on the direction of the lighting nor on the local normal to the surface of the room. The second is a specular component p s (n, Z), which depends on the direction normal to the surface of the part and the direction of illumination. In other words, the BRDF function is such that, for a pixel of a reference image, the reference reflectance value is given by is the The reference vector for the illumination direction is n, the reference vector for the direction normal to the part surface from the pixel of the considered reference image is r, the material roughness of the part is v, and v is the viewing direction (i.e., the direction from which the camera captured the image). The viewing direction is typically the same when assuming that the images are taken in orthographic views.

[0159] Thus, the first neural network can either directly produce the reflectance reference value p(n, Z), or separately produce the Lambertian component p d and the specular component p s (n, Z), which are then summed to obtain the reflectance reference value.

[0160] In one embodiment, it is possible to perform an overlay of simulated reflectance reference values ​​and / or simulated normal direction and / or illumination direction reference vectors, typically via a numerical simulation tool configured to generate such simulated reference values ​​and / or vectors.

[0161] In one embodiment, defect detection is performed only on pixels in the images of the part to be inspected whose grayscale value, on a scale from 0 to 255 (for an 8-bit encoded image), is greater than or equal to 5, or even greater than or equal to 10, and is less than or equal to 245, or even less than or equal to 240. This allows defect detection to be performed only on pixels in the images of the part to be performed on the images. that in the area of ​​each of the images of the room to be monitored where the camera has a linear response in light intensity, and therefore to limit the noise generated by the non-linearity of the camera.

[0162] In one embodiment, the distance between the intermediate images and the reference images of the first plurality of reference images is determined only for the pixels of these images whose intensity exceeds a predefined intensity threshold. This threshold can be defined by any method, for example by the operator based on their knowledge and experience in the field.

[0163] In one embodiment, related to Figure 8, the method according to the invention uses a 600 operation of augmenting the training database for training the detection network. Such an operation makes it possible to improve the learning of the training network on new data by having a reduced number of experimental data, i.e., the reference images of the first plurality of reference images.

[0164] This operation 600 includes a step 610 of obtaining the first plurality of reference images. These reference images are acquired via one or more cameras and one or more artificial lighting sources, for example, via the same camera and the same artificial lighting source(s) as those used to acquire the images of the part to be inspected. Preferably, for the same reference part, the reference images of the first plurality of reference images are acquired via the same camera.

[0165] This operation 600 also includes one or more of the augmentation mechanisms detailed previously. In particular, this operation may include augmentation 300 by embedding selected reference values ​​and reference vectors, generation 400 of the third plurality of reference images and / or generation 500 of the fourth plurality of reference images.

[0166] As an example, when the detection network detects the defect in the reflectance and normal direction maps from the images of the part to be inspected, operation 600 includes only the increase 300.

[0167] Alternatively, when the detection network detects the defect in the images of the part to be inspected, operation 600 includes generation 400 of the third plurality of reference images and / or generation 500 of the fourth plurality of reference images, the latter requiring the preliminary implementation of augmentation 300.

[0168] Furthermore, it is noted that operation 600 of augmenting the training database can be implemented as such, independently of process 100. This operation 600 is then typically implemented by computer as process 600 of augmenting the training database.

[0169] Another aspect of the invention relates to a device for detecting a defect on a metal part. This device is configured to implement the defect detection method 100 on the metal part, for example because it includes a processor adapted to implement the method 100.

[0170] In relation to figure 7, this device 10 can include a processor 11 and a memory 12. Instructions can be stored in this memory 12 which, when executed by the processor 11, lead the latter to implement the process 100.

[0171] The device 10 may also include an input interface 13 and / or an output interface 14. The input interface 13 can typically be used to obtain images of the part to be inspected, for example, when these images are produced by a third-party system, such as the acquisition system. The input interface 13 is therefore suitable for receiving the images of the part to be inspected. The output interface 14 can typically be used for issuing the alert. The output interface 14 is therefore suitable for issuing said alert.

[0172] In one embodiment, the device 10 is further or alternatively configured to use the training database augmentation operation 600 for training the detection network. For example, to this end, the device 10 may include instructions in its memory 12 which, when executed by the processor 11, cause the processor 11 to implement the augmentation operation 600. The input interface 13 can thus be used to obtain the first plurality of reference images. The input interface 13 is then adapted to receive the first plurality of reference images.

[0173] Device 10 can be included in the acquisition system.

[0174] Another aspect of the invention relates to a computer program product comprising instructions which, when the program is executed on a computer, cause the computer to implement the steps of operation 600 of increasing the training database for training the detection network.

[0175] Another aspect of the invention relates to a computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of operation 600 of augmenting the training database for training the detection network.

Claims

DEMANDS

1. A computer-implemented method (100) for detecting a defect on a metal part, the method (100) comprising: - Obtain (110) at least three images of the room, the images of the room being acquired via a camera and an artificial light source; - Detect (120) the defect in the part by analyzing the images of the part via a neural network, called a detection network, the detection network being a supervised training artificial neural network configured to detect the defect from the images of the part, the detection network being trained (200) on a training database constructed by means of a first neural network and a second neural network, the first neural network being an artificial neural network configured to determine (210), for each reference image of a first plurality of reference images, a reference reflectance map comprising reference reflectance values ​​and a reference normal direction map comprising reference vectors of normal directions, and the second neural network being an artificial neural network configured to determine (220),for each reference image of a first plurality of reference images, a reference vector of illumination direction and / or a luminous intensity of the artificial lighting source, each reference image of the first plurality of reference images being an image of a reference part among a plurality of reference parts, the first and second neural networks being self-supervised training and jointly optimized via a minimization of a distance between the first plurality of reference images and a plurality of intermediate images, the plurality of intermediate images being constructed (230) from the reference reflectance maps, the reference normal direction maps and the reference vectors of illumination directions.

2. A method (100) according to claim 1, wherein the training database is augmented (300) by generating (320) a plurality of pairs of new reference reflectance maps and new reference normal direction maps, the new reference reflectance and reference normal direction maps of each pair of new maps being generated by selecting (310) reference reflectance values ​​and reference normal direction vectors corresponding to a predetermined defect and originating respectively from a reference reflectance map and a reference normal direction map corresponding to a reference image of the first plurality of reference images comprising the predetermined defect, the augmentation being implemented by replacement, with the selected reference values ​​and reference vectors,of reflectance reference values ​​and normal direction reference vectors included in:, - A portion not corresponding to a predetermined defect in the respective reference reflectance and reference normal direction maps from which the reference values ​​and reference vectors for replacement are selected; or - A reference reflectance map and a reference normal direction map, respectively, corresponding to a reference image of the first plurality of reference images not containing any defects. [Claims] Method (100) according to claim 2, wherein the replacement by the selected reflectance reference values ​​and normal direction reference vectors is implemented by applying a geometric transformation of their distribution.

4. A method (100) according to any one of the preceding claims, wherein the detection network is configured to detect (120) the defect from a reflectance map comprising reflectance values ​​and a normal direction map comprising directions normal to the part, the reflectance and normal direction maps being determined (115) in the part images, the training database comprising the reference reflectance maps and reference normal direction maps determined by the first neural network. [Claims] Method (100) according to claim 4, wherein the reflectance mapping and normal directions to the part are determined in the part images via the first neural network.

6. A method (100) according to any one of claims 2 to 3, wherein the detection network is configured to detect (120) the defect directly from the part images, the training database comprising a second plurality of reference images, the second plurality of reference images being constructed from the reference reflectance maps and the reference normal direction maps.

7. A method (100) according to claim 6, wherein the training database is augmented by generating (400) a third plurality of reference images, each reference image of the third plurality of reference images being generated (420) from the reference reflectance maps and reference normal directions of one of the reference images of the first plurality of reference images comprising a predetermined defect and a modified illumination direction reference vector, the illumination direction reference vector comprising at least two reference values, the modified illumination direction reference vector being obtained by modifying at least one of the reference values ​​of said illumination direction reference vector,said at least one modified reference value being selected (410) from a predefined range of values ​​relating to a dimension corresponding to said reference value to be modified and including the reference value to be modified.

8. A method (100) according to any one of claims 2 to 3 and according to any one of claims 6 to 7, wherein the training database is augmented (500) by generating (520) a fourth plurality of reference images, each reference image of the fourth plurality of reference images being generated from the new reference reflectance maps and reference normal directions of a pair of new maps of the plurality of pairs of new maps, and of a modified illumination direction reference vector, the illumination direction reference vector comprising at least two reference values, the modified illumination direction reference vector being obtained by modifying at least one of the reference values ​​of said illumination direction reference vector, said at least one modified reference value being selected (510) from a predefined range of values ​​related to a dimension corresponding to said reference value to be modified and including the reference value to be modified of the illumination direction reference vector.

9. A method (100) according to any one of the preceding claims, wherein each reference image comprises a plurality of pixels, the first neural network accepting as input a predefined position of each pixel of the plurality of pixels relative to an artificial light source illuminating the reference piece corresponding to the reference image during the acquisition of said reference image, and wherein the second neural network accepts as input said reference image.

10. A method (100) according to any one of the preceding claims, wherein, for each of the reference parts in the plurality of reference parts, the first and second neural networks are jointly optimized from each of the reference images of said reference part among the first plurality of reference images.

11. Device (10) for detecting a defect on a metal part with complex geometry, the device (10) comprising a processor adapted to implement the method (100) according to any one of claims 1 to 10.

12. Computer program product comprising instructions which, when the program is executed on a computer, cause the computer to carry out the steps of the process (100) according to any one of claims 1 to 10.

13. A computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the process (100) according to any one of claims 1 to 10.

Images