THREE-DIMENSIONAL CONTROL PROCEDURE

Abstract

A method for three-dimensional inspection of a part from several radiographic images by an image analysis device comprising electronic circuitry adapted to implement the three-dimensional inspection method, the method comprises at least the following steps: - Step 1, Acquiring a plurality of radiographic images of a part by a source-detector system, each image comprising information on the radiographic transmission properties of the part; - Step 3, Aligning each image with a view of a digital model of the part; - Step 4, Detecting indications on each image and positioning the detected indications with bounding boxes; - Step 5, Building a model of the multi-view geometry using the alignment of each image with a respective view of the digital model of the part;- Step 6, Consolidate the detected indications with bounding boxes, using the multi-view geometry model by constructing, step 6.1, a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting, step 6.2, a candidate group to retain;- Step 7, Locate each three-dimensional indication;- Step 8, Characterize the dimensions of each indication in a relevant image selected from the set of images from the bounding boxes used to position the detected indications;- Step 9, Use the characterization of each indication for further processing by a third-party device.;

Description

Title of the invention: THREE-DIMENSIONAL CONTROL METHOD Technical field

[0001] The invention relates to the field of non-destructive testing of parts. STATE OF PRIOR ART

[0002] In the field of non-destructive testing, particularly of metal parts, it is well known to use computer vision methods and conventional image processing methods or methods based on statistical learning methods.

[0003] In particular, a method for non-destructive testing of metal parts is known which uses a module for classification and precise localization of indications. It is specified that in this document, by indication, is meant a characteristic of an image which requires a sanction. An indication can be sanctioned as being an anomaly or as being benign. The module is specific to each area of ​​the part which has distinctive characteristics. This method is particularly suitable for the control of a heterogeneous part. This method uses the learning of a neural network using two-dimensional radiographic images allowing the localization of image indications.

[0004] Thus, it is known to automatically recognize indications on a two-dimensional image. However, the transition from the characterization of an indication on a two-dimensional image to the characterization of this same indication in a three-dimensional aeronautical part is traditionally carried out manually by operators expert in the analysis of X-ray images. Traditionally, this characterization follows a procedure described in a work instruction established for the sanction of the part. Conventionally, multiple images are made for each part. Each image describes the X-ray transmission properties of the material of the part from a different point of view. In this document, by "view" is meant the point of view of the inspected part from which its radiographic image is acquired.Once all the multi-view images have been acquired, the operators analyze all the images to find all the indications. Then, the operators determine which indications in the images correspond to the projections of the same indication in the part (also called three-dimensional indication). Then, the operator determines the location of the three-dimensional indications in the part and they are also characterized in their most relevant projection.

[0005] This method relies mainly on the expertise and talent of the operators, and is very time-consuming.

[0006] In this context, it is necessary to provide a method for three-dimensional inspection of a part which makes it possible to automatically characterize a three-dimensional indication from two-dimensional multi-view images, to assist in the reliable and rapid detection of indications in aeronautical parts. Statement of the invention

[0007] To this end, according to a first aspect, a method for three-dimensional control of a part from several radiographic images by an image analysis device comprising electronic circuitry adapted to implement the three-dimensional control method is proposed. The method comprises at least the following steps: - (Step 1) Acquiring a plurality of radiographic images of a part by a source-detector system, each image including information on the radiographic transmission properties of the part; - (Step 3) Align each image with a view of a digital model of the part; - (Step 4) Detect indications on each image and position the detected indications with bounding boxes; - (Step 5) Build a model of the multi-view geometry using the alignment of each image to a respective view of the digital model of the part; - (Step 6) Consolidate the detected indications with bounding boxes, using the multi-view geometry model by constructing (step 6.1) a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting (step 6.2) a candidate group to retain; - (Step 7) Locate each three-dimensional indication; - (Step 8) Characterize the dimensions of each indication in an image relevant selected from the set of images from the bounding boxes used to position the detected indications; - (Step 9) Use the characterization of each indication for further processing by a third-party device.

[0008] According to a particular arrangement, alignment step 3 comprises estimating (step 3.1) an optimal rigid body transformation of each view of the digital model of the part with respect to the source-detector system to obtain an image of the projection of each view of the digital model.

[0009] According to a particular arrangement, a projection operator is used to carry out step 3.1, the operator is a matrix of the form . [ R [t ] ' with K a matrix described by intrinsic projection parameters, and with R and t respectively the rotation and the translation of each view of the digital model.

[0010] According to a particular arrangement, step 3 comprises defining (step 3.2) a similarity metric between the calculated image of each view of the digital model and each acquired image.

[0011] According to a particular arrangement, step 6.1 of constructing a set of candidate groups comprises analyzing a location of the indications.

[0012] According to a particular arrangement, step 6.2 of selecting a candidate group comprises selecting the candidate groups containing the projections of the same three-dimensional indication.

[0013] According to a particular arrangement, the localization (step 7) comprises determining an average of point clouds per candidate group containing several indications.

[0014] According to a particular arrangement, the characterization step comprises selecting (step 8.1) a relevant image for estimating a length and a width of an indication, the relevant image being selected as a function of a three-dimensional location of the indication.

[0015] According to another aspect, there is provided an image analysis device which comprises electronic circuitry for implementing a method for three-dimensional inspection of a part from several radiographic images, the method comprising at least the following steps: (Step 1) Acquiring a plurality of radiographic images of a part by a source-detector system, each image comprising information on the radiographic transmission properties of the part; - (Step 3) Align each image with a view of a digital model of the part; - (Step 4) Detect indications on each image and position the detected indications with bounding boxes; - (Step 5) Build a model of the multi-view geometry using the alignment of each image to a respective view of the digital model of the part; - (Step 6) Consolidate the detected indications with bounding boxes, using the multi-view geometry model by constructing (step 6.1) a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting (step 6.2) a candidate group to retain; - (Step 7) Locate each three-dimensional indication; - (Step 8) Characterize the dimensions of each indication in a relevant image selected from the set of images from the bounding boxes used to position the detected indications; - (Step 9) Use the characterization of each indication for further processing by a third-party device.

[0016] According to another aspect, there is provided a computer program product comprising program code instructions for executing the method according to the invention.

[0017] According to another aspect, there is provided a non-transitory storage medium on which is stored a computer program comprising program code instructions for executing the method according to the invention, when said instructions are read from said non-transitory storage medium and executed by a processor.

[0018] BRIEF DESCRIPTION OF THE DRAWINGS.

[0019] The characteristics of the invention mentioned above, as well as others, will appear more clearly on reading the following description of at least one exemplary embodiment, said description being made in relation to the attached drawings, among which:

[0020] [Fig.l] schematically illustrates the progress of a control process;

[0021] [Fig.2] schematically illustrates the construction of a multi-view geometry;

[0022] [Fig.3] schematically illustrates an epipolar constraint for the projection of a point in a three-dimensional room;

[0023] [Fig.4] schematically illustrates a cutting of a digital model of a part;

[0024] [Fig.5] illustrates a candidate group of four indications;

[0025] [Fig.6] illustrates a first phase of creation of candidate groups;

[0026] [Fig.7] illustrates a second phase of creation of candidate groups;

[0027] [Fig.8] illustrates the construction of a three-dimensional score;

[0028] [Fig.9] illustrates the use of a three-dimensional score for a candidate group;

[0029] [Fig. 10] illustrates a proposed solution for a dimensional characterization;

[0030] [Fig. 11] illustrates the application of a Huber regression model to estimate a length;

[0031] [Fig.12] illustrates the application of a second-degree polynomial regression model for width estimation;

[0032] [Fig. 13] schematically illustrates a computer system suitable for implementing the method.

[0033] DETAILED DESCRIPTION OF EMBODIMENTS

[0034] Three-dimensional control method.

[0035] With reference to [Fig.l], according to a first aspect, a method 100 is proposed for three-dimensional control of a part from several radiographic images. by an image analysis device comprising electronic circuitry 200 adapted to implement the three-dimensional control method 100. The method 100 comprises at least the following steps: - (Step 1) Acquiring a plurality of radiographic images of a part by a source-detector system, each image including information on the radiographic transmission properties of the part; - (Step 3) Align each image with a view of a digital model of the part; - (Step 4) Detect indications on each image and position the detected indications with bounding boxes; - (Step 5) Build a model of the multi-view geometry using the alignment of each image to a respective view of the digital model of the part; - (Step 6) Consolidate the detected indications with bounding boxes, using the multi-view geometry model by constructing (step 6.1) a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting (step 6.2) a candidate group to retain. - (Step 7) Locate each three-dimensional indication; - (Step 8) Characterize the dimensions of each indication in an image relevant selected from the set of images from the bounding boxes used to position the detected indications; - (Step 9) Use the characterization of each indication for further processing by a third-party device.

[0036] Step 1 - Image Acquisition

[0037] As indicated previously, the control method 100 comprises a first step (step 1) consisting of acquiring a plurality of radiographic images of a part by a source-detector system, each image comprising information on the radiographic transmission properties of the part.

[0038] According to one embodiment, the images are acquired using an X-ray radiography method. Thus, according to the example presented here, the X-ray acquisition is carried out in a tomographic cabin. The tomographic cabin consists of an X-ray generator which emits a beam passing through a part to be explored before being analyzed, after attenuation, by an X-ray detection system. An acquired image, or an image obtained after calibration of any inhomogeneities of the detector and / or extraction of any background information, is called a projection. It is specified that a calibration can be carried out for a wide variety of phenomena. A set of images of the same view is acquired without moving the part and the images are averaged in order to reduce acquisition noise.

[0039] According to a particular arrangement, several X-ray images are obtained for different views of the part, in order to have an optimized view of the part to be checked. In other words, several images are produced according to different orientations around the part. The views (i.e. the different orientations) are selected to have an optimized signal-to-noise ratio allowing the detection of indications with the lowest radio contrasts.

[0040] Step 2 - Pretreatments

[0041] According to a particular arrangement, pre-processing (step 2) may possibly be applied to the images in order to facilitate subsequent analysis:

[0042] - Filtering: use of filters to enhance characteristic patterns of image indications and reduce noise.

[0043] - Masking a region of interest on each image. According to an arrangement particular, the masking is obtained thanks to an automated detection of the air and the collimator,

[0044] - Data normalization: in order to facilitate learning and obtain a faster convergence, the data is normalized to have a mean of 0 and a standard deviation of 1.

[0045] Step 3 - Alignment

[0046] As indicated above, the inspection method then comprises a step 3 consisting of aligning each image, which describes the X-ray transmission properties of the material for a view of the inspected part, with a corresponding view of a digital model of the part. The alignment aims to match each image with the digital model of the part.

[0047] It is specified that by view of the digital model of the part, it is understood a point of view of the digital model of the part, for example, the digital model of the ideal part.

[0048] For this, according to a particular arrangement, alignment step 3 comprises estimating (step 3.1) an optimal rigid body transformation of each view of the digital model of the part with respect to the source-detector system to obtain an image by the projection of each view of the digital model. This image is here called the calculated image. This transformation is described by a rotation and a translation in a three-dimensional space.

[0049] In a particularly advantageous manner, a projection operator is used to carry out step 3.1. According to a particular arrangement, the projection operator is a matrix of the form ar R ], with K a matrix described by intrinsic projection parameters, for example the focal length and the position of the projected source in each image, and with R and t respectively the rotation and translation of each view of the digital model of the part, therefore its rigid body transformation. The calculated image is then obtained by a digital radiograph calculation algorithm, for example a "ray tracing" algorithm or the Siddon Jacobs algorithm, from the view of the digital model of the part, the digital model of the part and the projection operator.

[0050] In other words, to estimate the transformation of the view of the digital model of the part, a projection operator is necessary in order to project the three-dimensional information of the digital model into a two-dimensional plane of a two-dimensional image. The matrix K can be calibrated upstream by calibration procedures specific to the inspection of parts, or it can be constructed from intrinsic parameters prescribed in a control range of the part.

[0051] In a particularly advantageous manner, alignment step 3 also comprises defining (step 3.2) a similarity metric between the calculated image of each view of the digital model and each image from a view of the inspected part. It is specified that a similarity metric is a value that describes the similarity between two images. This similarity can for example be evaluated from a difference in gray levels, or from the correlation of the two images or from information that one image holds on the other. In other words, in step 3.2, a similarity metric is defined between the projected information of the digital model and the information in the view of the part to be inspected.In a particularly clever way, it is the optimization of this similarity metric with respect to the rotation and translation parameters of the digital model of the part that allows the alignment of a view of the inspected part with a view of the digital model of the part, in other words, the alignment of an acquired image with the digital model of the part.

[0052] According to a particular arrangement, a first realization of the alignment makes use of a similarity measurement between a projection of three-dimensional salient points in the two-dimensional plane of the image and estimated corresponding points in the image. In this mode, the three-dimensional salient points are selected beforehand, by view, in the digital model of the part. The projection is done by view from the matrix K and a set of initial parameters for R and t. This initialization can be derived from parameters prescribed by a control range, for example the pose of the part prescribed for the acquisition of the image. The estimation of the corresponding points in the image can be done by a local correlation between an image calculated from a view of the digital model and the corresponding acquired image.The image can be, for example, calculated from the digital model of the part with parameters prescribed by a control range. using the projection operator defined for the digital model view. The so-called local correlation corresponds to a correlation operator, for example the two-dimensional normalized cross-correlation, between a neighborhood centered around a salient point projected into the computed image and a prescribed search region around the same point in the acquired image. The estimated correspondence of the salient point in the image corresponds in this case to the position where the local correlation is maximum. Once the salient points have been projected into the computed image, and the corresponding points have been estimated in the acquired image, the optimization consists of minimizing their distance with respect to the parameters of a rigid body transformation of the three-dimensional positions of the salient points. This can be done, for example, with an implementation of the PnP (Perspective and Point) method.

[0053] According to another arrangement, it is possible to use a similarity measure between an image calculated from a view of the digital model of the part and the image of the part to be inspected. In this case, the optimization consists of maximizing the similarity between these two projections with respect to the parameters of a rigid body transformation of the digital model of the part. One embodiment of this optimization consists of using the procedure minimizing the residual (difference) between the calculated image, for example by a simulation method, and the acquired image. Another embodiment of this optimization consists of using a similarity measure such as those derived from information theory, for example the realizations of the metrics of mutual information based on entropy. The estimation of the similarity measure can be restricted to a prescribed region of interest. The optimization is done per view from K and a set of initial parameters.In one embodiment, the initial parameters may be derived from the parameters prescribed by the control range, for example the pose of the part prescribed for the acquisition of the view. In another embodiment, the initial parameters may be derived from the result of the first realization of the alignment described in the preceding paragraph.

[0054] Step 4 - Detection of indications

[0055] As indicated previously, the method comprises (step 4) detecting indications on each image and positioning the detected indications with bounding boxes.

[0056] It is specified that by bounding box, it is understood a rectangle which encompasses the indication. A bounding box is defined by a height, a width and a position.

[0057] According to a particular arrangement, the detection of indications is done in two steps: a learning step 4.1 and a prediction step 4.2 on a production line. According to an exemplary embodiment, the learning step 4.1 can be performed using a neural network. According to this arrangement, to accomplish the detection of two-dimensional indications, a model is learned in a supervised manner. Supervised learning involves a database of images annotated by an operator. Each indication present in an image is labeled as follows: - position of the indication with a bounding box; - label of the indication; - dimensions of the indication (the dimension consists of the width and length of the identified indication, as will be described below; this information will be used in particular in step 7 of dimensional characterization).

[0058] According to a particular arrangement, it is possible to label all the indications with a single class or with several classes depending on the type of indication present (for example: inclusion, excess thickness, residue, or excess thickness). This way of labeling makes it possible not only to detect the indications and to label them according to predefined classes (for example inclusions, over thickness, under thickness). It is specified that this database can be made up of images representative of the variability of the production images. Preferably, the number of images with indications to be inserted into the database is sufficient to be able to learn representative characteristics of each indication.

[0059] According to a particular arrangement, a database balancing phase may possibly be carried out. Indeed, it may be more difficult to obtain samples of certain types of indications and / or certain areas to be inspected. A very unbalanced number of samples may be detrimental during statistical learning. To compensate for this imbalance, transformations may be added to the images (for example, rotations, addition of noise or blur, modification of contrast, change of brightness, or cropping). It is also possible to opt for the use of simulated images provided that they are representative of real data. According to another arrangement, it is possible to opt for using loss functions during learning which more strongly penalize errors on minority samples.

[0060] After applying the pre-processing mentioned during the implementation of step 2, the X-ray images are acquired, as well as the annotations made manually by the operators (nature and two-dimensional location of each indication) are given as inputs to the neural network. According to the example presented here, these entire images are of size 1024x1024 pixels. However, other sizes can be used.

[0061] According to a particular arrangement, the neural network used is a convolutional network designed for the classification and localization of patterns of interest on images. According to a particular arrangement, the network is trained using classical gradient descent algorithms (an optimization algorithm that allows finding the minimum of any convex function by progressively converging towards it). The two tasks (classification and two-dimensional localization with bounding boxes) are performed in parallel by back-propagation of two combined loss functions, where two refers to the two tasks performed in parallel: classification and two-dimensional localization. This approach is performed several times, in order to update the network weights in an increasingly optimal iterative manner.

[0062] In more detail, the procedure for updating weights (network parameters) through gradient descent algorithms generally follows the following formula:

[0063] et+1=et-a7()\(x,y,et)

[0064] Where represents the network parameters at time t, a(x, y; 0) represents the loss function specific to the problem to be solved with x the input data and y the expected output, and a represents a hyper-parameter which controls the speed of convergence.

[0065] For training the classification heads of the network to determine the type of target indication, traditional loss functions of the "focal loss" or Cross Entropy type can be used. For training the two-dimensional localization heads of the network to locate the target object in the image, the functions commonly used for regression problems can be used (for example Smooth L1 or L1).

[0066] Step 4.2 of prediction with the learned neural network is applied to all the parts to be inspected. Consequently, some X-ray transmission radiographic images of the part to be inspected are used as input to the model learned in step 4.1 in order to obtain the prediction results, i.e. the classification and two-dimensional localization (with bounding boxes) of the possible indications present in each image.

[0067] According to a particular arrangement, the output of this network is given for each image of the part individually and there is no joint analysis of the different projections of the same part nor quantification of the position of the indications detected in the part (three-dimensional localization).

[0068] Finally, all the indications found with the model are used as input for the following steps.

[0069] Step 5 - Building a model of the multi-view geometry

[0070] Next, the method comprises a step 5 which consists of constructing a model of the multi-view geometry using the alignment of each image on a respective view of the digital model of the part.

[0071] As shown schematically in Fig. 2, the multi-view geometry of the scanner-part system is constructed, which will subsequently make it possible to exploit the epipolar constraint for the multi-view consolidation which will be described below. This geometry is constructed from the pose of the part, given by a rotation p0) and a translation which models view 1 of the part, estimated by step 3 of alignment of images calculated with the acquired images.

[0072] According to the embodiment shown schematically in Fig. 2, the multi-view geometry is constructed such that the coordinate system of the digital model of the ideal part described by the unit vectors j and the origin °, is the reference system. The radiographic images corresponding to each view are placed around the part. For this, the orientations of its horizontal and vertical axes, and of the normal vector to the plane which contains them are calculated relative to the coordinate system of the digital model of the ideal part. The position of the X-ray source (C) per view relative to the coordinate system of the digital model of the ideal part is also calculated. It is specified that the embodiment shown schematically in [Fig.2] illustrates a three-view geometry.

[0073] Let / (yO)) be the intensity at the image position a3j)c P2 of the image of the view \ hereinafter called image \ The X-ray detector being of a discrete nature, the positions x of an element of the sensor (pixel) are defined by the coordinates (xu, xY) of its center. Let G p2 be the projection of the three-dimensional point XG P3 on the image 2

[0074] with pü) _ . £ p0) | j the projection matrix, ph) a matrix with the projection parameters intrinsic to the CT scanner per view (e.g., constructed with the values ​​prescribed by the control range such as the focal length). The projection of X into image j, i.e., the correspondence of in image j, must lie in a straight line called the epipolar line:

[0075]

[0076] with pü'jj the fundamental matrix from image 2 to image j. This matrix is ​​calculated from ph), pU) the projection matrices for images 2, j, and £>0) the position of the X-ray source for image 2, using for example a formalization known for stereovision.

[0077] Step 6 - Multi-view consolidation

[0078] Step 6 includes consolidating the detected indications with bounding boxes, using the multi-view geometry model by constructing (step 6.1) a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting (step 6.2) a candidate group to retain. In other words, step 6 consists of consolidating the indications detected on each image into groups containing the image indications that correspond to the projection of the same three-dimensional indication in the room.

[0079] According to a particular arrangement, step 6.1 of constructing a set of candidate groups comprises analyzing a location of the indications. This arrangement allows each candidate group to contain two-dimensional indications which are potentially the projections of the same three-dimensional indication.

[0080] According to a particular arrangement, step 6.2 of selecting a candidate group comprises selecting the candidate groups comprising the projections of the same indication.

[0081] [Fig.3] illustrates the problem of correspondence between boxes which encompass image indications which correspond to projections of the same three-dimensional indication in the room.

[0082] As indicated previously, according to a particular arrangement, consolidation step 6 comprises two steps. In a first step 6.1, a set of candidate projection groups is constructed by analyzing the location of the two-dimensional indications. In a second step 6.2, the candidate groups containing the projections of the same three-dimensional indication are selected. The selection uses a three-dimensional proximity criterion and the location relative to the digital model of the part of the three-dimensional position estimated by group. We will detail each of these two steps below.

[0083] Step 6.1: Selection of a candidate group:

[0084] The epipolar constraint given by the multi-view geometry and a zoning of the digital model of the ideal part are used to calculate a set of candidate groups of image indications. Indeed, each candidate group contains two-dimensional indications which are potentially the projections of the same three-dimensional indication in the part. The information of the digital model of the ideal part and / or of a division into zones of interest of the digital model of the part (see [Fig.4]) is added to the epipolar constraint. It is specified that in [Fig.4], these zones are referenced Z1 to Z7.

[0085] The candidate groups are obtained from the analysis of the location of the two-dimensional indications in the projections. Fig. 3 illustrates the correspondence of two projections of the same three-dimensional indication 300 in the case where the two-dimensional indication is contained by a bounding box 301. The correspondence is done by the matching of the two-dimensional points of the different images, for example the centers 302 of the bounding boxes. For this, the distance

[0086]

[0087] epipolar is exploited. The epipolar line 304 of a projection of a bounding box center 305 in image 1 is plotted in image j using the fundamental matrix given by the multi-view geometry and the distance 303 to this epipolar line 304 for the box centers in image j is calculated. An epipolar confidence score is given to each center in image j. This score is calculated by erfc. the complementary error function: pd, With d the epipolar distance, j(J) an indication detected in image j, the mean and &d the standard deviation value of a normal distribution of epipolar distances. d is calculated for a point in jO), for example the center of its bounding box. Pj is different from zero in the case where the points put in correspondence are not exactly the projection of the same point in the three-dimensional indication, as illustrated in Figure 5. The standard deviation &d is calibrated to the expected epipolar distance variability. This variability is explained by the expected uncertainty in the results of an alignment chain. In one embodiment, Ud and ® d are estimated per view from the analysis of the variations of the installation of the part (see Section Other Embodiments). an indication in image 1 is matched with jU), an indication in image J if p(dP thd AND if the two image indications are intersected with the

[0088]

[0089]

[0090]

[0091]

[0092] projection of the same area into the digital model of the ideal room: if p(^P) > a 0) for at least one Zk (0, otherwise with thd = 0.05 the threshold for the distance d to be in the 95% confidence interval, the projection in image 1 of the k-th zone of the digital model of the part, and ^j(i) q 2^) the intersection between the indication jU) and the zone £©. In one embodiment, jCO p 2^) is defined as the number of pixels contained by the bounding box of jW and by the projection in image 1 of the k-th zone of the model digital part. Finally, the following equation is used to construct a consolidation matrix between the detected indications for a part: f for i, i {1, A}, i^j, Lmejl.....L}, l*m JU l 0, otherwise with N the number of views and L the number of indications detected for a part. f indicates whether the 1-th indication in image 1 and the n-th indication in image j can be matched. This allows the estimation of candidate groups.

[0093] It should be noted that the consolidation matrix fjm is not symmetrical: it is possible O116 f ( B® # f ( B^ ) • In addition, the areas in which the part is cut can be quite large. This implies that the constraints of the equation used to construct a consolidation matrix can be satisfied, but that the association is not correct (see example in [Fig.5]). For these reasons, a final three-dimensional consolidation step is proposed.

[0094] Once the first candidate groups are obtained, all possible combinations of detection associations are calculated. This is illustrated by Figs. 6 and 7. First, the first candidate groups are obtained according to the method previously presented ([Fig.6]). According to the example presented here, it is observable that the consolidation matrix is ​​not symmetrical: detection 1 (column "Detection id") has been associated with detection 2 (column "Match id"). However, this last detection has been associated with detections 4 and 5.

[0095] As a result, all possible combinations of detection associations are then calculated from these first candidate groups ([Fig.7]). With reference to [Fig.7], this allows the following candidate groups to be created: {[1,2,4,5],[1,2,4],[1,2,5], [1,4,5],[2,4,5],[1,2],[1,4],[1,5],[2,4],[2,5],[4,5]}.

[0096] Note that combinations where more than one bounding box of the same image is present are not accepted. For example, detections 2 and 3 cannot be present in the same group, because they are on the same image.

[0097] Step 6.2 of selecting a candidate group:

[0098] In step 6.2 of selecting a candidate group, the candidate group to be retained (called consolidated group) is selected. For this, a consolidation score r (or “3D Score”) is calculated for each of the candidate groups. This score r corresponds to the radius of the smallest sphere containing the closest points between all the lines going from the three-dimensional position of the center of the two-dimensional indications in the images to the positions of the corresponding X-ray sources (see Figs. 8 and 9). The average of these points, called Xm, gives the estimated 3D position of each candidate group.

[0099] According to the example presented here, consolidation step 6.2 begins with the group(s) that contains the largest number of indications. Thus, in the example of Figs. 6 and 7, there are two groups with four indications: [1,2,4,5] and [1,3,4,5]. Among these groups, the group having the minimum 3D score is chosen (in the example presented here it is the group [1,3,4,5]), then its 3D score is compared with a threshold thr. In one embodiment, the threshold thr is estimated by analyzing the variations of the part installation (see Section Other Implementation Methods). If r < thr this group is retained. Otherwise, the group containing the second largest number of indications is chosen. The same approach by comparing r with thr. This is done until a candidate group is found that meets the constraint: r — thr. In the case where no group is retained, the detections can remain without consolidation.

[0100] Step 7 - Location of three-dimensional indications

[0101] As indicated previously, the method then comprises a step 7 consisting of locating each three-dimensional indication.

[0102] The localization (step 7) comprises determining an average of point clouds per candidate group containing several two-dimensional indications, which are projections of the three-dimensional indications.

[0103] The location of the three-dimensional indications corresponds to the averages Xm (described in step 6.2) of the point clouds per consolidated group containing several two-dimensional indications (see Figs 8 and 9).

[0104] Step 8 - Characterization of dimensions

[0105] Then, the method comprises (step 8) characterizing the dimensions of each indication in a relevant image selected from the set of images from the bounding boxes used to position the detected two-dimensional indications.

[0106] According to a particular arrangement, the characterization step comprises identifying a relevant image for measuring a length and a width of an indication using the length and the width of the bounding box of this indication.

[0107] Since the same indication can be visible on several images, it is necessary to identify the image from the most relevant view to make this measurement. This image, known as the main image, is identified using the three-dimensional location described in step 7. Indeed, the image to be used depends on the three-dimensional location of the indication to be characterized.

[0108] Once the main image has been identified, the method moves on to a measurement step (length and width). To do this, the method exploits the strong relationship between the size of the bounding boxes (obtained in step 4) and the measurement of the indications. According to a particular arrangement, a regression model (for example: linear, polynomial, or Huber model) is used. Indeed, the regression model allows us to describe the relationship between one or more independent variable(s) (for example, the height or width of the box) and a dependent variable (for example, the length or width of the indication in mm). According to a particular arrangement, a regression model for each dimension of the indication is used: a first model to estimate the length of the indication and a second model to estimate its width (see [Fig. 10]).

[0109] According to a particular arrangement, the two models are previously adjusted with annotated indications (length and width) by inspectors and detected by the detection model (see Figs 11 and 12). It should be noted that these data are obtained from step 4.1. Once the regression models are adjusted, they are used in the production line on the detected indications which are found in the main image (information obtained thanks to the three-dimensional localization of step 7).

[0110] Step 9 - Use

[0111] The method then comprises using (step 9) the characterization of each indication for subsequent processing by a third-party device.

[0112] Image analysis device

[0113] According to another aspect, there is provided an image analysis device comprising electronic circuitry (computer system 200) adapted to implement a method 100.

[0114] As shown diagrammatically in [Fig. 13], the computer system 200 may comprise, connected by a communication bus 210: a processor 201; a random access memory 202; a read-only memory 203, for example of the ROM (“Read Only Memory” in English) or EEPROM (“Electrically-Erasable Programmable Read Only Memory” in English) type; a storage unit 204, such as a hard disk HDD (“Hard Disk Drive” in English), or a storage media reader, such as an SD (“Secure Digital” in English) card reader; and an input-output interface manager 205.

[0115] The processor 201 is capable of executing instructions loaded into the RAM 202 from the ROM 203, an external memory, a storage medium (such as an SD card), or a communications network. When the computer system 200 is powered on, the processor 201 is capable of reading instructions from the RAM 202 and executing them. These instructions form a computer program enabling the implementation, by the processor 201, of the method 100.

[0116] All or part of the method 100 can thus be implemented in software form by executing a set of instructions by a programmable machine, for example a DSP (Digital Signal Processor) type processor or a microcontroller, or be implemented in hardware form by a machine or a dedicated component, for example an FPGA (Field Programmable Gate Array) or ASIC (Application-Specified Integrated Circuit) component. Generally speaking, the computer system 200 comprises electronic circuitry adapted and configured to implement, in software and / or hardware form, the method in relation to the computer system 200 in question.

[0117] Computer program product

[0118] According to another aspect, there is also provided a computer program product comprising program code instructions for executing the method 100.

[0119] Storage medium

[0120] According to another aspect, there is also provided a non-transitory storage medium on which the computer program comprising program code instructions for executing the method 100 is stored.

Claims

1.

2. Claims Method (100) for three-dimensional control of a part from several radiographic images by an image analysis device comprising electronic circuitry adapted to implement the three-dimensional control method, the method being characterized in that it comprises at least the following steps: - (Step 1) Acquiring a plurality of radiographic images of a part by a source-detector system, each image including information on the radiographic transmission properties of the part; - (Step 3) Align each image with a view of a digital model of the part; - (Step 4) Detect indications on each image and position the detected indications with bounding boxes; - (Step 5) Build a model of the multi-view geometry using the alignment of each image to a respective view of the digital model of the part; - (Step 6) Consolidate the detected indications with bounding boxes, using the multi-view geometry model by constructing (step 6.1) a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting (step 6.2) a candidate group to retain; - (Step 7) Locate each three-dimensional indication; - (Step 8) Characterize the dimensions of each indication in a relevant image selected from the set of images from the bounding boxes used to position the detected indications; - (Step 9) Use the characterization of each indication for further processing by a third-party device. Method (100) according to claim 1 wherein the alignment step 3 comprises estimating (step 3.1) an optimal rigid body transformation of each view of the digital model of the part with respect to the source-detector system to obtain an image of the projection of each view of the digital model.

3. The method (100) of claim 2, wherein a projection operator is used to perform step 3.1, the operator is a matrix of the form • [ R |t ] ' with K a matrix described by intrinsic projection parameters, and with R and t respectively the rotation and translation of each view of the digital model.

4. Method (100) according to one of claims 2 or 3, in which step 3 comprises defining (step 3.2) a similarity metric between the calculated image of each view of the digital model and each acquired image.

5. A method (100) according to any one of claims 1 to 4, wherein step 6.1 of constructing a set of candidate groups comprises analyzing a location of the indications.

6. Method (100) according to any one of claims 1 to 5, in which step 6.2 of selecting a candidate group comprises selecting the candidate groups containing the projections of the same three-dimensional indication.

7. Method (100) according to any one of claims 1 to 6 in which the localization (step 7) comprises determining an average of point clouds per candidate group containing several indications.

8. A method (100) according to any preceding claim, wherein the characterizing step comprises selecting (step 8.1) a relevant image for estimating a length and a width of an indication, the relevant image being selected based on a three-dimensional location of the indication.

9. Image analysis device characterized in that it comprises electronic circuitry (200) for implementing a method (100) for three-dimensional control of a part from several radiographic images, the method comprising at least the following steps: - (Step 1) Acquire a plurality of radiographic images of a part by a source-detector system, each image comprising information on the radiographic transmission properties of the part; - (Step 3) Align each image with a view of a digital model of the part; - (Step 4) Detect indications on each image and position the detected indications with bounding boxes; - (Step 5) Construct a model of the multi-view geometry using the alignment of each image with a respective view of the digital model of the part; - (Step 6) Consolidate the detected indications with bounding boxes, using the model of the multi-view geometry by constructing (Step 6.1) a set of candidate groups, each group containing projections of a potential three-dimensional indication, and selecting (Step 6.2) a candidate group to retain; - (Step 7) Locate each three-dimensional indication; - (Step 8) Characterize the dimensions of each indication in a relevant image selected from the set of images from the bounding boxes used to position the detected indications; - (Step 9) Use the characterization of each indication for further processing by a third-party device.

10. A computer program product comprising program code instructions for executing the method (100) according to any one of claims 1 to 9, when said computer program product is executed by a processor.

11. A non-transitory storage medium having stored thereon a computer program comprising program code instructions for executing the method (100) according to any one of claims 1 to 9, when said instructions are read from said non-transitory storage medium and executed by a processor.

Images