Method for real-time display of a signal for non-destructive testing of a mechanical part

Description

[0001] The present invention lies in the field of visualization in 3D space of augmented reality data. The invention is described here in the field of non-destructive testing.

[0002] The invention is presented here, by way of illustration and in a non-limiting manner, in the field of non-destructive testing with an ultrasonic sensor. Based on the description of the invention, it will be clear to a person skilled in the art that the principle of the invention can be applied to other types of sensors to visualize data related to the sensor.

[0003] Non-destructive testing by ultrasound is a non-invasive method of detecting defects in a part, based on the emission of ultrasound and the detection of their reflection linked to the acoustic interfaces encountered.

[0004] A suitable sensor emits ultrasound at a frequency (usually between 500 kHz and 100 MHz) chosen according to the nature of the part to be tested. The sensor must be in direct contact with the part so that the propagated waves are not slowed down by the impedance of the air between the sensor's emission point and the part.

[0005] The waves are reflected on the acoustic interfaces encountered: contours of the room, interior defects.

[0006] The sensor, placed in contact with the part to be checked, intercepts the waves re-emitted by any possible defect.

[0007] The detected waves are converted into signals by an associated electronic assembly of the sensor. Software assembles these signals to form an image of the interior of the part. Image analysis makes it possible to discriminate between echoes due to a defect and those related to the geometry of the part.

[0008] In non-destructive testing, an operator sweeps the sensor over a pre-defined examination area of the part to be tested. It is imperative that the entire area be scanned by the sensor. The entire coordinates of the signal emission point are a key element in verifying that the entire area has been scanned. Furthermore, in addition to knowing that the entire area has been scanned, it is advantageous for the operator to detect during testing whether an area requires special attention.

[0009] The technical background includes Marc Dubois' article "Hololens for Augmented Reality NDT" published on the ULTRASONICS forum on 10-30-2026, as well as document DE 10 2016 003 543 A1.

[0010] The technical challenge is how to visualize 3D-positioned data in augmented reality while giving the user the impression of seeing inside the room. Indeed, a 3D surface displayed as a hologram appears to float in space, even if its position has been precisely defined relative to the 3D volume of the room. This shows that a simple hologram display is not sufficient to give the impression of depth and allow the brain to interpret the data as if it were inside the room.

[0011] The invention aims to overcome all or part of the problems mentioned above by proposing a visualization method and device making it possible to present the acquired data, i.e. the spatialized signals, in real time in the form of a 3D holographic surface positioned in the 3D volume where they were acquired.

[0012] To this end, the invention relates to a method for real-time visualization of a non-destructive testing signal of a mechanical part, the signal being emitted by a non-destructive testing device comprising: an optical motion tracking system to which a reference frame is linked, a sensor holder, a rigid body, a non-destructive testing sensor secured to the sensor holder and fixedly connected to the rigid body, a computer, the visualization method being executed by an augmented reality visualization device opposite the mechanical part, to which an augmented reality reference is linked, the method being characterized in that it comprises the following steps: Moving the non-destructive testing sensor over an examination area of the mechanical part; Simultaneously with the step of moving the non-destructive testing sensor, emission from an emission point along an emission axis and reception of the signal by the sensor; Determination, by the computer, of a cutout of an occlusion inside the mechanical part, the cutout being centered around the emission point; Determination, by the computer, of a signal visualization surface constructed from the ultrasonic paths of the signal, this surface being located inside the cutout; Visualization, on the augmented reality visualization device, ▪ of a real view of the mechanical part,of the sensor holder and the non-destructive testing sensor, ▪ a holographic 3D representation of the mechanical part, the sensor holder and the non-destructive testing sensor, superimposed on the real view, ▪ a holographic representation of the cutout of the occlusion of the part and the signal viewing surface, superimposed on the real view, the occlusion being achieved at least partially by superimposing the holographic 3D representation of the mechanical part (7') on the real view of the mechanical part (7), the cutout (80) of the occlusion crossing the depth of the mechanical part in its holographic 3D representation (7') until reaching the signal viewing surface (81), the signal viewing surface (81) being a quadric determined so as to correspond to a mesh constituted by the paths of said signals propagating in the mechanical part. ∘

[0013] Advantageously, the orientation of the cutout is linked to the location of the augmented reality viewing device.

[0014] Advantageously, the visualization of 3D holographic representations is carried out transparently.

[0015] Advantageously, the step of determining the intersection surface comprises the following steps: Calculation of the paths taken by the signal emitted by the sensor; Creation of a 3D mesh representative of the mechanical part, the sensor holder, the non-destructive testing sensor, and the paths taken by the signal; Mapping the paths taken by the signal onto the 3D mesh.

[0016] Advantageously, the visualization method according to the invention first comprises a step of calibrating the visualization device in augmented reality in the reference frame.

[0017] Advantageously, the visualization method according to the invention comprises, prior to the step of calibrating the augmented reality visualization device in the reference frame, a step of calibrating the non-destructive testing device.

[0018] The invention also relates to a device for real-time visualization of a non-destructive testing signal of a mechanical part, the device comprising a non-destructive testing device comprising: an optical motion tracking system to which a reference frame is linked, a sensor holder, a first rigid body, a non-destructive testing sensor secured to the sensor holder and fixedly connected to the first rigid body, the sensor being intended to be moved over an examination area of the mechanical part, and to emit from an emission point and receive the signal along an emission axis a computer, the device (10) further comprising an augmented reality display device opposite the mechanical part, to which an augmented reality frame is linked, the non-destructive testing device being characterized in that the computer is configured to determine a cutout of the occlusion of the mechanical part, the cutout being centered around the emission point; determine a signal viewing surface constructed from the ultrasonic paths of the signal, this surface being located inside the cutout; and the augmented reality viewing device is configured to display: ▪ a real view of the mechanical part, the sensor holder and the non-destructive testing sensor, ▪ a holographic representation of the mechanical part, the sensor holder and the non-destructive testing sensor, superimposed on the real view, ▪ a holographic representation of the cutout of the occlusion of the part and the signal viewing surface, superimposed on the real view.the occlusion being carried out at least partially by a superposition of the holographic 3D representation of the mechanical part (7') on the real view of the mechanical part (7), the cutout (80) of the occlusion crossing the depth of the mechanical part in its holographic 3D representation (7') until reaching the signal display surface (81), the signal display surface (81) being a quadric determined so as to correspond to a mesh constituted by the paths of said signals propagating in the mechanical part.

[0019] Advantageously, the control device according to the invention further comprises a pointing device comprising a tip and fixedly connected to a second rigid body, the pointing device being capable of determining the position of points on a surface.

[0020] The invention also relates to a computer program comprising instructions which cause the device according to the invention to execute the steps of the method according to the invention.

[0021] The invention also relates to a computer-readable recording medium on which said computer program is recorded.

[0022] The invention will be better understood and other advantages will appear on reading the detailed description of an embodiment given by way of example, a description illustrated by the attached drawing in which: there [ Fig.1 ] schematically represents a non-destructive testing device according to one embodiment of the invention; the [ Fig.2 ] schematically represents a sensor associated with a sensor holder of a non-destructive testing device according to an embodiment of the invention; the [ Fig.3 ] schematically represents a pointing device according to one embodiment of the invention; the [ Fig.4 ] schematically represents a mechanical part capable of being checked by the non-destructive testing device of the invention; the [ Fig.5 ] schematically represents a flowchart of the sub-steps of a calibration step usable in the invention; the [ Fig.6 ] schematically represents a transformation (composed of a translation and / or a rotation) allowing one to move from an RA frame to an RB frame; the [ Fig.7 ] schematically represents the upper surface of a calibration block implemented in the calibration method according to the invention; the [ Fig.8 ] schematically represents the sensor, sensor holder and first rigid body of the calibration device according to the invention; the [ Fig.9 ] schematically represents the upper surface of a calibration block implemented in the calibration method according to the invention to determine the length and width of the sensor; the [ Fig.10 ] schematically represents a flowchart of the steps of the method for real-time visualization of a non-destructive testing signal of a mechanical part according to the invention; the [ Fig.11 ] illustrates the display obtained in the augmented reality visualization device with the visualization method according to the invention; the [ Fig.12 ] schematically represents the QR code implemented in the calibration step of the augmented reality visualization device in the reference frame of the visualization method according to the invention; the [ Fig.13 ] schematically represents the transformation matrices in the visualization method according to the invention; the [ Fig.14 ] schematically represents the transformation matrices in the visualization method according to the invention.

[0023] For the sake of clarity, the same elements will have the same references in the different figures. For better visibility and to enhance understanding, the elements are not always shown to scale. The term user, operator or inspector is used to refer to the physical person who performs the non-destructive testing, i.e. who manipulates and scans the sensor (and sensor holder) on the mechanical part.

[0024] The subject of the invention is based on a visualization device and method that allows the user to visualize data (ultrasonic signals) positioned in 3D in augmented reality by giving the user the impression of seeing inside the part. Augmented reality consists of superimposing virtual elements on the real world. The device includes in particular an optical positioning system, and an augmented reality visualization device. And it is necessary to carry out a recalibration of the two 3D environments (real 3D scene and 3D representation of the data) by applying augmented reality rendering techniques such as transparencies, occlusions, cutting planes in a material recess. The objective of the invention is to allow the controller a complete immersion in the part and thereby a focus on the non-destructive testing signals positioned precisely in the geometry.

[0025] There figure 1 schematically represents a non-destructive testing device 10 according to one embodiment of the invention. The device 10 for non-destructive testing of a mechanical part comprises an optical motion tracking system 1 which has the function of tracking the movement of an object in space and more particularly the movement of rigid bodies as described below.

[0026] The optical motion tracking system 1 is associated with an orthonormal reference frame R 0 = (0, u 0 , v 0 , n 0 ) where O is the origin of the reference frame and u 0 , v 0 , n 0 are normalized vectors orthogonal to each other.

[0027] The optical motion tracking system 1 determines the Cartesian coordinates and orientation of a rigid body in the orthonormal frame of the optical motion tracking system 1.

[0028] The optical motion tracking system 1 comprises at least two cameras and one or more infrared emitters. Other types of optical system can be used within the scope of the invention, for example a laser-based optical system and / or with non-volume markers of the pellet type. It is important to specify that the system 1 can be a passive motion tracking system (optical or non-optical).

[0029] The non-destructive testing device 10 comprises a first rigid body 2 connected to a probe, or sensor, 3 and a sensor holder 8. The sensor 3 is integral with the sensor holder 8 fixedly connected to the first rigid body 2. The first rigid body 2, the sensor holder 8 and the sensor 3 are fixedly connected and form an indivisible assembly during non-destructive testing. The first rigid body 2 comprises at least three infrared-reflecting spherical targets located at distinct positions. The first rigid body 2 is associated with an orthonormal reference frame RC = (C, u C , v C , n C ) where C is the origin of the reference frame and u C , v C , n C are normalized vectors orthogonal to each other.

[0030] In a preferred embodiment, the first rigid body 2 comprises six spherical targets. The sensor 3 is, for example, a single-element ultrasonic probe. It comprises an emitting and receiving surface, called the active surface, 31. The active surface 31 is a rectangle with a flat surface. Alternatively, the sensor 3 is of another type, for example an eddy current probe. Generally speaking, an active surface is any surface emitting or receiving physical signals belonging to a non-destructive testing sensor. For example, in the case of a single-element ultrasonic contact sensor, this corresponds to the surface of the piezoelectric. In the case of a single-element ultrasonic sensor with a “Plexiglas” shoe, this corresponds to the surface of the shoe through which the ultrasonic signals are emitted.

[0031] The control device 10 comprises an augmented reality viewing device 16 opposite the mechanical part 7, to which an augmented reality (AR) marker is linked. It is through this viewing device 16 that the operator sees the actual mechanical part, and it is on this viewing device 16 that the holographic 3D representations (detailed later) are displayed, superimposed on the view of the mechanical part on the viewing device 16.

[0032] These holographic 3D representations can form occlusions of the mechanical part 7 in the sense that they are totally or partially superimposed on the actual mechanical part 7. In particular, a holographic 3D representation of the mechanical part can be superimposed on the actual mechanical part 7 and thus form an occlusion.

[0033] An occlusion can consist of the envelope of the virtual representations of the mechanical part to be controlled and / or the sensor and / or the sensor holder

[0034] It is possible to add occlusion information such as coverage areas or welds and also to cut this occlusion in order to reveal signals.

[0035] The control device 10 comprises a computer 6 configured to a. determining a cutout of the occlusion of the holographic representation of the mechanical part 7, the cutout being centered around the emission point; b. determining a projection (or visualization) surface of the signals constructed from the ultrasonic paths of the ultrasonic signals delivered by the sensor 3. This surface corresponds to the intersection of the plane containing the emission axis in the cutout of the occlusion. In other words, the signal visualization surface is a quadric, or quadratic surface, calculated so as to correspond as closely as possible to the mesh constituted by the ultrasonic paths.

[0036] The term occlusion means hiding real objects behind virtual objects. Occlusion occurs when an object in a 3D space blocks the view of another object. In augmented reality, computer-generated objects are placed in a real scene to provide additional information or modify the nature of the real objects. Thus, the virtual objects and the real scene must be perfectly aligned in order to maintain high levels of realism and allow the objects to behave as they would under normal conditions. In particular, in the context of the invention, a holographic representation of the mechanical part 7 is superimposed on the real part 7 and thus constitutes a total or partial occlusion.

[0037] The occlusion cutout corresponds to an intersection between a 3D surface, for example an ellipsoid, and the signal viewing surface.

[0038] The calculator 6 makes it possible in particular to assemble the signals received by the sensor to form an image of the interior of the room.

[0039] And the augmented reality viewing device 16 is configured to display: a. a real view of the mechanical part 7, the sensor holder 8 and the non-destructive testing sensor 3, b. a holographic representation in the form of occlusions of the mechanical part 7', the sensor holder 8' and the non-destructive testing sensor 3', superimposed on the real view, c. a holographic representation of the cut-out of the occlusion of the part and the signal viewing surface, superimposed on the real view.

[0040] The control device 10 can be calibrated according to a method known to those skilled in the art, for example as disclosed in document FR 3 087 254 B1. In the following, and by way of example and in a non-limiting manner of the invention, the calibration of the control device 10 is carried out according to another method.

[0041] The control device 10 may comprise a calibration block and, in our calibration example, the computer may be configured to carry out the following steps which will be explained below: determining, from three points on an upper flat surface of the calibration block, the length and width of the calibration block; determining a first transformation matrix from the reference frame (R 0 ) to a frame of the block (RB ) linked to the calibration block; determining, from three points on the first rigid body (2), the length and width of the sensor when it is arranged on the upper flat surface of the calibration block; determining a second transformation matrix making it possible to move from a frame of reference (RH ) linked to the sensor holder to a frame of reference (RS ) linked to the sensor.

[0042] The calculator is further configured to determine a third transformation matrix making it possible to move from the reference frame (R 0 ) to the frame (RS ) linked to the sensor.

[0043] The invention is described for tracking one sensor, but it also applies to tracking multiple sensors, simultaneously and independently.

[0044] The non-destructive testing device comprises a computer 6 connected to the optical motion tracking system 1 and to a control module 5. The computer 6 is for example a computer or an electronic card. It comprises in particular a processor executing a computer program implementing the method which will be described and a memory for storing the results. It also comprises input and output interfaces and can be associated with a display screen.

[0045] The connection between the computer 6 and the optical motion tracking system 1 may be wired or wireless. Similarly, the connection between the computer 6 and the control module 5 may be wired or wireless.

[0046] There figure 2 schematically represents a sensor 3 associated with a sensor holder 8 of a non-destructive testing device 10 according to an embodiment of the invention. The non-destructive testing sensor 3 is integral with the sensor holder 8 fixedly connected to the first rigid body 2. More precisely, the sensor is inserted into a sensor holder, which is integral with a rigid body allowing it to be located in the frame of reference of the optical positioning system. The lower surface of the sensor extends substantially in the same plane as the lower surface of the sensor holder.

[0047] Advantageously, the non-destructive testing device comprises a pointing device 4.

[0048] There figure 3 schematically represents a pointing device 4 according to one embodiment of the invention. The pointing device 4 comprises a second rigid body 41 and a precision tip 42. The second rigid body 41 comprises at least three infrared-reflecting spherical targets located at distinct positions. The shape of the second rigid body 41, i.e. the exact positioning of the spheres relative to each other, is known in advance. The second rigid body 41 and the precision tip 42 are fixedly connected and form an indivisible whole. The origin of the second rigid body 41 has been previously configured to correspond to the tip 42. Thus, the origin of the second rigid body 41 which will be measured by the optical motion tracking system 1 as explained below corresponds exactly to the physical point pointed at with the pointing device.

[0049] In a preferred embodiment, the second rigid body 41 comprises seven spherical targets.

[0050] The non-destructive testing device comprises a control module 5 provided with at least one actuation button 51. Preferably the control module 5 is mounted on the pointing device 4 to facilitate its use.

[0051] There figure 4 schematically represents a mechanical part 7 capable of being checked by the non-destructive testing device of the invention. The mechanical part to be checked 7 comprises an examination zone 71 defined on the surface of the mechanical part 7. The examination zone 71 extends over all or part of the mechanical part. The examination zone is on a part of the mechanical part which has a surface of known geometric shape, such as a flat surface, a cylindrical surface or even a conical surface, for example. In all cases, the geometric shape of the examination zone can be represented by an analytical function. Alternatively, it is also possible to work from a mesh of the part originating for example from CAD software (abbreviation for computer-aided design).

[0052] There figure 5 schematically represents a flowchart of the sub-steps of a calibration step usable in the invention. In the following, we will use the following letters: R to qualify a reference frame C to qualify the center (or origin) of a reference frame I, J and K to qualify the unit vectors of a reference frame respectively along the x, y and z axes of this same reference frame P or Q to qualify a point V to qualify a vector T to qualify a transformation matrix L to qualify a length W to qualify a width O to qualify the optical positioning system and the associated world reference frame B to qualify the calibration block and the associated block reference frame H to qualify the sensor holder and the associated sensor holder reference frame S to qualify the sensor and the associated sensor reference frame

[0053] To express a vector V 1 in the RA frame, the notation A < V 1 is used. Note that a vector is used to express both a position and a displacement.

[0054] The transformation (composed of a translation and / or a rotation) allowing to pass from a RA frame to a RB frame is defined by the transformation matrix A< TB . The figure 6 schematically represents such a transformation.

[0055] This 4×4 transformation matrix is composed as follows: T B <none / > <mprescripts / > <none / > A = I B <none / > <mprescripts / > <none / > A → J B <none / > <mprescripts / > <none / > A → K B <none / > <mprescripts / > <none / > A → C B <none / > <mprescripts / > <none / > A → 0 0 0 1 = I x B <none / > <mprescripts / > <none / > A J x B <none / > <mprescripts / > <none / > A K x B <none / > <mprescripts / > <none / > A C x B <none / > <mprescripts / > <none / > A I y B <none / > <mprescripts / > <none / > A J y B <none / > <mprescripts / > <none / > A K y B <none / > <mprescripts / > <none / > A C y B <none / > <mprescripts / > <none / > A I z B <none / > <mprescripts / > <none / > A J z B <none / > <mprescripts / > <none / > A K z B <none / > <mprescripts / > <none / > A C z B <none / > <mprescripts / > <none / > A 0 0 0 1 A< I B , A< J B And A< K B respectively denote the unit vectors along the axes x B , y B And z B of the landmark R B and expressed in the reference R A .

[0056] A< C B is the vector expressing the origin of the reference frame R B in the benchmark R A .

[0057] The motion tracking system 1 is capable of locating rigid bodies 2, 41 in space. A rigid body is a non-deformable collection of spherical markers reflecting infrared rays and thus locatable by the optical positioning system. After calibration, the motion tracking system can associate with a rigid body an origin reference frame which is used to qualify the position and orientation of rigid bodies in the reference frame of the motion tracking system.

[0058] It should be noted that in order to be correctly located, rigid bodies must be located within the solid angle seen by the motion tracking system.

[0059] In particular, the motion tracking system is provided with a factory-calibrated tool for obtaining the coordinates of a point in space in the frame of reference of said motion tracking system. In the following, this tool is referred to as the second rigid body 41.

[0060] Two different rigid bodies are used in the calibration method of the invention: the second rigid body 41 previously described and the first rigid body 2 used to locate the sensor holder 8. It is this latter first rigid body 2 which is the subject of the calibration procedure described in this calibration step. The purpose of this calibration step is to be able to quickly, simply and accurately determine the geometric transformation (rotation and / or translation) existing between the origin of the first rigid body 2 as located by the motion tracking system and the point of emission of the ultrasounds by the sensor 3, independently of the sensor holder used. In other words, this calibration makes it possible to determine the geometric transformation, in the form of a transformation matrix, between the origin of the first rigid body 2 and a point of the sensor regardless of the shape of the sensor holder necessarily arranged between these two elements.

[0061] The control device 10 according to the invention comprises: an optical motion tracking system 1 to which a reference frame (R0) is linked, a sensor holder 8, a first rigid body 2, a non-destructive testing sensor 3 secured to the sensor holder 8 fixedly connected to the first rigid body 2, a computer 6, an augmented reality visualization device 16 facing the mechanical part 7, to which an augmented reality (AR) frame is linked.

[0062] Prior to the step of calibrating the augmented reality visualization device 16 (detailed below) in the reference frame (R0), the visualization method of the invention may comprise a step 1000 of calibrating the non-destructive testing device. The step 1000 of calibrating a non-destructive testing device for a mechanical part 7 may comprise, by way of example, the following steps: determination (100), from three points on an upper planar surface of a calibration block 14, of the length and width of the calibration block; determination (110) of a first transformation matrix of the reference frame (R 0 ) to a frame of the block (RB ) linked to the calibration block; arrangement (115) of the sensor 3 on the upper planar surface of the calibration block 14; determination (120), from three points on the first rigid body (2), of the length and width of the sensor 3; determination (130) of a second transformation matrix making it possible to move from a frame of reference (RH ) linked to the sensor holder to a frame of reference (RS ) linked to the sensor.

[0063] It is recalled that the calibration step is compatible with the visualization process detailed below. However, another calibration step can be implemented instead of step 1000.

[0064] Advantageously, the calibration step may further comprise a step 140 of determining a third transformation matrix making it possible to move from the reference frame (R 0 ) to the frame (RS ) linked to the sensor. The third transformation matrix is obtained by multiplying the second transformation matrix with the first transformation matrix.

[0065] The purpose of steps 100 and 110 is to calibrate the calibration block 14 itself, i.e. to determine its dimensions (length and width) and to associate with it a reference frame RB linked to the calibration block. The motion tracking system and the calibration block must be firmly fixed in order to avoid any modification of their relative positions and orientations. Then, an operator uses the motion tracking system and the second rigid body 41 in order to acquire three positions on the surface of the calibration block 14. These three positions Q 1 , Q 2 andQ 3 are previously marked, advantageously but not necessarily, by small holes created for this purpose on the flat upper surface of the calibration block 14, as shown in the figure 7 Advantageously, these three positions are located at three vertices of a rectangle delimited by the blockers 13.

[0066] There figure 7 schematically represents the upper surface of a calibration block implemented in the calibration step according to the invention. The calibration block is a part having a flat upper surface on which an area has been delimited by blockers 13. The dimensions of this area, rectangular in shape, are known precisely. As will become clear later, the blockers 13 serve as a stop to correctly position the sensor and sensor holder assembly at a vertex, preferably a right angle, of the area.

[0067] The calibration step may comprise, prior to step 110 of determining the first transformation matrix of the reference frame R 0 to a frame of the block RB linked to the calibration block, a step 105 of determining the frame of the block RB linked to the calibration block in the reference frame R 0 .

[0068] More precisely, the calculator 6 receives three vectors as input: Vector defining the first position in the reference frame R O : O< Q 1 Vector defining the second position in the reference frame R O : O< Q 2 Vector defining the third position in the reference frame R O : O< Q 3

[0069] Calculator 6 calculates the following data: Block length: L B Width of the bloc : W B Transformation matrix allowing to move from the reference frame R O at the block marker R B : O< T B

[0070] The calculation steps are detailed below.

[0071] The vector defining the center of the calibration block in the reference frame R O is calculated: C B <none / > <mprescripts / > <none / > O → = Q 1 <none / > <mprescripts / > <none / > O → + Q 3 <none / > <mprescripts / > <none / > O → 2

[0072] The vector defining the length of the calibration block in the world frame R O is calculated: L B <none / > <mprescripts / > <none / > O → = Q 1 <none / > <mprescripts / > <none / > O → − Q 2 <none / > <mprescripts / > <none / > O →

[0073] The vector defining the width of the calibration block in the world frame R O is calculated: W B <none / > <mprescripts / > <none / > O → = Q 3 <none / > <mprescripts / > <none / > O → − Q 2 <none / > <mprescripts / > <none / > O →

[0074] The length of the calibration block can thus be calculated: L B = L B <none / > <mprescripts / > <none / > O →

[0075] The width of the calibration block can also be calculated: L B = W B <none / > <mprescripts / > <none / > O →

[0076] From this data, it is possible to calculate the three unit vectors of the block frame R B in the reference frame R O : I B <none / > <mprescripts / > <none / > O → = L B <none / > <mprescripts / > <none / > O → L B J B <none / > <mprescripts / > <none / > O → = W B <none / > <mprescripts / > <none / > O → W B K B <none / > <mprescripts / > <none / > O → = I B <none / > <mprescripts / > <none / > O → ∧ J B <none / > <mprescripts / > <none / > O →

[0077] Finally, the transformation matrix allowing to move from the reference frame R O at the block marker R B is calculated: T B <none / > <mprescripts / > <none / > O = I B <none / > <mprescripts / > <none / > O → J B <none / > <mprescripts / > <none / > O → K B <none / > <mprescripts / > <none / > O → C B <none / > <mprescripts / > <none / > O → 0 0 0 1

[0078] The purpose of steps 120 and 130 is to determine the dimensions of the sensor (length and width), as well as the transformation matrix between the first rigid body located on the sensor holder and the center of the sensor, as shown in the figure 8 , which schematically represents the sensor, sensor holder and first rigid body of the calibration device according to the invention.

[0079] There figure 9 schematically represents the upper surface of a calibration block implemented in the calibration step to determine the length and width of the sensor.

[0080] To carry out steps 120 and 130, the sensor 3 is placed (step 115) on the upper flat surface of the calibration block 14. The operator acquires three positions of the first rigid body present on the sensor holder, as visible in the figure 9 . These three positions P 1 , P 2 and P 3 correspond to the upper-left, lower-left and lower-right corners respectively. Note that it is possible to choose a different combination of corners than this one.

[0081] Note that the sensor holder is designed in such a way that it is the sensor itself and not the sensor holder which comes into contact with the edges (blockers) of the calibration block when the sensor holder is brought to the positions P 1 , P 2 and P 3 .

[0082] In addition to the previously available data, the calculator 6 receives three vectors as input: Vector defining the first position in the reference frame R O : O< P 1 Vector defining the second position in the reference frame R O : O< P 2 Vector defining the third position in the reference frame R O : O< P 3

[0083] Calculator 6 calculates the following data: Sensor length: L S Sensor width: W S Transformation matrix allowing to pass from the sensor holder reference R H to the sensor mark R S : H< T S

[0084] The calculation steps are detailed below.

[0085] The following vectors are calculated: P 21 <none / > <mprescripts / > <none / > O → = P 1 <none / > <mprescripts / > <none / > O → − P 2 <none / > <mprescripts / > <none / > O → P 23 <none / > <mprescripts / > <none / > O → = P 3 <none / > <mprescripts / > <none / > O → − P 2 <none / > <mprescripts / > <none / > O →

[0086] The sensor length is calculated: L S = L B − P 21 <none / > <mprescripts / > <none / > O →

[0087] The sensor width is calculated: W S = W B − P 23 <none / > <mprescripts / > <none / > O →

[0088] Finally, the three unit vectors of the sensor frame RS in the reference frame RO are calculated: I S <none / > <mprescripts / > <none / > O → = I B <none / > <mprescripts / > <none / > O → J S <none / > <mprescripts / > <none / > O → = J B <none / > <mprescripts / > <none / > O → K S <none / > <mprescripts / > <none / > O → = K B <none / > <mprescripts / > <none / > O →

[0089] It is then possible to calculate the transformation matrix allowing the transition from the RH sensor-holder frame to the RO reference frame, the sensor-holder being in P 3: T H <none / > <mprescripts / > <none / > O = I S <none / > <mprescripts / > <none / > O → J S <none / > <mprescripts / > <none / > O → K S <none / > <mprescripts / > <none / > O → P 3 <none / > <mprescripts / > <none / > O → 0 0 0 1 T O <none / > <mprescripts / > <none / > H = T H <none / > <mprescripts / > <none / > O − 1

[0090] In other words, the calibration step detailed here comprises, prior to step 125 of determining the RS reference frame linked to the sensor in the RH reference frame linked to the sensor holder, a step 123 of determining a fourth transformation matrix making it possible to move from the RH reference frame linked to the sensor holder to the reference frame R 0 .

[0091] The next step is to calculate the coordinates of the center of the sensor in the reference frame R O (the calculation is done for the sensor holder in P 3 ): C S <none / > <mprescripts / > <none / > O → = Q 3 <none / > <mprescripts / > <none / > O → − L S × I S <none / > <mprescripts / > <none / > O → 2 − W S × J S <none / > <mprescripts / > <none / > O → 2

[0092] It is also appropriate to calculate the three unit vectors of the sensor frame RS in the sensor holder frame RH: I S <none / > <mprescripts / > <none / > H → = T O <none / > <mprescripts / > <none / > H × I S <none / > <mprescripts / > <none / > O → J S <none / > <mprescripts / > <none / > H → = T O <none / > <mprescripts / > <none / > H × J S <none / > <mprescripts / > <none / > O → K <mprescripts / > <none / > H S → = I <mprescripts / > <none / > H S → ∧ J <mprescripts / > <none / > H S →

[0093] Finally, we calculate the center of the RS sensor frame in the RH sensor holder frame: C S <none / > <mprescripts / > <none / > H → = T O <none / > <mprescripts / > <none / > H × C S <none / > <mprescripts / > <none / > O →

[0094] In other words, the calibration step comprises, prior to step 130 of determining the second transformation matrix making it possible to move from an RH reference frame linked to the sensor holder to an RS reference frame linked to the sensor, a step 125 of determining the RS reference frame linked to the sensor in the RH reference frame linked to the sensor holder.

[0095] Finally, we can calculate the transformation matrix allowing us to move from the RH sensor-holder frame to the RS sensor frame: T S <none / > <mprescripts / > <none / > H = I S <none / > <mprescripts / > <none / > H → J S <none / > <mprescripts / > <none / > H → K S <none / > <mprescripts / > <none / > H → C S <none / > <mprescripts / > <none / > H → 0 0 0 1

[0096] As a reminder, this calibration step is given for information purposes only. Other calibrations are also compatible with the visualization process detailed below. In what has just been described, the calibration of the sensor holder can be carried out directly using a tool to record the positions of different characteristic points. The use of the calibrated calibration block naturally allows the integration of a coplanarity constraint which increases the accuracy of the calibration. It is the combination of the calibration of the calibration block and the sensor which gives a particularly interesting aspect to this calibration process. This results in great simplicity and speed of implementation as well as a significant improvement in the accuracy of the calibration compared to a conventional method.

[0097] After this preliminary step of calibrating the non-destructive testing device 10, the heart of the invention relating to the display of the data is detailed below.

[0098] The object of the invention is to visualize in augmented reality the signals directly inside the mechanical part in the field of non-destructive testing. To do this, the invention is based on two aspects. On the one hand, the elements of the scene must be presented in the form of holographic occlusions so that the brain can correctly interpret the position and depth of the holographic 3D surface inside the part. And on the other hand, a cutout is made within the occlusion to allow the visualization of this holographic 3D surface. The orientation of this cutout is linked to the relative position of the operator. More precisely, the orientation of the cutout 80 is linked to the location of the augmented reality visualization device.Thus, this cutout and the signal visualization surface constructed from the ultrasonic paths face the augmented reality visualization device and therefore the operator's eyes. The major advantage of the invention is to allow the user to understand the positioning of the signals inside the room and to perform real-time visualization of spatialized data in augmented reality.

[0099] There figure 10 schematically represents a flowchart of the steps of the method for real-time visualization of a non-destructive testing signal of a mechanical part according to the invention. The signal is emitted by the non-destructive testing device 10 described above. According to the invention, the visualization method comprises the following steps: Moving (step 200) the non-destructive testing sensor 3 over an examination area of the mechanical part 7; Simultaneously with the step 200 of moving the non-destructive testing sensor 3, emitting (step 210) from an emission point F along an emission axis and receiving (step 220) the signal by the sensor 3; Determining (step 230) a cutout 80 of an occlusion inside the mechanical part 7, the cutout 80 being centered around the emission point F; Determining (step 240) a signal display surface 81 constructed from the ultrasonic paths of the signal, this surface being located inside the cutout 80;Visualization (step 250), on the augmented reality visualization device 16, ∘ of a real view of the mechanical part 7, of the sensor holder 8 and of the non-destructive testing sensor 3, ∘ of a holographic 3D representation of the mechanical part 7', of the sensor holder 8' and of the non-destructive testing sensor 3', superimposed on the real view, ∘ of a holographic representation of the cutout 80 of the occlusion of the part and of the signal visualization surface 81, superimposed on the real view. ;

[0100] There figure 11 illustrates the display obtained in the augmented reality visualization device with the visualization method according to the invention. The cutout 80 corresponds to a 3D surface of substantially semi-hemispherical or ellipsoidal shape. In other words, the cutout 80 is a representation of a portion of a volume in the mechanical part 7, around the point F. The reference 85 represents a weld in the mechanical part 7.

[0101] The cutout 80 is a 3D surface that crosses the depth of the occlusion until it reaches an acquisition surface of the ultrasonic paths which corresponds to a surface 81 for viewing the ultrasonic signals. Thus, the cutout of the occlusion corresponds to the intersection between the 3D surface and the viewing surface.

[0102] For example, the cutout 80 is made in the holographic 3D representation of the mechanical part 7' as illustrated in figure 11 in the case where the occlusion is due at least partially to the superposition of the holographic 3D representation and the real view of the mechanical part.

[0103] The cutout 80 is predetermined so that the operator can visualize the paths of the ultrasonic signals inside the mechanical part in its 3D holographic representation 7', that is to say inside this cutout 80 in the part. In particular, the position of the ellipsoid is centered on a point of the sensor chosen as a reference. The rotation and the dimensions of the ellipsoid are calculated according to the position of the operator relative to the sensor and the part, in order to allow the visualization of all the signals by this same operator.

[0104] Thus, the visualization method makes it possible to obtain a display in the augmented reality visualization device 16. The operator sees, through the visualization device 16, the mechanical part 7, the sensor holder 8 and the sensor 3. In addition to these real objects, the display comprises a holographic 3D representation 7' of the mechanical part, a holographic 3D representation 8' of the sensor holder, and a holographic 3D representation 3' of the sensor. These three holographic 3D representations are superimposed on the display of the real objects. In addition to the real objects and these holographic 3D representations superimposed on the real objects, the display comprises the holographic 3D representation of the cutout of the occlusion 80 and of the intersection surface 81 or surface for viewing the ultrasonic signals, superimposed on the holographic 3D representations superimposed on the real objects.This surface corresponds to a quadric calculated as close as possible to the mesh formed by the ultrasonic paths.

[0105] In the rest of the document, we will refer to the following benchmarks: Reference of the augmented reality visualization system under the name RA reference: R RA. Reference of the target under the name QRCode reference: R QR.

[0106] The visualization method according to the invention first comprises a step 190 of calibrating the augmented reality visualization device 16 in the reference frame (R 0 ).

[0107] There figure 12 schematically represents the QR code implemented in step 190 of calibrating the augmented reality viewing device in the reference frame of the viewing method according to the invention. In order to define the position of the known elements in the world frame of reference in the frame of reference of the augmented reality viewing device, the augmented reality viewing device must be calibrated. The augmented reality viewing device establishes an anchor, i.e. a frame of reference at the location of the QR code. During calibration, this frame of reference is positioned in the world frame of reference, defined by the optical positioning system. Calibration makes it possible to establish the relationship between the two worlds.

[0108] In the world of augmented reality visualization device, the R QR mark associated with QR Code 86 is defined as shown in the figure 12 . To calibrate this R QR reference frame in the world reference frame, the three positions C QR , C 1 and C 2 in the world reference frame are acquired. Then, the QR Code 86 is acquired by the augmented reality visualization device and an internal transformation QR< T RA is carried out which, combined with the matrix O< T QR , allows the positioning in the RA reference frame of all the elements positioned and tracked in the world reference frame. In a similar way to what was detailed previously for the calibration step 1000, we have here the following relationships: T QR <none / > <mprescripts / > <none / > O = I QR <none / > <mprescripts / > <none / > O → J QR <none / > <mprescripts / > <none / > O → K QR <none / > <mprescripts / > <none / > O → C QR <none / > <mprescripts / > <none / > O → 0 0 0 1

[0109] The vector O< I QR in the world frame is: I QR <none / > <mprescripts / > <none / > O → = C QR C 1 → C QR C 1 →

[0110] The vector O< J QR in the world frame is: J QR <none / > <mprescripts / > <none / > O → = C QR C 2 → C QR C 2 →

[0111] The vector O< K QR in the world frame is: K <mprescripts / > <none / > O Q R → = I <mprescripts / > <none / > O Q R → ∧ J <mprescripts / > <none / > O Q R →

[0112] There figure 13 schematically represents the transformation matrices in the visualization method according to the invention.

[0113] Calibration step 1000 defines and describes how to obtain the following transformation matrices by calibration: the transformation matrix allowing to move from the sensor reference R F to the world landmark R O : F< T O . By inversion of this matrix, we also have O< T F ; the transformation matrix allowing to move from the part reference R P to the world landmark R O : P< T O . By inversion of this matrix, we also have O< T P . the transformation matrix allowing to move from the RA reference frame R RA to the world landmark R O : RA< T O . By inversion of this matrix, we also have O< T RA .

[0114] There figure 14 schematically represents these transformation matrices in the visualization method according to the invention.

[0115] We can position the origin F of sensor 3 in the part reference R P : P F <none / > <mprescripts / > <none / > P = T O <none / > <mprescripts / > <none / > P P F <none / > <mprescripts / > <none / > O

[0116] The informationI spatially linked to this point F such as the ultrasonic paths and the sector scan reconstructed from these and the physical signals are then localizable in the reference frame of the augmented reality visualization device R RA : P I <none / > <mprescripts / > <none / > RA = RA T O P I <none / > <mprescripts / > <none / > O

[0117] In order to clarify the different elements of the 3D scene to be visualized, we can subsequently designate as "main object" the 3D object containing the signals to be visualized by the operator and as "secondary objects" the 3D objects relative to which the main object is positioned (in particular the part and the sensor). Note that, in the case of augmented reality, each of the objects described above is visualized in the form of a hologram and is perfectly superimposed on a real object, thus creating an occlusion of this real object.

[0118] As already mentioned, the objective of the method of the invention is to allow the user to visualize the main object constituted from ultrasound data (sector scan for example) and to position it in 3D in augmented reality by giving the user the impression of visualizing it inside the room.

[0119] For a human to visually interpret the positioning of this main object in 3D visual space correctly, displaying the hologram of this main object is not enough. The hologram will appear to be floating and its spatial relationship with the volume of the room will be lost. In our case described here, the volume in question is the volume crossed by the ultrasonic signals used to construct this main object.

[0120] The invention consists of displaying the hologram of this main object accompanied by a set of graphic occlusions corresponding to the holograms of the secondary objects. The secondary objects are the part and the sensor (accompanied by its sensor holder) in particular. So that these occlusions do not completely hide the real objects, the occlusions advantageously have a medium level of transparency. The graphic occlusions of the part (possibly including a weld to be checked) and of the sensor are therefore produced with textures and colors having a percentage of transparency.

[0121] In order for the brain to interpret the main object as part of the internal integrity of the part (and possibly the weld), this main object must appear on the graphic occlusion of the part. In order to achieve this objective, a transparent opening of the graphic occlusions of the part is made. This is the cutout 80 of the occlusion formed by the virtual representation of the main object. This opening is centered on the sensor (more precisely the point F of the sensor, the point relative to which the main object to be displayed is located). This opening or cutout 80 is also a 3D object and part of the surface of its volume is merged with the surface of the main object.

[0122] In order not to disturb the operator with information located outside his field of view (for example the rear of the part), the visible data, and therefore the direction of the opening, are selected according to the position and orientation of the operator's eye. Thus, if the operator views the holograms from the front or the rear of the part, the holograms remain correctly oriented. This is made possible by calibrating the viewing device 16 with respect to the motion tracking device 1.

[0123] Step 240 of determining the intersection surface 81 or of visualizing the ultrasonic paths comprises the following steps: Calculation (step 241) of the paths taken by the signal emitted by the sensor 3; Creation (step 242) of a 3D mesh representative of the mechanical part 7, the sensor holder 8, the non-destructive testing sensor 3, and the paths taken by the signal; Mapping (step 243) of the paths taken by the signal onto the 3D mesh.

[0124] In other words, step 240 allows the visualization of a sector scan within the augmented reality visualization device 16. This is the intersection surface 81 of the plane containing the emission axis in the cutout 80 of the occlusion.

[0125] The signal viewing surface 81 is determined on the one hand from the characteristics of the sensor, such as its position relative to the part to be imaged and the characteristics of the signals emitted, in particular their orientations in the part. It is determined on the other hand from the cutout 80 of the part as being included in this cutout 80 and representing, for example, the bottom surface of this cutout 80.

[0126] The signal viewing surface 81 changes with each new signal acquisition, particularly if the sensor position and / or signal orientations change.

[0127] Thanks to the invention, it is thus possible for the operator to visualize the secondary objects (i.e. real objects: mechanical part 7 to be checked, the sensor holder 8 and the sensor 3), on which is superimposed the main object containing the signals to be visualized, i.e. the assembly formed by the cutout 80 and the surface 81.

[0128] In a particular embodiment, the steps of the method according to the invention are implemented by computer program instructions. Consequently, the invention also relates to a computer program on an information medium, this program being capable of being implemented in a computer, this program comprising instructions adapted to the implementation of the steps of a method as described above.

[0129] This program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form. The invention also relates to a computer-readable information medium, and comprising computer program instructions adapted to the implementation of the steps of a method as described above.

[0130] The information carrier may be any entity or device capable of storing the program. For example, the carrier may include a storage medium, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording medium, for example a floppy disk or a hard disk.

[0131] On the other hand, the information carrier may be a transmissible carrier such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio or by other means. The program according to the invention may in particular be downloaded from a network such as the Internet.

[0132] Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method according to the invention.

[0133] It will more generally appear to those skilled in the art that various modifications can be made to the embodiments described above, in light of the teaching which has just been disclosed to them. In the following claims, the terms used should not be interpreted as limiting the claims to the embodiments set out in the present description, but should be interpreted to include all equivalents which the claims are intended to cover by virtue of their wording and the prediction of which is within the reach of those skilled in the art based on their general knowledge.

Claims

1. A method for visualising, in real time, a signal for the non-destructive testing of a mechanical part (7), the signal being emitted by a non-destructive testing device comprising: - an optical motion tracking system (1) to which a reference coordinate system (R0) is tied, - a sensor holder (8), - a rigid body (2), - a non-destructive testing sensor (3) integral with the sensor holder (8) connected fixedly to the rigid body (2), - a computer (6), the visualisation method being executed by an augmented reality visualisation device (16) facing the mechanical part (7), to which an augmented reality coordinate system (RA) is tied, the method being characterised in that it comprises the following steps: - moving (200) the non-destructive testing sensor (3) over an examination area of the mechanical part (7); - simultaneously with the step (200) of moving the non-destructive testing sensor (3), emitting (210) the signal from a point of emission along an emission axis and receiving (220) the signal by way of the sensor; - determining (230), by way of the computer (6), a cut-out (80) of an occlusion inside the mechanical part (7), the cut-out (80) being centred around the point of emission; - determining (240), by way of the computer (6), a signal visualisation surface (81) constructed from the paths of the signal, this surface being located inside the cut-out (80); - visualising (250), on the augmented reality visualisation device: ▪ a real view of the mechanical part (7), of the sensor holder (8) and of the non-destructive testing sensor (3), ▪ a holographic 3D representation of the mechanical part (7'), of the sensor holder (8') and of the non-destructive testing sensor (3'), which are superimposed on the real view, ▪ a holographic representation of the cut-out (80) of the occlusion of the part and of the signal visualisation surface (81), which are superimposed on the real view, - the occlusion being created at least partially by superimposing the holographic 3D representation of the mechanical part (7') on the real view of the mechanical part (7), - the cut-out (80) of the occlusion passing through the depth of the mechanical part in its holographic 3D representation (7') until it reaches the signal visualisation surface (81), - the signal visualisation surface (81) being a quadric that is determined so as to correspond to a mesh formed by the paths of said signals propagating in the mechanical part.

2. The visualisation method according to claim 1, wherein the orientation of the cut-out (80) is tied to the location of the augmented reality visualisation device.

3. The visualisation method according to any one of claims 1 or 2, wherein the visualisation (250) of the holographic 3D representations is created in transparency.

4. The visualisation method according to any one of claims 1 to 3, wherein the step (240) of determining the visualisation surface (81) comprises the following steps: - computing (241) the paths travelled by the signal emitted by the sensor; - creating (242) a 3D mesh representative of the mechanical part (7), of the sensor holder (8), of the non-destructive testing sensor (3), and of the paths travelled by the signal; - plating (243) the paths travelled by the signal onto the 3D mesh.

5. The visualisation method according to any one of claims 1 to 4, comprising, beforehand, a step (190) of calibrating the augmented reality visualisation device (16) in the reference coordinate system (R0).

6. The visualisation method according to claim 5, comprising, prior to the step (190) of calibrating the augmented reality visualisation device (16) in the reference coordinate system (R0), a step of calibrating (1000) the non-destructive testing device.

7. A device (10) for visualising, in real time, a signal for the non-destructive testing of a mechanical part (7), the device (10) comprising a non-destructive testing device comprising: - an optical motion tracking system (1) to which a reference coordinate system (R0) is tied, - a sensor holder (8), - a first rigid body (2), - a non-destructive testing sensor (3) integral with the sensor holder (8) connected fixedly to the first rigid body (2), the sensor being designed to move over an examination area of the mechanical part (7), and to emit the signal from a point of emission and receive the signal along an emission axis, - a computer (6), the device (10) further comprising an augmented reality visualisation device (16) facing the mechanical part (7), to which an augmented reality coordinate system (RA) is tied, the computer (6) being configured to - determine a cut-out of the occlusion (80) of the mechanical part (7), the cut-out (80) being centred around the point of emission; - determine a signal visualisation surface (81) constructed from the paths of the signal, this surface being located inside the cut-out (80); and the augmented reality visualisation device (16) is configured to display: ▪ a real view of the mechanical part (7), of the sensor holder (8) and of the non-destructive testing sensor (3), ▪ a holographic representation of the mechanical part (7'), of the sensor holder (8') and of the non-destructive testing sensor (3'), which are superimposed on the real view, ▪ a holographic representation of the cut-out (80) of the occlusion of the part and of the signal visualisation surface (81), which are superimposed on the real view, - the occlusion being created at least partially by superimposing the holographic 3D representation of the mechanical part (7') on the real view of the mechanical part (7), - the cut-out (80) of the occlusion passing through the depth of the mechanical part in its holographic 3D representation (7') until it reaches the signal visualisation surface (81), - the signal visualisation surface (81) being a quadric that is determined so as to correspond to a mesh formed by the paths of said signals propagating in the mechanical part.

8. The device according to claim 7, further comprising a pointing device (4) containing a tip (42) and connected fixedly to a second rigid body (41), the pointing device (4) being able to determine the position of points on a surface.

9. A computer program containing instructions that cause the device according to claim 7 or 8 to execute the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium on which the computer program according to claim 9 is stored.