SURFACE INSPECTION PROCEDURES

Description

[0001] The present invention relates to the field of control and more particularly that of robotic control applications using an optical matrix sensor.

[0002] In industry, it is known to embed imaging devices such as optical matrix sensors on robots. For many applications, it is necessary to know precisely the positions of the effectors embedded on the robots. In the case of an optical sensor, the position of the optical center of the imaging device serves as an optical reference for the robot.

[0003] A common application example is surface inspection using thermography. For large parts, it is necessary to perform multiple acquisitions from different viewpoints using an infrared camera positioned on a robotic arm.

[0004] It is known to use matrix sensor control (e.g. in the infrared range) on composite parts, but mainly in the laboratory or in production on surfaces with relatively simple geometry. By relatively simple geometry we mean the absence of curvatures or variations in relief at the surface level.

[0005] The development of a method for controlling parts with complex geometry in industrial conditions requires mastery of: the area visualized in relation to the position and orientation of the matrix sensor embedded on an industrial robot, the design of the robot's trajectory while respecting the influencing parameters of the control process.

[0006] Controlling the viewing area is based on the precise positioning of the surface to be controlled at a given focusing distance between the surface and the optical centre of the camera, and according to a depth of field of the camera.

[0007] The design of the robot trajectory is often carried out by learning or by experimental methods directly on the part to be controlled.

[0008] FR 2 940 449 A1 and US 2002 / 169 586 A1 disclose similar methods.

[0009] A control method according to claim 1 is provided.

[0010] This process allows the robot to automatically define waypoints and, consequently, a predefined trajectory allowing it to successively move the camera to the acquisition points. The advantage of this process is that it can be carried out entirely in a virtual environment, whereas the usual procedure consists of creating a trajectory by experimental learning directly on the part.

[0011] According to the invention, the generation of the three-dimensional virtual model of the sharpness volume comprises the operations of: loading, in the virtual environment, a three-dimensional model of the device, generation of a truncated pyramid of which: o the summit is the optical center C, o the angular aperture (or aperture cone) is that of the optics, o two opposite faces respectively define a clear first plane PPN and a final clear shot DPN whose spacing corresponds to the depth of field PC of optics.

[0012] This three-dimensional virtual model of the sharpness volume allows the optical parameters to be represented in a simple and virtual way. It is directly linked to the optical characteristics.

[0013] According to the invention, the surface is located between the first net plane PPN and the last clear shot DPN of each unit model of three-dimensional virtual model of the sharpness volume.

[0014] This particular positioning is facilitated by the use of a three-dimensional virtual model of the sharpness volume, and makes it possible to guarantee a sharp image at each acquisition during surface control.

[0015] According to the invention, the generation of the three-dimensional virtual model of the sharpness volume comprises an operation of dividing said three-dimensional virtual model of the sharpness volume into a working area strictly included therein, and a peripheral overlapping area surrounding the working area. In the tiling operation, the unit models of the three-dimensional virtual model of the sharpness volume are distributed so as to overlap two by two in said peripheral areas.

[0016] Generating a work area makes it easier and faster to position the unit volumes of the sharpness volume. The work area allows you to distinguish an overlapping area in which the unit volumes overlap. This operation also allows the operator to have control over the desired level of overlap.

[0017] According to a particular feature, the position of each unit model of the three-dimensional virtual model of the sharpness volume is defined at least by the distance d between a singular point P of the three-dimensional virtual model of the surface to be controlled and its orthogonal projection onto the first net plane PPN or on the last net plane DPN. This feature allows the operator to have control over the distance between the device and the surface to be controlled. In fact, depending on the geometric characteristics of the surface to be controlled, it may be relevant to set the distance d under constraint. Controlling this distance makes it possible to control the spatial resolution of the images displayed.

[0018] According to another characteristic, the singular point P can be the barycenter of the three-dimensional virtual sharpness volume model.

[0019] According to a particular feature, in the tiling operation, the position of each unit model of the three-dimensional virtual model of the sharpness volume is defined by the angle between an axis X associated with the three-dimensional virtual model of the sharpness volume and the normal N to the surface of interest at the point of intersection of the axis X and the surface. The axis X is for example an axis of symmetry of the three-dimensional virtual model of the sharpness volume. This characteristic allows the operator to have control over the angular orientation of each unit model of the three-dimensional virtual model of the sharpness volume. This thus makes it possible to control the orientation of the shot on certain areas of the surface to be controlled.

[0020] The invention will be better understood and other details, characteristics and advantages of the invention will appear on reading the following description given by way of non-limiting example with reference to the drawings in which: There FIG.1 is an illustration of a camera mounted on a carrier robot by means of tooling. The FIG.2 is a perspective view of a camera mounted on a tool, and the associated sharpness volume. The FIG.3 is a side view of a camera mounted on a tool, and the associated sharpness volume. The FIG.4 is a perspective view of an example of a sharpness volume. The FIG.5 is a side view of the example volume of sharpness of the FIG.4 . There FIG.6 is a perspective view of an example of a surface to be controlled. The FIG.7 is an illustration of the surface of the FIG.7 after the paving operation. The FIG.8 is an illustration of the positioning of the camera, for each position of a unit model of the three-dimensional virtual model of the sharpness volume. The FIG.9 illustrates an example of positioning a unit model of the three-dimensional virtual model of the sharpness volume relative to a surface as a function of a distance. The FIG.10 illustrates an example of positioning a unit model of the three-dimensional virtual model of the sharpness volume relative to a surface as a function of an angle.

[0021] The present invention relates to a method of controlling a surface 1 of interest of a piece 2 by means of a camera 3 mounted on a carrier robot 4. Mounting the device 3 on the carrier robot 4 can for example be achieved using a tool 5 as illustrated by the FIG.1 .

[0022] The room2 can for example be a mechanical part.

[0023] The device 3 includes a sensor and optics associated with an optical center C , at an angular aperture and a depth of field PC and defining a volume of sharpness 6, as illustrated by the FIG.3 .

[0024] This process includes the operations of: loading, into a virtual design environment (e.g. a virtual computer-aided drawing environment), a three-dimensional virtual model of the surface 1 of interest, as illustrated by the FIG.6 , generation, in the virtual environment, of a three-dimensional virtual model of the sharpness volume 6, as illustrated by the FIG.2 , tiling, in the virtual environment, of the surface model 1of interest by means of a plurality of unit models of said three-dimensional virtual model of the sharpness volume 6, as illustrated in the FIG.7 , for each position of said unit models of the three-dimensional virtual model of the sharpness volume 6, calculation of the corresponding position, called acquisition, of the shooting device 3.

[0025] For each position of said unit models, it is then possible to automatically calculate passage points for the robot, and consequently a predefined trajectory allowing it to successively move the camera to the acquisition points.

[0026] For each position of a unit model of the three-dimensional virtual model of the sharpness volume 6, the position of the optical axis of the device 3of the corresponding shooting differs. Three optical axes are shown as examples, respectively Y, Y' And Y'' on the FIG.7 . They are not necessarily parallel to each other because the unit models are not necessarily oriented in the same way relative to the surface 1.

[0027] According to the invention, the generation of the three-dimensional virtual model of the sharpness volume 6 includes the operations of: loading of a three-dimensional model of the device 3, generation of a truncated pyramid of which: o the summit is the optical center C of the device 3, o the angular aperture is that of the optics, noted alpha, o two opposite faces respectively define a clear foreground PPN and a final clear shot DPN whose spacing corresponds to the depth of field PC of optics.

[0028] We can refer to the FIG.3 to locate the positions of the sharp foreground PPN and last clear shot DPN of the sharpness volume 6. The plans PPN And DPN are located on either side of a plane L (called the focusing plane) of a focusing distance. This operation thus makes it possible to import the geometric characteristics of the device 3 shooting in the virtual environment. The use of a truncated pyramid makes it easy to integrate the positions of the foreground in focus PPN and last clear shot DPN , and the angular aperture of the optics. The angular aperture is shown on the FIG.4 by a pyramidal cone with rectangular section, on which two angles denoted alpha1 and alpha2 can be defined. Angle alpha1 is defined by a first triangle comprising an edge of the rectangular section and the optical center C, angle alpha2 being defined by a second triangle adjacent to the first triangle and comprising an edge of the rectangular section and the optical center C.

[0029] According to the invention, the surface 1 is located, when tiling, between the first net plane PPN and the last clear shot DPN of each unit model of the three-dimensional virtual model of the sharpness volume 6, as illustrated on the FIG.9 et FIG.10 . This configuration makes it possible to guarantee, for each corresponding acquisition position of each unit model of the three-dimensional virtual model of the sharpness volume 6, generation, by the camera 3, of a clear image.

[0030] Geometric characteristics of the camera 3 are supplier data. We can cite: the pixel dimensions of an image provided by the device 3 : the number n h of horizontal pixels, the number n v of vertical pixels, the distance p between the centers of two adjacent pixels on the sensor, the focus distance I , the angular aperture of the optics.

[0031] The focus distance I is defined by the user. The geometry of the sharpness volume 6 can be adjusted by a calculation to manage overlap zones 7. Each position of a unit model of the three-dimensional virtual model of the sharpness volume 6 on the surface 1 corresponds to a shooting position.

[0032] Thus, during this operation, the generation of the three-dimensional virtual model of the sharpness volume 6 further includes an operation of dividing the three-dimensional virtual model of the sharpness volume 6 in a work area 8 strictly included in it, and a peripheral area 7 recovery 7 surrounding the work area 8. An example of sharpness volume 6 divided into a work area 8 and an overlap area 7 is illustrated in FIG.4 and in FIG.5 . Note that this is an example and the overlap areas may have different geometry and dimensions than shown in the FIG.4 et FIG.5 .

[0033] The geometry and dimensions of the work area 8 are governed by the geometry of the sharpness volume 6generated and a parameter relating to the desired percentage of overlap in each image. This parameter can be modulated by an operator. This division step makes it easy to manage the desired level of overlap between two acquisitions.

[0034] For each type of sensor, equations allow the dimensions of the working area to be calculated 8.

[0035] As an example, the following equations are given for applications in the visible domain and in particular in the case of the use of silver sensors.

[0036] Calculation of the working area at a distance I focusing is governed by equations (1) and (2) which allow the horizontal field of view HFOV and vertical field of view VFOV in millimeters to be calculated respectively: HFOV = l h f . l avec l h = . n h . p VFOV = l v f . l avec l v = n v . p n h being the number of horizontal pixels, n v the number of vertical pixels and p the distance between the centers of two adjacent pixels on the acquired images.

[0037] Depth of field PC is the difference between the distance of C in the last clear shot DPN, noted [ C,DPN ], and the distance from C in the clear foreground PPN, noted [ C,PPN ], as illustrated by equation (3): PC = C DPN − C PPN

[0038] The equations for determining distances [ C,DPN ] And [ C,PPN ] vary depending on the sensor. For example, for a film camera, these distances are calculated by equations (4) and (5) in which D is the diagonal of the sensor calculated by equation (6), c is the perimeter of the circle of confusion defined by equation (7), and H is the hyperfocal distance: C DPN = H . l H − l C PPN = H . l H + l D = n h . p 2 + n v . p 2 c = D 1730 H = f 2 N . c

[0039] The variables calculated by equations (4) to (8) may vary depending on the type of sensor used. They are given here as an example.

[0040] In case the operator has selected a non-zero overlap percentage, the positions of the sharpness volume 6 are defined so as to overlap two by two in the overlap zones 7 during the surface paving operation 1. An example of overlap between sharpness volumes 6 is illustrated in the FIG.7 .

[0041] Using a sharpness volume allows control of the viewing area and facilitates the integration of certain constraints such as the distance between the shooting device 3 and the surface 1, normality to the surface, centering on a particular point on the surface 1, control of work areas 8and recovery 7.

[0042] According to a particular characteristic, the position of each unit model of the three-dimensional virtual model of the sharpness volume 6 is defined at least by a distance d which can be the distance d1 between a singular point P of the three-dimensional model of the surface 1 of interest and its orthogonal projection onto the PPN plane, as shown in the FIG.9 . This distance can also be the distance d2 between this point P and its orthogonal projection on the last plane DPN as shown in the FIG.10 . According to an exemplary embodiment, in the tiling operation, the position of each unit model of the three-dimensional virtual model of the sharpness volume 6 can also be defined, by the angle between an axis Xassociated with the three-dimensional virtual model of the sharpness volume 6 and normal N on the surface 1 of interest at the point of intersection of the axis X and the surface 1. This scenario is illustrated by the FIG.10 . In the particular case of the FIG.9 , this angle is zero because the normal N is confused with the axis X. The axis X can for example be an axis of symmetry of the three-dimensional virtual model of the sharpness volume, as illustrated in the FIG.9 et FIG.10 . Indeed, it is essential to know this angular orientation because the position and orientation of the robot are given in relation to the part's reference frame.

Claims

1. A method for controlling a surface (1) of interest of a part (2) by means of a camera (3) intended to be mounted on a carrying robot (4), the camera (3) comprising a sensor and optics associated with an optical centre (C), with an angular aperture alpha and with a depth of field (PC) and defining a sharpness volume (6), the method comprising the following operations: a) loading, in a virtual design environment, a three-dimensional virtual model of the surface (1) of interest, b) generating, in the virtual environment, a three-dimensional virtual model of the sharpness volume (6), c) paving, in the virtual environment, the model of the surface (1) of interest by means of a plurality of unit models of said three-dimensional virtual model of the sharpness volume (6), d) for each position of said unit models (6), calculating the corresponding position, called the acquisition position, of the camera (3), the method wherein the generation of the three-dimensional model of the sharpness volume (6) comprises the operations of: - loading, in the virtual environment, a three-dimensional model of the camera (3) and its tooling (5), - generating a truncated pyramid of which: o the top is the optical centre (C), o the angular aperture is that of the optics noted alpha, o two opposing faces each define a first sharp plane (PPN) and a last sharp plane (DPN), the spacing of which corresponds to the depth of field (PC) of the optics, the method wherein the surface (1) is located between the first sharp plane (PPN) and the last sharp plane (DPN) of each unit model sharpness volume model (6), the method in which the generation of the three-dimensional virtual model of the sharpness volume (6) comprises an operation of dividing the sharpness volume model (6) into a working area (8) strictly included therein, and a peripheral overlapping area (7 ) surrounding the working area (8); and in that in the paving operation, the unit models of the sharpness volume model (6) are distributed so as to overlap two by two in said peripheral areas (7).

2. A method according to claim 1, wherein in the paving operation, the position of each unit model of the three-dimensional virtual model of volume of sharpness (6) is defined at least by the distance d between a singular point P of the three-dimensional model of the surface (1) of interest and its orthogonal projection on one of the planes (PPN) or (DPN).

3. A method according to claim 2, wherein the singular point P is the barycenter of the three-dimensional virtual model of volume of sharpness.

4. A method according to one of the preceding claims, wherein in the paving operation, the position of each unitary sharpness volume model (6) is defined by the angle between an X-axis associated with the sharpness volume model and the normal N to the surface (1) of interest at the point of intersection of the X-axis and the surface (1).

5. A method according to claim 4, wherein the X-axis is an axis of symmetry of the sharpness volume model (6).