To-be-detected object surface image acquisition device, image selection method and defect detection method

Abstract

The invention discloses a to-be-detected object surface image acquisition device, an image selection method and a defect detection method. The to-be-detected object surface image acquisition device comprises a projector, a diffusion plate, a reflector and a camera. The projector is suitable for projecting stripes; the diffusion plate is suitable for diffusing fringes projected by the projector to the surface of the to-be-measured object; the reflector moves between a first position and a second position relative to the to-be-measured object, and at the first position, the reflector shields a light path from the projector to the diffusion plate. And at the second position, the reflecting mirror avoids a light path from the projector to the diffusion plate. The camera is suitable for receiving a signal reflected by the surface of the to-be-measured object after the diffusion plate enters the surface of the to-be-measured object so as to obtain a phase deflection technology (PMD) image, or receiving a signal diffused by the surface of the to-be-measured object after the reflection mirror enters the surface of the to-be-measured object so as to obtain a fringe projection profile measurement (FPP) image. A phase deflection technology and a fringe projection profile measurement technology are fused in one set of defect detection system, rapid switching of two three-dimensional measurement defect detection technologies is realized for products with complex materials and different surface roughness, and the defect detection system is ensured to be high in efficiency, high in detection capability, wide in applicability and high in detection rate.

Description

Technical Field

[0001] The present invention relates to the field of image acquisition, and in particular to a device for acquiring a surface image of an object to be tested, an image selection method and a defect detection method. Background Art

[0002] FPP technology can form stripes on rough surfaces (i.e. sand surface) to detect defects, but it is difficult to detect shiny objects and is prone to overexposure. However, products on the market that require defect detection often have both sand surface materials and shiny surface materials, which makes it difficult to use FPP for full inspection; at the same time, products of a single material such as building materials, coatings, metal parts, etc., if there are glue-like high-reflective defects on them, using only FPP detection will result in missed detection in this solution. Summary of the invention

[0003] The purpose of the present invention is to overcome the above-mentioned defects or problems existing in the background technology and to provide a device for acquiring an image of the surface of an object to be tested, an image selection method and a defect detection method.

[0004] To achieve the above objectives, the present invention and its preferred embodiments adopt the following technical solutions, but the embodiments are not limited to the following solutions:

[0005] Solution 1: A device for acquiring a surface image of an object to be measured, comprising:

[0006] a projector adapted to project the stripes;

[0007] A diffusion plate, which is suitable for diffusing the fringes projected by the projector onto the surface of the object to be measured;

[0008] a reflector, which moves relative to the object to be measured between a first position and a second position, wherein in the first position, the reflector blocks the light path from the projector to the diffuser and reflects the stripes projected by the projector to the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured, and in the second position, the reflector avoids the light path from the projector to the diffuser;

[0009] The optical path distance from the stripes of the projector to the diffuser is equal to the optical path distance from the stripes of the projector to the surface of the object to be measured when the reflector is in the first position;

[0010] and

[0011] The camera is suitable for receiving a signal reflected by the surface of the object to be measured after being incident on the diffuser to obtain a phase deflection technology (PMD) image, or receiving a signal diffused by the surface of the object to be measured after being incident on the surface of the object to be measured from the reflector to obtain a fringe projection profile measurement (FPP) image.

[0012] Solution 2, based on Solution 1, the reflector is adapted to slide or rotate to be located at a first position and a second position.

[0013] Solution 3, based on Solution 1, further includes a driving member, and the driving member is adapted to drive the reflector to move.

[0014] Solution 4, an apparatus for obtaining an image of a surface of a to-be-measured object, includes:

[0015] A first projector adapted to project fringes;

[0016] A second projector adapted to project fringes;

[0017] A diffuser adapted to diffuse the fringes projected by the first projector onto the surface of the to-be-measured object;

[0018] A reflector adapted to reflect the fringes projected by the second projector onto the surface of the to-be-measured object in a direction perpendicular to the surface of the to-be-measured object;

[0019] A first camera adapted to receive the signal reflected from the surface of the to-be-measured object after the light incident from the diffuser onto the surface of the to-be-measured object, so as to obtain a Phase Measuring Deflectometry (PMD) image;

[0020] A second camera adapted to receive the signal diffused from the surface of the to-be-measured object after the light incident from the reflector onto the surface of the to-be-measured object, so as to obtain a Fringe Projection Profilometry (FPP) image.

[0021] Solution 5, obtain a Phase Measuring Deflectometry (PMD) image and a Fringe Projection Profilometry (FPP) image by using an apparatus for obtaining an image of a surface of a to-be-measured object according to any one of Solutions 1 to 3, or an apparatus for obtaining an image of a surface of a to-be-measured object according to Solution 4;

[0022] Select the Phase Measuring Deflectometry (PMD) image and / or the Fringe Projection Profilometry (FPP) image based on the contrast of the imaging.

[0023] Solution 6, based on Solution 5, the obtaining of the contrast of the imaging includes the following steps:

[0024] Collect one Phase Measuring Deflectometry (PMD) image and one Fringe Projection Profilometry (FPP) image respectively;

[0025] Obtain the first contrast and root mean square error (RMSE) of the Phase Measuring Deflectometry (PMD) image and the Fourier Ptychographic Profilometry (FPP) image: Extract the gray values of the middle row in the row direction of the vertical stripe image, fit a sine curve, and calculate the RMSE of the PMD image and the FPP image; Crop the PMD image and the FPP image to a size of 2Π periods, and obtain the adjacent maximum gray value m and minimum gray value n within one period of the PMD image and the FPP image respectively, to obtain the first contrast.

[0026] Calculate the contrast of the imaging: If m > 255 or RMSE > the first threshold, discard the first contrast k of the corresponding image. If only one image satisfies m ≤ 255 and RMSE ≤ the first threshold, directly use this k value as the contrast of the imaging. If the contrasts of both images satisfy m ≤ 255 and RMSE ≤ the first threshold, take the average of the first contrasts of the PMD image and the FPP image as the contrast of the imaging.

[0027] Scheme Seven, based on Scheme Six, the first threshold is 20.

[0028] Scheme Eight, based on Scheme Five, when the contrast of the imaging is greater than the second threshold, select PMD imaging; when the contrast of the imaging is less than the third threshold, select the FPP image; when the contrast of the imaging is less than or equal to the second threshold and greater than or equal to the third threshold, select both the PMD image and the FPP image simultaneously.

[0029] Scheme Nine, based on Scheme Eight, the second threshold is 20 and the third threshold is 5.

[0030] Scheme Ten, a defect detection method, including an image selection method as described in any one of Schemes Five to Nine.

[0031] When only selecting the PMD image as the detection image of the defect, take the detection result of the PMD image as the final detection result.

[0032] When only selecting the FPP image as the detection image of the defect, take the detection result of the FPP image as the final detection result.

[0033] When selecting both the PMD image and the FPP image as the detection images of the defect, when both detect defects, it is judged as a real defect.

[0034] From the above description of the present invention and its preferred embodiments, it can be seen that, compared with the prior art, the technical solution of the present invention and its preferred embodiments have the following beneficial effects due to the adoption of the following technical means:

[0035] 1. In solution 1 and its preferred embodiment, a device for acquiring an image of the surface of an object to be measured includes a projector, a diffuser, a reflector and a camera.

[0036] The projector is suitable for projecting stripes; the diffuser is suitable for diffusing the stripes projected by the projector to the surface of the object to be measured; the reflector moves between a first position and a second position relative to the object to be measured. In the first position, the reflector blocks the light path from the projector to the diffuser, and reflects the stripes projected by the projector to the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured. Compared with the traditional symmetrical layout, it can ensure a high signal-to-noise ratio and good contrast of the image, and after passing through the reflector, the light intensity decreases, and it is not easy to overexpose the high-reflection area on the image. In the second position, the reflector avoids the light path from the projector to the diffuser; the light path distance from the projector stripes to the diffuser is equal to the light path distance from the projector stripes to the surface of the object to be measured when the reflector is in the first position, so that no focusing is required when switching. The camera is suitable for receiving the signal reflected by the surface of the object to be measured after the diffuser is incident on the surface of the object to be measured to obtain a phase deflection technology PMD image, or receiving the signal diffused by the surface of the object to be measured after the reflector is incident on the surface of the object to be measured to obtain a fringe projection profilometry FPP image. The use of projector + diffuser to replace the programmable light source in PMD technology is cost-effective, can save space well, and is easy to integrate. The phase deflection technology and fringe projection profilometry are integrated into a defect detection system. For products with complex materials and different surface roughness, the two three-dimensional measurement defect detection technologies can be quickly switched to ensure that the defect detection system is efficient, has high detection capabilities, wide applicability, and high detection rate. For FPP technology, the image signal-to-noise ratio is improved and it is not easy to overexpose; for PMD technology, due to the vertical projection, there is no tilt angle between the optical axis and the projection plane, and there is no deformation, so the fringe image has no transmission distortion. This device saves at least one workstation for product defect detection, and is convenient for image collection, effectively saving time for industrial assembly line detection, improving efficiency, and reducing the cost of product defect detection. It can realize two three-dimensional measurement defect detection technologies without refocusing under the premise of using only one projector and camera, and the fringe focus is clear, without distortion, and the image quality is good, saving hardware and acquisition costs.

[0037] 2. In the second solution and its preferred embodiment, the reflector is suitable for sliding or rotating to be located at the first position and the second position, which is simple and convenient.

[0038] 3. Solution 3 and its preferred embodiment further include a driving member, which is suitable for driving the reflector to move, so as to realize automatic image acquisition.

[0039] 4. In Solution 4 and its preferred embodiments, an apparatus for acquiring an image of the surface of an object to be measured includes a first projector, a second projector, a diffuser, a reflector, a first camera, and a second camera. The first projector is adapted to project fringes; the second projector is adapted to project fringes; the diffuser is adapted to diffuse the fringes projected by the first projector onto the surface of the object to be measured; the reflector is adapted to reflect the fringes projected by the second projector onto the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured; the first camera is adapted to receive the signal reflected from the surface of the object to be measured after the signal incident on the surface of the object to be measured from the diffuser, so as to obtain a Phase Measuring Deflectometry (PMD) image; the second camera is adapted to receive the signal diffused from the surface of the object to be measured after the signal incident on the surface of the object to be measured from the reflector, so as to obtain a Fringe Projection Profilometry (FPP) image. The form of using a projector + diffuser has a high cost performance, can well save space, and is convenient for system integration. By integrating the Phase Measuring Deflectometry and the Fringe Projection Profilometry in a set of defect detection systems, for products with complex materials and different surface roughnesses, it is ensured that the defect detection system has high efficiency, high detection ability, wide applicability, and high detection rate. For the Fringe Projection Profilometry (FPP) technology, the signal-to-noise ratio of the image is improved and it is not easy to overexpose; for the Phase Measuring Deflectometry (PMD) technology, there is no transmission distortion in the fringe image. In this embodiment, the two methods can be focused independently, and there is no need to switch the position of the reflector.

[0040] 5. In Solution 5 and its preferred embodiments, an image selection method is to acquire a Phase Measuring Deflectometry (PMD) image and a Fringe Projection Profilometry (FPP) image through the above-mentioned apparatus for acquiring an image of the surface of an object to be measured; and select the Phase Measuring Deflectometry (PMD) image and / or the Fringe Projection Profilometry (FPP) image based on the contrast of the imaging, so as to select a suitable image for defect detection according to the roughness of different surfaces of the object to be measured, prevent missed detections, and ensure the detection rate. Of course, in addition to being used for defect detection, it can also be used for other purposes, such as surface construction, etc.

[0041] 6. In Solution 6 and its preferred embodiments, the acquisition of the contrast of the imaging includes the following steps:

[0042] Collect one Phase Measuring Deflectometry (PMD) image and one Fringe Projection Profilometry (FPP) image respectively;

[0043] Obtain the first contrast and root mean square error (RMSE) of the Phase Measuring Deflectometry (PMD) image and the Fourier Ptychographic Profilometry (FPP) image: Extract the gray values of the middle row in the row direction of the vertical stripe image, fit a sine curve, and calculate the RMSE of the PMD image and the FPP image; Crop the PMD image and the FPP image to a size of 2Π periods, and obtain the adjacent maximum gray value m and minimum gray value n within one period of the PMD image and the FPP image respectively to obtain the first contrast.

[0044] Calculate the contrast of the imaging: If m > 255, it means the image is overexposed, or if RMSE > the first threshold, it means the image does not have good sinusoidality, then the first contrast k of the corresponding image is discarded. If only one image satisfies m ≤ 255 and RMSE ≤ the first threshold, then directly use this k value as the contrast of the imaging. If the contrasts of both images satisfy m ≤ 255 and RMSE ≤ the first threshold, then take the average value of the first contrasts of the PMD image and the FPP image as the contrast of the imaging.

[0045] 7. In Solution Seven and its preferred embodiments, the first threshold is 20.

[0046] 8. In Solution Eight and its preferred embodiments, since the contrast of the bright surface material is higher, when the contrast of the imaging is greater than the second threshold, select the PMD image as the defect detection image, which is more accurate; when the contrast of the imaging is less than the third threshold, select the FPP image as the defect detection image, which is more accurate; when the contrast of the imaging is less than or equal to the second threshold and greater than or equal to the third threshold, select both the PMD image and the FPP image as the defect detection images to prevent over-screening.

[0047] 9. In Solution Nine and its preferred embodiments, the second threshold is 20 and the third threshold is 5.

[0048] 10. In Solution Ten and its preferred embodiments, for a defect detection method including the above image selection method, when only selecting the PMD image as the defect detection image, use the detection result of the PMD image as the final detection result, which is more accurate; when only selecting the FPP image as the defect detection image, use the detection result of the FPP image as the final detection result, which is more accurate; when simultaneously selecting the PMD image and the FPP image as the defect detection images, when both detect defects, it is judged as a real defect to prevent over-screening. Description of the Drawings

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0050] Figure 1 It is a schematic diagram of an implementation manner of the device for obtaining the surface image of the measured object in the first embodiment;

[0051] Figure 2 It is a schematic diagram of another implementation manner of the device for obtaining the surface image of the measured object in the first embodiment;

[0052] Figure 3 It is a framework diagram of the PMD technology system in the first embodiment;

[0053] Figure 4 It is a framework diagram of the FPP technology system in the first embodiment;

[0054] Figure 5 It is a flowchart of the defect detection method in the first embodiment.

[0055] Main reference numerals description:

[0056] Projector 1; Diffuser 2; Reflector 3; Camera 4; Specific implementation manners

[0057] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are the preferred embodiments of the present invention and should not be regarded as excluding other embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0058] In the claims, the description and the above-mentioned accompanying drawings of the present invention, unless otherwise clearly defined, when using terms such as "first", "second" or "third", etc., are all used to distinguish different objects and not for describing a specific order.

[0059] In the claims, the description and the above-mentioned drawings of the present invention, unless otherwise clearly defined, for orientation terms, such as the use of terms "center", "horizontal", "longitudinal", "level", "vertical", "top", "bottom", "inner", "outer", "upper", "lower", "front", "rear", "left", "right", "clockwise", "counterclockwise", etc. to indicate the orientation or position relationship are based on the orientation and position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, so it should not be construed as limiting the specific protection scope of the present invention.

[0060] In the claims, the description and the above-mentioned drawings of the present invention, unless otherwise clearly defined, if the terms "fixed connection" or "fixedly connected" are used, they should be understood in a broad sense, that is, any connection method without displacement relationship and relative rotation relationship between the two, that is to say, it includes non-detachable fixed connection, detachable fixed connection, being integrated into one body, and being fixedly connected through other devices or elements.

[0061] In the claims, the description and the above-mentioned drawings of the present invention, if the terms "comprise", "have" and their variants are used, are intended to mean "including but not limited to".

[0062] Embodiment 1:

[0063] Reference Figures 1 - 5 , an apparatus for obtaining an image of the surface of an object to be measured, comprising a projector 1, a diffuser 2, a reflector 3, and a camera 4.

[0064] The projector 1 is adapted to project fringes, and the fringes are finally projected onto the surface of the object to be measured. The camera 4 receives the fringe image reflected or diffused by the surface of the object to be measured, and determines whether there are defects by judging whether the fringes are distorted, or performs three-dimensional construction of the surface of the object to be measured according to the distortion.

[0065] The diffuser 2 is adapted to diffuse the fringes projected by the projector 1 onto the surface of the object to be measured;

[0066] The reflector 3 moves between a first position and a second position relative to the object to be measured. In the first position, the reflector 3 blocks the optical path from the projector 1 to the diffuser 2, so as to reflect the fringes projected by the projector 1 onto the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured. The perpendicular reflection of the fringes on the product can ensure high signal-to-noise ratio and good contrast of the image compared with the traditional symmetric layout method. And after passing through the reflector 3, the light intensity decreases, and it is not easy to overexpose the high-reflection area on the image. In the second position, the reflector 3 avoids the optical path from the projector 1 to the diffuser 2;

[0067] The mirror 3 is adapted to slide or rotate to be in a first position and a second position. For example, the mirror 3 can move vertically. Through an automatic adjustment device installed in the vertical direction, the mirror 3 is clamped and moved upward until it reaches the second position where it is out of the projection range of the projector 1. The sine fringes projected by the projector 1 directly hit the diffuser plate 2. Conversely, when it moves to the first position, the mirror 3 reflects the fringes projected by the projector 1 to the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured. In this process, the mirror 3 is driven by a driving member to facilitate automation. In other embodiments, the mirror 3 can be switched to a transmissive mirror by rotating a two-hole type automatic electric filter wheel by 180°. The transmissive mirror is adapted to allow the fringes to pass through to the diffuser plate 2. By switching the mirror 3 and the transmissive mirror in this way, the switching speed is fast and the efficiency is high.

[0068] Moreover, the optical path distance from the fringes of the projector 1 to the diffuser plate 2 is equal to the optical path distance from the fringes of the projector 1 to the surface of the object to be measured when the mirror 3 is in the first position. That is, it is ensured that when the mirror 3 is in the second position, the distance from the projector 1 to the diffuser plate 2 is equal to the distance from the projector 1 to the surface of the object to be measured when the mirror 3 is in the first position, so that manual focusing is not required during switching, ensuring the final obtained image quality. During implementation, ensuring that S1 and S2 in the figure are equal can achieve this.

[0069] The camera 4 is adapted to receive the signal reflected from the surface of the object to be measured after the diffuser plate 2 is incident on the surface of the object to be measured to obtain a Phase Measuring Deflectometry (PMD) image, or receive the signal diffused from the surface of the object to be measured after being incident from the mirror 3 on the surface of the object to be measured to obtain a Fringe Projection Profilometry (FPP) image.

[0070] Principle of Phase Measuring Deflectometry (PMD): In the prior art, generally, structured light such as sine fringes and binary fringes is projected through an LCD display / programmable light source. The camera 4 receives the sine fringes reflected from the surface of the mirror to be measured, and the fringes will be distorted at the defective places. The camera 4 collects the distorted fringes, and then uses the phase slope mapping relationship to extract the wrapped phase map, as well as the standard intensity map (gray average), specular reflection map, diffuse reflection map, and gloss ratio map; the phase is unwrapped to obtain the absolute phase map, and then 2.5D feature maps such as gradient map and curvature map can be obtained. The gradient information of the mirror to be measured is calculated from the distorted fringe map. Finally, according to the surface shape reconstruction algorithm of the gradient data, the surface shape of the mirror to be measured is reconstructed. The phase measuring deflectometry has the advantages of simple structure and large dynamic range, high detection accuracy, easy implementation of in-situ or on-line detection without contacting the component, simple calibration process, and can obtain two-dimensional and three-dimensional data information at the same time. It can effectively detect products with complex materials and different surface roughnesses. In this embodiment, an innovative form of using the projector 1 plus the diffuser plate 2 as the light source can also achieve the fringe projection and defect detection effects of the programmable light source.

[0071] Technical principle of fringe projection profilometry (FPP): By projecting phase-shifted fringes onto the surface of the object to be measured, the fringes will deform due to the height variation of the object surface. By collecting these deformed fringe images and processing these images to obtain the phase representing the height information of the object, the three-dimensional contour information of the object can be finally calculated.

[0072] The main difference between the two is that the fringe projection of FPP is imaged on the surface of the object to be measured, while the PMD fringe is imaged on the surface of the light source. This means that, on the one hand, the projector 1 of the FPP technology is difficult to detect bright surface objects and is prone to overexposure; and it has no parallax, so it is difficult to replace PMD. On the other hand, the PMD technology is difficult to form fringes on a rough surface (i.e., sand surface), so it is difficult to detect objects made of sand surface materials and cannot replace FPP. However, the product inspection requirements in modern manufacturing are high and the difficulty is great. The products to be inspected often have both bright surfaces and sand surfaces. In this embodiment, defect detection is carried out based on the fusion of phase deflectometry (PMD) and fringe projection (FPP) technologies. This solution can greatly improve the inspection efficiency, reduce the inspection cost, and ensure the quality of defect detection.

[0073] In this embodiment, two layout methods of the device for obtaining the surface image of the object to be measured are provided:

[0074] ① Refer to Figure 1 , the projector 1 projects fringes in the horizontal direction, the diffuser 2 forms an angle of 45° with the horizontal plane, and the light diffused by the diffuser 2 hits the surface of the object to be measured. The fringes of the highly reflective product are reflected by the mirror surface and received by the camera 4. This method is the phase deflectometry (PMD), which can efficiently detect the defects of bright surface objects.

[0075] The reflector 3 forms an angle of 45 degrees with the horizontal plane. The fringes projected by the projector 1 are reflected by the reflector 3 and imaged on the surface of the object to be measured. After the fringes are imaged on the surface of the object to be measured, they are diffused and received by the camera 4. This method is the fringe projection profilometry (FPP), which has the advantages of non-contact, high precision, fast measurement speed, good measurement flexibility, etc., and can efficiently detect the defects of sand surface objects.

[0076] ② Refer to Figure 2, the projector 1 projects stripes in a direction inclined upward relative to the horizontal plane (the stripes projected by the projector 1 at an angle α with the surface of the object to be measured). The diffuser 2 is at a 90-degree angle to the projection direction of the projector 1. The stripes projected by the projector 1 are perpendicular to the diffuser 2, and the stripes are perpendicularly projected onto the diffuser 2. The perpendicular projection can also be understood as an equidistant projection, which can be approximately regarded as an orthogonal projection. Under orthogonal projection, the size of the object does not change with depth, so there will be no deformation caused by perspective scaling. The main source of perspective distortion is the tilt angle between the optical axis and the projection plane, which causes the projected image to have a trapezoidal deformation. When projecting perpendicularly, the optical axis is orthogonal to the projection plane, and the projection ratio of each point remains the same, without deformation caused by perspective projection. Therefore, this method enables the stripes to be globally focused, without perspective distortion, and subsequent distortion correction is not required, which has more advantages. The light diffused by the diffuser 2 hits the surface of the object to be measured, and the stripes of the highly reflective product are received by the camera 4 after specular reflection.

[0077] Reference Figure 3 、 Figure 4 , the mirror 3 is inclined to the horizontal plane, and the mirror 3 is perpendicular to the angular bisector of the α angle. The stripes projected by the projector 1 are imaged on the surface of the object to be measured after being reflected by the reflective mirror 3, and the stripes are perpendicularly reflected on the surface of the object to be measured, and finally its diffuse reflection component is received by the camera 4.

[0078] The above device for obtaining the surface image of the object to be measured is used to obtain the phase deflection technique PMD image and the fringe projection profilometry FPP image, and is used for defect detection of products.

[0079] Reference Figure 5 , before product defect detection, it is necessary to first select which image to perform defect detection on. Specifically, the image selection method includes the following steps:

[0080] First, collect one phase deflection technique PMD image and one fringe projection profilometry FPP image respectively, and extract the ROI region, such as the region where the defect is located;

[0081] Obtain the first contrast and root mean square error of the phase deflection technique PMD image and the fringe projection profilometry FPP image: Specifically, extract the gray values of the middle row in the row direction of the vertical stripe image, fit a sine curve, and calculate the root mean square error RMSE of the phase deflection technique PMD image and the fringe projection profilometry FPP image; Crop the phase deflection technique PMD image and the fringe projection profilometry FPP image into images of 2Π cycle size, and respectively obtain the maximum gray value m (i.e., the gray value of the bright stripe) and the minimum gray value n (i.e., the gray value of the dark stripe) adjacent within one cycle of the phase deflection technique PMD image and the fringe projection profilometry FPP image, to obtain the first contrast It is used to represent the surface roughness of the object to be measured; among them, the phase deflection technology (PMD) image and the fringe projection profilometry (FPP) image are sinusoidal structured light images. The period of the sinusoidal fringe is 2Π. There are several periods in one image, and the maximum and minimum gray values (i.e., amplitudes) of each period are slightly different. Therefore, divide the periods and select one period.

[0082] Calculate the contrast of the image: If m > 255, it means the image is overexposed, and then the first contrast k of this image is discarded. If RMSE > the first threshold, as in this embodiment, the first threshold is 20, and RMSE > 20, it means the image does not have good sinusoidality, and then the first contrast k of this image is discarded. If the image satisfies m ≤ 255 and RMSE ≤ the first threshold, the corresponding first contrast k can be used for calculating the contrast of the image. If only one image satisfies m ≤ 255 and RMSE ≤ the first threshold, directly use the k value corresponding to this image as the contrast of the image. If the contrasts of both images satisfy m ≤ 255 and RMSE ≤ the first threshold, then take the average value of the first contrasts k of the phase deflection technology (PMD) image and the fringe projection profilometry (FPP) image as the contrast of the image.

[0083] Select the phase deflection technology (PMD) image and / or the fringe projection profilometry (FPP) image based on the contrast of the image as the detection image for defects, and obtain the final detection result: Specifically, when the contrast of the image is greater than the second threshold, select the PMD image as the detection image for defects. According to the prior experiments, the contrast of the bright surface material is higher. In this embodiment, the second threshold is set to 20. When the surface roughness, that is, the contrast of the image, is greater than 20, it is determined as a high-reflectivity surface. The electric filter wheel receives the trigger signal and turns to the position of the transmission mirror, and only collects the PMD image. Run the PMD defect detection algorithm, that is, use the four-step phase-shifting method to extract the wrapped phase and the exponential time method to unwrap the phase, and take the detection result of the phase deflection technology (PMD) image as the final detection result.

[0084] When the contrast of the image is less than the second threshold and further less than the third threshold, it is determined as a low-roughness frosted surface, and select the fringe projection profilometry (FPP) image as the detection image for defects; in this embodiment, the third threshold is 5; the electric filter wheel receives the trigger signal and turns to the position of the mirror 3, and only collects the FPP image. Run the FPP defect detection algorithm, that is, use the four-step phase-shifting method to extract the wrapped phase and the traditional Gray code combined with the phase-shifting method to unwrap the phase, and take the detection result of the fringe projection profilometry (FPP) image as the final detection result; when the contrast of the image is less than or equal to the second threshold and greater than or equal to the third threshold, select the phase deflection technology (PMD) image and the fringe projection profilometry (FPP) image as the detection images for defects at the same time. When defects are detected in both, it is judged as a real defect. When only one defect is detected, it is judged as a false positive.

[0085] This algorithm automatically determines which working mode to use based on the contrast of the imaging, ensuring the defect detection effect and detection rate of products with diverse materials, and at the same time avoiding missed detections and over-screening.

[0086] Among them, the first threshold, the second threshold, and the third threshold are calculated based on the contrast of the surfaces with different roughnesses of two types of watch middle frame products on the market. The contrast of the bright surface products is high, basically far greater than 20; the contrast of the sanded surface products is low, between 1.8 and 2.2, and the values of 3 or 4 occasionally appear, so the third threshold is set to 5. It should be understood that the first threshold, the second threshold, and the third threshold can be set for the products to be tested with different roughnesses. For example, before setting, the roughness of the product can be tested first, that is, the contrast is calculated in the first step, and then appropriate thresholds are set according to different products to be tested.

[0087] In summary, this embodiment has the following beneficial effects:

[0088] 1) Using the form of the projector 1 + diffuser 2 to replace the programmable light source in the PMD technology, it has high cost performance; and because the programmable light source is large in volume and requires a light source controller, it is not convenient for system integration. This method can well save space and is convenient for system integration.

[0089] 2) Integrating the phase deflection technology and the fringe projection profilometry in a set of defect detection systems, for products with complex materials and different surface roughnesses, realizing the rapid switching of two three-dimensional measurement defect detection technologies, ensuring that the defect detection system has high efficiency, high detection ability, wide applicability, and high detection rate. It fully exerts the defect detection ability of PMD for highly reflective, mirror, and mirror-like objects and FPP for sanded surface material objects. This solution integrates two technologies, which can greatly improve the defect detection effect of products with different materials on the market, ensure the detection rate, and avoid missed detections. For the FPP technology, the image signal-to-noise ratio is improved and it is not easy to overexpose; for the PMD technology, there is no transmission distortion in the fringe image. The good image quality further improves the defect detection efficiency and ensures the effect and detection rate of the defect test. It can be widely applied to the manufacturing industry, the automotive industry including welding detection and body detection, the electronic manufacturing industry including PCB detection and semiconductor detection, medical devices, the consumer goods industry, packaging and printing, etc. The coverage range is wide and the application prospect is high.

[0090] 3) This device saves at least one station for the defect detection of products, and is convenient for image acquisition, effectively saving the time of industrial production line detection, improving efficiency, and at the same time reducing the cost of product defect detection.

[0091] 4) It can implement two three-dimensional measurement defect detection technologies without refocusing, with clear fringe focus, no distortion, good image quality, and cost savings in hardware and acquisition, by only using one projector 1 and one camera 4. It can automatically identify the image acquisition method (PMD / FPP) and achieve fast switching, and also has the ability to detect product defects with different surface roughnesses. It can achieve automated and intelligent defect detection, automatically determine which working mode to use based on the contrast of the imaging, and quickly switch to the corresponding technology for image acquisition.

[0092] Compared with the prior art, the present embodiment has the following beneficial effects:

[0093] In an exemplary embodiment, a device for obtaining an image of the surface of an object to be measured includes a projector 1, a diffuser 2, a reflector 3, and a camera 4.

[0094] The projector 1 is adapted to project fringes; the diffuser plate 2 is adapted to diffuse the fringes projected by the projector 1 onto the surface of the object to be measured; the mirror 3 moves between a first position and a second position relative to the object to be measured. In the first position, the mirror 3 blocks the optical path from the projector 1 to the diffuser plate 2 and reflects the fringes projected by the projector 1 onto the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured. Compared with the traditional symmetric layout method, it can ensure high image signal-to-noise ratio and good contrast. After passing through the mirror 3, the light intensity decreases, and it is not easy to overexpose the high-reflection area on the image. In the second position, the mirror 3 avoids the optical path from the projector 1 to the diffuser plate 2; the optical path distance from the fringes of the projector 1 to the diffuser plate 2 is equal to the optical path distance from the fringes of the projector 1 to the surface of the object to be measured when the mirror 3 is in the first position, so that no focusing is required during switching. The camera 4 is adapted to receive the signal reflected from the surface of the object to be measured after the diffuser plate 2 is incident on the surface of the object to be measured to obtain a phase deflection technology (PMD) image, or receive the signal diffused from the surface of the object to be measured after the mirror 3 is incident on the surface of the object to be measured to obtain a fringe projection profilometry (FPP) image. Using the form of the projector 1 + diffuser plate 2 instead of the programmable light source in the PMD technology has high cost performance, can save space well, and is convenient for system integration. Integrating the phase deflection technology and the fringe projection profilometry in a set of defect detection systems can realize the rapid switching of the two three-dimensional measurement defect detection technologies for products with complex materials and different surface roughnesses, ensuring that the defect detection system has high efficiency, high detection ability, wide applicability and high detection rate. For the FPP technology, the image signal-to-noise ratio is improved and it is not easy to overexpose; for the PMD technology, since it is a vertical projection, there is no tilt angle between the optical axis and the projection plane, and there is no deformation, so there is no transmission distortion in the fringe image. This device saves at least one work station for the defect detection of products, is convenient for image acquisition, effectively saves the time for industrial assembly line detection, improves efficiency, and reduces the cost of product defect detection. It can realize the two three-dimensional measurement defect detection technologies without refocusing on the premise of only using one projector 1 and one camera 4, and the fringe focusing is clear, there is no distortion, the image quality is good, and the hardware and acquisition costs are saved.

[0095] In an exemplary embodiment, the mirror 3 is adapted to slide or rotate to be located at the first position and the second position, which is simple and convenient.

[0096] In an exemplary embodiment, a driving member is further included, and the driving member is adapted to drive the mirror 3 to move, which is convenient for realizing automatic image acquisition.

[0097] In an exemplary embodiment, an image selection method acquires a Phase Measuring Deflectometry (PMD) image and a Fringe Projection Profilometry (FPP) image through the above-mentioned device for obtaining the surface image of the object to be measured; and based on the contrast of the imaging, selects the PMD image and / or the FPP image as the detection image for defects, so as to select a suitable image for defect detection according to the roughness of different surfaces of the object to be measured, prevent missed detections, and ensure the detection rate.

[0098] In an exemplary embodiment, the acquisition of the contrast of the imaging includes the following steps:

[0099] Collect one PMD image and one FPP image respectively;

[0100] Obtain the first contrast and the root mean square error (RMSE) of the PMD image and the FPP image: Extract the gray values of the middle row in the row direction of the vertical stripe image, fit a sine curve, and calculate the RMSE of the PMD image and the FPP image; Crop the PMD image and the FPP image into images of 2Π period size, and respectively obtain the maximum gray value m and the minimum gray value n adjacent within one period of the PMD image and the FPP image, to obtain the first contrast.

[0101] Calculate the contrast of the imaging: If m > 255, it means the image is overexposed, or if RMSE > the first threshold, it means the image does not have good sinusoidality, then the first contrast k of the corresponding image is discarded; if only one image satisfies m ≤ 255 and RMSE ≤ the first threshold, directly use this k value as the contrast of the imaging; if the contrasts of both images satisfy m ≤ 255 and RMSE ≤ the first threshold, then take the average value of the first contrasts of the PMD image and the FPP image as the contrast of the imaging.

[0102] In an exemplary embodiment, the first threshold is 20.

[0103] In an exemplary embodiment, since the contrast of the bright surface material is higher, when the contrast of the imaging is greater than the second threshold, select PMD imaging as the detection image for defects, which is more accurate; when the contrast of the imaging is less than the third threshold, select the FPP image as the detection image for defects, which is more accurate; when the contrast of the imaging is less than or equal to the second threshold and greater than or equal to the third threshold, select the PMD image and the FPP image simultaneously as the detection images for defects to prevent over-screening.

[0104] In an exemplary embodiment, the second threshold is 20 and the third threshold is 5.

[0105] In an exemplary embodiment, a defect detection method includes the above-described image selection method. When only the phase deflectometry (PMD) image is selected as the detection image for defects, using the detection result of the PMD image as the final detection result is more accurate. When only the fringe projection profilometry (FPP) image is selected as the detection image for defects, using the detection result of the FPP image as the final detection result is more accurate. When both the PMD image and the FPP image are selected as the detection images for defects, if defects are detected in both, it is determined as a real defect to prevent over-screening.

[0106] Embodiment 2: Different from Embodiment 1, in this embodiment, the composition structure of the device for obtaining the surface image of the object to be measured is different. In this embodiment, the device for obtaining the surface image of the object to be measured includes a first projector, a second projector, a diffuser 2, a reflector 3, a second camera, and a second camera;

[0107] The first projector is adapted to project fringes, and the diffuser 2 is adapted to diffuse the fringes projected by the first projector onto the surface of the object to be measured; the second camera is adapted to receive the signal reflected from the surface of the object to be measured after the fringes incident on the surface of the object to be measured through the diffuser 2, so as to obtain a phase deflectometry (PMD) image;

[0108] The second projector is adapted to project fringes, and the reflector 3 is adapted to reflect the fringes projected by the second projector onto the surface of the object to be measured in a direction perpendicular to the surface of the object to be measured; the second camera is adapted to receive the signal diffused from the surface of the object to be measured after the fringes incident on the surface of the object to be measured from the reflector 3, so as to obtain a fringe projection profilometry (FPP) image. In this embodiment, the two methods can be focused independently, and there is no need to switch the position of the reflector 3. The second camera and the second camera can be respectively connected to two prototypes, and the two prototypes are respectively brought into the algorithm to analyze the defect detection effect and check whether there is any missed detection or over-screening.

[0109] The above description of the specification and embodiments is used to explain the protection scope of the present invention, but does not constitute a limitation on the protection scope of the present invention. Through the inspiration of the present invention or the above embodiments, those of ordinary skill in the art, combining common general knowledge, ordinary technical knowledge in the art, and / or existing technologies, through logical analysis, reasoning, or limited experiments, can obtain modifications, equivalent replacements, or other improvements to the embodiments of the present invention or some of its technical features, which should all be included within the protection scope of the present invention.

Claims

1. A device for acquiring a surface image of an object to be measured, characterized in that: include: A projector (1) adapted to project fringes; A diffusion plate (2), which is suitable for diffusing the stripes projected by the projector (1) onto the surface of the object to be measured; A reflector (3) moves relative to the object to be measured between a first position and a second position. In the first position, the reflector (3) blocks the light path from the projector (1) to the diffuser (2) and reflects the stripes projected by the projector (1) to the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured. In the second position, the reflector (3) avoids the light path from the projector (1) to the diffuser (2). The optical path distance from the stripes of the projector (1) to the diffuser (2) is equal to the optical path distance from the stripes of the projector (1) to the surface of the object to be measured when the reflector (3) is in the first position; and The camera (4) is adapted to receive a signal reflected by the surface of the object to be measured after being incident on the diffuser plate (2) to obtain a phase deflection technique (PMD) image, or to receive a signal diffused by the surface of the object to be measured after being incident on the surface of the object to be measured from the reflector (3) to obtain a fringe projection profilometry (FPP) image.

2. The device for acquiring a surface image of an object to be measured according to claim 1, characterized in that: The reflector (3) is suitable for sliding or rotating to be located at a first position and a second position.

3. The device for acquiring a surface image of an object to be measured according to claim 1, characterized in that: It also comprises a driving member, which is suitable for driving the reflecting mirror (3) to move.

4. A device for acquiring an image of a surface of an object to be measured, characterized in that: include: a first projector adapted to project fringes; a second projector adapted to project fringes; A diffusion plate (2), which is suitable for diffusing the stripes projected by the first projector onto the surface of the object to be measured; A reflector (3), which is suitable for reflecting the stripes projected by the second projector to the surface of the object to be measured in a manner perpendicular to the surface of the object to be measured; A first camera is adapted to receive a signal reflected by the surface of the object to be measured after the diffusion plate (2) is incident on the surface of the object to be measured, so as to obtain a phase deflection technology (PMD) image; The second camera is adapted to receive a signal diffused from the surface of the object to be measured after being incident from the reflector (3) to the surface of the object to be measured, so as to obtain a fringe projection profilometry (FPP) image.

5. An image selection method, characterized in that: Acquire a phase deflection technique (PMD) image and a fringe projection profilometry (FPP) image by using a surface image acquisition device for an object to be measured as described in any one of claims 1 to 3, or an surface image acquisition device for an object to be measured as described in claim 4; Imaging based on contrast selective phase deflectometry (PMD) images and / or fringe projection profilometry (FPP) images.

6. An image selection method as claimed in claim 5, characterized in that: Acquisition of imaging contrast includes the following steps: Collect one phase deflectometry (PMD) image and one fringe projection profilometry (FPP) image respectively; Obtain the first contrast and root mean square difference of the phase deflection technology PMD image and the fringe projection profilometry FPP image: extract the grayscale value of the middle row in the row direction of the vertical fringe image, and fit the sine curve to calculate the root mean square difference RMSE between the phase deflection technology PMD image and the fringe projection profilometry FPP image; crop the phase deflection technology PMD image and the fringe projection profilometry FPP image into 2Π period size images, and obtain the adjacent maximum grayscale value m and minimum grayscale value n within one period of the phase deflection technology PMD image and the fringe projection profilometry FPP image respectively, and obtain the first contrast Calculate the contrast of the imaging: if m>255 or RMSE>the first threshold, the first contrast k of the corresponding image is discarded; if only one image satisfies m≤255 and RMSE≤the first threshold, then this k value is directly used as the contrast of the imaging; if the contrasts of the two images both satisfy m≤255 and RMSE≤the first threshold, then the average of the first contrasts of the phase deflection technology PMD image and the fringe projection profile measurement FPP image is used as the contrast of the imaging.

7. An image selection method as claimed in claim 6, characterized in that: The first threshold is 20.

8. An image selection method as claimed in claim 5, characterized in that: When the contrast of the imaging is greater than the second threshold, PMD imaging is selected; when the contrast of the imaging is less than the third threshold, the fringe projection profilometry FPP image is selected; when the contrast of the imaging is less than or equal to the second threshold and greater than or equal to the third threshold, both the phase deflection technology PMD image and the fringe projection profilometry FPP image are selected.

9. An image selection method as claimed in claim 8, characterized in that: The second threshold is 20, and the third threshold is 5.

10. A defect detection method, characterized in that: comprising an image selection method as described in any one of claims 5 to 9, When only the phase deflection technology PMD image is selected as the defect detection image, the detection result of the phase deflection technology PMD image is used as the final detection result; When only the fringe projection profilometry FPP image is selected as the defect detection image, the detection result of the fringe projection profilometry FPP image is used as the final detection result; When both the phase deflection technology PMD image and the fringe projection profilometry FPP image are selected as defect detection images, if defects are detected in both, they are judged as real defects.

Images