Photometric three-dimensional imaging method and high-speed photometric three-dimensional imaging device

Abstract

The invention discloses a photometric three-dimensional imaging method and a high-speed photometric three-dimensional imaging device, and relates to the technical field of industrial detection.Partition light sources which are evenly distributed are arranged in the vertical direction of a sample to be detected, and the partition light sources are correspondingly provided with area-array cameras and prime lenses; the partitioned light source comprises four light sources which are independently controlled, each light source is turned on / off under a set program during image acquisition, only one light source is turned on / off each time, and the other light sources are in opposite states, namely turned off / on; the method comprises the following steps: acquiring a sample image in the on / off state of each light source, combining and registering two images acquired in the on / off state of two light sources, then calculating a uniform illumination image and two primary difference images, and further calculating a pseudo depth image and a contour image. On the basis of five images obtained through processing, high-speed detection of a single sample can be achieved within 50 ms, and the detection requirement in an actual industrial scene can be better met.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial detection, and in particular to a photometric stereo imaging method and a high-speed photometric stereo imaging device. Background Art

[0002] Photometric stereo is a computer vision technology used to reconstruct the surface of three-dimensional objects. It is suitable for the detection of samples with strong diffuse reflection characteristics and relatively flat surfaces, such as rubber, plastic, and frosted metal. By utilizing the changes in the direction of illumination and the changes in the intensity of light reflected from the surface of the object, photometric stereo can estimate the surface shape of the object (such as the normal); it does not require complex sensors, but only needs to take multiple images of the object through light sources in different directions to derive the three-dimensional information of the object's surface.

[0003] Since the same sample needs to obtain images illuminated from multiple directions for image synthesis, traditional photometric stereo imaging methods mostly use static shooting methods, which are difficult to meet the needs of high-speed industrial inspection scenarios. Some R&D teams use linear array cameras and rely on sample conveying devices to achieve time-sharing scanning and imaging of samples, which can only be used in linear conveying production lines and cannot be applied to rotating motion conveying platforms such as transparent turntables. Summary of the invention

[0004] In view of the technical defects of the existing photometric stereo imaging, the present invention aims to provide a photometric stereo imaging method and a high-speed photometric stereo imaging device.

[0005] The present invention protects a photometric stereo imaging method, wherein a uniformly distributed partitioned light source is arranged in the vertical direction of the sample to be measured, and a planar array camera and a fixed-focus lens are arranged corresponding to the partitioned light source;

[0006] The partition light source contains four independently controlled light sources. During image acquisition, each light source is turned on / off according to a predetermined program, and only one light source is turned on / off at a time, while the other light sources are in the opposite state, that is, off / on.

[0007] The sample images of each light source are collected in the on / off state, and the two images collected in the on / off state of the two light sources are combined and registered. Then, a uniform illumination map and two first-order difference maps are calculated, and then a pseudo depth map and a contour map are calculated.

[0008] The form of the light source is not limited, for example, a ring light source or a rectangular light source may be selected.

[0009] The present invention also protects a high-speed photometric stereo imaging device, which performs imaging processing on a sample to be tested based on the above-mentioned photometric stereo imaging method;

[0010] The system comprises a partitioned light source arranged above the sample stage, a surface array camera and a fixed-focus lens arranged above the partitioned light source, wherein the surface array camera is connected to an image calculation module and a display control module, and the image calculation module and the display control module interact with each other in a two-way manner.

[0011] The image calculation module is used to receive the original image of the area array camera and perform calculations to obtain various required processing images;

[0012] The display and control module is used to control the actions of the area array camera, the partitioned light source, and the image calculation module, and to display relevant data.

[0013] Furthermore, the display and control module is also connected to an audible and visual alarm module and / or a rejection module;

[0014] The sound and light alarm module gives the user reminders through sound and light information, including the working status of the equipment and whether the object to be tested has defects;

[0015] The rejection module is used to reject defective samples.

[0016] Furthermore, the sample stage is a rotating transport platform (see Figure 1 ) or a linear motion transport platform (refer to Figure 2 ); Furthermore, the table top of the transport platform is transparent, and the partitioned light source, the area array camera and the fixed-focus lens are arranged above and / or below the detection station, Figure 3 The rotating platform in the apparatus is transparent, and image acquisition components (partitioned light source, area array camera and fixed-focus lens) facing the top and bottom are respectively arranged at different workstations to perform sample inspection from two angles of pitch and elevation.

[0017] Compared with the prior art, the high-speed stereoscopic imaging device disclosed in the present invention does not require the sample to stop during the imaging process and can pass through the device at high speed. During this period, 4 partitioned exposure images are captured, and spatial position alignment and image synthesis are used to generate 5 images, including uniform illumination map, horizontal first-order difference map, vertical first-order difference map, pseudo-depth map and contour map. The single sample detection time can achieve high-speed detection of less than 50ms (i.e., more than 1200 samples / min), which can better meet the detection needs in actual industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A high-speed photometric stereo imaging device for a rotary motion conveying platform;

[0019] Figure 2 A high-speed photometric stereo imaging device for a linear motion conveying platform;

[0020] Figure 3 High-speed photometric stereo imaging device for bilateral image acquisition;

[0021] Figure 4 The photometric stereo imaging flow chart disclosed in Example 1;

[0022] Figure 5 Schematic diagram of the working state of the annular partitioned light source of Example 1 Figure 1 ;

[0023] Figure 6 Schematic diagram of the working state of the annular partitioned light source of Example 1 Figure 2 ;

[0024] Figure 7 Schematic diagram of the rectangular partition light source of Example 2 Figure 1 ;

[0025] Figure 8 Schematic diagram of the rectangular partition light source of Example 2 Figure 2 ;

[0026] Fig. 9 It is a main structure block diagram of Example 3;

[0027] Fig.10 This is the key control timing diagram of the present invention.

[0028] Fig.11 is the original image of the sample to be tested collected based on Example 1;

[0029] Fig.12 for Fig.11 The uniformly illuminated image is restored from the four images.

[0030] Fig.13 for Fig.11 A horizontal difference image of four images.

[0031] Fig.14 for Fig.11 A vertical difference image of four images.

[0032] Fig.15 for Fig.13 and Fig.14 Synthesize the calculated pseudo depth image.

[0033] Fig.16 for Fig.13 and Fig.14 Synthesize the calculated contour image. DETAILED DESCRIPTION

[0034] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively below in conjunction with the accompanying drawings and specific embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thoroughly understood.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The technical scheme of the present invention is described more clearly and completely below in conjunction with specific embodiments and comparative examples.

[0036] Example 1

[0037] A photometric stereo imaging method, in which a uniformly distributed partitioned light source is arranged in the vertical direction of the sample to be measured, and a surface array camera and a fixed-focus lens are arranged corresponding to the partitioned light source, such as Figure 1 , Figure 2 shown. Figure 1 , Figure 2 A rotary motion transport platform and a linear motion transport platform are respectively provided, but this embodiment is applicable to more than these two types of transport platforms. Figure 4 The complete process of the photometric stereo imaging method disclosed in this embodiment is given.

[0038] In this embodiment, the partitioned light source is a ring light source, and four independently controlled light sources form a ring. Figure 6 , Figure 6 .

[0039] Figure 5 As shown, during image acquisition, each light source is lit under a predetermined procedure, and only one light source is lit at a time, while the other light sources are in the opposite state, that is, off. The images acquired when 31 and 32 are lit are horizontal original images (2 images), and the images acquired when 33 and 34 are lit are vertical original images (2 images).

[0040] Figure 6 As shown, during image acquisition, each light source is turned off under a predetermined procedure, and only one light source is turned off at a time, while the other light sources are in the opposite state, i.e., on. The images acquired when 32 and 31 are turned off are horizontal original images (2 images), and the images acquired when 33 and 34 are turned off are vertical original images (2 images).

[0041] Set the two horizontal original images to be I 1 with I 2 , the two vertical original images are I 3 with I 4First, the images are registered as described in equations (1) to (3):

[0042]

[0043]

[0044] In formula (1), (x, y) represents image I n The image coordinates (n=1, 2, 3 or 4); (x' / z', y' / z') represent the transformed coordinates; T represents the coordinate transformation matrix, which is given by (2), where n1 represents the parameters corresponding to the transformation from image n to image 1.

[0045] The coordinate transformation matrix T can be obtained by means of geometric path calculation, feature point matching calculation, artificial intelligence learning, etc. When the movement speed of the device is stable, it only needs to be calculated once and can be directly used for other image calculations. It does not need to be calculated in each measurement, which can greatly shorten the detection time.

[0046] Formula (3) gives the image after image registration:

[0047] I n '(u,v)=I n (x,y) (3)

[0048] In formula (3), u = x' / z', v = y' / z', which are the coordinates of the image after registration.

[0049] Based on the registered image, the uniform illumination image and the first difference image are calculated as shown in the following equations (4) and (5):

[0050]

[0051]

[0052] Among them, I normal is a uniform illumination map, I h is the horizontal first difference graph, I v It is a vertical first difference diagram.

[0053] Based on I h ,I v , the pseudo depth map I can be calculated by formula (6) depth Since the position and radiation calibration of the camera and light source are not performed, this image is called a pseudo depth map. In most defect detection scenarios, the determined depth of the defect is not of concern, so the calibration process can be omitted, reducing the complexity of equipment debugging and the time consumption of calculation.

[0054]

[0055] Based on I h ,Iv , the contour map I can be calculated by formula (7) contour .

[0056]

[0057] Example 2

[0058] Different from Example 1, in this embodiment, the partitioned light source is a rectangular light source, and four independently controlled light sources form a square. Figure 7 , Figure 8 .

[0059] Figure 7 As shown, during image acquisition, each light source is lit under a predetermined procedure, and only one light source is lit at a time, while the other light sources are in the opposite state, that is, off. The images acquired when 71 and 72 are lit are horizontal original images (2 images), and the images acquired when 73 and 74 are lit are vertical original images (2 images).

[0060] Figure 8 As shown, during image acquisition, each light source is turned off under a predetermined procedure, and only one light source is turned off at a time, while the other light sources are in the opposite state, i.e., on. The images acquired when 71 and 72 are turned off are horizontal original images (2 images), and the images acquired when 73 and 74 are turned off are vertical original images (2 images).

[0061] The processing method of the original image is the same as that of Embodiment 1 and will not be described again here.

[0062] Example 3

[0063] A high-speed photometric stereo imaging device performs imaging processing on a sample to be tested based on the photometric stereo imaging method disclosed in embodiments 1 and 2.

[0064] Reference Fig. 9 The high-speed photometric stereo imaging device includes a partitioned light source arranged above the sample stage, an array camera and a fixed-focus lens arranged above the partitioned light source, the array camera is connected to an image calculation module and a display control module, and the image calculation module and the display control module interact with each other in a two-way manner.

[0065] The image calculation module is used to receive the original image of the area array camera and perform calculations to obtain various required processing images (see Example 1 for details).

[0066] The display and control module is used to control the actions of the area array camera, the partitioned light source, and the image calculation module, and to display relevant data. Fig. 9 The monitor provides display function, and the mouse and keyboard provide input and control functions.

[0067] The display and control module is also connected to an audible and visual alarm module and / or a rejection module; the audible and visual alarm module gives the user reminders through audible and visual information, including the working status of the equipment and whether the object to be tested has defects; the rejection module is used to reject defective samples.

[0068] Fig. 9 The conveying module in the embodiment is the transport platform, and two forms of rotational motion and linear motion are given in Example 1. The transport platform table can be transparent, and the partitioned light source, the area array camera and the fixed focus lens are arranged above and / or below the detection station. Figure 3 The rotating platform in the apparatus is transparent, and image acquisition components (partitioned light source, area array camera and fixed-focus lens) facing the top and bottom are respectively arranged at different workstations to perform sample inspection from two angles of pitch and elevation.

[0069] Fig. 9 The external trigger module in the system is used to trigger the imaging of the area array camera. After the external trigger module is triggered, the display and control module collects the trigger signal and sends a working instruction to the partition light source and the area array camera. The power module provides working power for other modules.

[0070] Fig.10 A key control timing diagram is given, which includes Fig. 9 External trigger signal, light source control signal, camera control signal, and rejection control signal. Fig.10 T 1 The light source trigger delay; T 2 T is the time the light source is turned on; 3 T is the camera exposure time; 4 The frame interval for partition shooting; T 5 The duration of enabling the external trigger signal, the rising edge is effective; T 6 To eliminate the control signal delay; T 7 For different sample detection speeds, different control timings are configured. For example, in an actual application where the sample detection speed is 1200 pieces / min, T 2 Can be 1ms, T 3 Can be 50μs, T 4 Can be 2ms, T 6 The minimum value can be 50ms, T 7 It can be 5ms.

[0071] The technical effects of the present invention are demonstrated below with reference to specific examples.

[0072] Fig.11 Four original images (81-84 horizontal original images and 2 vertical original images) taken during the movement of the sample to be tested are given, and the light source illumination method is as follows Figure 6 When capturing image 81, Figure 6 The middle light source areas 31, 33 and 34 are all on, and 32 is off; when capturing image 82, Figure 6 The middle light source areas 32, 33 and 34 are all on, and 31 is off; when capturing image 83, Figure 6 The middle light source areas 31, 32 and 34 are all on, and 33 is off; when capturing image 84, Figure 6 The middle light source areas 31, 33 and 33 are all on at the same time, and 34 is off.

[0073] Fig.12 The uniform light source map calculated after image registration is given. This map is equivalent to the image taken when the four partitions of the partition light source are lit at the same time. If the high-speed area array camera is a color camera, the synthetic uniform illumination map can detect color defects, but it is difficult to distinguish concave and convex defects from noisy features.

[0074] Fig.13 , Fig.14 The horizontal first difference image and the vertical first difference image are given respectively. From these two images, it can be seen that the composite image weakens the features of most fine dust 93 and shallow scratches 94, enhances the features of depressions 91 and protrusions 92, and makes it easier to extract the depth-related defects of depressions and protrusions from the noisy image information. For most industrial product defect detection, Fig.13 and Fig.14 This is sufficient to meet the feature extraction requirements, and the calculation only needs to end here, which greatly reduces the image synthesis time of the conventional photometric stereo method.

[0075] If the actual detection time is not a bottleneck, you can continue to synthesize pseudo depth images Fig.15 With contour map Fig.16 , the pseudo depth image can be displayed in one picture at the same time Fig.13 and Fig.14 The concave and convex defect characteristics in the two directions are more intuitive; Fig.16 It can better highlight the contours of samples and concave and convex features, and can be applied to certain needs that require character detection.

[0076] In summary, compared with the prior art, the high-speed stereoscopic imaging device disclosed in this embodiment does not require the sample to stop during the imaging process and can pass through the device at high speed. During this period, 4 partitioned exposure images are captured, and spatial position alignment and image synthesis are used to generate 5 images, including uniform illumination map, horizontal first-order difference map, vertical first-order difference map, pseudo-depth map and contour map. The single sample detection time can achieve high-speed detection of less than 50ms (i.e., more than 1200 pieces / min), which can better meet the detection needs in actual industrial scenarios.

[0077] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A photometric stereo imaging method, characterized in that: A uniformly distributed partitioned light source is arranged in the vertical direction of the sample to be tested, and an area array camera and a fixed-focus lens are arranged corresponding to the partitioned light source; The partition light source contains four independently controlled light sources. During image acquisition, each light source is turned on / off according to a predetermined program, and only one light source is turned on / off at a time, while the other light sources are in the opposite state, that is, off / on. The sample images of each light source are collected in the on / off state, and the two images collected in the on / off state of the two light sources are combined and registered. Then, a uniform illumination map and two first-order difference maps are calculated, and then a pseudo depth map and a contour map are calculated.

2. The photometric stereo imaging method according to claim 1, characterized in that: The light source is a ring light source or a rectangular light source.

3. A high-speed photometric stereo imaging device, characterized in that: Performing imaging processing on the sample to be tested based on the photometric stereo imaging method according to claim 1 or 2; The system comprises a partitioned light source arranged above the sample stage, a surface array camera and a fixed-focus lens arranged above the partitioned light source, wherein the surface array camera is connected to an image calculation module and a display control module, and the image calculation module and the display control module interact with each other in a two-way manner. The image calculation module is used to receive the original image of the area array camera and perform calculations to obtain various required processing images; The display and control module is used to control the actions of the area array camera, the partitioned light source, and the image calculation module, and to display relevant data.

4. The photometric stereoscopic imaging device according to claim 3, characterized in that: The display and control module is also connected to an audible and visual alarm module and / or a rejection module; The sound and light alarm module gives the user reminders through sound and light information, including the working status of the equipment and whether the object to be tested has defects; The rejection module is used to reject defective samples.

5. The photometric stereo imaging method according to claim 3, characterized in that: The sample stage is a rotary transport platform or a linear transport platform.

6. The photometric stereo imaging method according to claim 5, characterized in that: The table top of the transport platform is transparent, and the partitioned light source, the area array camera and the fixed-focus lens are arranged above and / or below the detection station.

Images