Cross arrangement of latitude and longitude color light sources and surface topography detection method

Abstract

The application discloses a cross-arranged longitude and latitude color light source and a surface topography detection method. The method uses a cross-arranged longitude and latitude color light source and surface topography detection. The light source is a semispherical dome. LED particle light sources are arranged in the longitude direction and the latitude direction inside the semisphere. In the latitude direction, blue-green-red or blue-cyan-green-yellow-red color light sources are arranged from the equator to the north pole. In the longitude direction, blue-green-red or blue-cyan-green-yellow-red color light sources are arranged at equal angles. The LED particle light sources are replaced by expanded color laser light sources or projection light sources controlled by a computer. The color can be switched according to needs. A controller is used to open the longitude light source and the latitude light source at different times, and images are collected respectively. The color light source information collected by the camera in the longitude and latitude directions is color coding information of the color light source. The application avoids the complexity and low adaptability of a model in a traditional modeling method and overcomes the field of view problem of a single small ball calibration in the traditional method.

Description

TECHNICAL FIELD

[0001] The application is a cross arrangement of longitude and latitude color light source and surface topography detection method, belonging to the field of machine vision. BACKGROUND

[0002] 3D vision is a research hotspot in the field of machine vision. With the development of vision sensor technology, computer technology and image processing technology, 3D vision methods are more and more, the precision and imaging speed are faster and faster, and some 3D vision methods have been widely used in engineering field, such as TOF vision, laser structured light vision, projection structured light vision, speckle laser binocular vision and so on. The 3D vision method based on photometry has the advantages of rich information and simple equipment, so it is also one of the 3D vision methods widely concerned by researchers, among which the vision method based on AOI light source has been well applied in the detection of circuit board welding points. However, there are still many difficulties to be solved in the 3D vision method based on photometry, such as the non-determination problem in 3D topography recovery algorithm, the calibration difficulty problem, the uncertainty problem of object surface reflection property and so on.

[0003] As for the AOI light source commonly used at present, it can only reflect the gradient size of the surface to be detected, but cannot reflect the gradient direction, specifically, it cannot judge whether the surface topography is convex or concave. The classic photometric stereoscopy recovers the surface topography, which needs to model the vision system, and needs to use the brightness information of monochromatic light or color light as the surface gradient recovery information, and the brightness signal is seriously disturbed by the environmental light and the reflection characteristics of the material surface. The application introduces the longitude direction color light source on the basis of the AOI light source, which can recover the gradient direction of the surface at the same time, and then the complete surface topography information can be obtained, and the gradient size information and the direction information are encoded by color, and the information is reliable.

[0004] In the surface topography detection method of the application, the calibration-look-up table method is used instead of the traditional modeling method to obtain the surface gradient size and gradient method of the target to be detected, which avoids the complexity and low adaptability of the traditional modeling method. At the same time, the interpolation method is combined to design a multi-position small ball calibration method, which realizes the detection task of large field of view and overcomes the field of view problem of traditional single small ball calibration.

[0005] The traditional surface topography detection method only designs one kind of calibration or modeling method for the same material, and often ignores the influence of different surface states of the same material on detection, for example, the oxidation and corrosion of the surface will change the reflectivity of the part, and then the standard recovery method will cause errors. The present application adopts a deep learning method, learns the relationship between the surface chroma and the surface gradient of the images of the same material and different surface states, so that when encountering a detection target with different parts and different surface states, the surface gradient is recovered by the deep learning model, and the surface topography detection precision and applicability are improved. SUMMARY

[0006] The present application provides a cross arrangement of longitude and latitude color light source and surface topography detection method to solve the problems of the prior art, and the technical scheme adopted by the present application is as follows:

[0007] The method of using cross arrangement of longitude and latitude color light source and surface topography detection is as follows: the light source is a semispherical dome, and LED particle light sources are arranged in the longitude direction and the latitude direction inside the hemisphere, wherein the latitude direction is arranged with blue-green-red or blue-cyan-green-yellow-red color light sources from the equator to the north pole, and the longitude direction is arranged with blue-green-red or blue-cyan-green-yellow-red color light sources at equal angles. The LED particle light source can be replaced by a color laser light source with beam expansion, or a projection light source controlled by a computer, and the color can be switched as needed. There is a controller to open the longitude light source and the latitude light source at different times, and the camera collects images respectively. The color coding information of the color light source is the color coding information of the color light source collected by the camera in the longitude and latitude directions.

[0008] At the same time, the color coding can be replaced by light source brightness time coding, and then a plurality of image groups are combined and decoded. The light sources of various colors in the longitude and latitude directions have independent switches, which can realize time-sharing imaging under different color illumination, prevent color aliasing, and facilitate more accurate color coding.

[0009] After the latitude light source image and the longitude light source image are obtained at different times, the gradient size of the light-reflecting surface can be inversely solved from the chroma of the latitude light source image, and the gradient direction of the light-reflecting surface can be inversely solved from the chroma in the longitude light source image. The normal direction of the light-reflecting surface can be solved from the gradient size and the gradient direction, and the surface topography detection of the object is realized.

[0010] Further, the relationship between the chroma of the latitude or longitude light source image and the surface gradient size and gradient direction is mainly determined by the calibration ball. The calibration ball is placed on the reference plane at the position to be detected, and the images are collected by opening the longitude and latitude light sources at different times. The direction of the visual system is defined as the z direction, the image row direction and the column direction are defined as the x and y directions respectively, and the coordinate axis zero value is the center. The relationship between the chroma and the gradient size is determined by the latitude image, and the expression is as follows:

[0011]

[0012]

[0013] where f is the distance between the camera and the surface of the measured object in the actual space; C is the chrominance information, which is determined by the three channels of R, G and B; k and η are proportional coefficients; is the gradient size.

[0014] The relationship between the chrominance and the gradient direction is determined by the longitude image, and the representation is as follows:

[0015]

[0016] arctan (θ) = η * C (4)

[0017] where θ is the gradient direction.

[0018] The surface normal vector of the measured object is represented as:

[0019]

[0020] where is the surface normal vector of the measured object.

[0021] Further, the above method of calibrating and calculating the surface gradient by a single ball can be changed to a method of calculating the surface gradient of a large field of view by multiple balls and interpolation. Specifically, multiple balls are placed at different positions on the reference plane, and multiple frames of images under illumination of different light sources are obtained. Then, the relationship between the chrominance of the images and the gradient of the balls at different positions is established. When the target to be detected falls outside the calibration balls, the relationship model at the new position is calculated by interpolation.

[0022] Further, the calibration ball can be selected according to the material of the measured object to adapt to different targets to be detected.

[0023] Further, for the same material, different surface states of the material, corresponding calibration balls are made, and the relationship between the image chrominance and the surface gradient is established respectively. Then, the relationship between the image chrominance and the surface gradient is learned by the method of deep learning, and an end-to-end model between the two is established. When encountering a detection target with different surface states at different parts, the surface gradient is back calculated by the deep learning model.

[0024] Further, the point-based detection formed by the mapping relationship between the calibration ball and the measured object can be changed to region-based detection. For different topographies of the region, a region segmentation network is used to directly establish a relationship model between the chrominance combination and the topography. In actual detection, a region is regarded as a whole detection, and the classification or overall recovery of the surface gradient is realized.

[0025] Further, the specific topographic defects of specific application sites can be detected and the boundaries are positioned by using a deep learning method, and the knowledge learned by the deep learning can be directly defect knowledge or feature knowledge such as the type and distribution of the rainbow strip, and then the feature knowledge is used to identify the defect type.

[0026] The positive effects of the present application are:

[0027] 1. The present application provides a surface topography detection method, which uses the chrominance or brightness information collected by the longitude and latitude direction light sources and the chrominance or brightness information of the calibration small ball to obtain the gradient direction and the gradient size, and the surface normal vector of the measured object is obtained by combining the two, compared with the general photometric stereoscopy method which recovers the surface gradient size from the reflection intensity, the present application recovers the surface gradient size from the chrominance, and the information is more reliable. The present application cross-fuses the longitude light source and the latitude light source, avoids the position matching of the image, the device is simple, easy to control, and the image acquisition speed is fast.

[0028] 2. The present application can select multiple light sources, and the encoding mode can select chrominance or brightness according to the actual situation, so that the application scene of the method is wider, and the topography detection precision is improved. For example, the laser light source has better direction type than the traditional LED light source, and the projection light source has better flexibility than the LED light source.

[0029] 3. The present application avoids the construction of a complex model by using the calibration lookup table method, and one method can be used for multiple reflection types of materials and surface states.

[0030] 4. The present application improves the topography detection result by using the deep learning method to establish the relationship between the chrominance and the surface gradient of different surface states of the same material in different parts.

[0031] 5. The present application directly establishes the relationship between the chrominance combination and the topography by using the region segmentation network for different topography regions, so as to realize region detection. The present application improves the reliability of topography detection. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings are included to provide a further understanding of the embodiments, and are incorporated in and constitute a part of this specification, and together with the description serve to explain the principles of the application, and the same reference numerals indicate the same components throughout the drawings.

[0033] Figure 1 It is a device schematic diagram of the method, wherein 1 is a longitude direction LED particle light source, 2 is a latitude direction LED particle light source, 3 is an industrial camera, 4 is a processor, and 5 is a controller.

[0034] Figure 2 It is a schematic diagram of a single calibration small ball on a reference plane.

[0035] Figure 3 is a schematic diagram of the distribution of a plurality of calibration balls on a plane. DETAILED DESCRIPTION

[0036] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred embodiments of the present application will be described in detail below with reference to the drawings, which constitute a part of the present application and serve to explain the principles of the embodiments of the present application, but are not used to limit the scope of the present application.

[0037] The embodiment discloses a cross-arranged longitude and latitude color light source and a surface topography detection method, and comprises the following steps:

[0038] 1) The method for cross-arranging longitude and latitude color light sources is as shown in Figure 1 The light sources form a hemispherical dome, and LED particle light sources are arranged in the longitude direction and the latitude direction inside the hemisphere. The blue-green-red or blue-cyan-green-yellow-red color light sources are arranged in the latitude direction from the equator to the north pole, and the blue-green-red or blue-cyan-green-yellow-red color light sources are arranged at equal angles in the longitude direction.

[0039] 2) According to Figure 1 According to actual requirements, the LED particles can be replaced by color laser light sources with beam expansion or projection light sources controlled by a computer. Color coding can be replaced by light source brightness time coding, and then a plurality of image groups are combined for decoding.

[0040] 3) The object to be detected is placed in the center under the longitude and latitude color light sources, and an industrial camera is adjusted to ensure clear imaging of the camera.

[0041] 4) The relationship between the chroma of the latitude or longitude light source image and the surface gradient size and gradient direction is determined by calibration balls, as shown in Figure 1 and Figure 2 The calibration balls of the same material and different surface states as the object to be detected are placed on the reference plane at the detection position, the longitude and latitude light sources are turned on at different times to collect images, the direction of the visual system is defined as the z direction, the image row direction and the column direction are defined as the x direction and the y direction respectively, and the coordinate axis zero value is the center. The relationship between the chroma and the gradient size is determined by the latitude image, and the representation is as follows:

[0042]

[0043]

[0044] Wherein f is the distance between the camera and the surface of the object to be detected in the actual space; C is the chroma information, which is determined by R, G and B three channels; k and η are proportional coefficients; is the gradient size.

[0045] The relationship between the chroma and the gradient direction is determined by the longitude image, and the representation is as follows:

[0046]

[0047] arctan(θ) = η*C (4)

[0048] where θ is the gradient direction.

[0049] The surface normal vector of the measured object can be expressed as:

[0050]

[0051] where is the surface normal vector of the measured object.

[0052] 5) When the object to be measured is larger than the calibration ball, select the calibration method with a large field of view. Place multiple small balls at different positions on the reference plane, as shown in Figure 3 , acquire multiple images under different light source irradiation, and then establish the relationship between the colorimetric value of the image and the gradient of the small ball at different positions. When the target to be detected falls outside the calibration ball, the relationship model of the new position is obtained by interpolation method. The interpolation method can refer to the following:

[0053] ω ij = (x i -x)(y i -y) (6)

[0054] f(x,y) = f(Q 11 )ω 22 -f(Q 21 )ω 12 -f(Q 12 )ω 21 -f(Q 22 )ω 11 (7)

[0055] where f(x,y) is the value to be measured; Q ij is the gradient size or gradient direction value of the calibration ball; x i , y i are the corresponding positions of Q ij ; ω ij is the weight coefficient, and x, y are the corresponding positions of the measured value.

[0056] 6) Through the method of deep learning, learn the relationship between the image colorimetric value and the surface gradient established by the calibration ball, and establish an end-to-end model between the two. When encountering a detection target, the target is sent into the model, and the surface gradient is output through the model.

[0057] 7) Different regions are divided according to different topographies obtained from images, and the chrominance combinations of different regions are divided into different categories, and the relationship between the chrominance combinations and the topographies is learned by a region segmentation network to realize the classification or overall recovery of the surface gradient. After the object to be measured is sent into the network, the surface gradient of the corresponding region is corrected according to the segmentation result.

[0058] 8) According to the requirement, the topography defect information or position information of the object to be measured is calibrated, the obtained surface gradient map is sent into a semantic segmentation network, and the topography defect is learned. The defect detection or boundary measurement of the object to be measured is realized.

[0059] Although the preferred embodiments of the present application are described above in combination with the drawings, the present application is not limited to the specific embodiments described above, and the specific embodiments described above are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection scope of the present application.

Claims

1. A cross-arranged latitude and longitude color LED light source, characterized by: The light source is a hemispherical dome, and LED particle light sources are arranged in the longitude direction and the latitude direction inside the hemispherical dome; wherein the latitude direction is arranged with blue-green-red or blue-cyan-green-yellow-red color light sources from the equator to the north pole; the longitude direction is arranged with blue-green-red or blue-cyan-green-yellow-red color light sources at equal angles; the trigger light source is opened by the controller in time with the longitude light source and the latitude light source, and the camera is used for image acquisition when the controller is triggered; the color coding information of the color light source is the color information of the longitude and latitude directions collected by the camera; the surface topography detection method based on the longitude and latitude color light sources is as follows: after the latitude light source image and the longitude light source image are obtained in time, the gradient size of the light-reflecting surface is reversely obtained from the chroma of the latitude light source image, the gradient direction of the light-reflecting surface is reversely obtained from the chroma in the longitude light source image, the normal direction of the light-reflecting surface is obtained from the gradient size and the gradient direction, and the three-dimensional topography of the light-reflecting surface is detected; The relationship between the chroma of the latitude or longitude light source image and the surface gradient size and the gradient direction is determined by the calibration ball; the calibration ball is placed on the reference plane at the position to be detected, and the images are collected by opening the longitude and latitude light sources in time; the z direction is defined as the shooting direction of the vision system, the x and y directions are defined as the image row direction and the column direction respectively, and the coordinate axis zero value is the center; the relationship between the chroma and the gradient size is determined by the latitude image, and the representation is as follows: (1) (2) where f is the distance between the camera and the surface of the measured object in the actual space; is the chrominance information, and is determined by the three channels; k, is a proportional coefficient; is the gradient size; The relationship between the chroma and the gradient direction is determined by the longitude image, and the representation is as follows: (3) (4) wherein is the gradient direction; The surface normal vector of the measured object is represented as: (5) wherein is the surface normal vector of the object being measured.

2. The cross-arranged latitude and longitude color LED light source according to claim 1, characterized in that, The LED particle is or the color laser light source is expanded.

3. The cross-arranged latitude and longitude color LED light source according to claim 1, characterized in that, The LED particle is or the projection light source is controlled by the computer and the color is switched as needed.

4. A cross-arranged latitude and longitude color LED light source according to any of claims 1-3, characterized in that, The color coding can be replaced by light source brightness time coding, and then a plurality of images are combined for decoding.

5. A cross-arranged latitude and longitude color LED light source according to any of claims 1-3, characterized in that, In the longitude and latitude color light source, the light sources of various colors in the longitude and latitude directions have independent switches, so that the images are collected in time under different color illuminations, color aliasing is prevented, and more accurate color coding is facilitated.

6. The cross-arrangement of latitude and longitude color light sources according to any of claims 1-3, characterized in that: For the latitude light source, the colors are arranged in the latitude direction and uniformly arranged in the longitude direction; for the longitude light source, the colors are arranged in the longitude direction and uniformly arranged in the latitude direction.

7. The cross-arrangement of latitude and longitude color light sources of claim 1, wherein, The method of single ball calibration and surface gradient calculation is changed into the method of multiple ball calibration and interpolation for calculating the surface gradient of a large field of view; a plurality of balls are placed at different positions on the reference plane, a plurality of images under the illumination of different light sources are obtained, and then the relationship between the chroma of the different position images and the ball gradient is established; when the target to be detected falls outside the calibration ball, the relationship model of the new position is calculated by interpolation.

8. The cross-arrangement of latitude and longitude color light sources of claim 1, wherein, The surface topography detection method, the calibration ball is selected according to the material of the measured object to adapt to different targets to be detected. The calibration method is used for substances of the same material and different surface states, and corresponding calibration balls are made; the relationship between image chroma and surface gradient is established respectively, and the relationship between image chroma and surface gradient is learned through a deep learning method, and an end-to-end model between the two is established, when a detection target with different surface states in different parts is encountered, the surface gradient is inversely solved by the deep learning model; the detection of the point as the target is changed into the detection of the region as the target, and the relationship model between the chroma combination and the topography is directly established by using a region segmentation network for different topographies, in actual detection, a region is regarded as a whole detection, and the classification or overall recovery of the surface gradient is realized; For specific topography defects in specific application places, a deep learning method is used to realize the detection and boundary positioning of the defects, the knowledge learned by the deep learning is directly defect knowledge or feature knowledge, and the feature knowledge is used to identify the defect type.

Images